EPA Contract EP-W-09-024
Work Assignment 1-06
December 2010
Environmental Technology
Verification Program
Environmental and Sustainable
Technology Evaluations
Report
ANDALYZE
LEAD-IN-PAINT TEST KIT
QUALITATIVE SPOT TEST KIT FOR LEAD IN PAINT
Prepared by
Battelle
Baiteiie
Tnc Business of Innovation
for
U.S. Environmental Protection Agency
En/ En/ En/
-------
EPA Contract EP-W-09-024
Work Assignment 1-06
December 2010
Environmental Technology
Verification Program
Environmental and Sustainable
Technology Evaluations
Report
ANDALYZE
LEAD-IN-PAINT TEST KIT
QUALITATIVE SPOT TEST KIT FOR LEAD IN PAINT
by
Stephanie Buehler, Dale Rhoda, and Bruce Buxton, Battelle
Julius Enriquez and Evelyn Hartzell, U.S. EPA
Battelle
Columbus, Ohio 43201
-------
Notice
Funding for this verification test was provided under Contract No. EP-W-09-024, Work
Assignments 4-16, 0-06, and 1-06, Office of Pollution Prevention, and Toxics, US EPA. The U.S.
Environmental Protection Agency, through its Office of Research and Development, managed
the research described herein. It has been subjected to the Agency's peer and administrative
review and has been approved for publication. Any opinions expressed in this report are those of
the author(s) and do not necessarily reflect the views of the Agency, therefore, no official
endorsement should be inferred. Any mention of trade names or commercial products does not
constitute endorsement or recommendation for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
Nation's land, air, and water resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, EPA's research program is providing data and technical support for solving
environmental problems today and building a science knowledge base necessary to manage our
ecological resources wisely, understand how pollutants affect our health, and prevent or reduce
environmental risks in the future.
The National Risk Management Research Laboratory (NRMRL) is the Agency's center for
investigation of technological and management approaches for preventing and reducing risks
from pollution that threaten human health and the environment. The focus of the Laboratory's
research program is on methods and their cost-effectiveness for prevention and control of
pollution to air, land, water, and subsurface resources; protection of water quality in public water
systems; remediation of contaminated sites, sediments and groundwater; prevention and control
of indoor air pollution; and restoration of ecosystems. NRMRL collaborates with both public
and private sector partners to foster technologies that reduce the cost of compliance and to
anticipate emerging problems. NRMRL's research provides solutions to environmental
problems by: developing and promoting technologies that protect and improve the environment;
advancing scientific and engineering information to support regulatory and policy decisions; and
providing the technical support and information transfer to ensure implementation of
environmental regulations and strategies at the national, state, and community levels.
This publication has been produced as part of the Laboratory's strategic long-term research plan.
It is published and made available by EPA's Office of Research and Development to assist the
user community and to link researchers with their clients.
Sally Gutierrez, Director
National Risk Management Research Laboratory
in
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We also would like to thank Al
Liabastre, ret. U.S. Army Center for Health Promotion and Preventive Medicine; David Jacobs,
National Center for Healthy Housing; Kenn White, American Industrial Hygiene Association;
Larry Franklin, Centers for Disease Control and Prevention, Coordinating Center for
Environmental Health and Injury Prevention, National Center for Environmental Health, Lead
Poisoning Prevention Branch; and Moira Lataille and Michael Crane, U.S. EPA for their careful
review of the test/quality assurance plan and this verification report. Quality assurance oversight
was provided by Michelle Henderson, U.S. EPA, and Zachary Willenberg and Rosanna Buhl,
Battelle.
IV
-------
Contents
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations and Acronyms viii
Chapter 1 Background 1
Chapter 2 Technology Description 2
Chapters Test Design and Procedures 4
3.1 Introduction 4
3.2 Test Facility 5
3.3 Test Procedures 5
3.3.1 Test Sample Collection, Storage, and Shipment 8
3.3.2 Test Sample Analysis Procedure 8
Chapter 4 Quality Assurance/Quality Control 11
4.1 Quality Control Samples 11
4.1.1 ICP-AES Blank Sample Results 11
4.1.2 ICP-AES Matrix Spike Samples and Calibration Verification Standards 11
4.1.3 Test Kit Quality Controls and Blank PEMs 12
4.2 Audits 13
4.2.1 Performance Evaluation Audits 13
4.2.2 Technical Systems Audit 13
4.2.3 Audit of Data Quality 13
Chapters Statistical Methods 15
5.1 False Positive and False Negative Rates 15
5.2 Precision 16
5.3 Sensitivity 16
5.4 Modeled Probability of Test Kit Response 16
5.4.2 Accounting for Measurement Error - SIMEX Background and Intuition 17
5.4.3 SIMEX Input and Analysis 18
5.4.4 Goodness of Fit 19
5.5 Matrix Effects 20
5.6 Operational Factors 20
Chapter 6 Test Results 21
6.1 False Positive and False Negative Rates 21
6.2 Precision 26
6.3 Sensitivity 30
6.4 Modeled Probability of Test Kit Response 31
6.5 Matrix Effect 42
-------
6.6 Operational Factors 42
Chapter 7 Performance Summary 45
Chapters References 48
Appendix A Performance Evaluation Materials Summary Information A-1
Al Preparation of Performance Evaluation Materials A-3
A2 Comparison of Expected vs. Actual Lead Concentrations of Performance Evaluation
Materials A-19
A3 Q A/QC Results for the ICP-AES Analysis of Performance Evaluation Materials A-22
Appendix B Vendor Comments B-l
Figures
Figure 2-1: Schematic representation of DNAzyme based lead sensing 2
Figure 2-2. ANDalyze Lead-in-Paint Test Kit Test extraction kit (left) and testing kit (right) 3
Figure 6-1. Probability curves that represent test kit results that are both perfect (red line) and
within RRP rule criteria (black solid line) 36
Figure 6-2. Probability curves with shaded region to denote performance results that meet RRP
rule false positive and negative criteria. Test kits with curves that fall within the shaded region
and avoid the white region meet the RRP rule 37
Figure 6-3. ANDalyze Lead-in-Paint Test Kit predicted probability of positive test result (solid
lines) with 90% prediction interval (dotted lines) for a white paint topcoat on various substrates.
38
Figure 6-4. ANDalyze Lead-in-Paint Test Kit predicted probability of positive test result (solid
lines) with 90% prediction interval (dotted lines) for a grey paint topcoat on various substrates.39
Figure 6-5. ANDalyze Lead-in-Paint Test Kit predicted probability of positive test result (solid
lines) with 90% prediction interval (dotted lines) for a red paint topcoat on various substrates.. 40
Tables
Table 3-1. PEMs Testing Scheme for Each Test Kit 6
Table 6-1. The number of panels in each false positive and false negative analysis category.... 22
Table 6-2. ANDalyze Lead-in-Paint Test Kit false positive results for panels with confirmed
lead levels < 0.8 mg/cm2 and false negative results for panels with confirmed lead levels > 1.2
mg/cm2 24
VI
-------
Table 6-3. ANDalyze Lead-in-Paint Test Kit false positive results for panels with confirmed
lead levels < 1 mg/cm2 and false negative results for panels with confirmed lead levels > 1
mg/cm2 25
Table 6-4. Actual lead levels and their replicate set labels 26
Table 6-5. The number of panels at each target level and the number in each replicate set bin . 27
Table 6-6. ANDalyze Lead-in-Paint Test Kit consistency results by operator type, lead type,
substrate, and lead level 29
Table 6-7. ANDalyze Lead-in-Paint Test Kit precision results by lead type and operator type .. 30
Table 6-8. ANDalyze Lead-in-Paint Test Kit sensitivity results - lowest lead level for which the
kit gave consistent positive results (mg/cm2) 30
Table 6-9. Average and standard deviation of ANDalyze Lead-in-Paint Test Kit results
compared to the concentration of the replicate sets 31
Table 6-10. ANDalyze Lead-in-Paint Test Kit univariate associations between probability of
positive response and explanatory variables 32
Table 6-11. ANDalyze Lead-in-Paint Test Kit multivariable Stata SIMEX logistic regression
parameter estimates 33
Table 6-12. ANDalyze Lead-in-Paint Test Kit modeled probability of positive test results and
upper 95% prediction bound when lead level = 0.8 mg/cm2 33
Table 6-13. ANDalyze Lead-in-Paint Test Kit modeled probability of positive test results, lower
95% prediction bound, and corresponding conservative estimate of the false negative rate when
lead level = 1.2 mg/cm2 34
Table 6-14. ANDalyze Lead-in-Paint Test Kit false positive and negative threshold values (95%
confidence) based on the modeled probability of test results 41
vn
-------
List of Abbreviations and Acronyms
AMS
ASTM
CCV
COC
CRM
DOT
EPA
ESTE
ETV
ICP-AES
LCS
mg/cm2
HL
mL
mm
MSDS
NLLAP
PE
PEM
ppb
PT
QA
QC
QCS
QMP
RRP
SOP
ISA
Advanced Monitoring Systems
American Society for Testing and Materials
continuing calibration verification
chain of custody
certified reference material
Department of Transportation
U.S. Environmental Protection Agency
Environmental and Sustainable Technology Evaluations
Environmental Technology Verification
inductively coupled plasma-atomic emission spectrometry
laboratory control spike
milligrams per centimeter squared
microliter
milliliter
millimeter
material safety data sheets
National Lead Laboratory Accreditation Program
performance evaluation
performance evaluation material
parts per billion
performance test
quality assurance
quality control
quality control sample
quality management plan
Renovation, Repair, and Painting
standard operating procedure
technical systems audit
Vlll
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted according to rigorous quality assurance
(QA) protocols to ensure that data of known and adequate quality are generated and that the
results are defensible.
This verification test was conducted under the U.S. EPA ETV program. Testing was performed
by Battelle, which served as the verification organization under the Environmental and
Sustainable Technology Evaluations (ESTE) arm of ETV. Battelle evaluated the performance of
qualitative spot test kits for lead in paint.
This verification test was developed with the support of a stakeholder technical panel. A
voluntary stakeholder technical panel consisting of individuals from the American Industrial
Hygiene Association (Kenn White), U.S. Department of Housing and Urban Development
(Warren Friedman), National Institute for Occupational Safety and Health (Kevin Ashley), U.S.
Army Center for Health Promotion and Preventative Medicine (Al Liabastre), National Center
for Healthy Housing (David Jacobs), National Association of Homebuilders (Matt Watkins), the
U.S. Consumer Product Safety Commission (Joanna Matheson), the Center for Disease Control
and Prevention (Larry Franklin), and U.S. EPA (Paul Carroll and Moira Lataille) was formed for
this verification test. Participants on this panel were reviewed and approved by EPA. This panel
gave input during the entire ETV process, including providing guidance and input on the
development of the performance evaluation materials used in this test, on the development of the
test design and test/QA plan, and comments on this report.
-------
Chapter 2
Technology Description
This report provides results for the verification testing of the Lead-in-Paint Test Kit for lead-
based paint by ANDalyze. The following is a description of the Lead-in-Paint Test Kit, based on
information provided by the vendor. The information provided below was not verified in this
test.
The ANDalyze Lead-in-Paint Test Kit utilizes a sensor/fluorimeter platform to quantitatively
detect lead in paint. The test is based on a sensing technology which uses DNA to identify lead.
Research done at the University of Illinois, Urbana Champaign used combinatorial biology to
identify a particular DNA sequence that specifically binds to lead ion (Pb2+) and catalyzes the
cleavage of another DNA sequence. These special DNA sequences capable of performing
catalysis are called DNAzymes (DNA enzymes). ANDalyze has converted this patented
technology into a test kit for lead. The DNA sequence specific for Pb2+ is linked to
fluorophores/quencher pair as depicted in Figure 2-1. Two strands of DNA, an enzyme strand
(shown in green) linked to a quencher and a substrate strand (shown in black) linked to a
fluorophore are held together by DNA hybridization. The fluorescence of the fluorophore is
quenched due to its close proximity to the quencher. In the presence of lead, the DNAzyme
catalyzes the cleavage of the substrate strand which releases the cleaved fragment containing the
fluorophore into solution thereby enhancing the fluorescence. The increased level of
fluorescence upon reaction with lead can be measured using a fluorimeter. The rate of this
increase is proportional to the lead concentration.
Substrate .... .^ % Fluorophore
I I I 11 I I nTTTT
Enzyme I II II II. LLLJJ^ Quencher
I
Pb2*
L /
I \
Figure 2-1: Schematic representation of DNAzyme based lead sensing
-------
The Lead-in-Paint Test Kit consists of two parts: the extraction kit (shown in Figure 2-2, left)
and the testing kit (shown in Figure 2-2, right). The extraction kit includes a razor blade, ruler,
drill bit, plastic tissue grinder, plastic transfer pipette, weighing paper and a bottle of 25% nitric
acid. A drill required for drilling the paint sample from surfaces may be purchased from
ANDalyze if the user does not already own one. The testing kit includes a portable fluorimeter
instrument and the following consumables: dried sensor in a plastic housing, syringe, glass tube,
30 milliliter (mL) plastic tube with test buffer, and 10 microliter (jiL) fixed volume pipette with
tips. A lead paint standard for calibrating the instrument is also supplied with the kit.
Figure 2-2. ANDalyze Lead-in-Paint Test Kit Test extraction kit (left) and testing kit (right)
To extract soluble lead (as Pb2+) from a dry paint surface, a 1.2 cm2 area of paint is either drilled
using a drill fitted with a 1A inch drill bit or cut using a razor blade. The selection of a drill or
manual scraping device is dependent upon the type of surface that is being tested. The entire
paint sample is transferred into a plastic tissue grinder to which 2 mL 25% nitric acid is added
using the plastic pipette. The paint chips are then ground to a fine powder by rotating the pestle
of the tissue grinder for approximately 2-5 minutes which results in Pb2+ being extracted into
the acidic solution. The test is performed by first transferring and mixing 10 jiL volume (using
the fixed volume pipette) of the acidified Pb2+ extract into the 30 mL plastic tube containing 20
mL of testing buffer. This is the test solution. A glass tube is inserted into the sample chamber of
the fluorimeter and sensor housing is placed on the glass tube. Using the syringe, 0.7 mL of test
solution is withdrawn and pushed through the sensor housing into the glass tube. The lead reacts
with the DNA-based sensor during this step. The housing is immediately removed, the lid is
closed and the START button is pressed. The lead concentration in paint is displayed on the
screen within 30 seconds in units of mg/cm2.
At the time of testing, a test kit included a fluorimeter at $1500 and consumables for 50 tests at
$300. Refill consumables could be purchased for further testing. Optional: A Craftsman drill
could be purchased from ANDalyze at a cost of $310 if the user does not own one. The
ANDalyze fluorimeter could be used for any other tests which utilize fluorescent sensing
methods.
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Qualitative Spot Test Kits for Lead in Paint1 Lead-based paints were commonly
used in houses in both interior and exterior applications prior to 1978, when the US government
banned the use of lead-based paint in residential applications. The term lead-based paint means
paint or other surface coatings that contain lead at contents that equal or exceed a level of 1.0
milligrams per centimeter squared (mg/cm2) or 0.5 percent by weight.2 This paint still exists in
many of these houses across the country. The accurate and efficient identification of lead-based
paint in housing is important to the Federal government as well as private individuals living in
residences containing such paints. Renovation, repair, and painting (RRP) activities may disturb
painted surfaces and produce a lead exposure hazard. Such disturbances can be especially
harmful to children and pregnant women as lead exposure can cause neurological and
developmental problems in both children and fetuses. In fact, because of the large amount of
pre-1978 housing stock, a report by the President's Task Force on Environmental Health Risks
and Safety Risks to Children found that approximately 24 million US dwellings were at risk for
lead-based paint hazards.3
There are lead-based paint test kits available to help home owners and contractors identify lead-
based paint hazards before any RRP activities take place so that proper health and safety
measures can be taken. However, many of these test kits have been found to have high rates of
false positives (i.e., test kit indicates that lead in excess of 1.0 mg/cm2 is present, while in fact
the true lead level is below 1.0 mg/cm2).4 This verification test was conducted in response to the
call of the Renovation, Repair, and Painting rule2 for an EPA evaluation and recognition program
for test kits that are candidates to meet the goal of a demonstrated probability (with 95%
confidence) of a false negative response less than or equal to 5% of the time for paint containing
lead at or above the regulated level, 1.0 mg/cm2 and a demonstrated probability (with 95%
confidence) of a false positive response less than or equal to 10% of the time for paint containing
lead below the regulated level, 1.0 mg/cm2. This test incorporated ASTM International's El828,
Standard Practice for Evaluating the Performance Characteristics of Qualitative Chemical Spot
Test Kits for Lead in Paint5 guidelines into the test design.
The objective of this verification test was to evaluate the performance of the test kits for the
detection of lead in paint. This evaluation assessed the capabilities of the lead paint spot test kits
against laboratory-prepared performance evaluation material (PEM) samples and compared the
-------
lead paint test kit results with those of a standard technique, inductively coupled plasma-atomic
emission spectrometry (ICP-AES). Additionally, this verification test relied on verification
testing staff observations to assess other performance characteristics of the lead paint test kits.
Only qualitative results (e.g., detect/non-detect of lead at specified levels) were considered for
each technology.
The ANDalyze Lead-in-Paint Test Kit was verified by evaluating the following parameters:
False positive and false negative rates
Precision
Sensitivity
Modeled probability of test kit response
Matrix effects
Operational factors.
Verification testing of the test kit was conducted from January to June 2010. This timeframe
included testing of the test kit and also completion of all ICP-AES and QC analyses. False
positive and negative rates were determined by comparing test kit responses to actual lead
concentrations of the PEM as determined through ICP-AES. Precision was determined by
reproducibility of responses for replicate samples. Sensitivity was determined as the lowest
detectable level of the test kit. The modeled probability and matrix effects were determined
using logistic regression models.
Operational factors such as ease of use, operator bias, average cost, average time for kit
operation, helpfulness of manuals, and sustainability metrics such as volume and type of waste
generated from the use of each test kit, toxicity of the chemicals used, and energy consumption
were determined based on documented observations of the testing staff and the Battelle
Verification Test Coordinator. Operational factors were described qualitatively, not
quantitatively; therefore, no statistical approaches were applied to the operational factors.
3.2 Test Facility
Laboratory analyses of the ANDalyze Lead-in-Paint Test Kit were conducted in Battelle
laboratories in Columbus, Ohio. No field testing was conducted during this technology
verification.
3.3 Test Procedures
Qualitative spot test kits for lead in paint were evaluated against a range of lead concentrations in
paint on various substrates through the use of PEMs. PEMs were 3 inch by 3 inch square panels
of wood (pine and poplar), metal, drywall, or plaster that were prepared by Battelle.6 Pine and
poplar were chosen for the wood panels as they are representative of woods most commonly
found in homes. Table 3-1 shows the PEMs prepared for each test kit. Poplar and pine PEMs
were distributed in random mixtures (e.g., two poplar and one pine or one poplar and two pine)
for each set of three wood PEMs listed in Table 3-1. Each PEM was coated with either white
lead (lead carbonate) or yellow lead (lead chromate) paint. The paint contained lead targeted at
-------
Table 3-1. PEMs Testing Scheme for Each Test Kit
Lead Type
Control Blank
White Lead
(Lead Carbonate)
Yellow Lead
(Lead Chromate)
Lead Level
(mg/cm2)
0
0.3
0.6
1.0
1.4
2.0
6.0
0.3
0.6
1.0
1.4
2.0
6.0
Substrate
Wood
Metal
Dry wall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Wood
Metal
Drywall
Plaster
Painted PEMs Subtotal
PEMs Analyzed Per Test Kit by Topcoat Color
White
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
156
Red-Orange
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
156
Grey-Black
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
156
Unpainted PEMs Subtotal (2 per each substrate)
Total
Total
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
468
8
476
Actual number of PEMs used to evaluate performance at specific lead levels varied based on actual concentrations observed during analysis.
-------
0.3, 0.6, 1.0, 1.4, 2.0, and 6.0 mg/cm2. These lead concentrations were chosen with input from
the stakeholder technical panel based on criteria provided in EPA's lead RRP rule as well as to
represent potential lead levels in homes. Paint containing no lead (0.0 mg/cm2) was also applied
to each substrate and tested.
Two different layers of paint were applied over the leaded paint. One was a primer designed for
adhesion to linseed oil-based paint and the second coat was a typical interior modern latex paint
tinted to one of three colors: white, red-orange, or grey-black. These colors were chosen by
EPA, with input from the stakeholder technical panel, based on the potential of certain colors to
interfere or not with lead paint test kit operations. The topcoat paint manufacturers'
recommended application thickness was used. Two coats at the recommended thickness were
applied. Details on the PEM production process can be found in Appendix A.
The ANDalyze Lead-in-Paint Test Kit for lead paint was operated by a technical and non-
technical operator. The technical operator was a Battelle staff member with laboratory
experience. The technical operator was trained by a representative of the vendor company in the
operation of its test kit. The same technical operator operated this test kit throughout testing.
Because this lead paint test kit is anticipated to be used by certified remodelers, renovators, and
painters, it was also evaluated by a non-technical operator. The non-technical operator was a
certified renovator with little to no experience with lead analysis. The non-technical operator
was also a college graduate. The non-technical operator was provided the instruction manual,
demonstrational DVD, and other materials (operational tip sheet, material safety data sheets
(MSDS)) typically provided by the vendor with the test kit for training. The non-technical
operator viewed the materials himself to understand how to operate the test kit. The non-
technical operator was also permitted to ask questions or clarifications of the vendor on the
operation of the test kit. This scenario approximated the training renovators are expected to
receive under the RRP rule.
Tests were performed in duplicate on each PEM by each operator, technical and non-technical
(i.e., two samples were taken from each PEM by each operator). Duplicates were tested in
succession by each operator on a given PEM. PEMs were analyzed blindly by each operator in
that the PEMs used for analysis were marked with a non-identifying number. Test kit operators
were not made aware of the paint type, lead level, or substrate of the PEM being tested. PEMs
were tested in random order (i.e., PEMs were placed in plastic bins and the operators arbitrarily
selected a PEM for analysis). To determine whether the substrate material affected the
performance of the test kits, two unpainted PEMs of each substrate were tested using each test
kit, in the same manner as all other PEMs (i.e., per the test kit instructions). Three PEMs at each
lead level, substrate, and topcoat color were prepared for use in this test. In total, 468 painted
PEMs were prepared for use in the verification test of each test kit.
Paint chip samples from each PEM were analyzed by a National Lead Laboratory Accreditation
Program (NLLAP) recognized laboratory, Schneider Laboratories, Inc., using ICP-AES to
confirm the lead level of each PEM used for testing. The paint chip samples for reference
analyses were collected by Battelle according to a Battelle SOP7, which was based on ASTM
E1729.8 The reference analyses confirmed the lead level of each PEM. Lead levels determined
through the reference analysis were used for reporting and statistical analyses.
-------
The procedures for collecting, storing, and shipping test samples are provided below.
3.3.1 Test Sample Collection, Storage, and Shipment
Chips of lead paint were taken from each PEM and sent for ICP-AES analysis at a NLLAP-
recognized laboratory, Schneider Laboratories, Inc. A glass screw-top vial was labeled with the
PEM identification number located on the back of the panel. The number was also recorded on
the Chain of Custody (COC) form. Sampling was performed per the Battelle SOP for collection
of dried paint samples for lead determination. All safety precautions and personal protective
equipment were used. A one inch square, metal template was placed adjacent to the tested area.
A utility knife was used to trace around the template. Tweezers and a utility knife were used to
scrape and remove the paint within the one inch area, using caution to minimize introduction of
the substrate into the paint sample. The topcoat and remaining paint were transferred to a
glassine weighing paper with the assistance of a paintbrush. The sample was then transferred
from the glassine paper into a glass vial using the paintbrush. All instruments and templates were
wiped with tissue paper and the bench top was cleaned and gloves were changed between each
sample to minimize contamination. The paint brush was carefully flicked and tapped over a trash
can to remove any residual lead dust. All wipes and gloves were disposed of as lead waste. The
vials were then collected into a zip-top bag and taped up securely for shipping. The bags and
COC were then shipped together using overnight delivery to Schneider Laboratories, Inc.
Paint chip samples were stored at room temperature as received by Schneider Laboratories, Inc.
and then analyzed by ICP-AES. Analytical results were reported to Battelle within 2-3 days.
Sample digests were stored separately by Schneider Laboratories, Inc. at room temperature.
PEMs were stored individually in zip-top bags. The back of each PEM was labeled with an
identifying number. The outside of the zip-top bag was labeled with the same number. Each
PEM was wrapped in a Kimwipe and each zip-top bag was sealed when not in use. The zip-top
bags containing the PEMs were housed in large plastic bins in the laboratory during testing.
3.3.2 Test Sample Analysis Procedure
At the beginning of each day of testing, prior to the analysis of any samples, the fluorimeter
apparatus for the lead measurement was calibrated using a three-point calibration. First the
fluorimeter was plugged into an electrical outlet and allowed to warm up for at least 5 minutes.
Then standard solutions of 0.2, 1.0 and 5.0 mg/cm2 lead were prepared as follows: 10 |jL of the
0.2 mg/cm2 standard, Stock Solution 1 as supplied with the test kit, were added to the pre-
prepared buffer Tube 1 and mixed. Calibration standards for the remaining two levels were
prepared similarly, with 10 |jL of the 1.0 mg/cm2 lead standard, Stock Solution 2, added to the
pre-prepared buffer tube 2 and 10 \\L of the 5.0 mg/cm2 lead standard, Stock Solution 3, added to
Tube3.
The following steps were taken to prepare the fluorimeter for calibration. On the meter display
panel of the fluorimeter, "Menu" was pressed. Then the operator scrolled down to "Calibration"
on the screen and pressed "Select" and then "Next". Once the fluorimeter was prepared, a clean
glass test tube was placed into the receptacle of the meter. Then a green sensor housing was
-------
placed over the test tube. Using the 1 mL sterile syringe supplied with the test kit, 0.7 mL of
solution from Tube 1 was transferred into the sensor housing. The syringe and housing were
removed and discarded and the fluorimeter lid was quickly closed. Immediately following the
closing of the lid on the fluorimeter, "OK" was pressed on the display panel. After a reading was
obtained, the glass test tube was removed and discarded. This process was repeated using Tubes
2 and 3. After the final calibration solution (Tube 3) was analyzed, a R2 value was generated by
the fluorimeter. If the R2 reading was above 0.98, the calibration was considered acceptable and
"Apply" was pressed on the meter display panel. If the R2 reading was below 0.98, the
calibration process was repeated.
Once the fluorimeter was calibrated, sampling was conducted. Paint samples were obtained
from the selected PEM using either the drill or scrape method, depending on the surface that was
being tested.
A modified half-inch spade bit attached to an 18 volt Craftsman cordless hand drill was used on
the wood and plaster PEMs. The spade bit was lightly placed on the PEM surface, and the drill
was activated to a slow speed to remove the paint and try to get as little of the substrate as
possible.
An X-actoฎ knife was used for collection on the metal and drywall PEMs. The ANDalyze Lead-
in-Paint Test Kit came with a razor blade for cutting up the paint sample and scraping paint from
the sample area of interest. The vendor indicated that it was not necessary to use this razor blade
if the user had a suitable replacement. The ANDalyze Lead-in-Paint Test Kit instructions called
for using a ruler to measure a 1.1 cm x 1.1 cm area on metal and drywall substrates for removal
of a paint sample. Because of the large number of samples being generated in this study, a
stainless steel template was used by both the technical and non-technical operator. The X-actoฎ
knife was used to trace around the outside of the stainless steel template. The paint sample was
then removed.
In both sampling methods, after removal from the PEM, the paint sample was cut up into small
pieces using the X-actoฎ knife and then placed into the labeled grinding tube. Once the paint
sample was removed and placed into the grinding tube, 2 mL of the 25% nitric acid solution was
added to the grinding tube using a 3 mL plastic transfer pipette. Per tips for successful operation
of the kit provided by the vendor, the paint sample was allowed to sit in the acid for 5 minutes.
This softened up the paint sample and allowed for easier grinding. After 5 minutes in the acid,
the pestle was screwed onto the grinding tube and the pestle was rotated, with an up and down
motion, or the pestle was held and the tube was rotated. The ANDalyze Lead-in-Paint Test Kit
instructions indicated that the paint sample should be ground until it turns into a powder, up to
seven minutes. It was determined during testing that 3 minutes was needed to grind each of the
samples for this test.
Because nearly 1000 samples were needed to be ground by each operator using the pestle and
grinding tube, there was concern that the operators might acquire a repetitive motion disorder
over the course of testing. To alleviate this concern, additional Battelle laboratory technicians
were brought in to perform the grinding step, in conjunction with the efforts of the operators, for
both the technical and non-technical operator. The technical and non-technical operator each
-------
calibrated the fluorimeter, removed the paint sample from the PEM, added acid to the pestle, and
analyzed the resulting extract (see below for details). However, for approximately half of the
samples analyzed on a given day, one to two additional Battelle laboratory technicians helped
conduct the grinding step for a given sample. It was noted on the data collection sheets when a
person other than the operator conducted the grinding. All grinding was performed for three
minutes.
After the grinding of the sample was complete, 10 |jL of the ground paint/acid mixture was
removed using a pipette and placed into a plastic tube containing 20 mL of the testing buffer.
The tube was capped and shaken to mix.
Next, the fluorimeter was prepared for analyzing the sample extract. A clean glass test tube was
placed into the receptacle of the fluorimeter. Then a green sensor housing was placed over the
test tube. Using a sterile 1 mL syringe supplied with the test kit, 0.7 mL of the lead test solution
was transferred into the test tube. The syringe and housing were removed and discarded and the
fluorimeter lid was quickly closed. "Start" was pressed on the fluorimeter display panel and the
sample was analyzed. The value displayed on the fluorimeter screen provided the concentration
(mg/cm2) of lead in the sample.
After each sample the pestle and grinding tube had to be cleaned. After emptying the pestle and
grinding tube of any lead solution, both components were cleaned with a brush and tap water.
Then 2 mL of the cleaning solution was placed into the grinding tube. Pestles were then placed
into tube. The pestle and grinding tube were allowed to sit for at least 30 minutes. Then the
pestle was removed and the cleaning solution was poured out. The brush was used to remove any
remaining debris, and the pestle and grinding tube were rinsed and completely dried before the
next use.
10
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed according to the quality management plan (QMP) for the
Battelle ETV Advanced Monitoring Systems (AMS) Center9, except where differences were
noted for ESTE per the EPA ETV Program QMP10, and the test/QA plan for this verification
test.1 Test procedures were as stated in the test/QA plan; however a deviation to the test/QA plan
was made during the ICP-AES analyses. For some sample runs, continuous calibration
verification (CCV) samples were run once every 20 instead of 10 samples. This deviation is
described below. This change was assessed to have no impact on the quality of the results as
described below. QA/QC procedures and results are described below. Additional information
on QA/QC outcomes for the PEMs is provided in Appendix A.
4.1 Quality Control Samples
Steps were taken to maintain the quality of data collected during this verification test. This
included analyzing specific quality control samples for the reference method (ICP-AES) and the
test kit.
4.1.1 ICP-AES Blank Sample Results
Various blank samples were analyzed for the ICP-AES analyses. Method blank samples were
analyzed in each set of 10-20 paint samples to ensure that no sources of contamination were
present. An initial calibration blank was analyzed at the beginning of each run and used for
initial calibration and zeroing the instrument. A continuing calibration blank was analyzed after
each CCV to verify blank response and freedom from carryover. No blank samples failed during
the analyses.
4.1.2 ICP-AES Matrix Spike Samples and Calibration Verification Standards
Initial calibration standards were run at the beginning of each set of analyses. The acceptance
criterion for the calibration coefficient of the calibration standards was > 0.998. If this criterion
was not met, the analysis was stopped and recalibration was performed before samples were
analyzed. A 500 parts per billion (ppb) CCV standard was analyzed at the beginning of each run
(following the initial calibration), at the end of each run, and every 10-20 samples. CCV
recoveries ranged from 96% to 108%. Per the test/QA plan, CCV sample frequency was once
every 10 samples. For most of the sample sets CCVs were performed with this frequency.
11
-------
However, for later sample sets CCVs were run once every 20 samples. CCV samples were used
to verify instrument performance. CCV samples were run every 10 samples as a preventative
measure so that large amounts of samples do not need to be re-run if a CCV sample fails. In the
course of this study, one CCV sample failed. All samples from the last passing CCV of that
sample set were re-analyzed.
A matrix spike sample and laboratory control sample (LCS), as well as duplicates of these
samples, were also analyzed. Duplicate samples were run once every 10-20 samples. Acceptable
recoveries for matrix spike samples were between 80-120%. Acceptable recoveries for LCS
samples were between 80-120%. Duplicate samples had acceptance criteria of ฑ25% relative
percent difference (RPD).
All matrix spike samples were performed as post-digestion spikes as there was insufficient
sample volume to perform a pre-digestion spike. Matrix spike recoveries ranged from 86% to
207%. Six matrix spike samples failed, with recoveries above the specified acceptance criteria.
In these instances, the lead concentration in the sample was well above the spike level. Matrix
spike results indicated that matrix interferences were not observed. Duplicate samples were
within the specified RPD.
LCS samples were analyzed once every 10-20 samples. LCS recoveries ranged from 17% to
225%. Schneider Laboratories, Inc. noted that LCS failures on one sample set were attributed to
improper spiking technique. Training on spiking procedures was immediately implemented by
Schneider Laboratories for all analysts spiking samples. All LCS failures occurred prior to a
revision to the Schneider Laboratories, Inc. SOP11 for analyzing paint samples written
specifically for this verification test. In the original version of the SOP, LCS samples were
prepared by spiking a known amount of lead onto a certified reference material (CRM). This
practice was changed because there were over-recovery issues. This was because the spike was
not >3x the background lead concentration because of the high lead concentrations in the actual
CRM samples. In the revised SOP, the LCS was prepared by spiking a piece of lead-free latex
paint. There were no LCS failures after that. In addition, a QC check sample containing only
the CRM, which had a known concentration of lead weighed out to a particular amount, was
analyzed with each sample set throughout the verification test. These QC check samples all
passed acceptance criteria.
4.1.3 Test Kit Quality Controls and Blank PEMs
As indicated in Section 3.3.2, quality control measures were built into the test procedures
through the calibration of the fluorimeter. All fluorimeter calibrations had to obtain an R2 value
of 0.98 or higher to pass calibration. All calibrations obtained for this test were above this R2
value. Painted PEMs containing no lead as well as each of the PEM substrates containing no
paint were also run as part of the verification test. All samples of PEM substrates containing no
paint returned negative results from the test kit (i.e., no lead was present). All of painted PEMs
containing no lead returned negative results.
12
-------
4.2 Audits
Three types of audits were performed during the verification test: a performance evaluation (PE)
audit of the reference method measurements made in this verification test, a technical systems
audit (ISA) of the verification test performance, and a data quality audit. Audit procedures are
described below.
4.2.1 Performance Evaluation Audits
A PE audit was conducted to assess the quality of the reference method measurements made in
this verification test. The reference method PE audit was performed by supplying an
independent, NIST-traceable lead paint standard (Reference Material 8680, panel CBS), to the
reference laboratory. The PE audit samples were analyzed in the same manner as all other
samples and the analytical results for the PE audit samples were compared with the nominal
concentration. The target criterion for this PE audit was in agreement with the analytical result
within 20% of the nominal concentration. The specified acceptable concentration range for the
NIST standard panel was 1.13 - 1.75 mg/cm2 (1.44 ฑ0.31 mg/cm2). The PE samples taken from
this standard panel were 1.38, 1.38, 1.19, and 1.31 mg/cm2. The PE audit result met the target
criterion. This audit was performed once at the start of the test.
4.2.2 Technical Systems A udit
The Battelle Quality Manager performed one TSA during this verification test to ensure that the
verification test was being performed according to the Battelle AMS Center and ETV Program
QMPs, the test/QA plan, any published reference methods, and standard operating procedures. In
the TSA, the Battelle Quality Manager reviewed the reference methods used, compared actual
test procedures with those specified or referenced in the test/QA plan, and reviewed data
acquisition and handling procedures. Also in the TSA, the Battelle Quality Manager observed
testing, observed reference method sample preparation and analysis, inspected documentation,
and reviewed technology-specific record books. He also checked standard certifications and
technology data acquisition procedures and conferred with the technical staff. A TSA report was
prepared. There were no findings. The records concerning the TSA are permanently stored with
the Battelle Quality Manager.
The EPA ETV Quality Manager also performed a TSA of both the reference laboratory and the
testing conducted at Battelle Columbus, OH facilities. No findings were reported in the TSA of
the reference laboratory, Schneider Laboratories, Inc. In the TSA of the lead paint test kit
evaluations at Battelle's Columbus, OH facilities, the EPA ETV Quality Manger cited two
findings. These findings were related to ease of use observations and were immediately and
adequately addressed and did not affect the quality of the test.
4.2.3 Audit of Data Quality
Records generated in the verification test received a one-over-one review (i.e., review by a
Battelle technical staff who did not generate the records) before these records were used to
calculate, evaluate, or report verification results. A Battelle technical staff member involved in
the verification test reviewed the data. Datasheets generated by the operators during testing were
13
-------
reviewed for completeness and errors. The person performing the review added his/her initials
and the date to a hard copy of the record being reviewed. At least 10% of the data acquired
during the verification test, including the ICP-AES results, were audited by Battelle. At least
25% of the ICP-AES data acquired during the verification test were audited by EPA. Battelle's
Quality Manager traced the data from the initial acquisition, through reduction and statistical
analysis, to final reporting to ensure the integrity of the reported results. All calculations
performed on the data undergoing the audit were checked. Minor transcription errors were
identified and corrected before the results were used for the calculations described in Chapter 5.
Battelle's and EPA's Quality Managers also reviewed the PEM ICP-AES results thoroughly to
ensure that all data quality indicators as stated in the test/QA plan were followed and that
reported results matched the data generated on the instrument. Findings were cited by the EPA
Quality Manager. Appropriate corrective actions were taken. Significant QA/QC concerns
identified during EPA's audit are discussed in Section 4.1.
14
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the performance factors listed in Section 3.1 are
presented in this chapter. The ANDalyze Lead-in-Paint Test Kit was evaluated for qualitative
results (i.e., positive/negative responses to samples). All data analyses were based on these
qualitative results. QC samples and unpainted PEM substrates were not included in any of these
analyses. Results are provided in Chapter 6.
5.1 False Positive and False Negative Rates
A false positive response was defined as a positive result when regulated lead-based paint was
not present. The test/QA plan1 defined false positive rates as being based on target lead levels at
and below 0.6 mg/cm2 with confirmed values not to exceed 0.8 mg/cm2. Because confirmed lead
levels of particular PEMs did not sometimes match target concentrations for those PEMs, false
positive rates were assessed on panels with confirmed lead levels at 0.8 mg/cm2 and lower.
Consistent with the EPA's April 22, 2008 RRP rule2, panels with an ICP-AES confirmed lead
level between 0.8 and 1.0 mg/cm2 were not used in the false positive analysis.
A false negative response was defined as a negative response when regulated lead-based paint
was present. The test/QA plan defined false negative rates as being based on target lead levels at
and above 1.4 mg/cm2 with confirmed values not to exceed 1.2 mg/cm2. Because confirmed
lead levels of particular PEMs did not sometimes match target concentrations for those PEMs,
false negative rates were assessed on panels with confirmed lead levels at 1.2 mg/cm2 and
higher. Consistent with the EPA's April 22, 2008 RRP rule, panels with an ICP-AES confirmed
lead level between 1.0 and 1.2 mg/cm2 were not used in the false negative analysis.
Based on stakeholder technical panel input, the EPA lead paint action level of 1.0 mg/cm2 lead
was included for analysis as part of the verification test. Though evaluations of test kit
performance based on this level is not in the EPA RRP rule, false positive and negative rates, in
addition to those stated above, were also calculated for each test kit based on 1.0 mg/cm2 lead.
Thus, false positive rates were assessed on PEMs with confirmed lead levels at 1.0 mg/cm2 and
lower and false negative rates were assessed on PEMs with confirmed lead levels at 1.0 mg/cm2
and higher. For panels that measure 1.0 mg/cm2, positive results were considered "correct" and
negative results were considered false negative. If the confirmed lead concentration of the PEM
was greater than 1.0 mg/cm2 (e.g., 1.1 mg/cm2), then negative results were considered false
15
-------
negatives. If the confirmed lead concentration of the PEM was less than 1.0 mg/cm2 (e.g., 0.9
mg/cm2), then positive results were considered false positives.
False positive and negative rates were calculated as shown in Equations 1 and 2, respectively:
, n v. ^ # of positive results ...
False Positive Rate = ^- (1)
total # of PEMs with lead level below 0.8 (or 1.0) mg/cm
# of negative results
False Negative Rate =
total # of PEMs with lead level above 1.2 (or 1.0) mg/cm2
5.2 Precision
Precision was measured by the reproducibility of responses for replicate samples within a group
of PEMs. Precision results were reported as the percentage of consistent responses from all
replicate sets for those paint types (see Equation 3). Responses were considered inconsistent if
25% or more of the replicates differed from the response of the other samples in the same group
of PEMs.
..,.. . , . # of consistent responses of replicate sets n^ ,..
Precision (% consistent results) = x 100 (3)
total number of replicate sets
5.3 Sensitivity
The sensitivity or lowest detectable lead level for each test kit was identified based on the
detection results across all PEM lead levels. The lowest PEM lead level with consistent (>75%)
positive or "detect" responses was considered the lowest detectable level. The identified lowest
detectable lead level was reported and discussed.
5.4 Modeled Probability of Test Kit Response
Logistic regression models were used to determine the probabilities of positive or negative
responses of the test kit at the 95% confidence level, as a function of lead concentration and
other covariates, such as substrate type, lead paint type, operator type, and topcoat color. An
evaluation of the bivariate relationship between the response variable and each candidate
explanatory variable was performed by fitting single covariate logistic models to assess the
predictive ability of each of the PEM parameters. Using the results from these bivariate
analyses, a parsimonious multivariate model was developed including a set of explanatory
variables which were most predictive of the probability of the test kit response variable. The
potential logistic regression model took the form below:
16
-------
logit(Pr
-------
concentration and measurement error. If we were able to remove the measurement error then we
would observe less variability in that independent variable.
There are two important points of intuition that will inform expectations about what is seen in the
SIMEX results. First, the data along the x-axis of a scatterplot would "tighten up" if
measurement error were removed. "Tightening up" the independent variable in a regression
analysis will result in a steeper slope or a regression coefficient with a larger magnitude. This is a
fundamental consequence of any technique that adjusts for measurement error in the independent
variable in a regression analysis. In the lead paint analysis, steeper logistic regression curves
will result from the SIMEX analysis than would result from a non-SIMEX analysis where lead
levels were considered to be fixed and known.
Second, when the statistical analyses acknowledge and account for the measurement error, then
the regression output prediction intervals may be wider than those for a non-SEVIEX analysis
where 'x' is considered to be fixed and known. For any given predicted value of the outcome
variable, the prediction interval will most likely be wider, or at least not narrower. But for a
fixed value of 'x', (such as 0.8 or 1.2 mg/cm2) whether the SIMEX prediction intervals are wider
or narrower than the non-SIMEX intervals depends on how much the slopes of the SIMEX and
non-SIMEX regression line differ. For typical logistic regression models, prediction intervals
are very narrow at the extreme low and high asymptotic ends of the x-axis, and only appreciably
wide in the region where the probability of the outcome is not near zero and not near one. So if
the SIMEX analysis has only a moderate impact on the slope then wider prediction intervals
might be observed at 0.8 and 1.2 mg/cm2. But if the slope changes dramatically, then 0.8 or 1.2
mg/cm2 might now be in the part of the prediction curve that is near zero or one and the SIMEX
prediction interval might be dramatically more narrow than a non-SIMEX interval.
Thus, the prediction curves for every SIMEX analysis are expected to be steeper than, or at least
not less steep than, a non-SIMEX analysis. However, the assessment of test kit performance is
based on the upper and lower bounds of prediction intervals at 0.8 and 1.2 .mg/cm2, respectively.
5.4.3 SIMEX Input and Analysis
During pre-production of the PEMs, replicate paint chip samples were analyzed from selected
metal PEMs that served as reference panels (see Appendix A). Three metal panels were
prepared for the pre-production homogeneity testing. Four paint chip samples, one from each
quadrant of the PEM, were taken and analyzed via ICP-AES for their lead levels. Data are
available on the coefficients of variation for these metal PEMs for both white and yellow lead.
These data are shown below in Table 5.1. Though these data did not come from actual PEMs
used during the lead paint test kit verification test, this information was used as a surrogate
measure of homogeneity variability on the PEMs.
For each PEM in the study, nine random pseudo-replicates were generated from a normal
distribution with a mean equal to the confirmed lead concentration for that panel, and a standard
deviation computed from the metal reference PEM data in Table 5-1 and indexed by the panel's
lead type and target lead level. The nine measurements were used as inputs to the Stata SIMEX
algorithm as if they were true replicate measurements.
18
-------
Table 5-1. Results from Final Homogeneity Testing for each Set of ETV PEMs
Target Lead Mean Levels CoV*
Lead Type Level ICP (mg/cm2) ICP
White Lead
Yellow Lead
0.3
0.6
1.0
1.4
2.0
6.0
0.3
0.6
1.0
1.4
2.0
6.0
0.30
0.65
0.99
1.56
1.85
5.97
0.30
0.62
1.07
1.42
1.92
6.88
13.3
7.1
3.9
7.2
5.6
14.2
9.6
4.1
11.0
4.1
10.1
5.2
* Coefficient of Variation (Standard Deviation/Mean x 100)
There are two user-specified parameters for the Stata SIMEX algorithm: 1) the number of
replicate measurements for the covariate measured with error, and 2) the number of bootstrap
samples used to estimate standard errors on regression parameters. In testing not detailed here,
the sensitivity of the SIMEX algorithm to different settings of these parameters was investigated.
It was determined that the qualitative results were not sensitive to the values used in the analysis.
The values used were nine pseudo-replicates per PEM and 199 bootstrap samples, respectively.
The predicted regression curves and associated prediction intervals were generated in the interval
0.0 to 6.0 mg/cm2 using Stata. The relevant prediction bounds (the upper bound at 0.8 mg/cm2
and lower bound at 1.2 mg/cm2) were assessed and the predicted false positive and false negative
rates based on these prediction bounds were determined.
5.4.4 Goodness of Fit
To assess whether the logistic regression models fit the data well, standardized Pearson residuals
were computed for every observation and those with an absolute value greater than two were
flagged and plotted versus lead level. Standardized Pearson residuals greater than two are
associated with observations that are not well fit by the model. In the logistic regression context
observations that are not well fit might be those with high lead levels where the kit results were
negative or very low lead levels where the kit results were positive. In the absence of categorical
variables the standardized Pearson residuals should be normally distributed, so we would expect
approximately 5% of the observations to have residuals with absolute value greater than two. In
this case there are categorical covariates so the residuals are not strictly expected to be
distributed normally but the proportion of observations with large residuals is still informative.
That proportion is reported in Section 6.4.
19
-------
5.5 Matrix Effects
The covariate-adjusted logistic regression model described in Section 5.4 was used to assess the
significance of PEM parameters and the interactions among them on the performance of the test
kits. PEM parameters were included in the model as explanatory variables associated with the Y;
response variable.
Comparison of the observed values of the response variable to predicted values obtained from
models with and without the predictor variable in question was the guiding principle in the
logistic regression model. The likelihood function is defined as
Y.\-xY (5)
where ;r(Y.) is the conditional probability of Y; =1 and [1 - ;r(Y.)] is the conditional probability
of Y;i =0 given the vector of explanatory variables (X). For purposes of assessing the
significance of a group of p predictor variables (where p can be 1 or more), we computed the
likelihood ratio test statistic, G, as follows:
G = -2 logs [likelihood without the p variables / likelihood with the p variables] (6)
Under the null hypothesis, this test statistic followed a chi-square distribution with p degrees of
freedom. If the test statistic was greater than the 95th percentile of the chi-square distribution,
then the group of variables, taken together, were statistically significant.
5.6 Operational Factors
There were no statistical calculations applicable to operational factors. Operational factors were
determined qualitatively based on assessments from the Operator (both technical and non-
technical) and the Battelle Verification Test Coordinator. Operational factors such as ease of
use, operator bias, average cost, average time for kit operation, and helpfulness of manuals, were
determined. Sustainability metrics such as volume and type of waste generated from the use of
each test kit, toxicity of the chemicals used, and energy consumption are discussed. This
discussion is based on how much waste was generated and what the waste was composed of,
information from the vendor on how the waste should be properly handled, a summary of the
pertinent MSDS information, when available, and noting whether the test kit used batteries, a
power supply, or no energy source was needed. Information on how many tests each kit could
perform as well as the shelf life of the test kit and chemicals used as part of the test kit was also
reported.
20
-------
Chapter 6
Test Results
The results for the ANDalyze Lead-in-Paint Test Kit are presented below for each of the
performance parameters. The interpretation of results for this test kit relied on the use of a
fluorimeter. A specific numerical response was provided by the fluorimeter, indicating the
actual lead concentration, in mg/cm2, of the test sample. All responses that indicated the
presence of lead at or above 1.0 mg/cm2 were considered positive for the purposes of the
statistical analyses presented in this section. All responses that indicated that lead was present
below the 1.0 mg/cm2 threshold were considered negative for the purposes of this report. Only
the qualitative results (i.e., positive or negative) were used in conducting the statistical analyses
presented here.
In this report each PEM is associated with three definitions of lead levels:
Target lead level - the expected concentration of each PEM as outlined in Table 3-1.
These target lead levels were 0, 0.3, 0.6, 1.0, 1.4, 2.0, or 6.0 mg/cm2.
Confirmed lead level - the concentration as measured by the reference laboratory using
ICP-AES analysis.
Closest target lead level - the target level that is closest to the confirmed level. If a panel
has a target lead level of 1.4 mg/cm2 and a confirmed lead level of 1.9 mg/cm2 then the
closest target level is 2.0 mg/cm2.
Under ideal circumstances the confirmed lead level would equal the target lead level, but this
was sometimes not the case. Analyses where lead level was a categorical variable (i.e.,
consistency, precision, and sensitivity analyses) characterized the panels by their closest target
lead level. Analyses where lead level was a continuous variable (i.e., the false positive/negative
and logistic regression analyses) characterized the panels by their confirmed lead level. Each
analysis described clearly which level was used to characterize the lead level.
6.1 False Positive and False Negative Rates
Observed false positive and negative rates were calculated based on confirmed lead levels as
measured though ICP-AES analysis. For example, if the PEM was confirmed to have a lead
level of 1.4 mg/cm2, and the test kit returned a negative result, this would be considered a false
negative. Table 3-1 details the target lead levels for the PEMs and the number of PEMs that
were anticipated at each lead level. Because of variations in PEM production, the confirmed
lead level of a particular PEM did not always match the target lead level. Table 6-1 compares
the number of PEMs at the confirmed and target lead levels used for the false positive and
negative analyses. The data are divided into three categories: those panels eligible for false
positive analysis (lead levels up to and including 0.8 mg/cm2), those excluded from false positive
and false negative analyses (lead levels between 0.8 and 1.2 mg/cm2) and those eligible for false
negative analysis (lead levels 1.2 mg/cm2 and above). If the confirmed lead levels had been
equal to the target lead levels, all of the numbers would lie along the shaded diagonal. Because
the confirmed levels sometimes differed significantly from the target levels, (i.e., the target lead
level was at 0.6 mg/cm2 but confirmed near 1.4 mg/cm2) some panels appear in the off-diagonal
21
-------
table entries and were therefore included in portions of the analysis other than those for which
they had been targeted.
Table 6-1. The number of panels in each false positive and false negative analysis category
Confirmed Lead Levels
Target
Lead
Levels
Eligible for False Positive Analysis
Excluded from Analysis
Eligible for False Negative Analysis
Total
Eligible
for False
Positive
Analysis
146
7
1
154
Excluded from
Analysis
22
43
17
82
Eligible
for False
Negative
Analysis Total
11
22
197
179
72
215
230 466
Tables 6-2 and 6-3 list the observed false positive and false negative rates for the ANDalyze
Lead-in-Paint Test Kit under two sets of conditions:
Table 6-2 shows the observed false positive results for panels with confirmed lead levels
< 0.8 mg/cm2 and observed false negative results for panels with confirmed lead levels >
1.2 mg/cm2, per the RRP ruling2.
Table 6-3 shows observed false positive results for panels with confirmed lead levels < 1
mg/cm2 and observed false negative results for panels with confirmed lead levels > 1
mg/cm2.
Results for both the technical and non-technical operator are presented. Results are presented as
overall rates (i.e., false positive and negative results across all applicable PEMs combined) and
also false positive and negative rates based on lead paint type (i.e., white or yellow lead),
substrate (i.e., drywall, metal, plaster, or wood), and topcoat paint color (i.e., grey red or white).
The observed overall false positive rate for the technical and non-technical operators, based on
confirmed lead levels of <0.8 mg/cm2 (see Table 6-2) was 4-5%. Observed false positive rates
across both operators based on PEM characteristics ranged from 0% for metal PEMs and PEMs
with a red topcoat to 8% for yellow lead PEMs and PEMs with a grey topcoat. The observed
false positive rates across different PEM factors were similar to the overall rates and were similar
between the two operators. Observed false positive rates were 10% or lower in all cases.
22
-------
Observed false negative rates for the ANDalyze Lead-in-Paint Test Kit were slightly higher than
the observed false positive rates and were close to two times higher than the desired RRP rule4 of
a 5% or lower false negative rate. Observed false negative rates for the technical operator were
9% overall. Observed false negative rates for substrate and topcoat color were similar to the
overall rates found for each operator. Observed false negative rates for the non-technical
operator were 12% overall with comparable observed false negative rates on the various PEM
sub-factors except for metal PEMs. The observed false negative rate for the non-technical
operator on metal PEMs was 22%.
The observed false positive rates for both the technical and non-technical operator using 1.0
mg/cm2 as the deciding concentration (see Table 6-3) were slightly higher than those found using
RRP rule concentration limits (PEMs with confirmed lead levels <0.8 mg/cm2) (see Table 6-2),
with overall observed false positive rates of 7% and 6%, respectively. The observed false
positive rate for white lead panels was twice as high as that for yellow lead panels for the
technical operator. The observed false negative rates were also higher overall for both operators
than those found on PEMs with confirmed lead levels >1.2 mg/cm2, 14% for the technical
operator and 19% for the non-technical operator. Observed false positive rates for the substrates
and topcoat colors were similar to the overall rate for the technical operator when panels were
divided based on 1.0 mg/cm2. As with Table 6-2, analysis of metal PEMs by the non-technical
operator resulted in a higher observed false negative rate.
23
-------
Table 6-2. ANDalyze Lead-in-Paint Test Kit false positive results for panels with
confirmed lead levels < 0.8 mg/cm2 and false negative results for panels with confirmed
lead levels > 1.2 mg/cm2
Overall
None
White
Yellow
Drywall
Metal
Plaster
Wood
Grey
Red
White
ANDalyze Lead
False Positives'
Non-technical
Technical Operator Operator
15 / 308 = 5%
0 / 70 = 0%
4/120 = 3%
9 / 118 = 8%
4/76 = 5%
0/94 = 0%
3/54 = 6%
6/84 = 7%
5/102 = 5%
3 / 104 = 3%
5 / 102 = 5%
12 / 308 = 4%
0 / 70 = 0%
7 / 120 = 6%
5 / 118 = 4%
5/76 = 7%
2/94 = 2%
1/54 = 2%
4/84 = 5%
8/102 = 8%
0 / 104 = 0%
4/102 = 4%
-in-Paint Test Kit
False Negatives"
Non-technical
Technical Operator Operator
41 / 462 = 9%
NA
23/232 = 10%
18/230 = 8%
13 / 124 = 10%
5 / 90 = 6%
11/138 = 8%
12 / 110 = 11%
10 / 158 = 6%
17 / 140 = 12%
14 / 164 = 9%
54 / 462 = 12%
NA
21 / 232 = 9%
33 / 230 = 14%
17 / 124 = 14%
20 / 90 = 22%
3 / 138 = 2%
14 / 110 = 13%
16 / 158 = 10%
21 / 140 = 15%
17 / 164 = 10%
'False positives on PEMs with confirmed lead levels < 0.8 mg/cm2
"False negatives on PEMs with confirmed lead levels > 1.2 mg/ cm2
NA: If the paint did not contain lead then a false negative is not possible, those entries are 'NA'
(not applicable).
24
-------
Table 6-3. ANDalyze Lead-in-Paint Test Kit false positive results for panels with
confirmed lead levels < 1 mg/cm2 and false negative results for panels with confirmed lead
levels > 1 mg/cm2
Overall
None
White
Yellow
Drywall
Metal
Plaster
Wood
Grey
Red
White
ANDalyze Le
False Positives'
Technical Non-technical
Operator Operator
29 / 398 = 7%
0 / 70 = 0%
19/172 = 11%
10 / 156 = 6%
7 / 94 = 7%
5 / 126 = 4%
6 / 68 = 9%
11/110 = 10%
8 / 134 = 6%
13 / 140 = 9%
8 / 124 = 6%
25 / 398 = 6%
0 / 70 = 0%
11/172 = 6%
14 / 156 = 9%
9 / 94 = 10%
3 / 126 = 2%
4 / 68 = 6%
9 / 110 = 8%
14 / 134 = 10%
4 / 140 = 3%
7 / 124 = 6%
;ad-in-Paint Test Kit
False Negatives"
Technical Non-technical
Operator Operator
77 / 536 = 14%
NA
39 / 262 = 15%
38 / 274 = 14%
26 / 144 = 18%
11/110 = 10%
24 / 164 = 15%
16 / 118 = 14%
21/178=12%
31/172 = 18%
25 / 186 = 13%
100 / 536 = 19%
NA
39 / 262 = 15%
61 / 274 = 22%
33 / 144 = 23%
33/110 = 30%
17 / 164 = 10%
17/118 = 14%
28 / 178 = 16%
42 / 172 = 24%
30 / 186 = 16%
'False positives on PEMs with confirmed lead levels < 1.0 mg/cm2
"False negatives on PEMs with confirmed lead levels > 1.0 mg/ cm2
NA: If the paint did not contain lead then a false negative is not possible, those entries are 'NA'
(not applicable).
Note that the observed false positive and negative rates presented in this section provide a
general representation of the ability of the ANDalyze Lead-in-Paint Test Kit to correctly identify
regulated lead paint when it is present or absent. The results presented in Table 6-2 provide rates
based on the cut-off concentration (0.8 or 1.2 mg/cm2) as well as all levels evaluated below or
above those concentrations. To evaluate test kit performance based on the RRP rule, lead paint
test kits should have a demonstrated probability (with 95% confidence) of a negative response at
or above the regulated lead level <5% of the time. Test kits should also have a demonstrated
probability (with 95% confidence) of a positive response below the regulated lead level <10% of
the time. Because the RRP rule also indicated that test kit performance would not be based on
lead levels between 0.8 and 1.2 mg/cm2, the false positive and negative probabilities assessed in
this report were then based around the excluded concentrations (of 0.8 and 1.2 mg/cm2). False
positive and negative rates associated with these criteria are discussed in Section 6.4.
25
-------
6.2 Precision
To compute precision, it is first necessary to compute the number of replicate sets with consistent
responses. Replicate sets are defined in the test/QA plan1 to be groups of panels with similar
lead levels. The target lead levels in this experiment were 0, 0.3, 0.6, 1.0, 1.4, 2, and 6 mg/cm2
but the lead levels that were achieved, as confirmed by ICP-AES, sometimes varied from those
target levels. To assemble replicate sets that represented the target lead levels, the panels were
assigned to the replicate set that was nearest their confirmed lead level. In other words, if a
particular panel was targeted for 0.3 mg/cm2 but was measured to have 0.9 mg/cm2 then it was
assigned to the replicate set nearest 0.9 mg/cm2, which is the set labeled 1.0 mg/cm2. Table 6-4
shows the thresholds that defined the replicate set bins as well as the range of measured levels
that fell in each bin.
Table 6-4. Actual lead levels and their replicate set labels
Replicate Set Bin
Label (mg/cm2)
(Closest Target Lead
Level)
0
0.3
0.6
1
1.4
2
6
Bin Thresholds
(mg/cm2)
Targeted to have zero lead
0 < Confirmed Lead Level < 0.45
0.45 < Confirmed Lead Level < 0.8
0.8 < Confirmed Lead Level < 1.2
1.2
-------
Table 6-5. The number of panels at each target level and the number in each replicate set
bin
Replicate Set Bin
(Target level that is closest to the panel's actual measured lead level)
0.3 0.6
1.4
Target Lead Level
(mg/cm2)
0
0.3
0.6
1.0
1.4
2.0
6.0
35
62
5
1
-
-
-
Total 35 68
-
7
37
6
1
-
-
51
-
3
19
43
16
1
-
82
-
-
9
16
46
5
-
76
-
-
2
6
8
62
-
78
-
-
-
-
1
3
72
76
Total
35
72
72
72
72
71
72
466
Table 6-6 lists consistency results for the ANDalyze Lead-in-Paint Test Kit by operator type,
lead type, substrate, and lead level. Each table entry lists the number of test results with those
characteristics (N) as well as the proportion of the results that were positive for lead (Pos). Table
entries where the proportion is below 25% or above 75% are 'consistent', meaning that more
than three-quarters of the results were the same (negative or positive). Table entries where the
proportion of positive results ranges from 25% to 75% are considered to be 'inconsistent'.
Inconsistent entries are shaded in the tables. Overall consistency results across all substrates for
white and yellow lead panels for each operator type are also provided in the last row of Table 6-
6. Results across both operators and lead paint types are provided in the last column of the table.
Overall inconsistencies for the non-technical operator were found at the 1.4 mg/cm2 lead level
for white lead PEMs (see the last row of Table 6-6). This was also true for drywall, metal, and
wood substrates, with consistencies ranging from 50-75% at the 1.4 mg/cm2 lead level for white
lead PEMs. Overall inconsistencies were also found at the 1.0 mg/cm2 lead level for the yellow
lead PEMs. However, the lead level of inconsistencies on yellow-lead PEMs varied across
substrates, with only the 0.0 and 0.3 mg/cm2 lead levels not showing any inconsistencies
regardless of substrate type. Overall inconsistencies for all PEMs for the non-technical operator
were found at the 1.0 and 1.4 mg/cm2 lead level.
Inconsistencies for the technical operator overall and across all white and yellow lead PEMs
were at the 1.0 mg/cm2 lead level. Inconsistencies were also found at 0.6 mg/cm2 on yellow lead
PEMs and 1.4 mg/cm2 on white lead PEMs for some substrates. Across both operators and white
and yellow lead PEMs, the test kit was inconsistent at 0.6 mg/cm2 for drywall (28% consistent)
and 1.4 mg/cm2 for drywall, metal, and wood (74%, 72%, and 70% consistent, respectively)
Across both operators and all substrates and lead paint type, the ANDalyze Lead-in-Paint Test
Kit was inconsistent at only 1.0 mg/cm2.
27
-------
The consistency results provided in Table 6-6 were used to calculate precision. Precision was
estimated for panels with no lead, white lead, and yellow lead and broken out by type of operator
and then aggregated across both types of operators. For any column in Table 6-6, the precision
is simply the proportion of consistent (unshaded) table entries in the rows for the four different
substrates. The 'All' rows are not counted in the precision calculation because those table entries
are summaries of the entries for the four substrates. Thus, precision was calculated as:
# of unshaded table entries in the
n fn, ... 7. N drywall,metal,plaster,and wood sections (7)
Preciswn(% consistent results) =
total entries in those sections
Table 6-7 lists the results of the precision calculations for the ANDalyze Lead-in-Paint Test Kit.
Higher proportions of consistent results indicate more consistency and higher precision.
The ANDalyze Lead-in-Paint Test Kit was precise on PEMs (100%) that contained no lead. The
precision of the non-technical operator was higher than that of the technical operator on white
lead PEMs (85% vs. 73%), while the results were reversed for the yellow lead PEMs, with the
technical operator having a precision of 81% while the non-technical operator only had a
precision of 66%. The overall precision across both operators was similar (79% and 73%) for
both lead paint types.
28
-------
Table 6-6. ANDalyze Lead-in-Paint Test Kit consistency results by operator type, lead type,
substrate, and lead level
ANDalyze Lead-in-Paint Test Kit
Lead Type
DRYWALL
0
0.3
0.6
1
1.4
2
6
METAL
0
0.3
0.6
1
1.4
2
6
PLASTER
0
0.3
0.6
1
1.4
2
6
WOOD
0
0.3
0.6
1
1.4
2
6
ALL
0
0.3
0.6
1
1.4
2
6
NON-TECHNICAL
None
N Pos
18 0%
18 0%
16 0%
18 0%
70 0%
White
N Pos
16 0%
12 17%
18 0%
24 75%
22 100%
18 100%
16 13%
22 0%
28 18%
10 50%
14 93%
20 100%
2 0%
16 0%
6 17%
16 50%
16 94%
34 100%
20 100%
4 0%
16 0%
10 20%
20 15%
16 56%
20 95%
18 100%
64 3%
50 10%
82 20%
66 71%
90 98%
76 100%
Yellow
N Pos
2 0%
18 0%
10 30%
20 40%
18 67%
22 91%
20 85%
2 0%
18 0%
18 0%
24 13%
16 69%
12 67%
18 72%
6 0%
8 0%
24 29%
30 93%
16 100%
22 100%
20 0%
16 13%
14 50%
22 73%
16 100%
18 100%
62 0%
52 10%
82 30%
86 78%
66 91%
78 90%
Total
N Pos
20 0%
34 0%
22 23%
38 20%
42 71%
44 95%
38 92%
20 0%
34 6%
40 0%
52 15%
26 59%
26 80%
38 86%
18 0%
22 0%
14 8%
40 40%
46 94%
50 100%
42 100%
22 0%
36 0%
26 16%
34 33%
38 64%
36 98%
36 100%
80 0%
126 2%
102 10%
164 26%
152 74%
156 95%
154 95%
TECHNICAL
None
N Pos
18 0%
18 0%
16 0%
18 0%
70 0%
White
N Pos
16 0%
12 0%
18 33%
24 75%
22 91%
18 100%
16 0%
22 0%
28 36%
10 80%
14 93%
20 100%
2 0%
16 0%
6 0%
16 50%
16 75%
34 94%
20 100%
4 0%
16 13%
10 20%
20 25%
16 75%
20 90%
18 100%
64 3%
50 4%
82 35%
66 76%
90 92%
76 100%
Yellow
N Pos
2 0%
18 0%
10 40%
20 20%
18 78%
22 95%
20 100%
2 0%
18 0%
18 0%
24 38%
16 88%
12 100%
18 100%
6 0%
8 38%
24 33%
30 87%
16 100%
22 95%
20 0%
16 13%
14 29%
22 77%
16 100%
18 94%
62 0%
52 17%
82 30%
86 83%
66 98%
78 97%
Total
N Pos
20 0%
34 0%
22 18%
38 27%
42 76%
44 93%
38 100%
20 0%
34 0%
40 0%
52 37%
26 84%
26 96%
38 100%
18 0%
22 0%
14 19%
40 42%
46 81%
50 97%
42 98%
22 0%
36 6%
26 16%
34 27%
38 76%
36 95%
36 97%
80 0%
126 2%
102 12%
164 34%
152 79%
156 95%
154 99%
TOTAL
Total
N Pos
40 0%
68 0%
44 20%
76 23%
84 74%
88 94%
76 96%
40 0%
68 3%
80 0%
104 26%
52 72%
52 88%
76 93%
36 0%
44 0%
28 14%
80 41%
92 87%
100 99%
84 99%
44 0%
72 3%
52 16%
68 30%
76 70%
72 96%
72 99%
160 0%
252 2%
204 10%
328 30%
304 77%
312 95%
304 98%
N = number of test results in each bin of thetable
POS = Proportion of those N test results that were 'Positive' for the presence of lead.
Lead levels in the left-most column represent the target level closest to the measured level of lead in the panel.
Shaded cells represent 'inconsistent' results, i.e., % positive is between 25% and 75%
29
-------
Table 6-7. ANDalyze Lead-in-Paint Test Kit precision results by lead type and operator
type
No Lead White Lead Yellow Lead1
Non-technical
Technical
All
4/4 = 100% 22/26 = 85% 17/26 = 66%
4/4 = 100% 19/26 = 73% 21/26 = 81%
8/8 = 100% 41/52 = 79% 38/52 = 73%
1 Results were consistently negative across all lead levels for this test kit on yellow lead paint panels, even those
samples containing detectable levels of lead.
6.3 Sensitivity
Sensitivity was calculated using the bottom six rows in Table 6-6. These rows aggregate results
across all four substrates. For the white lead and yellow lead columns in these tables, the
sensitivity is the lowest lead level > 1 mg/cm2 that is consistently detected with positive results
(unshaded and > 75%). Ideally the kit would give consistently negative results for lead levels <
1 mg/cm2 and consistently positive results for levels > 1 mg/cm2 so the optimal sensitivity results
would be 1 across every row of Table 6-8.
Table 6-8. ANDalyze Lead-in-Paint Test Kit sensitivity results - lowest lead level for which
the kit gave consistent positive results (mg/cm2)
Lead Type
Sensitivity
Non-technical Operator
White Yellow Total
2.0 1.4 2.0
Technical Operator
White Yellow Total
1.4 1.4 1.4
All
Total
1.4
Across all lead paint types and operators, the ANDalyze Lead-in-Paint Test Kit generated
consistent positive results at 1.4 mg/cm2 lead. When sensitivity is evaluated by operator type,
consistently positive results were found at 1.4 mg/cm2 on white and yellow as overall for the
technical operator. Consistently positive responses were found at the 2.0 mg/cm2 lead level for
the non-technical operator on white lead PEMs and the 1.4 mg/cm2 lead level for yellow lead
PEMs. The overall sensitivity as determined through evaluations performed by the non-technical
operator to be at the 2.0 mg/cm2 lead level. This is higher than the sensitivity determined by
evaluations from the technical operator.
Note that the ANDalyze Lead-in-Paint Test Kit is quantitative in nature. That is, the test kit
provides the measured lead level in mg/cm2 of the sample being evaluated. As such, the
ANDalyze Lead-in-Paint Test Kit can provide results lower than 1.0 mg/cm2, and in fact did.
Table 6-9 presents the average concentration and standard deviation of the samples as indicated
by the ANDalyze Lead-in-Paint Test Kit for each of the replicate bin sets. As shown in Table 6-
9, the average values indicated by the ANDalyze Lead-in-Paint Test Kit are close to those of the
replicate bin sets. However, as the standard deviation indicates, the range of concentrations
30
-------
given for samples within a particular replicate set are sometimes outside of the bin thresholds
(see Table 6-4). Also, the concentrations given for PEMs in the 6.0 mg/cm2 replicate set by the
ANDalyze Lead-in-Paint Test Kit were sometimes small in comparison to the actual confirmed
lead levels in this bin. Confirmed lead levels went up to 15 mg/cm2, while the ANDalyze Lead-
in-Paint Test Kit readings never went higher than 10 mg/cm2. Concentrations indicated by the
test kit were up to 7.5 times lower than confirmed lead levels for a PEM. This is likely because
of the limited upper range of the calibration curve used for the fluorimeter.
Table 6-9. Average and standard deviation of ANDalyze Lead-in-Paint Test Kit results
compared to the concentration of the replicate sets
Replicate Set Bin
Label (mg/cm2)
(Closest Target
Lead Level)
0
0.3
0.6
1
1.4
2
6
Average ANDalyze Lead-in-Paint
Measured Lead Level
(mg/cm2)
BL1
0.089
0.49
0.86
1.46
2.17
3.49
ANDalyze Lead-in-Paint
Measured Lead Level
Standard Deviation
(mg/cm2)
0
0.25
0.69
0.29
0.79
0.87
2.07
1 Below limit on fluorimeter; indicative of no lead present
6.4 Modeled Probability of Test Kit Response
Table 6-10 lists the explanatory variables which had significant (p<0.05) univariate associations
with the probability of obtaining a positive test kit result. All potential explanatory variables
except for lead type and operator type showed a statistically significant univariate association
with the probability of a positive response. Lead level, substrate type, and topcoat color were
significant in the multivariable model after backward selection. Table 6-11 lists the parameter
estimates for the multivariable logistic regression models from the Stata SIMEX program. There
were no statistically significant interactions between categorical covariates.
Table 6-12 lists the modeled probability of a positive test result for the ANDalyze Lead-in-Paint
Test Kit when the lead level is 0.8 mg/cm2 (PREDICTION) along with the upper bound of a 95%
prediction interval (UPPER). That upper bound can be considered to be a worst-case estimate of
the false positive probability when the true lead level is 0.8 mg/cm2 (FALSE POS RATE).
Ideally the numbers in the UPPER/FALSE POS RATE column would be < 10%. Note that the
FALSE POS RATE in Table 6-12 is higher than those in Tables 6-2 and 6-3. In those earlier
31
-------
tables the rates considered panels at a variety of comparatively low lead levels so some cases
should have been easier for the kit to obtain the correct answer. In Table 6-12, the false positive
rate is evaluated only at 0.8 mg/cm2 so the rate does not benefit from the comparatively lower
lead concentrations. Evaluating at only this level also ensures that a test kit can adequately
perform at concentrations of lead paint closest to the current regulatory level.
Based on the upper prediction bound estimates shown in Table 6-12, the ANDalyze Lead-in-
Paint Test Kit did not meet the false positive criteria for any scenario at the 0.8 mg/cm2 lead
level. The lowest upper bound prediction expected to be achieved at this lead level is 19.6% for
a metal substrate with a red topcoat, and this rate is approximately two times the ideal rate.
Table 6-13 lists the modeled probability of a positive test result for the ANDalyze Lead-in-Paint
Test Kit when the lead level is 1.2 mg/cm2 (PREDICTION) along with the lower bound of a 95%
prediction interval (LOWER). The difference between the lower bound and 100% can be
considered to be a worst-case estimate of the false negative probability when the true lead level
is 1.2 mg/cm2 (FALSE NEG RATE). Ideally, for the purposes of the RRP rule, the numbers in
the FALSE NEG RATE column would be < 5%. Based on the lower bound estimates shown in
Table 6-13, the ANDalyze Lead-in-Paint Test Kit did not meet the false negative criterion (<5%)
at the 1.2 mg/cm2 lead level.
Note that the FALSE NEG RATE in Table 6-12 is higher than those in Tables 6-2 and 6-3. In
the earlier tables, the false negative rates considered panels at a variety of comparatively high
lead levels so some cases should have been easier for the kit to obtain the correct answer. In
Table 6-12, the false negative rate is evaluated only at 1.2 mg/cm2 so the rate does not benefit
from the comparatively higher lead concentrations. Evaluating at only this level also ensures
that a test kit can adequately perform at concentrations of lead paint closest to the current
regulatory level.
Table 6-10. ANDalyze Lead-in-Paint Test Kit univariate associations between probability
of positive response and explanatory variables
Explanatory Variable Significant Univariate Association? Included in Multivariable Model?
Lead Level Yes (p-value < 0.0001) Yes
Lead Type No (p-value = 0.9960) No
Operator Type No (p-value = 0.2116) No
Substrate Type Yes (p-value < 0.0001) Yes
Topcoat Color Yes (p-value = 0.0168) Yes
32
-------
Table 6-11. ANDalyze Lead-in-Paint Test Kit multivariable Stata SIMEX logistic
regression parameter estimates
Simulation extrapolation
Residual df = 1861
Variance Function: V(u) = u(l-u)
Link Function : g(u) = log(u/(l-u))
result |
Substrate :
drywall |
metal |
plaster |
Topcoat :
grey |
red |
lead level |
constant |
Bootstrap
Coef. Std. Err.
t
No. of obs
Bootstraps reps
Wald F(6,1861)
Prob > F
[Bernoulli]
[Logit]
P>l 1 1
[95% Conf
1868
199
15.86
0.0000
Interval]
(wood is the reference level)
-.1469332
-.4716426
.4060918
(white is the
.0398478
-.3201983
2.198884
-2.657648
1650832
1601065
.161856
reference
1474259
1320513
3316751
3875847
-0.89
-2.95
2.51
level)
0.27
-2.42
6.63
-6.86
0
0
0
0
0
0
0
374
003
012
787
015
000
000
-.470701
-.7856498
.0886534
-.2492896
-.5791826
1.54839
-3.417795
.1768345
-.1576354
.7235301
.3289852
-.061214
2.849378
-1.897502
Table 6-12. ANDalyze Lead-in-Paint Test Kit modeled probability of positive test results
and upper 95% prediction bound when lead level = 0.8 mg/cm2
TOPCOAT SUBSTRATE LEAD LEVEL
DRYWALL
METAL
PLASTER
WOOD
DRYWALL
METAL
RED
PLASTER
WOOD
DRYWALL
METAL
PLASTER
WOOD
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
0.8
PREDICTION UPPER (FALSE POS RATE)
26.8%
20.9%
38.9%
29.8%
20.3%
15.6%
30.7%
22.8%
26.0%
20.3%
37.9%
28.9%
32.4%
25.3%
45.1%
35.3%
25.4%
19.6%
37.1%
27.9%
31.6%
24.7%
45.2%
35.2%
33
-------
Table 6-13. ANDalyze Lead-in-Paint Test Kit modeled probability of positive test results,
lower 95% prediction bound, and corresponding conservative estimate of the false negative
rate when lead level = 1.2 mg/cm2
TOPCOAT SUBSTRATE LEAD LEVEL
DRYWALL
METAL
GREY
PLASTER
WOOD
DRYWALL
METAL
RED
PLASTER
WOOD
DRYWALL
1A,,,,Tr. METAL
WH TE
PLASTER
WOOD
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
PREDICTION
46.9%
38.9%
60.5%
50.5%
38.1%
30.8%
51.7%
41.6%
45.9%
38.0%
59.6%
49.5%
LOWER FALSE NEC RATE
40.6%
32.8%
55.0%
44.1%
32.4%
25.5%
46.0%
35.8%
40.4%
32.6%
53.8%
43.2%
59.4%
67.2%
45.0%
55.9%
67.6%
74.5%
54.0%
64.2%
59.6%
67.4%
46.2%
56.8%
As another means of reporting the results for the ANDalyze Lead-in-Paint Test Kit, modeled
probability curves were also plotted based on the results of the regression analysis. To better
understand the information being provided in these probability curves, a brief explanation is
presented here. Figure 6-1 shows that for the perfect or ideal test kit, the probability of a positive
test result would be a step function. The probability of a positive result would be zero below 1.0
mg/cm2 and 100% at or above 1.0 mg/cm2. Under the RRP rule, a test kit must yield a
demonstrated probability (with 95% confidence) of no more than 10% false positives at lead
concentrations below 0.8 mg/cm2 and a demonstrated probability (with 95% confidence) of no
more than 5% false negatives at concentrations above 1.2 mg/cm2. Figure 6-1 also shows a
performance curve for a hypothetical test kit that achieves those rates. The upper bound of the
90% prediction interval is at 10% at 0.8 mg/cm2 and the lower bound of the prediction interval is
at 95% at 1.2 mg/cm2.
One way to think of the test kit performance guidelines is in terms of regions of the probability
plots. Figure 6-2 demonstrates this concept. For the kit to be within limits set up by the RRP
rule, the probability curve must trace a path through the white region in the figure and must not
stray into the shaded regions. If the curve crosses the shaded region at the left side of the graph
then there are lead levels < 0.8 mg/cm2 where the false positive rate is > 10%. If the curve
crosses the shaded region at the right side of the graph then there are lead levels > 1.2 mg/cm2
where the false negative rate is > 5%. Either type of intersection between the curve and the
shaded region indicates that the kit does not meet the performance levels stipulated in the RRP
rule.
Note that results for the region between 0.8 and 1.2 mg/cm2 were not discussed in this report.
This is consistent with the RRP rule stipulation that lead concentrations between 0.8 and 1.2
mg/cm2 were not to be considered for the evaluation of the performance of lead paint test kits.
34
-------
Figures 6-3 through 6-5 show the predicted probability of obtaining a positive test result using
the ANDalyze Lead-in-Paint Test Kit over the full range of explanatory variables along with the
bounds of a 90% prediction interval. Note that the upper and lower bounds of the 90%
prediction interval may also be considered to be upper and lower 95% prediction bounds for one-
sided inference. In every instance the upper end of the probability curves (above 1.2 mg/cm2)
pass through the shaded regions of the plot until above 2.0 mg/cm2. The low ends of the curves
(below 0.8 mg/cm2) cross out of the shaded regions well below 0.8 mg/cm2.
In every instance, both the upper end of the probability curves (above 1.2 mg/cm2) and the lower
end (below 0.8 mg/cm2) pass through the shaded regions of the plot, indicating that the kit's false
positive and false negative performance do not meet the RRP rule requirements.
35
-------
tO
8.
0>
ฃ
.Q
CO
o
a!
100%-
80%-
60%-
40%-
20%-
95%
Perfect Kit
Acceptable Kit: Upper 90% Bound (1)
Acceptable Kit
Acceptable Kit: Lower 90% Bound (2)
1 Upper bound is above and left of the "Acceptable Kit" curve
2 Lower bound is below and right of the "Acceptable Kit" curve
10%
0
0.8 { 1.2
Lead Level (mg/cm^Z)
Figure 6-1. Probability curves that represent test kit results that are both perfect (red line) and within RRP rule criteria (black
solid line).
36
-------
tO
8.
0>
ฃ
.Q
CO
o
a!
100% H
80%-
60%-
40% H
20%-^
95%
10%
0
0.8 i 1.2
Lead Level (ing/cm^)
Figure 6-2. Probability curves with shaded region to denote performance results that meet RRP rule false positive and
negative criteria. Test kits with curves that fall within the shaded region and avoid the white region meet the RRP rule.
37
-------
Topcoat = WHITE
ฑd
3
1
0)
M-
!5
o
a
100% H
75% -1
50% J
25% j
0%J ^
ioQ%j^_^_
75% -J
50%^
25% -J
1
0%-|
DRYWALL
__Jjeฑ=jj i
i&y j
r ^
**^
PLASTER
I
^' ^
ฃ* ^5
-------
Topcoat = GREY
ฑd
3
1
0)
'35
M-
0
^^j
I^M
A
O
a.
\ DRYWALL | METAL
100% H
75% -J
50% -J
25%^
^^ฃฃM*> 1 _.ปซ*SJ
/""
0%-|
75% -J
50%^
25% -J
PLASTER
*ฃ&&
WOOD
^^^^ I
,-/
** 1 ฃ&*"
o%^ i
^
0 0.8|l.2 2 30 0.8|l.2 2' 3
Lead Level (mg/cm^Z)
95%
10%
95%
10%
Figure 6-4. ANDalyze Lead-in-Paint Test Kit predicted probability of positive test result (solid lines) with 90% prediction
interval (dotted lines) for a grey paint topcoat on various substrates.
39
-------
W)
100% 1
75%-i
Jg 50%-
QC
0>
25% -
75% -
50%
25% -
Topcoat = RED
DRYWALL
I
wSStf^'
PLASTER
o
METAL
WOOD
2 3 0 -
Lead Level (mg/cm^Z)
95%
10%
95%
10%
Figure 6-5. ANDalyze Lead-in-Paint Test Kit predicted probability of positive test result (solid lines) with 90% prediction
interval (dotted lines) for a red paint topcoat on various substrates.
40
-------
Based on the modeled probabilities shown in Figures 6-3 through 6-5, threshold values for false
positive and negative rates were established for the ANDalyze Lead-in-Paint Test Kit. For the
false positive rate, this threshold value is the lead level, with 95% confidence, below which the
ANDalyze Lead-in-Paint Test Kit would yield fewer than 10% false positive results. For the
false negative rate, this threshold value is the lead level, with 95% confidence, above which the
ANDalyze Lead-in-Paint Test Kit would yield fewer than 5% false negative results. These
threshold values are then the lead levels where the ANDalyze Lead-in-Paint Test Kit is predicted
to meet the false positive and negative criteria set forth in the RRP rule.
Table 6-14 presents the false positive and negative threshold values for the ANDalyze Lead-in-
Paint Test Kit. Threshold lead levels are provided for each topcoat and substrate combination
shown in Tables 6-12 and 6-13.
Table 6-14. ANDalyze Lead-in-Paint Test Kit false positive and negative threshold values
(95% confidence) based on the modeled probability of test results
FALSE POSITIVE
TOPCOAT SUBSTRATE THRESHOLD (mg/cm2)
DRYWALL
METAL
GREY
PLASTER
WOOD
DRYWALL
METAL
RED
PLASTER
WOOD
DRYWALL
METAL
WHITE
PLASTER
WOOD
NA
0.18
NA
NA
0.16
0.37
NA
0.09
NA
0.18
NA
NA
FALSE NEGATIVE
THRESHOLD (mg/cm2)
3.13
3.35
2.79
3.05
3.33
3.55
2.99
3.25
3.13
3.34
2.79
3.05
OVERALL
0.20
3.15
NA in the FALSE POSITIVE THRESHOLD column means that the false positive rate was > 10% for all lead
levels.
NA in the FALSE NEGATIVE THRESHOLD column means that the false negative rate was > 5% for all lead
levels.
Table 6-14 indicates that overall, across all factors, the false positive threshold for the ANDalyze
Lead-in-Paint Test Kit is 0.20 mg/cm2. That is, this test kit is predicted, with 95% confidence, to
not yield fewer than 10% false positive results until lead levels reach 0.20 mg/cm2 or lower. The
overall false negative threshold for the ANDalyze Lead-in-Paint Test Kit is 3.12 mg/cm2. False
positive and negative thresholds for individual combinations of factors were similar to the
overall false positive and negative thresholds across all factors of significance. A false positive
41
-------
threshold could not be established for the plaster substrate with either a grey or white topcoat.
The red topcoat plaster had a false positive threshold of 0.0 mg/cm2.
Standardized Pearson residuals were calculated to assess goodness of fit of the logistic regression
models. For the ANDalyze Lead-in-Paint Test Kit model, 95.1% of the residuals had absolute
values smaller than two.
6.5 Matrix Effect
The matrix effects for the ANDalyze Lead-in-Paint Test Kit were evaluated with results in Table
6-11. The variables that were retained in the multivariable logistic regression model each add
significant explanatory power to their respective models. Those variables are significantly
associated with the probability of obtaining a positive test result from the kits tested in this study.
For the ANDalyze Lead-in-Paint Test Kit, Table 6-11 indicates that after controlling for the
significant covariates, the likelihood of a positive test result is positively and significantly
associated with: higher lead levels, plaster, drywall, and metal substrates, and grey and white
topcoats.
6.6 Operational Factors
A tip sheet was also provided with the instructions, containing helpful hints in the test kit
operation. The technical operator found the ANDalyze Lead-in-Paint Test Kit instructions to be
clear, informative, and easy to follow. The non-technical operator received no training from the
vendor and relied solely on the test kit instructions, tip sheet, and instructional DVD for his
understanding of the operation of the test kit. He did not believe that the kit was easy to follow
based solely on the supporting information.
Both the technical and non-technical operator stated that a significant amount of training and
possibly previous experience or laboratory knowledge would be needed to successfully operate
this test kit. The calibration step and number of complicated steps (including multiple pipetting
steps) and pieces of equipment needed for the operation of this test kit could make it difficult for
an average contractor or renovator to feel comfortable with the test kit and to conduct successful
evaluations without extensive training. It was the opinion of the non-technical operator that the
kit would be too complicated for the typical lead removal contractor. As an example, the
grinding step was established as 3 minute per sample for this verification test. The ANDalyze
Lead-in-Paint Test Kit instructions indicate that 10 minutes or more could be needed to grind a
single paint sample. The completeness of grinding is subjective and is left to the operator to
determine. The cleaning of the grinding tubes was a time-consuming step as each tube had to
soak, filled with cleaning solution, for at least 30 minutes after each use. The use of the drill and
drill bit was not complicated, but care must be taken to operate the drill as slowly as possible so
as not to remove much, if any, of the substrate.
The ANDalyze Lead-in-Paint Test Kit, as supplied for this verification test, included a drill,
modified half-inch drill bit, fluorimeter, razor blades, ruler, 15 mL grinding tubes and pestles,
nitric acid, plastic pipettes, calibration solution and buffers, sensor housings, 30 mL plastic tubes
pre-filled with testing buffer, 10 x 76 millimeter (mm) glass test tubes, 1 mL syringes, 10 |jL
42
-------
mini-pipette and disposable tips, and grinder cleaning solution. The test kit instructions
indicated that the kit should be stored at room temperature and out of direct sunlight. The
housing sensors are sensitive to light and needed to be stored out of direct light until their use.
Expiration dates were not supplied with any of the reagents. When new supplies were provided
for verification testing, mixing of the new and old reagents was discouraged. The grinding tubes
began to crack and wear down after repeated use and had to be thrown away in some instances.
The point of their disposal was determined by the operator.
The test kit instructions indicated that appropriate safety precautions should be taken when using
this test kit, such as wearing the necessary protective equipment and using caution when
handling the drill to prevent injury. The extraction solution is 25% nitric acid and the kit
instructions note that protective equipment should be used when handling the acid. Both the
technical and non-technical operators followed general laboratory safety procedures and wore a
lab coat, protective eyewear, and gloves at all times. A MSDS sheet was provided with the test
kit for the acid solution.
All reagents came prepared and ready to use. The solutions used for different steps were easily
identifiable within the kit. Storage conditions of the reagents were not marked on the containers,
although the ANDalyze Lead-in-Paint Test Kit instruction manual did indicate storage
requirements and a temperature range for test kit operation.
The waste generated for this test kit includes both liquid and solid waste. Solid waste included
pipette tips, glass test tubes, housing sensors, plastic tubes, and disposable plastic pipettes.
Liquid waste included approximately 2 mL of ground paint in nitric acid; 20 mL of testing
buffer, 3 mL of calibration solutions, and 2-3 mL of cleaning solution. The ANDalyze Lead-in-
Paint Test Kit tip sheet provided some waste disposal guidelines. It indicated that any used glass
test tubes should be disposed of into a sturdy container, such as a cardboard box, and labeled as
"Broken Glass". Any uncontaminated nitric acid solution should be neutralized with testing
buffer or baking soda and flushed down the sink. If a positive result is obtained with the test kit,
the sample would be assumed to contain lead. As such, any lead-containing waste, such as the
paint extract, would be considered lead waste. As such, the ANDalyze Lead-in-Paint Test Kit
instructions indicated that EPA and Department of Transportation (DOT) regulations pertaining
to disposal of lead waste should be followed and non-positive tests (indicating that no lead at the
regulated level is present) would be considered non-lead waste and disposed of with normal
waste procedures. (Note: Because regulations for the disposal of wastes generated from the use
of lead test kits may vary from state to state, EPA recommends that test kit users contact their
state government agency for proper waste disposal requirements.) After preparation, the
calibration tubes contain 60, 300, and 1500 ppb lead waste. The lead levels in these solutions are
very small and the vendor recommends that they be disposed in the drain by flushing with
enough water to dilute the lead to < 15 ppb (EPA limit for lead in drinking water). The vendor
recommends diluting each buffer tube with at least 100X the volume of water. That is, use 100
mL of tap water to dilute the 1 mL buffer containing lead. Similarly, the buffer solutions
containing the diluted lead paint extract from the sample should contain minimal levels of lead
and could be flushed down the drain with copious amounts of water.
Interpretation of the fluorimeter output for the ANDalyze Lead-in-Paint Test Kit was easy to
determine. Outputs were clear and well labeled. However, it was not clear how rugged the
fluorimeter itself would be for taking into the field if paint samples were to be evaluated onsite.
43
-------
The ANDalyze Lead-in-Paint Test Kit was viewed by both the technical and non-technical
operators to be complicated and difficult to use. Operation of the test kit took approximately 18
minutes by both the technical and non-technical operator, not including the grinder washing
procedure. A normal power supply was needed for the operation of the fluorimeter. As of the
writing of this report, the cost of the fluorimeter is $1500 and the cost for the drill is $310. The
cost for all necessary consumables to conduct 50 tests is $300.
44
-------
Chapter 7
Performance Summary
The observed overall false positive rate for the ANDalyze Lead-in-Paint Test Kit on PEMs with
confirmed lead levels of < 0.8 mg/cm2 was 4-5% for both the technical and non-technical
operator. Observed false positive rates across both operators based on PEM characteristics
ranged from 0% for metal PEMs and PEMs with a red topcoat to 8% for PEMs with a grey
topcoat. The observed false positive rates across different PEM factors (e.g., substrate type,
topcoat color, lead paint type) were similar to the overall rates and were similar between the two
operators. Observed false positive rates were 10% or lower in all cases.
Observed false negative rates for the technical operator were 9% overall. The observed false
negative rates for substrate and topcoat color were similar to the overall rates found for each
operator. Observed false negative rates for the non-technical operator were 12% overall with
comparable observed false negative rates on the various PEM sub-factors except for metal
PEMs. The observed false negative rate for the non-technical operator on metal PEMs was 22%.
Overall observed false positive rates on PEMs with confirmed lead levels <1.0 mg/cm2 for both
the technical and non-technical operator were slightly higher than those found on PEMs with
confirmed lead levels < 0.8 mg/cm2, with overall observed false positive rates of 7% and 6%,
respectively. The observed false negative rates were also higher overall for both operators than
those found on PEMs with confirmed lead levels > 1.2 mg/cm2, 14% for the technical operator
and 19% for the non-technical operator.
The ANDalyze Lead-in-Paint Test Kit produced consistent responses (either positive or
negative) across all substrates and paint types at all lead levels except one; the ANDalyze Lead-
in-Paint Test Kit results were inconsistent at 1.0 mg/cm2.
Results from the ANDalyze Lead-in-Paint Test Kit indicated 100% precision on PEMs that
contained no lead. The precision observed when the kit was operated by the non-technical
operator was higher than that of the technical operator on white lead PEMs (85% vs. 73%), while
the results were reversed for the yellow lead PEMs, with the technical operator having a
precision of 81% while the non-technical operator had a precision of 66%. The overall precision
across both operators was similar (79% and 73%) for both lead paint types.
Across all lead paint types and operators, the lowest lead level for which the ANDalyze Lead-in-
Paint Test Kit generated consistent positive results was 1.4 mg/cm2 lead. When sensitivity was
evaluated by operator type, consistently positive results were found at 1.4 mg/cm2 on white and
yellow as overall for the technical operator. Consistently positive responses were found at the
2.0 mg/cm2 lead level for the non-technical operator on white lead PEMs and the 1.4 mg/cm2
45
-------
lead level for yellow lead PEMs. The overall sensitivity as determined through evaluations
performed by the non-technical operator to be at the 2.0 mg/cm2 lead level. This is a higher than
the sensitivity determined by evaluations from the technical operator.
Under the RRP rule4, a test kit must yield a demonstrated probability (with 95% confidence) of
no more than 10% false positives at lead concentrations below 0.8 mg/cm2 and a demonstrated
probability (with 95% confidence) of no more than 5% false negatives at concentrations above
1.2 mg/cm2 to meet the rule criteria. Based on the upper bound estimates of the modeled
probability of the ANDalyze Lead-in-Paint Test Kit, the technology did not meet the false
positive criterion at the 0.8 mg/cm2 lead level. The lowest false positive rate expected to be
achieved at this lead level is 19.6% for a metal substrate with a red topcoat, and this rate is
approximately two times the 10% false positive rate specified in the RRP rule. All false negative
rates obtained using the ANDalyze Lead-in-Paint Test Kit results were above the 5% criterion
established by the RRP rule. False negative rates were predicted to range from 45.0 to 74.5%.
Based on the modeled probabilities, the overall false positive threshold value (i.e., the lead level,
with 95% confidence, below which the test kit would yield fewer than 10% false positive results)
for the ANDalyze Lead-in-Paint Test Kit is 0.20 mg/cm2. Across all factors of significance, the
overall false negative threshold (the lead level, with 95% confidence, above which the test kit
would yield fewer than 5% false negative results) for the ANDalyze Lead-in-Paint Test Kit is
3.12 mg/cm2.
After controlling for the significant covariates, the likelihood of a positive test result is positively
and significantly associated with: higher lead levels, drywall, metal and plaster substrates, and a
grey and white topcoat. It is not significantly and positively associated with red topcoats or
wood substrates.
The technical operator found the ANDalyze Lead-in-Paint Test Kit instructions to be clear,
informative, and easy to follow. The non-technical operator, however, did not. A tip sheet was
also provided with the instructions, containing helpful hints in the test kit operation. Both the
technical and non-technical operator stated that a significant amount of training and possibly
previous experience or laboratory knowledge would be needed to successfully operate this test
kit.
All reagents came prepared and ready to use. The solutions used for different steps were easily
identifiable within the kit. Storage conditions of the reagents were not marked on the containers,
although the ANDalyze Lead-in-Paint Test Kit instruction manual did indicate storage
requirements and a temperature range for test kit operation.
The ANDalyze Lead-in-Paint Test Kit, as supplied for this verification test, included, a drill,
modified half-inch drill bit, fluorimeter, razor blades, ruler, 15 mL grinding tubes and pestles,
nitric acid, plastic pipettes, calibration solution and buffers, sensor housings, 30 mL plastic tubes
pre-filled with testing buffer, 10 x 76 mm glass test tubes, 1 mL syringes, 10 |jL mini-pipette and
disposable tips, and grinder cleaning solution.
The waste generated for this test kit included both liquid and solid waste. Solid waste included
pipette tips, glass test tubes, housing sensors, plastic tubes, and disposable plastic pipettes.
Liquid waste included approximately 2 mL of ground paint in nitric acid; 20 mL of testing
46
-------
buffer, 3 mL of calibration solutions, and 2-3 mL of cleaning solution. The ANDalyze Lead-in-
Paint Test Kit tip sheet provided some waste disposal guidelines. It indicated that any used glass
test tubes should be disposed of into a sturdy container, such as a cardboard box, and labeled as
"Broken Glass". Any uncontaminated nitric acid solution should be neutralized with testing
buffer or baking soda and flushed down the sink. If a positive result is obtained with the test kit,
the sample would be assumed to contain lead. As such, any lead-containing waste, such as the
paint extract, calibration standards and tubes, and syringes, would be considered lead waste. As
such, the ANDalyze Lead-in-Paint Test Kit instructions indicated that EPA and DOT regulations
pertaining to disposal of lead waste should be followed and non-positive tests (indicating that no
lead at the regulated level is present) would be considered non-lead waste and disposed of with
normal waste procedures. (Note: Because regulations for the disposal of wastes generated from
the use of lead test kits may vary from state to state, EPA recommends that test kit users contact
their state government agency for proper waste disposal requirements.)
Operation of the test kit took approximately 18 minutes by both the technical and non-technical
operator, not including the grinder washing procedure. A normal power supply was needed for
the operation of the fluorimeter. At the time of the writing of this report, the cost of the
fluorimeter is $1500 and the cost for the drill is $310. The cost for all necessary consumables to
conduct 50 tests is $300.
47
-------
Chapter 8
References
1. Test/QA Plan for Verification of Qualitative Spot Test Kits for Lead in Paint, Battelle,
Columbus, Ohio, March 29, 2010.
2. "Lead Renovation, Repair, and Painting Program Final Rule", Federal Register, 73:78
(April 22, 2008), p.21692.
3. President's Task Force on Environmental Health Risks and Safety Risks to Children.
Eliminating Childhood Lead Poisoning: A Federal Strategy Targeting Lead Paint Hazards.
2000. Washington, DC.
4. NISTIR 6398, "Spot Test Kit for Detecting Lead in Household Paint: A Laboratory
Evaluation," NIST, May 2000.
5. ASTM E1828, "Standard Practice for Evaluating the Performance Characteristics of
Qualitative Chemical Spot Test Kits for Lead in Paint," ASTM International.
6. Revised Plan For Development And Production Of Performance Evaluation Materials For
Testing Of Test Kits For Lead In Paint Under The Environmental Technology Verification
Program, Battelle, Columbus, Ohio, September 2008.
7. Standard Operating Procedure for Collection of Dried Paint Samples for Lead Determination,
Battelle, Columbus, OH.
8. ASTM El 729, "Standard Practice for Field Collection of Dried Paint Samples for
Subsequent Lead Determination," ASTM International.
9. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center, U.S.
EPA Environmental Technology Verification Program, prepared by Battelle, Columbus, OH,
Version 7.0, 2008.
10. "Environmental Technology Verification Program Quality Management Plan", December
2002, EPA/600/R-03/021.
11. Standard Operating Procedure for Analysis of Lead in Paint Samples for Battelle, Schneider
Laboratories, Inc., Doc #111-044-10-011, January 20, 2010, revised February 24, 2010 and
April 25, 2010.
48
-------
12. Cook, J. and L. A. Stefanski. 1994. A simulation extrapolation method for parametric
measurement error models. Journal of the American Statistical Association 89: 1314-1328.
13. Hardin, J. W., H. Schmiediche, and R. J. Carroll. 2003a. The simulation extrapolation
method for fitting generalized linear models with additive measurement error. Stata Journal
3(4): 373-385.
14. Hardin, J. W. and R. J. Carroll. 2003b. Measurement error, GLMs, and notational
conventions. Stata Journal 3(4): 328-340.
15. Hardin, J. W., H. Schmiediche, and R. J. Carroll. 2003c. The regression-calibration method
for fitting generalized linear models with additive measurement error. Stata Journal 3(4):
360-371.
16. Stefanski, L. A. and J. Cook. 1995. Simulation extrapolation: The measurement error
jackknife. Journal of the American Statistical Association 90(432): 1247-1256.
49
-------
Appendix A
Performance Evaluation Materials
Summary Information
-------
List of Abbreviations and Acronyms
AMS
ASTM
CCV
CoV
CRM
DI
EPA
ESTE
ETV
FT
HVLP
ICP-AES
ICS
ICV
LCS
mg/cm2
mg/kg
mL
MSDS
NLLAP
PEM
ppb
QA
QC
QMP
RH
RPD
SOP
Advanced Monitoring Systems
American Society for Testing and Materials
continuing calibration verification
coeffi ci ent of vari ati on
certified reference material
deionized
U.S. Environmental Protection Agency
Environmental and Sustainable Technology Evaluations
Environmental Technology Verification
film thickness
high volume/low pressure
inductively coupled plasma-atomic emission spectrometry
interference check sample
initial calibration verification
laboratory control spike
micrograms per liter
microliters
milligrams per centimeter squared
milligrams per kilogram
milliliter
material safety data sheets
National Lead Laboratory Accreditation Program
performance evaluation material
parts per billion
quality assurance
quality control
quality management plan
relative humidity
relative percent difference
standard operating procedure
A-2
-------
Section Al
Preparation of
Performance Evaluation Materials
A-3
-------
Executive Summary
Battelle prepared a batch of performance evaluation materials (PEMs) for use in an
Environmental Technology Verification (ETV) program evaluation of the performance of lead
paint test kits. These PEMs encompass two lead types (white lead [lead carbonate] and yellow
lead [lead chromate]), four separate substrates (metal, wood, drywall, and plaster), and six lead
levels within each lead type (0.3, 0.6, 1.0, 1.4, 2.0, and 6.0 mg/cm2). The goal of the production
was to produce panels at a specified lead level with minimal variability across and within panels.
The study design called for a verification and homogeneity study involving inductively coupled
plasma (ICP) testing of the painted metal panels to determine applied lead levels. Initial
application procedures included spray application for paints at 2.0 and 6.0 mg/cm2, but testing
indicated that spray application yielded high variability in lead levels. As a result, the Battelle
team, in consultation with U.S. Environmental Protection Agency (EPA), decided to apply all
lead paint layers via drawdown bar, which enables more precision in the thickness of the paint
layer applied. Later in the development process, continued high variability measurements led to
the team's decision to include silica in the formulations of each lead paint to thicken the paint
and allow for a more even coating.
Verification and homogeneity testing was conducted for all 12 lead paints as well as the one no-
lead control paint. Verification testing determined the formulation and drawdown bar best suited
to yield a particular lead level. Homogeneity test results were assessed for proximity to target
lead levels, lead level range, and variability within and between panels. All paints passed
verification and homogeneity testing.
After completing the verification and homogeneity testing, base paint layers were applied for all
12 sets of lead paints (two lead types by six lead levels) and the no-lead paint. Paint chips were
sampled and analyzed from the metal reference panels within each set of PEMs. The metal
reference panel measurements met target specifications for all sets of PEMs. All nine sets of 468
panels each were appropriately labeled and packaged. All reference PEM concentrations and
homogeneity results were reviewed and approved by EPA prior to full-scale production of a set.
Study Design
The initial study design specified production of the ETV PEMs using six lead levels (0.3, 0.6,
1.0, 1.4, 2.0, and 6.0 mg/cm2), two lead types (white and yellow lead), four substrates (wood,
metal, drywall, and plaster), and three topcoat colors (white, red-orange, and grey-black), as
specified in Table A-l. For the wood substrates, both poplar and pine wood panels were
produced, segregated, and uniquely labeled to be consistent with the design in Table A-l.
The final design specified production of 624 panels for each of seven test kits for a total of 4,368
panels. Late in the development process, the planned evaluation design changed so that only 468
panels were required to test each of nine test kits for a total of 4,212 PEMs needed for the ETV
test.
A-4
-------
Table A-l: PEMs Produced for ETV Evaluation
Lead Type
Control Blank
White Lead
(Lead
Carbonate)
Yellow Lead
(Lead
Chromate)
Lead Level
(mg/cm2)
0
0.3
0.6
1.0
1.4
2.0
6.0
0.3,0.6,1.0,
1.4,2.0, 6.0
Substrate
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Wood-Poplar
Wood-Pine
Metal
Drywall
Plaster
Subtotal - Per Test Kit
# Samples Produced Per Test Kit by To
White
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
Red-Orange
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
Grey-Black
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2
2
4
4
4
2 panels per cell for Wood substrates, 4
panels per cell for other substrates (same
design as White Lead panels)
208
208
208
pcoat Color
Total
6
6
12
12
12
6
6
12
12
12
6
6
12
12
12
6
6
12
12
12
6
6
12
12
12
6
6
12
12
12
6
6
12
12
12
36
36
72
72
72
624
7 Test Kits
42
42
84
84
84
42
42
84
84
84
42
42
84
84
84
42
42
84
84
84
42
42
84
84
84
42
42
84
84
84
42
42
84
84
84
252
252
504
504
504
4,368
The original design plan called for a target lead concentration of 0.3 mg/cm for a set of PEMs.
During the writing of the ETV test/quality assurance plan, preliminary ICP results indicated that
the target level for this set of PEMs might be closer to 0.4 mg/cm2. The preliminary results were
used in the ETV test plan.
A-5
-------
Substrate Preparation
The ETV PEMs included four different substrate types - metal, wood, drywall, and plaster;
although two types of wood (pine and poplar) were utilized. The following bulleted lists
describe the steps taken to prepare each of the types of substrates.
Metal
Iron Phosphate Steel panels 0.032" x 3" x 3" were placed in an isopropyl alcohol bath and
carefully wiped and dried before being placed in plastic bags prior to coating.
The solvent wipe step was performed to ensure that residue oils/fingerprints from the
manufacturing processes were removed.
Wood
Wood (pine and poplar) was purchased in 4" widths, planed, and cut into 3" x 3" panels.
PEMs were placed into constant temperature and humidity conditioning rooms prior to
coating application to ensure uniform water content through each panel prior to coating.
[Note that plaster and drywall panels are less sensitive to water absorption prior to
coating.]
Drywall
4" x 8" x 3/8" gypsum drywall sheets were cut into 3" x 3" panels.
Plaster
Two j oint compound materials were evaluated for ease of application and smoothness to
ensure the best surface for coating. USG Joint Compound provided the smoothest
surface and was used to coat panels at about 1/32" thickness.
A 3 " x 4' strip of 3/8" thick gypsum drywall was placed into jig, then plaster joint
compound was smoothed over top surface to a precise 1/32" thickness. Plastered drywall
strips were then cut down into 3" x 3" panels.
Sealer Application to Drywall and Plaster PEMs
Stacks of drywall and plaster PEMs were sealed on cut edges with no lead latex
primer/sealer to eliminate dusting.
All panels were then placed in constant temperature and humidity conditioning rooms prior to
coating application.
Spray Application Facilities and Equipment
Battelle's laboratory includes a walk-in spray booth capable of this type of production as well as
air handling equipment and monitors to ensure the safety of Battelle staff. Although the 0.3 and
6.0 mg/cm2 white lead and no-lead paints were applied via spray application, all other
application of lead paint layers was performed using drawdown bars in a laboratory setting. All
topcoats were applied by spray application in the spray booth. Details on the equipment used in
these processes are listed below.
A-6
-------
Spray Booth
10' x 10' x 7.5' double door spray booth
Compressed air supply for spray equipment
Spray equipment consists of a high volume/low pressure (HVLP) gravity fed DeVilbiss
spray gun
Plastic sheeting covering walls and floor to minimize clean-up time
Conditioning Rooms
Constant temperature (75ฐFarenheit)) and humidity (50% relative humidity [RH])) rooms
for substrate conditioning (the variability in temperature and RH is not tracked in those
rooms)
Substrates were conditioned both before and after coating application. Wood substrates
were conditioned a minimum of two weeks prior to coating. All substrates were
conditioned a minimum of 48 hours after coating and before bagging and wrapping.
Plastic covering was placed on the floor to minimize clean-up time after transporting
drying racks from the coating application lab into the conditioning rooms.
Environmental Health and Safety
Battelle developed a health and safety plan related to producing lead-based paint and PEMs
coated with these paints. The plan was approved internally by appropriate environmental safety
and health personnel. Environmental monitoring during paint mixing and spraying activities
determined that lead exposure levels for workers were below Occupational Safety and Health
Administration standards. Some of the components of the safety plan included:
All staff and any visitors were required to have documented hazard communication
training on lead.
Baseline and post-work blood-lead levels were obtained for those Battelle staff that
conducted the paint mixing and spray painting.
Respirators were used during leaded paint production
Spray application operations staff were required to have a physical, appropriate training,
and to pass a respirator fit test.
The interior of the spray booth was covered with plastic or other material that could be
easily removed and was then disposed of as hazardous waste.
The area in front of the booth was set up as a change-out area where personal protective
equipment, such as coveralls, etc., could be removed without spreading lead outside of
the area.
Warning signs restricting access were posted at the paint booth door.
Preparation of Linseed Oil Based Leaded Paints
To formulate historically accurate lead-based paints to apply to PEMs, Battelle consulted
Bennett's The Chemical Formulary - A Collection of Valuable, Timely, Practical Commercial
Formulae and Recipes for Making Thousands of Products in Many Fields of Industry, Volume
VI.l The Chemical Formulary had been printed with revisions every year until at least 1998.
Sample formulations from this reference are listed below in Table A-2. Since the paints
produced for the ETV verification of lead test kits were being applied to metal, drywall, plaster
A-7
-------
and wood, Battelle used a combination of formulations from Chapter Thirteen - Paint, Varnish,
Lacquer and Other Coatings to ensure adhesion to all substrates. Battelle reviewed the various
relevant historical formulations and developed formulations to apply to the PEMs that would
work best for application to the four substrates being used, i.e. would provide the best adhesion
to the variety of substrates required while achieving desired target lead levels.
Table A-2: Paint Formulations from The Chemical Formulary
Floor Painting and Finishing (p.
281) (for raw wood)
Soft Paste White Lead, 100 Ib.
Raw Linseed Oil, 3 gal.
Turpentine, 2 gal.
Liquid Drier, 1 pt.
Plaster, Primer (p. 332)
White Lead, Semi-Paste, 100 Ib.
Interior Varnish, 4 gal.
Linseed Oil, Kettle Bodied, 2 gal.
Turpentine, % gal.
Exterior House Paint Pigments
White (p. 328)
35% Leaded Oxide, 45 Ib.
White Lead, 181b.
Titanium Dioxide, 15 Ib.
Inert, 22 Ib. (Battelle used Zinc
Oxide)
In preparing the lead-based paints for the PEMs, Battelle used a combination of raw and boiled
linseed oil to ensure realistic drying time and good adhesion to the variety of substrates. A
variety of other formulas in the reference also mix these two resins.
A similar formulation was also found in Charles Uebele' s Paint Making and Color Grinding: A
Practical Treatise for Paint Manufacturers and Factory Managers2. The excerpt below explains
the difference in formulation requirements based on the substrate to which the paint will be
applied.
"CHAPTER XXV - DIPPING PAINTS.
Dipping Paints for Wood or Metal require to be made specially for either surface, as that
intended for wood will not always serve the purpose for metal. The paint for wood
requires to contain a pigment that acts as a filler, while tin or smooth sheet iron or steel
does not necessarily need it, in fact, it is best without it for certain metallic surfaces. The
function of a dipping paint is, first of all, to economize in labor, to cover uniformly
any article immersed in it, and to dip freely without leaving fringes of paint at the edges
and dry equally all over the surface thus coated.
Metal Preservative Red may be made by grinding a base of 40 pounds bright red oxide of
95 per cent, purity, 8 pounds red lead, 2 pounds zinc chr ornate, 25 pounds floated silex or
silica in 25 pounds raw linseed oil thinning same with 5 gallons raw linseed oil, 1 gallon
hard gum japan and Y% gallon turps. This will produce 12 gallons of paint weighing a
trifle over II pounds per gallon. By substituting a long stock of hard gum varnish for part
of the 5 gallons raw oil a hard drying product will be the result. "
In support of achieving consistent application of the lead-based paints in terms of film thickness
and lead level, Battelle investigated additions of various elements to mitigate settling and
improve application. Silicon dioxide was selected for this purpose because it was present in pre-
1978 leaded paints, is used for thickening and anti-settling properties in modern paint
formulations, and achieved the most consistent results. Battelle established the historical
A-8
-------
precedent for including silica in paints in a technical report submitted to EPA on February 19,
20093.
The primer and topcoats applied to the PEMs on top of the lead-based paints (or base paint for
the no-lead panels) all contain some form of Diatomaceous Silica, as well. The primer and three
topcoats applied are listed below.
Sherwin Williams brand PreRite Bonding Primer
Sherwin Williams Classic 99 Interior Satin Latex color Extra White
Sherwin Williams Classic 99 Interior Satin Latex color 7047 Software (Grey)
Sherwin Williams Accents Interior Satin Latex color 6867 Fireworks (red-orange)
Section 2 of the primer Material Safety Data Sheet (MSDS) specifies that the primer contains 9%
quartz. Quartz is referred to as "Crystalline Silica" in Section 11 of the MSDS. The MSDSs for
the three topcoats specify Cristobalite (CAS 14464-46-1) as an ingredient, which is a synonym
for silicon dioxide and also referred to as Crystalline Silica in Section 11. All panels have some
level of silica in the topcoat layers.
The paint formulations used for this effort were based on historical records. Primary ingredients
included zinc oxide, raw and boiled linseed oil, turpentine, Japan drier, either lead carbonate or
lead chromate, and titanium dioxide (used to balance the levels of lead). Nine different paint
formulations were produced as dictated by the two lead pigments (lead carbonate and lead
chromate) and the six different lead levels in addition to the zero lead level control. The
formulations were designed to consistently achieve the lead levels required when applied at
typical wet film builds.
The paint formulations are shown in Tables A-3 and A-4 below. Since the molecular
compositions of the two lead pigments are different, the formulations have accounted for these
differences by adjusting the load levels. However, the formulations for the 0% lead chromate
and carbonate were the same because no lead pigment was used in either.
A-9
-------
Table A-3. White Lead (Lead Carbonate) Paint Formulations
0% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7.0
7.0
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
OLead
14.79
0.00
6.16
1.48
0.15
2.16
0.05
24.8
0 Lead % by wt.
59.67%
0.00%
24.86%
5.97%
0.60%
8.70%
0.20%
100%
Sample reduced to 60% solids, 0% of TS-100 silica added then sprayed to thickness.
0.3% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.3 Lead
14.85
1.49
4.95
1.49
0.15
2.17
0.05
25.1
0.3 Lead % by wt.
59.08%
5.91%
19.69%
5.91%
0.59%
8.62%
0.20%
100%
Sample reduced to 60% solids, 0% of TS-100 silica added then sprayed to 3 milswet.
Gram wt
1491.75
0.00
621.56
149.18
14.92
217.55
5.04
2500
Gram wt
1477.08
147.71
492.36
147.71
14.77
215.41
4.97
2500
0.6% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
Sample reduced to
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.6 Lead
14.77
1.77
4.92
1.48
0.15
2.15
0.03
25.3
0.6 Lead % by wt.
58.45%
7.00%
19.47%
5.86%
0.59%
8.51%
0.12%
100%
70% solids, 0.7% of TS-100 silica added then drawdown with # 24 bar.
Gram wt
1461.22
175.11
486.74
146.42
14.84
212.70
2.97
2500
1.0% Lead Carbonate Paint Formulation
Materials
ZnO
PbC03
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
Thisformulation wi
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.0 Lead
14.40
3.00
4.80
1.44
0.14
2.10
0.05
25.9
1.0 Lead % by wt
55.53%
11.57%
18.51%
5.55%
0.56%
8.10%
0.19%
100%
I be used to produce 0.6% and 1.4% lead levels at different coating thick
Gram wt
832.88
173.52
277.63
83.29
8.33
121.46
2.89
1500
ess.
1.4% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.4 Lead
13.22
4.21
4.81
1.44
0.14
2.1
0.03
26.0
1.4 Lead % by wt
50.94%
16.22%
18.54%
5.55%
0.54%
8.09%
0.12%
100%
Gram wt
764.16
243.35
278.03
83.24
8.09
121.39
1.73
1500
Sample reduced to 70 % solids, 1.5% of TS-100 silica added then drawdown with # 54 bar.
2.0% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
2.0 Lead
12.88
6.08
4.12
1.41
0.14
2.06
0.05
26.7
2.0 Lead % by wt
48.16%
22.73%
15.41%
5.28%
0.53%
7.70%
0.19%
100%
Sample reduced to 65% solids, 1.5% of TS-100 silica added then drawdown with #40 bar.
Gram wt
722.42
340.98
231.17
79.20
7.92
115.50
2.80
1500
6.0% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
6.0 Lead
4.70
18.10
1.57
1.43
0.14
2.09
0.05
28.1
6.0 Lead % by wt
16.73%
64.49%
5.58%
5.09%
0.51%
7.43%
0.18%
100%
Sample reduced to 70% solids, 1% of TS-100 silica added then sprayed to thickness.
Gram wt
250.89
967.34
83.63
76.40
7.64
111.42
2.67
1500
A-10
-------
0% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7.0
7.0
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
OLead
14.79
0.00
6.16
1.48
0.15
2.16
0.05
24.8
0 Lead % by wt.
59.67%
0.00%
24.86%
5.97%
0.60%
8.70%
0.20%
100%
Sample reduced to 60% solids, 0% of TS-100 silica added then sprayed to thickness.
Gram wt
1491.75
0.00
621.56
149.18
14.92
217.55
5.04
2500
0.3% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.3 Lead
14.85
1.49
4.95
1.49
0.15
2.17
0.05
25.1
0.3 Lead % by wt.
59.08%
5.91%
19.69%
5.91%
0.59%
8.62%
0.20%
100%
Sample reduced to 60% solids, 0% of TS-100 silica added then sprayed to 3 mils wet.
Gram wt
1477.08
147.71
492.36
147.71
14.77
215.41
4.97
2500
0.6% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.6 Lead
14.77
1.77
4.92
1.48
0.15
2.15
0.03
25.3
0.6 Lead % by wt.
58.45%
7.00%
19.47%
5.86%
0.59%
8.51%
0.12%
100%
Sample reduced to 70% solids, 0.7% of TS-100 silica added then drawdown with # 24 bar.
Gram wt
1461.22
175.11
486.74
146.42
14.84
212.70
2.97
2500
1 .0% Lead Carbonate Paint Formulation
Materials
ZnO
PbC03
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
Thisformulation wi
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.0 Lead
14.40
3.00
4.80
1.44
0.14
2.10
0.05
25.9
1.0 Lead % by wt
55.53%
11.57%
18.51%
5.55%
0.56%
8.10%
0.19%
100%
Gram wt
832.88
173.52
277.63
83.29
8.33
121.46
2.89
1500
I be used to produce 0.6% and 1.4% lead levels at different coating thickness
1 .4% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.4 Lead
13.22
4.21
4.81
1.44
0.14
2.1
0.03
26.0
1.4 Lead % by wt
50.94%
16.22%
18.54%
5.55%
0.54%
8.09%
0.12%
100%
Gram wt
764.16
243.35
278.03
83.24
8.09
121.39
1.73
1500
Sample reduced to 70 % solids, 1.5% of TS-100 silica added then drawdown with # 54 bar.
2.0% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
2.0 Lead
12.88
6.08
4.12
1.41
0.14
2.06
0.05
26.7
2.0 Lead % by wt
48.16%
22.73%
15.41%
5.28%
0.53%
7.70%
0.19%
100%
Sample reduced to 65% solids, 1.5% of TS-100 silica added then drawdown with #40 bar.
Gram wt
722.42
340.98
231.17
79.20
7.92
115.50
2.80
1500
6.0% Lead Carbonate Paint Formulation
Materials
ZnO
PbCO3
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot*
ZC-XD13
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
6.0 Lead
4.70
18.10
1.57
1.43
0.14
2.09
0.05
28.1
6.0 Lead % by wt
16.73%
64.49%
5.58%
5.09%
0.51%
7.43%
0.18%
100%
Gram wt
250.89
967.34
83.63
76.40
7.64
111.42
2.67
1500
Sample reduced to 70% solids, 1% of TS-100 silica added then sprayed to thickness.
A-ll
-------
Table A-4. Yellow Lead (Lead Chromate) Paint Formulations
0.3% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.3 Lead
14.97
1.10
4.99
1.50
0.15
2.18
0.05
Total
24.9
0.3Lead% bywt.
60.03%
4.40%
20.01%
6.00%
0.60%
8.75%
0.20%
100%
Gram wt
1500.74
110.05
500.25
150.07
15.01
218.86
5.01
2500
Sample reduced to 70 % solids, 0.7% of TS-100 silica added then drawdown with #34 bar.
0.6% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TIO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.6 Lead
14.65
2.15
4.88
1.47
0.15
2.14
0.03
25.5
0.6Lead% bywt.
57.52%
8.44%
19.16%
5.77%
0.59%
8.40%
0.12%
100%
Gram wt
1437.97
211.03
478.99
144.29
14.72
210.05
2.94
2500
Sample reduced to 70 % solids, 1.5% of TS-100 silica added then drawdown with #24 bar.
1.0% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TIO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
This formulation will be used to produce 0
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.0 Lead
14.40
3.00
4.80
1.44
0.14
2.10
0.05
25.9
1.0 Lead % bywt
55.53%
11.57%
18.51%
5.55%
0.56%
8.10%
0.19%
100%
Gram wt
832.88
173.52
277.63
83.29
8.33
121.46
2.89
1500
ferent coatinq thickness.
Sample reduced to 70 % solids, 1% of TS-100 silica added then drawdown with #48 bar.
1.4% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.4 Lead
13.21
5.09
4.2
1.44
0.14
2.1
0.03
26.2
1.4 Lead % bywt
50.40%
19.42%
16.02%
5.49%
0.53%
8.01%
0.11%
100%
Gram wt
1260.02
485.50
400.61
137.35
13.35
200.31
2.86
2500
Sample reduced to 70 % solids, 1% of Aerosil 200 silica added then drawdown with #60 bar.
2.0% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
2.0 Lead
10.90
7.17
3.81
1.49
0.15
2.18
0.05
Total
25.8
2.0 Lead % bywt
42.32%
27.83%
14.81%
5.80%
0.58%
8.46%
0.19%
100%
Gram wt
1058.12
695.72
370.34
145.00
14.50
211.46
4.85
2500
Sample reduced to 70% solids, 1.5% of TS-100 silica added then drawdown with #42 bar.
6.0% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
6.0 Lead
1.65
21.43
0.55
1.51
0.15
2.20
0.05
Total
27.5
6.0 Lead % bywt
5.99%
77.84%
2.00%
5.47%
0.55%
7.98%
0.18%
100%
Gram wt
149.69
1946.00
49.90
136.75
13.68
199.43
4.54
2500
Sample reduced to 70% solids, 2% of TS-100 silica added then drawdown with #54 bar.
A-12
-------
0.3% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.3 Lead
14.97
1.10
4.99
1.50
0.15
2.18
0.05
Total
24.9
0.3 Lead % bywt.
60.03%
4.40%
20.01%
6.00%
0.60%
8.75%
0.20%
100%
Sample reduced to 70 % solids, 0.7% of TS-100 silica added then drawdown with #34 bar.
Gram wt
1500.74
110.05
500.25
150.07
15.01
218.86
5.01
2500
0.6% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TIO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
0.6 Lead
14.65
2.15
4.88
1.47
0.15
2.14
0.03
25.5
0.6 Lead % bywt.
57.52%
8.44%
19.16%
5.77%
0.59%
8.40%
0.12%
100%
Gram wt
1437.97
211.03
478.99
144.29
14.72
210.05
2.94
2500
Sample reduced to 70 % solids, 1.5% of TS-100 silica added then drawdown with #24 bar.
1.0% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TIO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.0 Lead
14.40
3.00
4.80
1.44
0.14
2.10
0.05
25.9
1.0 Lead % bywt
55.53%
11.57%
18.51%
5.55%
0.56%
8.10%
0.19%
100%
Gram wt
832.88
173.52
277.63
83.29
8.33
121.46
2.89
1500
This formulation will be used to produce 0.6% and 1.4% lead evelsat different coating thickness.
Sample reduced to 70 % solids, 1% of TS-100 silica added then drawdown with #48 bar.
1.4% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TIO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
Sample reduced to
GW
47.3
51
37
7.8
7.7
7
7
Total
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
1.4 Lead
13.21
5.09
4.2
1.44
0.14
2.1
0.03
26.2
1.4 Lead % bywt
50.40%
19.42%
16.02%
5.49%
0.53%
8.01%
0.11%
100%
Gram wt
1260.02
485.50
400.61
137.35
13.35
200.31
2.86
2500
70 % solids, 1% of Aerosil 200 silica added then drawdown with #60 bar.
2.0% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TIO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
2.0 Lead
10.90
7.17
3.81
1.49
0.15
2.18
0.05
Total
25.8
2.0 Lead % bywt
42.32%
27.83%
14.81%
5.80%
0.58%
8.46%
0.19%
100%
Gram wt
1058.12
695.72
370.34
145.00
14.50
211.46
4.85
2500
Sample reduced to 70% solids, 1.5% of TS-100 silica added then drawdown with #42 bar.
6.0% Lead Chromate Paint Formulation
Materials
ZnO
PbCrO4
TiO2
Linseed Oil
Boiled Linseed Oil
Turpentine
Japan Drier
GW
47.3
51
37
7.8
7.7
7
7
Lot#
ZC-X013
1401047-267
931407T.12
83734
83404
83304
PJD40
Supplier
The Carry Co.
American Elements
DuPont
Recochem Inc.
Recochem Inc.
Recochem Inc.
Barr
6.0 Lead
1.65
21.43
0.55
1.51
0.15
2.20
0.05
Total
27.5
6.0 Lead % bywt
5.99%
77.84%
2.00%
5.47%
0.55%
7.98%
0.18%
100%
Gram wt
149.69
1946.00
49.90
136.75
13.68
199.43
4.54
2500
Sample reduced to 70% solids, 2% of TS-100 silica added then drawdown with #54 bar.
A-13
-------
Paint Formulation Procedures
The paint samples were produced using standard painting production procedures in the Battelle
laboratories, including pre-mixing, media grinding of pigment and binder resin, and paint
letdown with resin and solvents. This procedure has been used for paint production both in the
laboratory and in commercial paint manufacturing for over 50 years. The equipment utilized in
this procedure includes the following:
Variac that controls the speed of the dispersator
High speed dispersator using a 5" diameter blade on the end of the mixing shaft
Ice bath and ice
Balance
Paint cans
Medium paint filters
Red Devil paint shaker
Following are the detailed steps in the paint formulation procedure:
1. Add enough turpentine to cover mixing blade.
2. Start mixer at low speed.
3. Add zinc oxide slowly for 3-5 minutes, increasing mixing speed as needed to maintain
appropriate grind viscosity as visually evaluated by an operator skilled in the art.
4. Add turpentine as needed to keep the batch rolling.
5. Mix additional 10 minutes after addition of zinc oxide.
6. Add lead pigment slowly for 2-4 minutes, increasing mixing speed as needed.
7. Add turpentine as needed to keep the batch rolling.
8. Mix additional 10 minutes after addition of lead pigments.
9. Add titanium dioxide slowly for 3-5 minutes, increasing mixing speed as needed.
10. Add turpentine as needed to keep the batch rolling.
11. Mix for 60-90 minutes, or until batch viscosity decreases, determined by rolling action of
the batch.
12. Check Hegman, if < 5 continue to mix, and check Hegman every 10 minutes.4
13. When Hegman reaches ^ or > 5, start the let down, which includes adding all remaining
liquid raw materials after the pigment and extenders have been dispersed adequately.
14. Add boiled and raw linseed oil slowly and decrease mixing speed.
15. Add turpentine to wash out linseed oil container.
16. Mix additional 10 minutes after addition of linseed oils.
17. Add Japan drier drop wise to batch.
18. Mix additional 10 minutes after addition of Japan drier.
19. Tare quart cans.
20. Filter batch with medium paint filters into tared quart cans.
21. Note net weight and log book number of batch on quart cans.
22. Yields about ll/2 quarts of lead paint per batch.
23. Allow paint to set overnight.
24. Shake paint with Red Devil paint shaker for about 10 minutes take samples for % solids
check.
25. Check paint solids with moisture balance and record average of three test results on
formulation sheet.
A-14
-------
26. Store paint in aluminum cans in laboratory hood until future use.
Verification and Homogeneity Studies
Various batches of paint were prepared for the initial verification tests - one targeting each lead
level. Each paint was applied via drawdown or hand spraying to 3.5" x 5" metal panels attached
to a wooden rack. For each paint type and concentration batch, panels were coated to determine
proper film thickness, formulations, and drawdown bars to use, if applicable, to achieve each
desired lead level. Subsequently, homogeneity panels were coated to investigate ability to
achieve target lead levels and variability within and across panels. Verification and homogeneity
studies were performed on metal panels only due to ease and accuracy of sample extraction, i.e.,
it was easiest to obtain a 1 inch square sample from the metal surface which led to the most
accurate measurements of lead content in the sampled area, which was critical for verification
purposes.
After drying, paint chip samples were obtained from the metal panels following ASTM E1729.5
Laboratory analysis for lead by inductively coupled plasma-atomic emission spectrometry (ICP-
AES) was planned and conducted at an independent National Lead Laboratory Accreditation
Program (NLLAP)-accredited laboratory, Schneider Laboratories, Inc. ICP-AES testing was
conducted on three panels for each lead level with four samples obtained from each panel,
referred to as Homogeneity Panels since the primary purpose of the samples was to assess
consistency of lead levels across and within panels. The paint chips were digested using EPA
Method 3050B6 and the ICP-AES analysis was conducted following EPA Method 6010C7 as
well as the Schneider Laboratories, Inc. ICP SOP.8 The laboratory electronically reported lead
level measurements along with quality control (QC) sample results. Laboratory spike and
duplicate results as well as calibration verification sample results were supplied and reviewed for
each batch of samples analyzed. Acceptable recoveries for spike samples ranged from 80% to
120%. Acceptable recoveries for calibration verification samples were 90-110%. Acceptable
duplicate samples had a relative percent difference of 25% or less. Percent recoveries for
calibration verification samples ranged from 93-110%. Recoveries for QC spike samples ranged
from 92-115%. All duplicate samples had less than 25% relative percent difference. There were
no QC failures or problems.
Film thickness measurements were obtained by Battelle for each paint sample taken. Results of
the final batches of homogeneity samples for each set of PEMs are included in Table A-5.
Results were evaluated to determine correspondence to target lead levels and level of variability
as measured by the coefficient of variation (CoV), the standard deviation divided by the mean.
The production plan, agreed to in advance, specified a minimum acceptability of a CoV of less
than 15 percent. Following analysis, the results were forwarded to EPA with recommendations
regarding ability to proceed with production or the need for additional homogeneity testing. The
results shown in Table A-5 met the acceptability requirements and were thus deemed acceptable
for proceeding with the production of sets of PEMs at each lead level.
A-15
-------
Table A-5. Results from Final Homogeneity Testing on Metal Substrates for Each Set of
ETV PEMs
Lead
Type
White
Lead
Lead
Chromate
Target Lead
Level
0.3
0.6
1.0
1.4
2.0
6.0
0.3
0.6
1.0
1.4
2.0
6.0
Mean Levels
ICP (mg/cm2)
0.30
0.65
0.99
1.56
1.85
5.97
0.30
0.62
1.07
1.42
1.92
6.88
FT (mils)
0.79
0.95
1.26
1.72
1.48
1.94
1.16
0.98
1.50
1.89
1.38
1.81
CoV*
ICP
13.3
7.1
3.9
7.2
5.6
14.2
9.6
4.1
11.0
4.1
10.1
5.2
FT*
6.1
5.7
3.4
3.5
7.0
8.3
4.0
9.1
7.4
6.8
2.4
3.3
* Coefficient of Variation (Standard Deviation/Mean x 100)
** Film thickness
Production Application of Lead Paint Coatings
Based on the results from the Verification and Homogeneity Study summarized in Table A-5,
production proceeded using the paint formulation and application method (spray or a particular
size drawdown bar) that achieved the target lead levels. During production application,
reference panels were coated along with the production panels at a rate of 18 for each set of 468
panels. For sets of PEMs that were sprayed, reference panels were placed at previously-
assigned, randomly selected locations on the racks containing all the PEMs awaiting paint
application. For sets of PEMs that had the lead paint applied via drawdown bar, production
panels were drawdown in sets of two to three panels each for the wood, metal and drywall
substrates and one at a time for the plaster substrates. At the discretion of the operator, a
reference panel was prepared approximately every 10 sets or 25 panels.
Metal panels were used as the reference panels since metal panels yield the most accurate
measurements of film thickness and lead levels. The reference PEMs were tested for film
thickness during application and for lead level by ICP analysis after the paint had dried. This test
procedure was used to check that the application process resulted in appropriate lead levels.
Despite the use of the metal substrate only for the reference panels, the lead levels and paint
thickness on these reference panels served as representative of the coatings applied to all wood,
drywall, plaster, and metal substrate panels.
Table A-6 presents the average lead levels, CoV, minimum, and maximum of each set of 18
reference panel measurements. Most sets are very close to target lead levels, such as the 2.03
mg/cm2 average for the 2.0 mg/cm2 target yellow lead set, the 0.32 mg/cm2 for the 0.3 mg/cm2
target yellow lead set, and the 0.64 mg/cm2 average for the 0.6 mg/cm2 target white lead set.
There also were a few sets that were a bit off target, but were sufficient to meet the verification
A-16
-------
needs. Despite the high average lead level of 9.2 mg/cm2, the 6.0 mg/cm2 white lead PEMs were
accepted by EPA because they still met the needs of the verification for a set of PEMs at a high
lead level. In the 0.6 mg/cm2 yellow lead batch, the measured lead levels of 17 of the 18
reference panels ranged from 0.51 to 0.66 mg/cm2, yielding a mean of 0.55 mg/cm2, and a CoV
of 7.5%. Because only one reference panel of 18 yielded a high lead level, the set of panels was
accepted.
Table A-6. Reference Panel Results from Final Production for Each Set of ETV PEMs
Lead Type
No Lead
White
Lead
(Lead
Carbonate)
Yellow
Lead
(Lead
Chromate)
Target Lead
Level
0.0
0.3
0.6
1.0
1.4
2.0
6.0
0.3
0.6
1.0
1.4
2.0
6.0
Lead Levels
Mean (mg/cm2)
0.00
0.40
0.64
1.00
1.48
2.29
9.18
0.32
0.57
1.00
1.39
2.03
5.15
CoV
8.2
17.8
13.5
5.1
8.0
5.6
31.2
13.1
16.6
7.1
12.0
9.4
9.6
Range
Min
0.002
0.234
0.425
0.918
1.322
2.018
5.65
0.252
0.511
0.879
1.194
1.483
3.929
Max
0.003
0.505
0.761
1.095
1.748
2.525
18.4
0.428
0.920*
1.148
1.601
2.314
6.247
* Next highest measurement was 0.659
Topcoating
The linseed oil based paints were applied to the PEMs and stored in the constant temperature and
humidity rooms during a four to seven day drying time. The panels were then all topcoated with
Sherwin Williams brand Prep Rite bonding Primer to ensure good adhesion between the linseed
oil based paint and the latex emulsion topcoat paints. The final latex paint topcoat was then
applied to the PEMs. The topcoat paints are described in more detail below:
Primer - Sherwin Williams Prep Rite bonding primer, diluted with deionized (DI) water
at a ratio of 3:1 parts by volume. Spray application was done with a 50 percent overlap on
the PEMs in both horizontal and vertical directions with a total wet film build of
approximately 4-5 mils (a measure of dry film thickness). The PEMs then were allowed
1-2 hours to air dry before top coats were applied.
Top coat number 1 is Sherwin Williams Classic 99 interior satin latex; color Extra White,
diluted with DI water at a ratio of 3:1 parts by volume. Spray application was done with a
50 percent overlap on the PEMs in both horizontal and vertical directions, with a total
wet film build of approximately 4-6 mils. Then the PEMs were allowed to air dry for
three days. The PEMs were then bagged for further testing.
Top coat number 2 is Sherwin Williams Classic 99 interior satin latex; color 7047
(software gray), diluted with DI water at a ratio of 3:1 parts by volume. Spray application
was done with a 50 percent overlap on the PEMs in both horizontal and vertical
A-17
-------
directions for a total wet film build of approximately 4-6 mils. Then the PEMs were
allowed to air dry for three days. The PEMs were then bagged for further testing.
Top coat number 3 is Sherwin Williams Color Accents interior satin latex; color 6867
(Fireworks orange red), diluted with DI water at a ratio of 3:1.5 parts by volume. Spray
application was done with a 50 percent overlap on the samples in both horizontal and
vertical directions for a total wet film build of approximately 4-6 mils. The PEMs were
then allowed to air dry for three days. The PEM samples were then bagged for further
testing.
PEM Labeling, Packing and Storage
The PEMs were stored in the constant temperature and humidity conditioning rooms prior to
being packed up for transfer to the evaluation location. Each PEM was labeled on the back with
an individual identification number, wrapped in a single laboratory towel to protect the front
surface, and placed inside an individual zip seal bag also labeled with the identification number.
References
1. Bennett, H. The Chemical Formulary - A Collection of Valuable, Timely, Practical
Commercial Formulae and Recipes for Making Thousands of Products in Many Fields of
Industry, Volume VI. 1943. Chemical Publishing Co., Inc. Copyright 1943.
2. Uebele, Charles. Paint Making and Color Grinding: A Practical Treatise for Paint
Manufacturers and Factory Managersa The Trade Papers Publishing Co., Ltd., 1913.
3. Battelle. Addition of Silica to Lead-based Paints Used for Production of PEMs in
Support of ETV Evaluation of Lead Test Kits: References. Technical report submitted to
EPA on February 19, 2009.
4. ASTM D1210-05(2010), "Standard Test Method for Fineness of Dispersion of Pigment-
Vehicle Systems by Hegman-Type Gage," ASTM International.
5. ASTM El 729, "Standard Practice for Field Collection of Dried Paint Samples for
Subsequent Lead Determination," ASTM International.
6. United States Environmental Protection Agency, "Method 3050B: Acid Digestion of
Sediments, Sludges, and Soils", SW846 Online, Revision 2. December 1996.
7. United States Environmental Protection Agency, "Method 6010C: Inductively Coupled
Plasma-Atomic Emission Spectrometry", SW846 Online, Revision 3. February 2007.
8. Schneider Laboratories, Inc. ICP SOP Document# III-017-08-002
A-18
-------
Section A2
Comparison of Expected vs. Actual Lead Concentrations
of Performance Evaluation Materials
A-19
-------
The following tables present a comparison of the expected vs. confirmed lead concentration for
each PEM used during the testing of the lead test kits. Expected concentrations are based on lead
levels defined for sets of PEMs during the PEM production process. That is, PEMs were being
made at expected lead concentrations of 0, 0.3, 0.6, 1.0, 1.4, 2.0, or 6.0 mg/cm2. These are the
expected lead levels as defined in the test/quality assurance (QA) plan. Confirmed
concentrations are based on ICP-AES results from individual paint chip samples taken from each
PEM during testing (see Section 3.3. lin the test/QA plan).
Table A-7 presents the results by substrate and across all PEMs. Table A-8 presents the results
by lead type. The average and standard deviation for the confirmed lead levels, as well as the
CoV, are presented for each expected concentration level.
Table A-7. Confirmed lead level statistics for PEMs compared to expected lead level
concentrations by substrate type.
Substrate
Drywall
Metal
Plaster
Wood
All
N
Confirmed Lead
Confirmed Lead
CoV (%)
N
Confirmed Lead
Confirmed Lead
CoV (%)
N
Confirmed Lead
Confirmed Lead
CoV (%)
N
Confirmed Lead
Confirmed Lead
CoV (%)
N
Confirmed Lead
Confirmed Lead
CoV (%)
Level:
Level:
Level:
Level:
Level:
Level:
Level:
Level:
Level:
Level:
Average
StdDev
Average
StdDev
Average
StdDev
Average
StdDev
Average
StdDev
Expected PEM
0
144
0.00
0.00
143.16
144
0.00
0.01
368.04
140
0.00
0.01
258.82
144
0.00
0.02
470.54
572
0.00
0.01
451.36
0.3
288
0.34
0.07
20.59
288
0.31
0.07
22.91
290
0.44
0.18
40.17
288
0.32
0.16
48.20
1154
0.35
0.14
38.91
0.6
282
0.83
0.22
26.01
288
0.56
0.14
24.39
292
1.25
0.53
42.40
288
0.72
0.22
30.83
1150
0.84
0.41
48.34
Lead Level (mg/cm
1
290
1.15
0.21
17.95
288
0.85
0.10
11.31
296
1.65
0.84
50.92
275
1.07
0.31
28.80
1149
1.18
0.55
46.76
1.4
296
1.48
0.29
19.79
288
1.26
0.19
14.91
288
1.79
0.65
36.22
282
1.45
0.29
20.29
1154
1.50
0.44
29.38
2)
2
288
2.52
0.33
13.22
288
1.91
0.28
14.49
288
2.91
0.85
29.24
284
2.39
0.71
29.56
1148
2.43
0.69
28.49
6
292
9.04
2.32
25.69
286
8.18
1.86
22.76
284
10.11
3.01
29.81
288
8.71
1.59
18.29
1150
9.01
2.36
26.24
CoV = Coefficient of Variation (Standard Deviation/Mean x 100)
Table A-7 indicates that overall confirmed lead levels were similar to expected concentrations.
However, there are substrate types for which, comparatively, the confirmed lead levels were
higher than the expected levels. Average confirmed levels for drywall and plaster PEMs were
higher than expected levels, especially when compared to average confirmed lead levels from
metal and wood. The PEMs used in the verification test were produced mainly using a
drawdown technique (for all panels except no lead, 0.3 mg/cm2 and 0.6 mg/cm2 white lead).
This involved applying the paint to the PEM and pulling it down with a specially designed bar.
Being porous substrates, it is possible that the plaster and drywall panels absorbed some of the
A-20
-------
paint, causing more paint to be applied to the PEM to accommodate the thickness required on the
PEM. This would then lead to higher lead concentrations on these substrates. The most
significant potential impact of this effect can be seen on the plaster PEMs. This potential effect
is based on observations during the production of the PEMs but has not been studied or
confirmed.
Table A-8. Confirmed lead level statistics for PEMs compared to expected lead level
concentrations by lead paint type.
Lead Type
None
White
Yellow
All
N
Confirmed Lead
Confirmed Lead
CoV (%)
N
Confirmed Lead
Confirmed Lead
CoV (%)
N
Confirmed Lead
Confirmed Lead
CoV (%)
N
Confirmed Lead
Confirmed Lead
CoV (%)
Level:
Level:
Level:
Level:
Level:
Level:
Level:
Level:
Average
StdDev
Average
StdDev
Average
StdDev
Average
StdDev
Expected PEM
0
572
0.00
0.01
451.36
572
0.00
0.01
451.36
0.3
576
0.30
0.08
25.56
578
0.40
0.16
40.64
1154
0.35
0.14
38.91
0.6
574
0.88
0.41
46.53
576
0.80
0.40
49.95
1150
0.84
0.41
48.34
Lead Level
1
573
1.24
0.72
58.18
576
1.13
0.30
26.32
1149
1.18
0.55
46.76
(mg/cm
1.4
578
1.53
0.52
34.08
576
1.46
0.33
22.87
1154
1.50
0.44
29.38
2)
2
576
2.36
0.58
24.60
572
2.51
0.78
31.25
1148
2.43
0.69
28.49
6
572
8.37
2.05
24.47
578
9.64
2.48
25.75
1150
9.01
2.36
26.24
CoV = Coefficient of Variation (Standard Deviation/Mean x 100)
The results in Table A-8 show that there was no significant difference in confirmed lead levels
between white and yellow lead PEMs. The CoVs values were all <50% at all levels except 0.0
mg/cm2. The larger CoV at this level is reflective of small changes around the zero lead level
and most likely represent ICP-AES measurement variability near the detection limit, since no
lead was used in preparing these PEMs. It should be noted, as discussed in Section Al of this
appendix, that the PEMs prepared at the expected lead level of 6.0 mg/cm2 were known to be on
average higher than 6.0 mg/cm2 and that it was purposefully decided to accept the variation
present at this expected lead level.
Though there were some differences between the confirmed and expected lead levels, it should
be noted that when evaluated for proper responses, test kit results were compared to confirmed
lead levels. That is, test kit results were always compared to the actual PEM lead levels, not the
expected.
A-21
-------
Section A3
QA/QC Results for the ICP-AES Analysis of
Performance Evaluation Materials
A-22
-------
Summary of Lead Level Confirmation ICP-AES Analysis of PEMs
All paint chip samples from the PEMs used in this verification test were analyzed using ICP-
AES by Schneider Laboratories, Inc.
Sample preparation procedures followed the SOP generated by Schneider Laboratories, Inc. for
this study (Schneider Laboratories, Inc., SOP Battelle Paint Samples, Doc # 111-044-10-001).
Information on how QC samples were spiked and final concentrations is provided in the SOP.
Three versions of this SOP (the original and two revisions) were used dated 1/20/10, 2/24/10,
and 4/25/10. Approximately 27% of the PEMs were analyzed prior to the 2/24/10 revision to the
SOP. In the 2/24/10 version, revisions were made to clarify that post-digestion matrix spikes and
duplicates were being evaluated. Additionally, the laboratory control spike (LCS) procedures
changed such that a separate LCS and a QC check sample were now being performed.
Originally, in the 1/20/10 version, the LCS was prepared by spiking the QC check sample, which
was a certified reference material (CRM) (as stated in Section 6.11.2 of the 1/20/10 SOP)
containing a known quantity of lead. This practice was changed because there were recovery
issues. The spike concentration of 1000 micrograms per liter (ng/L) was not >3x the background
lead concentration because of the high lead concentrations in the actual CRM samples (4630
milligrams per kilogram [mg/kg]). Thus, as of the 2/24/10 SOP, one LCS, one QC check
sample, and one QC check sample duplicate were being evaluated for every 20 samples. The
LCS (Blank Paint QC) sample in the 2/24/10 SOP was defined as a piece of non-lead containing
paint that was spiked with lead solution to a resulting concentration of 1000 [ig/L. The QC check
sample in the 2/24/10 SOP contained 10 mg of the CRM, a known lead-containing material. The
QC check (CRM) was purchased to contain 4630 ฑ 266 mg/kg lead. To prepare the sample, 10
mg of the CRM was weighed out and diluted to 10 mL, resulting in a final concentration of 4.630
mg/L.
The 4/25/10 revision of the SOP clarified the acceptance criteria for the LCS samples, as it did
not appear to be clearly defined in previous versions.
Because of the high lead concentration in the PEM samples, dilutions were made to the samples
prior to initial analysis. The dilutions were prepared by spiking 10 microliters (|jL) of the
original digested sample into 9.990 milliliters (mL) of reagent water for a 1:1000 dilution. The
samples were thoroughly mixed by inverting, and then analyzed for lead content. If the result
was below the reporting limit, the sample was reanalyzed either non-diluted or at a lower dilution
level. If samples were rerun at a different dilution level, this was noted in the QC summary
report for that particular sample set.
The MDL for lead was 2.91 |ig/L.
The reporting limit was 40 |ig/L. Therefore all blank results should be <40 |ig/L.
A-23
-------
Summary of Quality Control Measures for PEMs ICP-AES Analysis
QC procedures were performed in accordance with the quality management plan (QMP) for the
Battelle ETV Advanced Monitoring Systems (AMS) Center, except where differences were
noted for Environmental and Sustainable Technology Evaluations (ESTE) per the EPA ETV
Program QMP, and the test/QA plan for this verification test. Test procedures were conducted as
stated in the test/QA plan; however a deviation to the test/QA plan was made during the ICP-
AES analyses. For some sample runs, continuous calibration verification (CCV) samples were
run once every 20 instead of 10 samples. This deviation is further described below. This change
was assessed to have no impact on the quality of the results as described below. QC results for
the analysis of paint chip samples from the PEMs are described below.
ICP-AES Blank Sample Results
Various blank samples were analyzed for the ICP-AES analyses. Method blank samples were
analyzed in each set of 10-20 paint samples to ensure that no sources of contamination were
present. An initial calibration blank was analyzed at the beginning of each run and used for
initial calibration and zeroing the instrument. A continuing calibration blank was analyzed after
each CCV to verify blank response and freedom from carryover. No blank samples failed QC
during the analyses.
Calibration Verification Standards
Initial calibration standards were run at the beginning of each set of analyses. The acceptance
criterion for the calibration coefficient of the calibration standards was > 0.998. If this criterion
was not met, the analysis was stopped and recalibration was performed before samples were
analyzed. A 500 parts per billion (ppb) CCV standard was analyzed at the beginning of each run
(following the initial calibration), at the end of each run, and every 10-20 samples. CCV
recoveries ranged from 96% to 108%. Per the test/QA plan, CCV sample frequency was once
every 10 samples. For most of the sample sets, CCVs were performed with this frequency.
However, for later sample sets, CCVs were run once every 20 samples. CCV samples are used
to verify instrument performance and are evaluated usually at a specified frequency as a
preventative measure so that large amounts of samples do not need to be re-run if a CCV sample
fails. In the course of this study, only one CCV sample failed, and it was when the CCV was
being run once every 10 samples. All samples from the last passing CCV of that sample set were
re-analyzed.
QC samples also included an initial calibration verification (ICV) standard and interference
check sample (ICS). Both samples were 500 ppb. ICV samples were analyzed once at the
beginning of each sample run and were required to have percent recoveries between 90-110% to
be acceptable. ICS samples were analyzed at the beginning and end of every run and every 10-
20 samples. ICS samples had to have percent recoveries between 80-120% to be acceptable. All
reported ICV and ICS samples met the acceptance criteria. Recoveries for ICV samples ranged
from 96% to 108%. Recoveries for ICS samples ranged from 93% to 112%.
A-24
-------
Matrix Spike Samples/Duplicates
Matrix spike samples, as well as duplicates of these samples, were analyzed once every 10-20
samples. Acceptable recoveries for matrix spike samples were between 80-120%. Duplicate
samples had acceptance criteria of ฑ25% relative percent difference (RPD).
All matrix spike samples were performed as post-digestion spikes as there was insufficient
sample volume to perform a pre-digestion spike. Matrix spike recoveries ranged from 86% to
207%. Six matrix spike samples failed with recoveries above the specified acceptance criteria.
In these instances, the lead concentration in the sample was well above the spike level. Matrix
spike results indicate that matrix interferences were not observed. Duplicate samples were
within the specified RPD.
LCS Samples
LCS samples were analyzed once every 10-20 samples. Acceptable recoveries for LCS samples
were between 80-120%. LCS recoveries ranged from 17% to 225% . Schneider Laboratories,
Inc. noted that LCS failures on one sample set were attributed to improper spiking technique.
Training on spiking procedures was immediately implemented by Schneider Laboratories, Inc.
for all analysts spiking samples. All LCS failures occurred prior to a revision to the Schneider
Laboratories, Inc. SOP for analyzing paint samples for this verification test. In the original
version of the SOP, LCS samples were prepared by spiking a known amount of lead onto a
CRM. This practice was changed on 2/24/10 because there were recovery issues. The spike was
not >3x the background lead concentration because of the high lead concentrations in the actual
CRM samples. In the revised SOP, the LCS was prepared by spiking a piece of lead-free latex
paint. There were no LCS failures after that In addition, a QC check sample containing only
the CRM, which has a known concentration of lead weighed out to a particular amount, was
analyzed with each sample set throughout the verification test. These QC samples all passed
acceptance criteria.
A-25
-------
Appendix B
Vendor Comments
-------
ANDalyze, Inc. submitted the following comments on the draft report. These comments
have not been reviewed by Battelle or U.S. EPA for accuracy, and do not necessarily reflect
the opinions or views of U.S. EPA. Any questions regarding the comments in this section
should be addressed to the vendor.
The ANDalyze test kit is unique because on performing a lead test the instrument screen displays
a quantitative result (lead amount in units of mg/cm2) instead of a positive/ negative response.
We believed that this would be the best option for our product as the user can get an estimate of
the amount of lead in the paint. However, the quantitative results put additional stringency on our
product when false positive and false negative criteria are analyzed by ETV testing.
In the brief description that follows, ANDalyze shows that the false negative criteria can be
met for all substrates (except metal) if a minor change is made to the display of the
instrument
- For this report, ETV considered any result > 1 to be a "positive response" and any result
< 1 to be "negative response". We propose that results > 0.1 be considered positive and
results < 0.1 be considered negative response.
ANDalyze will change only the display on the fluorimeter to indicate "positive" or
"negative" instead of a numerical result. When the fluorimeter calculates lead
concentration to be more than 0.1 mg/cm2, the display will be "positive" and when lead
concentration is calculated to be less than 0.1 mg/cm2 the display will be "negative".
Please note the proposed modification does NOT in any way change the chemical tests
that were performed during the ETV program or the software code which calculates the
lead amount. Therefore tests performed during ETV remain valid.
With this proposed modification, we recalculated the predicted false positive and false
negative rates. The results are pasted below (table 6-2, 6-13 were re analyzed)
Based on the modeled probability of test response, the ANDalyze lead in paint test kits
will meet the "false negative" requirement for all substrates except metal (see Re-
analyzed Table 6-13 in the following page)
Please contact ANDalyze customer service for further comments/ clarifications at
Email: Info@andalyze.com
Toll free in the US: 888.388.0818
Phone: +1 217.328.0045
B-2
-------
Table 6-2 Re-analyzed: ANDalyze Lead-in-Paint Test Kit false positive results for panels
with confirmed lead levels < 0.8 mg/cm2 and false negative results for panels with
confirmed lead levels > 1.2 mg/cm2
Overall
NONE
WHITE
YELLOW
DRYWALL
METAL
PLASTER
WOOD
GREY
RED
WHITE
False Postive Rate
OPERATOR_TYPE
TECHNICAL
111/310=36%
1/70=1%
49/122=40%
61/118=52%
39/78=50%
24/94=26%
17/54=31%
31/84=37%
35/104=34%
34/104=33%
42/102=41%
NON-
TECHNICAL
89/310=29%
0/70=0%
35/122=29%
54/118=46%
27/78=35%
14/94=15%
18/54=33%
30/84=36%
32/104=31%
27/104=26%
30/102=29%
False Negative Rate
OPERATOR_TYPE
TECHNICAL
2/460=0%
NA
1/230=0%
1/230=0%
0/122=0%
1/90=1%
0/138=0%
1/110=1%
0/156=0%
2/140=1%
0/164=0%
NON-
TECHNICAL
3/460=1%
NA
0/230=0%
3/230=1%
1/122=1%
2/90=2%
0/138=0%
0/110=0%
1/156=1%
0/140=0%
2/164=1%
Table 6-13 Re-analyzed: ANDalyze Lead-in-Paint Test Kit modeled probability of positive
test results, lower 95% prediction bound, and corresponding conservative estimate of the
false negative rate when lead level = 1.2 mg/cm2
TOPCOAT
GREY
RED
WHITE
SUBSTRATE
DRYWALL
METAL
PLASTER
WOOD
DRYWALL
METAL
PLASTER
WOOD
DRYWALL
METAL
PLASTER
WOOD
LEADJ.EVEL
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
1.2
PREDICTION
98%
94%
98%
98%
98%
94%
97%
97%
97%
94%
97%
97%
LOWER
96%
91%
96%
96%
96%
91%
95%
96%
96%
90%
95%
95%
false
negative
rate
4%
9%
4%
4%
4%
9%
5%
4%
4%
10%
5%
5%
------- |