United States Office of Research and EPA/600/R-02/057
Environmental Protection Development September 2002
Agency Washington, D.C. 20460
4>EPA Environmental Technology
Verification Report
Lead in Dust Wipe Measurement
Technology
Palintest
Scanning Analyzer SA-5000
System
on\l
Oak Ridge National Laboratory
-------
THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
ET
AM^
V
r/EPA
oml
Oak Ridge National Laboratory
Verification Statement
TECHNOLOGY TYPE:
APPLICATION:
TECHNOLOGY NAME:
COMPANY:
ADDRESS:
WEB SITE:
E-MAIL:
ANODIC STRIPPING VOLTAMMETRY
MEASUREMENT OF LEAD IN DUST WIPES
Scanning Analyzer SA-5000 System
Palintest
21 Kenton Lands Road
Erlanger, KY 41018
www.palintestusa.com
info@palintestusa.com
PHONE: (800)835-9629
FAX: (859) 341-2106
The U.S. Environmental Protection Agency (EPA) has created the Environmental Technology
Verification Program (ETV) to facilitate the deployment of innovative or improved environmental
technologies through performance verification and dissemination of information. The goal of the ETV
Program is to further environmental protection by substantially accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-quality,
peer-reviewed data on technology performance to those involved in the design, distribution, financing,
permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized standards and testing organizations and stakeholder groups
consisting of regulators, buyers, and vendor organizations, with the full participation of individual
technology developers. The program evaluates the performance of innovative technologies by developing
test plans that are responsive to the needs of stakeholders, conducting field or laboratory tests (as
appropriate), collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are
conducted in accordance with rigorous quality assurance protocols to ensure that data of known and
adequate quality are generated and that the results are defensible.
Oak Ridge National Laboratory (ORNL) is one of the verification organizations operating under the
Advanced Monitoring Systems (AMS) Center. AMS, which is administered by EPA's National Exposure
Research Laboratory (NERL), is one of six technology areas under ETV. In this verification test, ORNL
evaluated the performance of lead in dust wipe measurement technologies. This verification statement
provides a summary of the test results for Palintest's Scanning Analyzer SA-5000 system.
EPA-VS-SCM-50
The accompanying notice is an integral part of this verification statement.
September 2002
-------
VERIFICATION TEST DESCRIPTION
This verification test was designed to evaluate technologies that detect and measure lead in dust wipes. The
test was conducted at the Capitol Community Technical College in Hartford, CT, from November 5
through November 9, 2001. The vendors of commercially-available, field portable technologies blindly
analyzed 160 dust wipe samples containing known amounts of lead, ranging in concentration from <2 to
1,500 |_lg/wipe. The experimental design was particularly focused on important clearance standards, such
as those identified in 40 CFR Part 745.227(e)(8)(viii) of 40 |_lg/ft2 for floors, 250 |_lg/ft2 for window sills,
and 400 |-lg/ft2 for window troughs. The samples included wipes newly-prepared and archived from the
Environmental Lead Proficiency Analytical Testing Program (ELPAT). These samples were prepared from
dust collected in households in North Carolina and Wisconsin. Also, newly-prepared samples were
acquired from the University of Cincinnati (UC). The UC dust wipe samples were prepared from National
Institute of Standards and Technology (NIST) Standard Reference Materials (SRMs). The results of the
lead analyses generated by the technology were compared with results from analyses of similar samples by
conventional laboratory methodology, in a laboratory that was recognized as proficient by the National
Lead Laboratory Accreditation Program (NLLAP) for dust testing. Details of the test, including a data
summary and discussion of results, may be found in the report entitled Environmental Technology
Verification Report: Lead in Dust Wipe Detection Technology— Palintest, Scanning Analyzer SA-5000
System, EPA/600/R-02/057.
TECHNOLOGY DESCRIPTION
The Scanning Analyzer SA-5000 system uses the electrochemical technique of stripping analysis to
specifically determine the concentration of lead in a solution. Anodic stripping analysis is a two step
process. The first step is called the deposition step and involves the electro-deposition of lead on to a
disposable mercury-film electrode. The deposition is achieved by cathodic deposition at a fixed potential
and time. Following the fixed deposition time, the system enters the second step, the stripping or
measurement step. The stripping step involves scanning the potential anodically using a potential-time
waveform. During this anodic scan the deposited lead is reoxidized and stripped out of the electrode.
The current and potential are measured during the anodic scan and the resulting voltammogram contains
a peak whose potential is specific to lead and whose height is proportional to the concentration of lead in
the solution. The peak height is converted from a current to a concentration using one of many
calibration curves stored in the instrument. No user calibration is required because each batch of
electrodes is checked during manufacture and assigned an eight figure calibration code. The calibration
code is used to select the calibration curve which matches the electrode batch. Reporting limits during
this verification test were 25 |_lg/wipe.
VERIFICATION OF PERFORMANCE
The following performance characteristics of the SA-5000 were observed:
Precision: Precision, based on the average percent relative standard deviation (RSD), was 5% for the
ELPAT samples and 8% for the UC samples. A technology's performance is considered very precise if
the average RSD is less than 10%, but acceptable as long as the average RSD is less than 20%.
Accuracy: Accuracy was assessed using the estimated concentrations of the ELPAT and UC samples.
The number of results for the ELPAT samples that were reported within the acceptance ranges that have
been established for those samples is one measure of accuracy. The SA-5000 reported results within the
acceptance ranges for all 72 ELPAT samples (> 25 |_lg/wipe). The average percent recovery value (SA-
5000 reported result/estimated ELPAT concentration) for all samples reported above 25 |_lg/wipe was
91%. For the UC samples, the average percent recovery was 80%. This negative bias was statistically
significant, but within the acceptable bias range of 100% ± 25%. For the NLLAP laboratory results, the
EPA-VS-SCM-50 The accompanying notice is an integral part of this verification statement. September 2002
-------
average percent recovery values were 98% and 88%, respectively, for the ELPAT and UC samples. The
negative bias for both the ELPAT and UC samples was statistically significant.
Comparability: A comparison of the SA-5000 results and the NLLAP-recognized laboratory results was
performed for all samples (ELPAT and UC) that were reported above 25 |_lg/wipe. The correlation
coefficient (r) for the comparison of UC samples was 1.000 [slope (m) = 0.839, intercept = 5.539], and for
the ELPAT samples was 0.995 [m = 0.926, intercept = 6.506]. While the slopes for both the ELPAT and
UC samples were statistically different than 1.00, the correlation coefficients show a strong linear
agreement (i.e., r values greater than 0.990) with the NLLAP laboratory data.
Detectable blanks: All twenty samples prepared at concentrations < 2 |_lg/wipe were reported
correctly by the SA-5000 as < 25 |-lg/wipe. Performance was also assessed at concentrations near the
reporting limits of the technology. Two sets of four ELPAT samples with estimated concentrations of 16.9
and 17.6 |-lg/wipe were all reported by the SA-5000 as < 25 |_lg/wipe. For the set of four ELPAT samples at
29.8 |_lg/wipe, the SA-5000 reported results between 28 and 32 |_lg/wipe.
False positive results: A false positive (fp) result is one in which the technology reports a result that is
above the clearance level when the true (or estimated) concentration is actually below. For the ELPAT
samples, the SA-5000 did not produce any fp results out of a possible 12 results. For the UC samples, the
SA-5000 did not produce any out of a possible 38 fp results. By comparison, the NLLAP laboratory did not
report any fp results on the UC, but had 2 of!2 possible fp results on the ELPAT samples.
False negative results: A false negative (fn) result is one in which the technology reports a result that is
below the clearance level when the true (or estimated) concentration is actually above. For the ELPAT
samples, the SA-5000 reported 17 of a possible 28 m results. For the UC samples, the SA-5000 reported 22
out of a possible 22 fn results. By comparison, the NLLAP laboratory reported 7 out of a possible 28 m
results for the ELPAT samples, and 16 out of a possible 19 fn results for the UC samples.
Completeness: Completeness is defined as the percentage of measurements that are judged to be usable
(i.e., the result is not rejected). An acceptable completeness rate is 95% or greater. The SA-5000 generated
results for all 160 dust wipe samples, for a completeness of 100%.
Sample Throughput: Sample throughput is a measure of the number of samples that can be processed
and reported by a technology in a given period of time. A single analyst (a Palintest expert) was able to
prepare and analyze 80 samples per 12-hour day.
EPA-VS-SCM-50 The accompanying notice is an integral part of this verification statement. September 2002
-------
Overall Evaluation: The overall performance was characterized as having an acceptable amount of
negative bias, very precise, and in good agreement with an NLLAP-recognized laboratory's results. The
verification team found that the SA-5000 was relatively simple for the trained analyst to operate in the
field, requiring less than an hour for initial setup. As with any technology selection, the user must
determine if this technology is appropriate for the application and the project data quality objectives.
Additionally, ORNL and ETV remind the reader that, while the ETV test provides valuable information
in the form of a snapshot of performance, state, tribal, or federal requirements regarding the use of the
technologies (such as NLLAP recognition for analysis of clearance samples where required) need to be
followed. For more information on this and other verified technologies, visit the ETV web site at
http: //www. epa. gov/etv.
Gary J. Foley, Ph.D.
Director
National Exposure Research Laboratory
Office of Research and Development
W. Franklin Harris, Ph.D.
Associate Laboratory Director
Biological and Environmental Sciences
Oak Ridge National Laboratory
NOTICE: EPA verifications are based on evaluations of technology performance under specific, predetermined criteria
and appropriate quality assurance procedures. EPA and ORNL make no expressed or implied warranties as to the
performance of the technology and do not certify that a technology will always operate as verified. The end user is solely
responsible for complying with any and all applicable federal, state, and local requirements. Mention of commercial
product names does not imply endorsement or recommendation.
EPA-VS-SCM-50
The accompanying notice is an integral part of this verification statement.
September 2002
-------
EPA/600/R-02/057
September 2002
Environmental Technology
Verification Report
Lead in Dust Wipe Measurement
Technology
Palintest
Scanning Analyzer SA-5000 System
By
Amy B. Dindal
Charles K. Bayne, Ph.D.
Roger A. Jenkins, Ph.D.
Oak Ridge National Laboratory
Oak Ridge, Tennessee 37831-6120
Eric N. Koglin
U.S. Environmental Protection Agency
Environmental Sciences Division
National Exposure Research Laboratory
Las Vegas, Nevada 89193-3478
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and Development (ORD),
funded and managed, through Interagency Agreement No. DW89937854 with Oak Ridge National
Laboratory, the verification effort described herein. This report has been peer and administratively reviewed
and has been approved for publication as an EPA document. Mention of trade names or commercial products
does not constitute endorsement or recommendation for use of a specific product.
-------
Table of Contents
Notice ii
List of Figures v
List of Tables vi
Acknowledgments vii
Abbreviations and Acronyms viii
Section 1 — Introduction 1
Section 2 — Technology Description 2
General Technology Description 2
Analytical Procedure 2
Section 3 — Verification Test Design 3
Objective 3
Testing Location and Conditions 3
Drivers and Objectives for the Test 3
Summary of the Experimental Design 3
University of Cincinnati Sample Description 4
Sample Randomization 5
Precision 6
Accuracy 6
Comparability 7
Detectable Blanks 7
False Positive/Negative Results 7
Completeness 8
Sample Throughput 8
Ease of Use 8
Cost 8
Miscellaneous Factors 8
Section 4 — Laboratory Analyses 9
Background 9
NLLAP Laboratory Selection 9
Laboratory Performance 10
Section 5 — Technology Evaluation 16
Objective and Approach 16
Precision 16
Comparability 17
False Positive/False Negative Results 18
Completeness 21
Sample Throughput 21
Ease of Use 21
Cost Assessment 21
iii
-------
SA-5000 Costs 21
Labor 22
Equipment 22
Laboratory Costs 22
Sample Shipment 22
Labor, Equipment, and Waste Disposal 22
Cost Assessment Summary 23
Miscellaneous Factors 23
Summary of Performance 23
Section 6 — References 25
Appendix 26
IV
-------
List of Figures
1. Palintest's Scanning Analyzer SA-5000 System 2
2. Distribution of sample concentrations 6
3. Plot of DataChem reported values versus estimated values, shown for concentrations < 500 |_lg/wipel 1
4. False negative probabilities for DataChem reporting average concentrations at a target concentration
level of 40 |_lg/wipe 13
5. False negative probabilities for DataChem reporting average concentrations at a target concentration
level of 250 |_lg/wipe 14
6. False negative probabilities for DataChem reporting average concentrations at a target concentration
level of 400 |_lg/wipe 14
7. Plot of the SA-5000 average concentration versus the DataChem average concentration for all samples
(n=21), shown for concentrations < 500 |_lg/wipe 18
8. Comparison of the false negative probabilities for the SA-5000 and DataChem at a target
concentration level of 40 |_lg/wipe 19
9. Comparison of the false negative probabilities for the SA-5000 and DataChem at a target
concentration level of 250 |_lg/wipe 20
10. Comparison of the false negative probabilities for the SA-5000 and DataChem at a target
concentration of 400 |_lg/wipe 20
-------
List of Tables
1. Summary of DataChem Pre-Test Results 10
2. Summary of DataChem percent recovery values by sample source 11
3. Summary of DataChem precision estimates by sample source 11
4. False Positive/False Negative Error Rates for DataChem Measurements 13
5. Summary of the Linear Regression Constants and Recovery Data for DataChem's Measurements
Versus the Estimated Concentrations at the Clearance Levels 15
6. Precision of the SA-5000 Analyzer 16
7. Accuracy of SA-5000 Analyzer 16
8. Linear regression constants for the plots of the SA-5000 versus the estimated values and versus the
DataChem average measurements 17
9. False Positive/False Negative Error Rates for SA-5000 Measurements 19
10. Summary of the Linear Regression and Recovery Data for the SA-5000 Response versus the
Estimated Concentrations 21
11. Estimated analytical costs for lead dust wipe samples 22
12. Performance Summary for the Scanning Analyzer SA-5000 System 24
VI
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the verification test,
analyze the data, and prepare this report. In particular, we recognize: Patricia Lindsey, Capitol Community
Technical College, for providing a site for the verification test; Sandy Roda (University of Cincinnati) and
Laura Hodson (Research Triangle Institute) for coordination of sample preparation; the inorganic analytical
laboratory in EPA/Region 1 (North Chelmsford, MA) for the analysis of quality control samples; and the
expertise of the technical advisory panel, including Kenn White (American Industrial Hygiene Association),
John Schwemberger, Dan Reinhart, Oksana Pozda, and Darlene Watford (EPA/Office of Pollution
Prevention and Toxics), Paul Carroll (EPA/Region 1), Sharon Harper (EPA/Research Triangle Park), Peter
Ashley, Warren Friedman, Gene Pinzer, and Emily Williams (U.S. Department of Housing and Urban
Development); Paul Halfmann and Sharon Cameron (Massachusetts Childhood Lead Poisoning Prevention
Program), Kevin Ashley (National Institute for Occupational Safety and Health); Walt Rossiter and Mary
McKnight (National Institute of Standards & Technology); Bill Gutknecht (Research Triangle Institute), and
Bruce Buxton (Battelle Memorial Institute). The authors would specifically like to thank Kevin Ashley and
John Schwemberger for serving as peer reviewers of this report. The authors also acknowledge the
participation of Palintest, in particular, Steve Birch and David Miller.
For more information on the Lead in Dust Wipe Measurement Technology Verification contact:
Eric N. Koglin Roger A. Jenkins
Project Technical Leader Program Manager
Environmental Protection Agency Oak Ridge National Laboratory
Environmental Sciences Division Chemical Sciences Division
National Exposure Research Laboratory P.O. Box 2008
P.O. Box 93478 Oak Ridge, TN 37831-6120
Las Vegas, Nevada 89193-3478 (865) 574-4871
(702) 798-2332 jenkinsra@ornl.gov
koglin.eric@epa.gov
For more information on Palintest's Scanning Analyzer SA-5000 System, contact:
David Miller
Palintest USA
21 Kenton Lands Road
Erlanger, KY41018
1-800-835-9629
info@palintestusa.com
www.palintestusa.com
-------
Abbreviations and Acronyms
AIHA American Industrial Hygiene Association
AMS Advanced Monitoring Systems Center, ETV
ASTM American Society for Testing and Materials
ASV Anodic Stripping Voltammetry
CDC Centers for Disease Control and Prevention
CFR Code of Federal Regulations
CL Clearance level for lead (40, 250, or 400 |_lg/wipe)
ELPAT Environmental Lead Proficiency Analytical Testing program
EPA U. S. Environmental Protection Agency
ETV Environmental Technology Verification Program
ETVR Environmental Technology Verification Report
fn false negative result
fp false positive result
ICP-AES Inductively coupled plasma-atomic emission spectrometry
NERL National Exposure Research Laboratory, U.S. EPA
NIOSH National Institute for Occupational Safety and Health, CDC
NIST National Institute of Standards & Technology
NLLAP National Lead Laboratory Accreditation Program
OPPT Office of Pollution Prevention and Toxics, U.S. EPA
ORNL Oak Ridge National Laboratory
QA quality assurance
QC quality control
RSD relative standard deviation
RTI Research Triangle Institute
SD standard deviation
SRM Standard Reference Material
UC University of Cincinnati
-------
Section 1 — Introduction
The U.S. Environmental Protection Agency (EPA)
created the Environmental Technology Verification
Program (ETV) to facilitate the deployment of
innovative or improved environmental technologies
through performance verification and dissemination
of information. The goal of the ETV Program is to
further environmental protection by substantially
accelerating the acceptance and use of improved and
cost-effective technologies. ETV seeks to achieve
this goal by providing high-quality, peer-reviewed
data on technology performance to those involved in
the design, distribution, financing, permitting,
purchase, and use of environmental technologies.
ETV works in partnership with recognized standards
and testing organizations and stakeholder groups
consisting of regulators, buyers, and vendor
organizations, with the full participation of
individual technology developers. The program
evaluates the performance of innovative
technologies by developing verification test plans
that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate),
collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in
accordance with rigorous quality assurance (QA)
protocols to ensure that data of known and adequate
quality are generated and that the results are
defensible.
ETV is a voluntary program that seeks to provide
objective performance information to all of the
participants in the environmental marketplace and to
assist them in making informed technology
decisions. ETV does not rank technologies or
compare their performance, label or list technologies
as acceptable or unacceptable, seek to determine
"best available technology," or approve or
disapprove technologies. The program does not
evaluate technologies at the bench or pilot scale and
does not conduct or support research. Rather, it
conducts and reports on testing designed to describe
the performance of technologies under a range of
environmental conditions and matrices.
The program now operates six centers covering a
broad range of environmental areas. ETV began
with a 5-year pilot phase (1995-2000) to test a wide
range of partner and procedural alternatives in
various technology areas, as well as the true market
demand for and response to such a program. In these
centers, EPA utilizes the expertise of partner
"verification organizations" to design efficient
processes for conducting performance tests of
innovative technologies. These expert partners are
both public and private organizations, including
federal laboratories, states, industry consortia, and
private sector entities. Verification organizations
oversee and report verification activities based on
testing and QA protocols developed with input from
all major stakeholder/customer groups associated
with the technology area. The verification described
in this report was administered by the Advanced
Monitoring Systems (AMS) Center, with Oak Ridge
National Laboratory (ORNL) serving as the
verification organization. (To learn more about
ETV, visit ETV's Web site at
http://www.epa.gov/etv.) The AMS Center is
administered by EPA's National Exposure Research
Laboratory (NERL).
The verification of a field analytical technology for
measurement of lead in dust wipe samples is
described in this report. The verification test was
conducted in Hartford, Connecticut, from November
5 through November 9, 2001. The performance of
the Palintest's Scanning Analyzer SA-5000 system
was determined under field conditions. The
technology was evaluated by comparing its results to
estimated concentration values and with results
obtained on similar samples using a recognized
laboratory analytical method. For background
information, reports on performance of this and
other anodic stripping voltammetry systems can be
found in other published reports (EPA, 1996, Ashley
etal.,2001).
-------
Section 2 — Technology Description
In this section, the vendor (with minimal editorial changes by ORNL) provides a description of the
technology and the analytical procedure used during the verification testing activities.
General Technology Description
The Scanning Analyzer SA-5000 system (Figure
1) uses the electrochemical technique of anodic
stripping analysis to specifically determine the
concentration of lead in
a solution. Stripping
analysis is a two step
process. The first step
is called the deposition
step and involves the
electro-deposition of
lead on to a disposable
mercury-film electrode.
The deposition is
achieved by cathodic
deposition at a fixed
potential and for a fixed FiS"re L palintest's
, ., r-. Scanning Analyzer SA-
length of time. _... „ ° J
IT iT *u f j 5000 System.
Following the faxed
deposition time, the system enters the second step,
the stripping or measurement step. The stripping
step involves scanning the potential anodically
using a potential-time waveform. During this
anodic scan, the deposited lead is reoxidized and
stripped off of the electrode. The current and
potential are measured during the anodic scan and
the resulting voltammogram contains a peak
whose potential is specific to lead and whose
height is proportional to the concentration of lead
in the solution. The peak height is converted from
a current to a concentration using one of many
calibration curves stored in the instrument. No
user calibration is required because each batch of
electrodes is checked during manufacturing and
assigned an eight figure calibration code. The
calibration code is used to select the calibration
curve which matches the electrode batch.
Analytical Procedure
The following is the procedure that was followed
by Palintest during the verification test.
1. Place the dust wipe into a 50 mL sonicator
tube. Using apipettor add 15 mL (3x5 mL) of
25% nitric acid to the sonicator tube. Use a new
crushing rod to push the wipe down into the tube
ensuring it is covered by the acid. Continue to
push the wipe into the acid solution until trapped
air bubbles in the wipe have been released.
2. Place the tube in the ultrasonicator. Fill the
ultrasonicator with warm water (45-50 °C) so
that the level of water in the sonicator is at least 1
cm above the level of liquid in the tube.
3. Sonicate the tube for 30 minutes then remove
the tube and place in a rack.
4. Take the same crushing rod as previously used
and repeat the mixing of the wipe in the tube.
Replace the tube in the ultrasonicator and sonicate
for an additional 15 minutes.
5. Remove the cap and carefully add deionized or
distilled water to the 50 mL mark. Using the same
crushing rod as previously used, mix the wipe and
solution to ensure complete distribution of the
extract. Replace the cap and mix well by shaking.
6. Take a 5 mL screw capped test tube and pour a
portion of the solution into the tube filling to the 5
mL mark.
7. Add one SoluPrep SP-B tablet, crush and mix
until completely dissolved.
8. Test the sample with the scanning analyzer.
Switch on the instrument. Select Dust from the
menu. Key in the correct calibration code shown
on the electrode pack. Open the foil strip
containing an electrode and insert into the
connector. Insert the electrode into the sample.
The instrument automatically starts the test and
the result is displayed after 45 seconds.
-------
Section 3 — Verification Test Design
Objective
The purpose of this section is to describe the
verification test design. It is a summary of the test
plan (ORNL, 2001).
Testing Location and Conditions
The verification of field analytical technologies for
lead in dust wipes was conducted at the Capitol
Community Technical College in Hartford,
Connecticut. The test was conducted in the
basement of a classroom building. The temperature
and relative humidity were monitored during field
testing, but remained fairly constant. The average
temperature and relative humidity over the four days
of testing were 68 °F and 32%, respectively.
Drivers and Objectives for the Test
The purpose of this test was to evaluate the
performance of field analytical technologies that are
capable of analyzing dust wipe samples for lead
contamination. This test provides information on the
potential applicability of field technologies to EPA
standards for dust clearance testing. The
experimental design was designed around the three
clearance standards of 40 |-lg/ft2 for floors, 250
|~lg/ft2 for window sills, and 400 |-lg/ft2 for window
troughs that are outlined in 40 CFR Part
745.227(e)(8)(viii) (CFR, 2001).
The primary objectives of this verification were to
evaluate the field analytical technologies in the
following areas: (1) how well each performs relative
to a conventional, fixed-site analytical method for
the analysis of dust wipe samples for lead; (2) how
well each performs relative to results generated in
previously rounds of ELPAT testing (described in
the next section), and (3) the logistical and
economic resources necessary to operate the
technology. Secondary objectives for this
verification were to evaluate the field analytical
technology in terms of its reliability, ruggedness,
cost, range of usefulness, sample throughput, data
quality, and ease of use. Note that this verification
test does not provide an assessment of the selection
of locations for dust samples in a facility or an
assessment of the way that dust samples are
collected. The planning for this verification test
follows the guidelines established in the data quality
objectives process.
Summary of the Experimental
Design
All of the samples analyzed in this verification test
were prepared gravimetrically. At the time of the
test, both of the wipes utilized in the test
(PaceWipe™ and Palintest Dust Wipe™) were on
the list of wipes recommended for lead testing by
the American Society for Testing and Materials
(ASTM, 1996). Initial consideration was given to
conducting the test in a real-world situation, where
the technologies would have been deployed in a
housing unit that had been evacuated due to high
levels of lead contamination. In addition to the
safety concern of potentially subjecting participants
to lead exposure, the spatial variability of adjacent
samples would have been expected to be so great
that it would be much larger than the anticipated
variability of these types of technologies, thereby
making it difficult to separate instrument/method
variability and sampling variability. The availability
of we 11-characterized samples derived from "real-
world" environments made the use of proficiency
testing samples (so-called "ELPAT" samples) and
other prepared samples an attractive alternative.
ELPAT and Blank Sample
Description
In 1992, the American Industrial Hygiene
Association (AIHA) established the Environmental
Lead Proficiency Analytical Testing (ELPAT)
program. The ELPAT Program is a cooperative
effort of the American Industrial Hygiene
Association (AIHA), and researchers at the Centers
for Disease Control and Prevention (CDC), National
Institute for Occupational Safety and Health
(NIOSH), and the EPA Office of Pollution
Prevention and Toxics (OPPT). The ELPAT
program is designed to assist laboratories in
improving their analytical performance, and
therefore, does not specify use of a particular
analytical method. Participating laboratories are
sent samples to analyze on a quarterly basis. The
reported values must fall within a range of
acceptable values in order for the laboratory to be
deemed proficient for that quarter.
Research Triangle Institute (RTI) in Research
Triangle Park, NC, is contracted to prepare and
distribute the lead-containing paint, soil, and dust
-------
wipe ELPAT samples. For the rounds of testing
which have occurred since 1992, archived samples
are available for purchase. Some of these samples
were used in this verification test. Because the
samples have already been tested by over one-
hundred laboratories, a certified concentration value
is supplied with each sample. This certified value
represents a pooled measurement of all of the results
submitted, with outliers excluded from the
calculation.
The following description, taken from an internal
RTI report, briefly outlines how the samples were
prepared. RTI developed a repository of real-world
housedust, collected from multiple homes in the
Raleigh/Durham/Chapel Hill area, as well as from
an intervention project in Wisconsin. After
collection, the dust was sterilized by gamma
irradiation, and sieved to 150 |_lm. A PaceWipe™
was prepared for receiving the dust by opening the
foil pouch, removing the wet folded wipe and
squeezing the excess moisture out by hand over a
trash can. The wipe was then unfolded and briefly
set on a Kimwipe™ to soak up excess moisture.
The PaceWipe was then transferred to a flat plastic
board to await the dust. After weighing a 0.1000 ±
0.0005 g portion of dust on weighing paper, the pre-
weighed dust was gently tapped out onto the
PaceWipe. The wipe was then folded and placed in
a plastic vial, which was then capped. All vials
containing the spiked wipes were stored in a cold
room as a secondary means of retarding mold
growth until shipment.
Before use in the ELPAT program, RTI performed a
series of analyses to confirm that the samples were
prepared within the quality guidelines established
for the program. The data quality requirements for
the ELPAT samples were: 1) the relative standard
deviation of the samples analyzed by RTI must be
10% or less; 2) the measured concentrations must be
within 20% of the target value that RTI was
intending to prepare; and 3) analysis by an
accredited laboratory must yield results within ±
20% of the RTI result. Ten samples were analyzed
by RTI and nine samples were sent to the Wisconsin
Occupational Health Laboratory for independent,
confirmatory analysis. All ELPAT samples used in
this test met the data quality requirements described
above. The estimated concentration for an ELPAT
sample used in this evaluation was the certified
("consensus") value (i.e., an analytically derived
result).
RTI prepared the blank samples using the same
preparation method as the ELPAT samples, but the
concentration of lead was less than 2 |_lg/wipe, well
below the expected reporting limits of the
participant technologies.
University of Cincinnati Sample
Description
The ELPAT samples consisted of dust mounded in
the center of a PaceWipe. The University of
Cincinnati (UC) prepared "field QC samples" where
the dust was sprinkled over the wipe, more similar
to how a wipe would look when a dust wipe sample
is collected in the field. The sample was prepared by
weighing, so the concentrations can be estimated. In
a typical scenario, UC sends these control samples
to a laboratory along with actual field-collected
samples as a quality check of the laboratory
operations. Because the samples are visually
indistinguishable from an actual field sample, are
prepared on the same wipe, and are shipped in the
same packaging, the laboratory blindly analyzes the
control samples. This provides the user with an
independent assessment of the quality of the
laboratory's data.
A cluster of twenty UC samples prepared at the key
clearance levels were added to the experimental
design, primarily so that an abundance of data
would exist near the clearance levels, in order to
assess false positive and false negative error rates.
For Palintest, the UC samples were prepared on
their own Palintest DustWipes™. The UC wipe
samples were prepared using National Institute of
Standards & Technology (NIST) Standard
Reference Materials (SRMs). NIST SRM 2711 was
used to prepare the 40 |_lg/wipe samples, and NIST
SRM 2710 was used to prepare the 250 and 400
|-lg/wipe samples. Both SRM 2711 and SRM 2710
are Montana Soil containing trace concentrations of
multiple elements, including lead. Some NIST SRM
materials that are spiked on dust wipes are known to
have low extraction recoveries when prepared by
standard analytical methods (e.g., lead silicates
cannot be extracted unless hydrofluoric acid is used)
(Ashley et al., 1998). These particular SRMs are not
known to contain lead silicates or to give lower lead
recoveries. However, it is important to note the
-------
possibility of such when using NIST SRMs for lead
dust wipe analysis, since similar SRMs (e.g.,
Buffalo river sediment from Wyoming) do show
recoveries in the low 90% range (Ashley et al.,
1998).
Because accurate and precise estimated
concentrations for the UC samples were imperative,
ORNL imposed the following data quality
requirements for the UC-prepared wipe samples: 1)
each estimated concentration had to be within a ±
10% interval of the target clearance level; 2)
additional quality control (QC) samples (at least 5%
of the total samples ordered) were to be prepared
and analyzed by UC as a quality check prior to
shipment of the samples; and 3) the relative standard
deviation of the QC samples had to be < 10%. It is
important to note here the reason why the data
quality requirements between the UC and ELPAT
samples were different. The data quality
requirements for the ELPAT samples (i.e., ± 20% of
the target value) were established by the ELPAT
program. Since archived samples were being used,
those data quality requirements could not be
changed.
As a quality check of the sample preparation
process, UC prepared an additional nine samples
(5% of the total number ordered). UC extracted and
analyzed the samples following internal procedures
(nitric/hydrochloric acid extraction, followed by
atomic absorption spectrometry - see EPA, 1996 for
Methods 3 05 OB and 601 OB) and provided those
results to ORNL. For the nine samples (three at each
of the three clearance levels), the average percent
recovery (i.e., UC measured concentration/UC
estimated concentration x 100%) was 97% (median
value = 96%, standard deviation = 3%, range = 94%
to 103%). Additionally, 18 randomly-selected
samples (six at each of the three clearance levels)
were analyzed by the EPA Region 1 laboratory in
North Chelmsford, MA, as an independent quality
control check of the accuracy and precision of UC's
sample preparation procedure (nitric acid digestion
followed by ICP/AES analysis - EPA 1996). The
average percent recovery (EPA Region 1 reported
concentration/UC estimated concentration x 100%)
was 91% (median 90%, standard deviation = 3%),
with a range of values from 86% to 97%. The
average recovery determined from the EPA Region
1 analyses (91%) was lower than that which was
determined by UC (97%), but both values were
within the data quality requirement of 100 ± 10%.
Based on these data, ORNL determined that the UC
sample preparation process met the established data
quality criteria and was deemed acceptable for use
in the determination of false positive/false negative
error rates.
Distribution and Number of Samples
A total of 160 samples were analyzed in the
verification test. Figure 2 is a plot containing the
distribution of the sample concentrations that were
analyzed in this study. Twenty samples were
prepared by the University of Cincinnati at +/- 10%
of each of the three clearance levels (3 test levels x
20 samples = 60 samples total). Research Triangle
Institute prepared 20 "blanks" at lead concentrations
< 2 |_lg/wipe. These samples are noted as such in
Figure 2. The remaining samples in Figure 2 are
ELPAT samples. For most of the ELPAT samples,
four samples were analyzed at each concentration
level (16 test levels x 4 samples each = 64 samples
total). There were two concentration levels (at 49
and 565 |_lg/wipe) where eight samples were
analyzed. While the set of samples at each
concentration level were prepared using
homogeneous source materials and an identical
preparation procedure, ELPAT samples cannot be
considered true "replicates" because each sample
was prepared individually. However, these samples
represent four samples prepared similarly at a
specified target concentration, with an estimated
value calculated from more than 100 analyses of
similarly prepared samples.
Sample Randomization
The samples were packaged in 20-mL plastic
scintillation vials and labeled with a sample
identifier. Each participant received the same suite
of samples, but in a randomized order. The samples
were distributed in batches of 16. Completion of
chain-of-custody forms documented sample transfer.
-------
25 n
Clearance Levels at 40, 250, 400 ug/wipe
20
—
"E.
S
cs
^
St.
o
L.
£
I
15 —
10 —
5--
'in
Jk
c
_ro
1-
00
0
ro
O
j
<3-
^
-
CD
CD
in
00
00
i-
T —
CM
CO
s
•\l
CD
ro
M
O
)
in
.n
•\i
CO
•N
p
r? CD oo in in ro
<-> 0 T- CD 0 00
^ ^f ^f in oo ^3-
0
m
CM
v
o
in
CM
o
o
Estimated Concentrations of Test Samples (ug/wipe)
Figure 2. Distribution of sample concentrations.
Description of Performance Factors
In Section 5, technology performance is described in
terms of precision, accuracy, completeness, and
comparability, which are indicators of data quality
(EPA, 1996). False positive and negative results,
sample throughput, and ease of use are also
described. Each of these performance characteristics
is defined in this section.
Precision
Precision is the reproducibility of measurements
under a given set of conditions. Standard deviations
estimated at each concentration level can be used to
establish the relationship between the uncertainty
and the average lead concentration. Standard
deviation (SD) and relative standard deviation
(RSD) for "replicate" results are used to assess
precision, using the following equation:
RSD = (SD/average concentration) * 100% .
(Eq. 1)
The overall RSD is characterized by two summary
values:
mean — i.e., average;
range — i.e., the highest and lowest RSD values
that were reported.
The average RSD may not be the best representation
of precision, but it is reported for convenient
reference. An average RSD value less than 10%
indicates that the measurements are very precise.
RSDs greater than 20% should be viewed as
indicators of larger variability and possibly non-
normal distributions. The uncertainty in the
analytical measurements will include influences
from both the preparation (i.e., extraction) and
measurement steps.
Accuracy
Accuracy is a measure of how close the measured
lead concentrations are to estimated values of the
true concentration. The estimated values for the
ELPAT samples are the certificate values that are
reported on the certificate of analysis sheet provided
with the samples. The ELPAT certified values
represent an average concentration determined by
more than 100 accredited laboratories that
participated in previous rounds of ELPAT testing.
The UC estimated value is the concentration
reported by UC for individual samples, calculated
by the amount of NIST-traceable material loaded on
the dust wipes. The accuracy and precision of the
UC value was assessed by an independent
laboratory analyzing randomly selected QC samples.
An EPA laboratory in Region 1 analyzed 10% of the
total number of samples prepared by UC at each of
-------
the three concentration levels and confirmed that the
process used to prepare the samples met the pre-
determined data quality objective of accuracy within
a ± 10% interval of the estimated value.
Accuracy of the field technology measurements was
statistically tested using t-tests or non-parametric
tests at the 5% significance level. These statistical
tests compared the average results with the overall
estimated values using the precision of the sample
measurements. Bias was quantified by computing
the percent recovery for four similar samples or a
single sample using the equation:
percent recovery = [measured amount(s)/estimated
value] x 100% (Eq. 2)
Accuracy was assessed using both the ELPAT and
UC estimated concentrations. The comparison to the
ELPAT value represents how close the technology
reported results to the consensus value, which
represents the amount of "recoverable" lead in the
sample. Because the UC samples were prepared
gravimetrically from samples of known lead content,
the comparison to the UC samples represents how
close the technology reported results to an absolute
lead value. Comparison to the gravimetric values
reveals any bias imposed by the tested sampling and
analytical method.
The optimum percent recovery value is 100%.
Percent recovery values greater than 125% indicate
results that are biased high, and values less than
75% indicate results that are biased low. A small but
statistically significant bias may be detectable for a
field technology if precision is high (i.e., low
standard deviation). The field technology can still
have acceptable bias with an average percent
recovery in the interval of 75% to 125%. Bias
within the acceptable range can usually be corrected
to 100% by modification of calibration methods.
Comparability
Comparability refers to how well the field tech-
nology and conventional laboratory data agree. The
difference between accuracy and comparability is
that accuracy is judged relative to a known value,
comparability is judged relative to the results of a
laboratory procedure, which may or may not report
the results accurately. Because true "replicates"
were not available for use in this study, the averages
from similar samples measured by the technology
were compared with corresponding averages
measured by the laboratory for all target
concentration levels. A correlation coefficient
quantifies the linear relationship between two
measurements (Draper and Smith, 1981). The
correlation coefficient is denoted by the letter r; its
value ranges from -1 to +1, where 0 indicates the
absence of any linear relationship. The value r = -1
indicates a perfect negative linear relation (one
measurement decreases as the second measurement
increases); the value r = +1 indicates a perfect
positive linear relation (one measurement increases
as the second measurement increases). Acceptable r
values are 0.990 or greater. The slope of the linear
regression line, denoted by the letter m, is related to
r. Whereas r represents the linear association
between the vendor and laboratory concentrations,
m quantifies the amount of change in the vendor's
measurements relative to the laboratory's
measurements. A value of+1 for the slope indicates
perfect agreement. Values greater than 1 indicate
that the vendor results are generally higher than the
laboratory, while values less than 1 indicate that the
vendor results are usually lower than the laboratory.
Detectable Blanks
Twenty samples in the test were prepared at < 2
|J,g/wipe, below the anticipated reporting limits of
both the field technologies and the laboratory. Any
reported lead for these samples is considered a
"detectable blank". Performance was also assessed
at concentrations near the reporting limits of the
technology.
False Positive/Negative Results
A false positive (fp) result is one in which the
technology detects lead in the sample above a
clearance level when the sample actually contains
lead below the clearance level (Keith et al., 1996). A
false negative (fh) result is one in which the
technology indicates that lead concentrations are
less than the clearance level when the sample
actually contains lead above the clearance level. For
example, if the technology reports the sample
concentration to be 35 |_lg/wipe, and the true
concentration of the sample is 45 |_lg/wipe, the
technology's result would be considered a fh at the
40 |_lg/wipe clearance level. Accordingly, if the
technology reports the result as 45 |_lg/wipe and the
true concentration is 35 |_lg/wipe, the technology's
result would be a fp at the 40 |_lg/wipe clearance
level.
-------
A primary objective for this verification test was to
assess the performance of the technology at each of
the three clearance levels of 40, 250, and 400
|J,g/wipe, and estimate the probability of the field
technology reporting a fp or fn result. For each
clearance level, the probabilities of fn were
estimated as curves that depend on a range of
concentrations reported about the clearance level.
These error probability curves were calculated from
the results on the 60 UC samples at concentrations ±
10% of each clearance level. In order to generate
probability curves to model the likelihood of false
negative results, it was assumed that the estimated
concentration provided by UC was the true
concentration. However, this evaluation did not
include the gravimetric preparation uncertainty in
the UC estimated concentration. This error is likely
to be much smaller than other sources of
measurement error (e.g., extraction efficiency and
analytical).
The fp/fn evaluation also included a comparison to
the ELPAT sample results. The "estimated" value
for the UC and ELPAT samples are defined
differently. The UC value is based on weight of the
NIST-traceable material, while the ELPAT
estimated value is the average analytical reported
value from more than 100 accredited laboratories.
The UC sample estimated lead content is determined
gravimetrically, which should be closer to the "true"
concentration than an analytical measurement that
includes preparation and instrumental errors. In
contrast, determining the technology's fp/fn error
rates relative to the ELPAT estimated
concentrations represents a comparison to typical
laboratory values. One limitation of using the
ELPAT sample is that concentrations covered a
wider overall distribution of lead levels. Thus, the
availability of sample concentrations that were
tightly (i.e., +/- 10%) clustered about the clearance
levels was limited. In order to perform a broader
fp/fn analysis, the range of lead levels in the ELPAT
samples that bracketed the pertinent clearance levels
was extended to ± 25% of the target concentration.
Completeness
Completeness is defined as the percentage of
measurements that are judged to be usable (i.e., the
result is not rejected). An acceptable completeness
rate is 95% or greater.
Sample Throughput
Sample throughput is a measure of the number of
samples that can be processed and reported by a
technology in a given period of time. Sample
throughput is reported in Section 5 as number of
samples per day per number of analysts.
Ease of Use
A significant decision factor in purchasing an
instrument is how easy the technology is to use.
Several factors are evaluated and reported on in
Section 5:
• What is the required operator skill level (e.g.,
technician or advanced degree)?
• How many operators were used during the test?
• Could the technology be run by a single person?
• How much training would be required in order
to run this technology?
• How much subjective decision-making is
required?
Cost
An important factor in the consideration of whether
to purchase a technology is cost. Costs involved
with operating the technology and a typical
laboratory analyses are estimated in Section 5. To
account for the variability in cost data and
assumptions, the economic analysis is presented as a
list of cost elements and a range of costs for sample
analysis. Several factors affect the cost of analysis.
Where possible, these factors are addressed so that
decision makers can independently complete a site-
specific economic analysis to suit their needs.
Miscellaneous Factors
Any other information that might be useful to a
person who is considering purchasing the
technology is documented in Section 5 under
"Observations". Examples of information that might
be useful to a prospective purchaser are the amount
of hazardous waste generated during the analyses,
the ruggedness of the technology, the amount of
electrical or battery power necessary to operate the
technology, and aspects of the technology or method
that make it user-friendly or user-unfriendly.
-------
Section 4 — Laboratory Analyses
Background
EPA regulations (40 CFRPart 745.227(e)(8)(vii))
specify that residences and child occupied facilities
built before 1978 that have undergone an abatement
must pass clearance testing (CFR 2001). These EPA
regulations also state in 40 CFR Part 745.227(f)(2)
that dust samples for clearance must be analyzed by
a laboratory recognized by EPA (CFR 2001). Many
EPA-authorized state and tribal lead programs have
the same or similar requirements. EPA's vehicle for
recognizing laboratory proficiency is the National
Lead Laboratory Accreditation Program (NLLAP).
Although the NLLAP was initially designed to
accredit fixed site laboratories, in August 1996 the
NLLAP was modified so that mobile laboratory
facilities and testing firms operating portable testing
technologies could also apply for accreditation.
Despite this modification, the NLLAP list of
accredited laboratories has almost exclusively
consisted of fixed site laboratories. One possible
outcome of this ETV test is that more mobile
laboratory facilities and testing firms operating
portable testing technologies will apply for NLLAP
accreditation. In order to assess whether the field
portable technologies participating in this
verification test produce results that are comparable
to NLLAP-recognized data, an NLLAP-recognized
laboratory was selected to analyze samples
concurrently with the field testing.
NLLAP Laboratory Selection
NLLAP was established by the EPA Office of
Pollution Prevention and Toxics under the
legislative directive of Title X, the Lead-Based Paint
Hazard Reduction Act of 1992. In order for
laboratories to be recognized under the NLLAP,
they must successfully participate in the ELPAT
Program and undergo a systems audit. The
acceptable range for the ELPAT test samples is
based upon the reported values from participating
laboratories. Acceptable results are within three
standard deviations from the consensus value. A
laboratory's performance is rated as proficient if
either of the following criteria are met: (1) in the last
two rounds, all samples are analyzed and the results
are 100% acceptable; or (2) three-fourths (75%) or
more of the accumulated results over four rounds are
acceptable.
The NLLAP required systems audit must include an
on-site evaluation by a private or public laboratory
accreditation organization recognized by NLLAP.
Some of the areas evaluated in the systems audit
include laboratory personnel qualifications and
training, analytical instrumentation, analytical
methods, quality assurance procedures, and record
keeping procedures.
The list of recognized laboratories is updated
monthly. ORNL obtained the list of accredited
laboratories in July 2001. The list consisted of
approximately 130 laboratories. Those laboratories
which did not accept commercial samples and those
located on the U.S. west coast were automatically
eliminated as potential candidates. ORNL
interviewed at random approximately ten
laboratories and solicited information regarding
cost, typical turnaround time, and data packaging.
Based on these interviews and discussions with
technical panel members who had personal
experience with the potential laboratories, ORNL
selected DataChem (Cincinnati, OH) as the fixed-
site laboratory. As a final qualifying step, DataChem
blindly analyzed 16 samples (8 ELPAT and 8
prepared by UC) in a pre-test study. As shown in
Table 1 below, DataChem passed the pre-test by
reporting concentrations that were within 25% of
the estimated concentration for samples above the
reporting limit.
Laboratory Method
The laboratory method used by DataChem was hot
plate/nitric acid digestion, followed by inductively
coupled plasma-atomic emission spectrometry (ICP-
AES) analysis. The preparation and analytical
procedures, as supplied by DataChem, can be found
in the test plan (ORNL, 2001). To summarize the
procedure, the wipe was digested in 2 mL of nitric
acid, heated in a hotblock for 1 hour at 95 °C,
diluted to 20 mL with distilled water, and analyzed
by ICP-AES. DataChem's procedures are
modifications of Methods 3 05 OB and 601 OB of EPA
SW-846 Method Compendium for the preparation
and analysis of metals in environmental matrices
(EPA, 1996). Other specific references for the
preparation and analysis of dust wipes are available
from the American Society for Testing and
Materials (ASTM, 1998).
-------
Table 1. Summary of DataChem Pre-Test Results
Sample
Type
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
UC
UC
UC
UC
UC
UC
UC
UC
DataChem
Reported Cone
(|_lg/wipe)
<20
<20
41
44
190
210
440
450
<20
<20
25
38
150
200
250
310
Estimated
Cone
(|_lg/wipe)
2.12
2.12
41.3
41.3
201.6
201.6
408.7
408.7
10.3
5.9
29.9
44
172.4
237.5
327.3
379
Percent
Recovery
n/a
n/a
99%
107%
94%
104%
108%
110%
n/a
n/a
84%
86%
87%
84%
76%
82%
Analysis
Order
16
12
6
3
15
9
2
13
4
1
14
10
11
7
5
8
Laboratory Performance
ORNL validated all of the laboratory data according
to the procedure described in the verification test
plan (ORNL, 2001). During the validation, the
following aspects of the data were reviewed:
completeness of the data package, correctness of the
data, correlation between "replicate" sample results,
and evaluation of QC sample results. Each of these
categories is described in detail in the verification
test plan. An evaluation of the performance of the
laboratory results through statistical analysis of the
data was performed and is summarized below. (See
Section 3 for a detailed description of how the
performance factors are defined and the calculations
that are involved.)
In Table 2, DataChem's reported values are
compared to the estimated values to determine
percent recovery (i.e., accuracy of the DataChem
results) for both the ELPAT and the UC samples.
The results are also shown graphically in Figure 3.
The average percent recovery for the ELPAT
samples was 98%, while the average for the UC
samples was 88%. Both Table 2 and Figure 3
indicate that the analytical results from the
University of Cincinnati wipe samples were
generally reported lower than the estimated value,
while the results for the ELPAT samples were closer
to the estimated value. The better agreement with
the ELPAT samples is not unexpected, given that
the ELPAT estimated concentrations represent
analytical consensus values that include typical
extraction inefficiencies and instrumental error.
The negative bias observed with the UC and the
ELPAT samples was statistically significant. The
cause of the negative bias for the UC samples could
be related to: 1) extraction inefficiencies (due to the
use of NIST SRMs that contain lead that is
unrecoverable with the extraction procedure which
was used) and/or, 2) typical analytical variation due
to preparation and measurement errors. Another
indication of accuracy is the number of individual
ELPAT results which were reported within the
acceptance ranges that have been established for
those samples. For the 72 ELPAT samples (> 20
|-lg/wipe), DataChem reported 71 (99%) within the
acceptable ranges of values.
The precision assessment presented in Table 3
indicates that the analyses were very precise. The
average RSD for the ELPAT samples was 7%, while
the average RSD for the UC samples was 8%. The
variability of the UC sample preparation process,
provided for reference of the minimal achievable
RSD for the UC samples, was 6%. A single
estimate of the ELPAT variability was not
determined, since the ELPAT samples were
comprised of 20 different batches of samples.
DataChem reported all 20 detectable blank samples
correctly as < 20 |-lg/wipe. In addition, DataChem
reported seven of the eight samples with estimated
concentrations of either 16.9 |-lg/wipe or 17.6
|-lg/wipe as less than their reporting limit of 20
|-lg/wipe and only one was incorrectly reported as 30
|-lg/wipe.
10
-------
Tabl
e 2. Summary of DataChem percent recovery values by sam
Statistic
n a
average % recovery
std dev
minimum % recovery
maximum % recovery
ELPAT
72
98
9
81
143
uc
60
88
5
74
96
1 excludes estimated values <20 |_lg/wipe(n=28)
le source
500 -i
400 -
300 -
g 200 H
4*
§ 100 H
"8
0 n
D ELPAT (n=72)
• UnivofCinci(n=60)
100 200 300 400
Estimated value (ug/wipe)
500
Figure 3. Plot of DataChem reported values versus estimated values, shown for concentrations < 500 |_lg/wipe.
Table 3. Summary of DataChem precision estimates by sample source
Sample Source
ELPAT
UC
UC preparation a
Number of
sample sets
18 a
3b
3C
average RSD
7
8
6
Min RSD
2
6
5
Max RSD
21
9
7
" 4 replicates in each sample set
b 20 replicates in each sample set
c The value represents the variability in the sample preparation process.
11
-------
An important evaluation parameter for the analysis
of dust wipe samples is how the method performs at
the clearance levels and the method's likelihood of
reporting false positive (fp) and false negative (fn)
results. Recall from the experimental design that 20
UC samples were prepared at ± 10% of each
clearance level of 40, 250, and 400 |_lg/wipe, for a
total of 60 UC samples. The ELPAT samples
covered a wider range of concentrations. There was
a total of 40 ELPAT samples that fell within a ±25%
interval of the target values that could be used for
the fp/fn assessment. The number of false negative
and false positive results reported by DataChem
relative to the UC and ELPAT estimated
concentrations is summarized in Table 4. There are
a specific number of possible fp and fn results. For
example, if the estimated lead level on the wipe is
less than the clearance level (CL), then it is not
possible to produce a false negative result; only a
false positive (i.e., > 40) result is possible. For the
UC samples, in every case where the estimated
concentration was less than the CL, DataChem
reported a result for that was also less than the CL,
indicating no fp results at any of the three CL.
DataChem reported two fp results for the ELPAT
samples out of a possible 12.
When the estimated concentration was above the
clearance level, however, DataChem sometimes
reported results as less than the clearance level.
DataChem reported a higher rate of fn results for the
UC samples than the ELPAT samples (16 of 19 vs 7
of 28 possible fn results, respectively). This finding
is not surprising, since the results reported above
indicated that DataChem's results were negatively
biased, or reported lower than the estimated values
for the UC samples. As stated in Section 3, it is
important to note that in this evaluation, the
estimated concentration of the UC samples is
assumed to be the "true"concentration, and the
uncertainty in gravimetric preparation for the UC
estimated concentration is not considered in the
evaluation.
Figures 4, 5, and 6 show models of the likelihood of
DataChem reporting a false negative result at each
of the clearance levels versus the true concentrations
of the UC samples. (Note that only the UC samples
must be used in generation of probability curves
because these estimated values are a closer
representation of the true lead concentration than the
ELPAT estimated concentration. See Song et al.,
2001.) These figures indicate that the likelihood of
DataChem reporting false negative results for the
UC samples at the exact clearance level is high, near
100% in all three cases. This means, for example,
that if DataChem reported a value as exactly 250
|-lg/wipe, the probability that the true concentration
is >250 is essentially 100%. Again, this is due to
the negative bias that was observed in the
measurement of the UC samples. The plots also
demonstrate that, due to the relatively high level of
precision of results reported by DataChem, the
performance is very minimally impacted by
performing replicate analyses, as the distribution of
false negative probabilities is very similar whether 1
or 5 measurements (in Figures 4, 5, and 6,
delineated as N = 1, N = 2, etc.) are performed. The
interpretation of these curves for use in a "real-
world" situation can be demonstrated by the
following example. Suppose that a user decides that
an acceptable level of risk for having false negative
results is 5%. Using Figure 4, 5% FN probability (y
= 0.05) corresponds to a "true" lead concentration of
54 |_lg/wipe (meaning if the true concentration of the
sample is 54 |_lg/wipe, there is only a 5% chance/risk
that DataChem will report the value as < 40
|-lg/wipe.)
By plotting DataChem's measured values versus the
estimated concentrations, the equations of the linear
regression lines can be calculated for each of the
three CL. The slope, intercept, and correlation
coefficient for the ELPAT and UC samples are
presented in Table 5. The user might like to know at
what reported value (and at what associated
probability) will DataChem be likely to report a
"clean" sample (i.e., there is a high probability that
the true concentration is < CL). For example, for the
UC samples, we know that a value reported by
DataChem as 39 |_lg/wipe is biased low and will
have a true concentration of > 40 (44.7 |_lg/wipe,
using the linear regression equation in Table 5). A
true concentration of 40 |_lg/wipe for a UC sample
would correspond to a reported value rounded to the
nearest whole number of 35 |_lg/wipe (see Table 5).
For an ELPAT sample, a true concentration of 40
|J,g/wipe corresponds to a DataChem reported value
of 40 |_lg/wipe, because the negative bias was not as
large for the ELPAT samples. Estimates of the
reported concentration at the 250 and 400 |_lg/wipe
levels are reported in Table 5. In both cases, the
reported concentrations for the ELPAT samples are
higher (i.e., closer to the clearance level) than those
of the UC samples. (Recall that the ELPAT
estimated values are consensus values of more than
12
-------
100 analytical measurements, while the UC
estimated values are weighed values, so this finding
is not surprising.)
The user is reminded that the data obtained during
this verification test represent performance at one
point in time. The data produced by DataChem at
some other time after the writing of this report may
or may not be similar to what has been produced
here. To understand a method's performance at
critical clearance levels, it is recommended that the
user perform their own assessment of the method's
performance by including samples of known
concentration (at or near the clearance levels) along
with the analysis of "real-world" samples.
Table 4. False Positive/False Negative Error Rates for DataChem Measurements
Evaluation Parameter
fp: # samples where
DataChem reported the
result as >CLa of the #
samples where the
estimated concentration
was < CL
fn: # samples where
DataChem reported the
result as < CL of the #
samples where the
estimated concentration
was > CL
Sample
Source
UC
ELPAT
UC
ELPAT
Number of Samples
40 |_lg/wipe
Oof 15
Oof 4
4 of 5
lof 12
250 |J,g/wipe
Oof 12
2 of 8
6 of 8
5 of 8
400 |J,g/wipe
Oof 14
OofOb
6 of 6
lof 8
Total
Oof 41
2 of 12
16 of 19
7 of 28
a CL = clearance level
b Because all eight ELPAT values were above 400 |_ig/wipe, no samples were available to assess fp results at this level.
True Pb Concentration (ug/wipe)
Figure 4. False negative probabilities for DataChem reporting average
concentrations at a target concentration level of 40 |ig/wipe.
13
-------
0.0
240
250
260
270
280
290
300
310
320
True Pb Concentration (ug/wipe)
Figure 5. False negative probabilities for DataChem reporting average concentrations at a
target concentration level of 250 |ig/wipe.
400
410
420
430
440
450
460
470
480
True Pb Concentration (ug/wipe)
Figure 6. False negative probabilities for DataChem reporting average concentrations
at a target concentration level of 400 |ig/wipe.
14
-------
Table 5. Summary of the Linear Regression Constants and Recovery Data for DataChem's
Measurements Versus the Estimated Concentrations at the Clearance Levels
Evaluation Parameter
n
slope
intercept
correlation coefficient
average % recovery
SD of % recovery
Reported concentration at
theCL
40 |_lg/wipe
UC
20
0.787
3.844
0.705
89%
6%
35
l-ig/wipe
ELPAT
16
1.612
-6.182
0.840
101%
13%
40
l-ig/wipe
250 |J,g/wipe
UC
20
0.912
-10.169
0.759
87%
5%
218
l-ig/wipe
ELPAT
16
0.578
90.826
0.549
96%
9%
234
l-ig/wipe
400 |J,g/wipe
UC
20
0.956
-26.826
0.916
89%
2%
355
l-ig/wipe
ELPAT
8
2.394
-575.771
0.492
100%
5%
382
l-ig/wipe
15
-------
Section 5 — Technology Evaluation
Objective and Approach
The purpose of this section is to present a statistical
evaluation of the SA-5000 data and determine the
technology's ability to measure lead in dust wipe
samples. This section includes an evaluation of
comparability to NLLAP-recognized laboratory
data. Other aspects of the technology (such as
accuracy, precision, cost, sample throughput,
hazardous waste generation, and logistical
operation) are also evaluated in this section. The
Appendix contains the raw data provided by the
vendor during the verification test that were used to
assess the performance of the SA-5000.
Precision
Precision is the reproducibility of measurements
under a given set of conditions. Precision was
determined by examining the results of blind
analyses for replicate samples with estimated
concentrations greater than the SA-5000's reporting
limits (25 |_lg/wipe). For the ELPAT samples,
precision was measured on each set of four samples
from a particular round of ELPAT archives. For the
18 sets of samples, the SA-5000's average RSD
value was 5%, with a range from 2 to 8%, indicating
that the SA-5000 measurements of the ELPAT
samples were very precise (see Table 6). For the UC
samples, 20 samples were analyzed at each of three
target concentration levels of 40, 250, and 400
|J,g/wipe. The average precision of the UC sample
measurements by the SA-5000 was 8% RSD. With
the expectation that UC was to prepare the samples
as close to the target concentrations as possible, the
allowable variability was 10% RSD. The actual
variability of the UC preparation process was an
average of 6% RSD.
Accuracy
Accuracy represents the closeness of the SA-5000's
measured concentrations to the estimated content of
spiked samples. One measure of accuracy is the
number of results for the ELPAT samples that were
reported within the acceptance ranges that have been
established for those samples. The SA-5000
reported the results for all 72 ELPAT samples (> 25
|J,g/wipe) within the acceptance ranges. The results
reported by the SA-5000 can also be compared to
the ELPAT certificate value, i.e., the average
Table 6. Precision of the SA-5000 Analyzer
Source
ELPAT
UC
UC prep c
No. of
sample
sets
18 a
3b
o
6
RSD, %
Average
5
8
6
Min
2
8
5
Max
8
9
7
" 4 replicates in each sample set
b 20 replicates in each sample set
c precision of UC sample preparation process
concentration reported by 100+ laboratories who
participated in previous rounds of ELPAT testing.
The average percent recovery of 91% reported by
the SA-5000 for the 72 ELPAT samples indicates a
statistically significant negative bias, but such is
well within the acceptable bias limits of 100 ± 25%
(Table 7). The recovery values range from 74 to
107%. The results for the UC samples were also
biased low, with an average percent recovery of
80%, and a range of values from 65 to 95%. When
comparing these results to those of the NLLAP
laboratory that was presented in Table 2, a similar
trend is observed, in that the UC sample results
were, on average, biased 10% lower than the
ELPAT results. The possible explanations for this
difference in performance include: 1) that ELPAT
"estimated" values are in fact consensus values from
a large number of laboratories that may be similar in
performance to DataChem and to the SA-5000; and
2) the reference material used to prepare the UC
samples may be more challenging than the ELPAT
reference material.
Table 7. Accuracy of SA-5000 Analyzer
Statistic
na
average % recovery
standard deviation
minimum % recovery
maximum % recovery
% recovery
ELPAT
72
91
8
74
107
UC
60
80
7
65
95
1 Excludes estimated values < 25 |_ig/wipe
16
-------
Another way to assess accuracy is to plot the results
obtained from the SA-5000 versus the estimated
values that are > 25 |_lg/wipe. The linear regression
constants for the plot of the ELPAT and UC data are
listed in Table 8. As expected, the conclusions
produced from this assessment are similar to the
above conclusions regarding the percent recovery
calculations. The UC samples were generally
reported lower than the ELPAT samples relative to
the estimated concentrations, but overall the sample
results had an acceptable amount of bias.
Comparability
Comparability refers to how well the SA-5000 and
the NLLAP-recognized laboratory data agreed. In
this evaluation, the laboratory results are not
presumed to be the "correct" answers. Rather, these
results represent what a typical fixed laboratory
would report for these types of samples. A direct
comparison of the SA-5000 results and the
laboratory results was performed for all ELPAT (>
25 |_lg/wipe) and UC samples. Because each wipe
was prepared individually, a true one-to-one
matching of SA-5000 and laboratory results could
not be performed. However, the average
concentrations of the samples prepared at specific
levels was compared for the SA-5000 and laboratory
results. In Table 8, the regression constants for the
average SA-5000 results versus the average
DataChem results for both the ELPAT and UC
values are presented. The difference between the
regression slopes (m = 0.926 for ELPAT and m =
0.839 for UC) and a slope with a perfect agreement
line (m = 1.000) is statistically significant, but the
correlation coefficients (r = 0.990 for ELPAT and r
= 1.000 for UC) show a strong linear relationship
between DataChem and SA-5000 results. To
illustrate the strong linear agreement between the
SA-5000 and NLLAP laboratory results, Figure 7 is
a plot of the average SA-5000 results versus the
average DataChem results for both ELPAT and UC
data. For clarity, only those values < 500 |_lg/wipe
are shown.
Detectable Blanks
Of the samples that were prepared at < 2 |_lg/wipe,
the SA-5000 correctly reported all 20 as < 25
|J,g/wipe, so no detectable blanks were reported.
Performance was also assessed at concentrations
near the reporting limit of 25 |_lg/wipe. Two sets of
four ELPAT samples with estimated concentrations
of 16.9 and 17.6 |_lg/wipe were all reported by the
SA-5000 as < 25 |_lg/wipe. For the set of four
ELPAT samples at 29.8 |lg/wipe, the SA-5000
reported results between 28 and 32 |_lg/wipe.
Table 8. Linear regression constants for the plots of the SA-5000 versus the estimated values and
versus the DataChem average measurements
Statistic
n
slope
(standard error)
intercept
(standard error)
r
versus estimated values
UC
60
0.754
(0.010)
3.466
(2.773)
0.995
ELPAT
72
0.916
(0.011)
3.757
(5.151)
0.995
versus DataChem average concentrations
UC
3
0.839
(0.001)
5.539
(0.224)
1.000
ELPAT
18
0.926
(0.023)
6.506
(10.860)
0.995
17
-------
500 n
400 -
It 300
e 2°° H
100
0
0 100 200 300 400
DataChem average concentration (ug/wipe)
500
Figure 7. Plot of the SA-5000 average concentration versus the DataChem
average concentration for all samples (n=21), shown for concentrations < 500
|ig/wipe.
False Positive/False Negative Results
Similar to the evaluation described and presented in
Section 4 for DataChem, the number of false
negative and false positive results reported by the
SA-5000 relative to the estimated concentrations of
both ELPAT and UC samples is summarized in
Table 9. For every case where the estimated
concentration was less than the clearance level (CL),
the SA-5000 reported a result that was also less than
the CL, indicating no fp results at any of the three
CL for both ELPAT and UC samples. When the
estimated concentration was greater than the
clearance level, however, the SA-5000 reported
many of the results as less than the clearance level.
The SA-5000 had more false negative errors relative
to the UC estimated concentrations (22 of 22
possible fn results) than when compared to the
ELPAT estimated concentrations (17 of 28 possible
fn results). This finding is not surprising, since the
accuracy results reported above indicated that the
SA-5000 results were more negatively biased for the
UC samples.
In Figures 8, 9, and 10, the false negative
probabilities at the three clearance levels are
compared for the DataChem and SA-5000 results for
the UC samples only (see previous discussion on
false negatives in Section 4). In these figures, the
two-sided 90% confidence intervals (not shown for
clarity) are used to express uncertainty on the false
negative curves. These confidence intervals overlap
for the SA-5000 and DataChem at the 40 |_lg/wipe
and the 250 |-lg/wipe clearance levels over the range
of true lead concentrations shown in the Figures 8
and 9. The overlapping confidence intervals
indicate the two methods are comparable when
considering their uncertainty. In Figure 10, the 90%
confidence intervals for the two methods only
overlap for part of the true lead concentration range
(425 to 472 |-lg/wipe). This result indicates that the
SA-5000 appears to be more prone to false negatives
at the 400 |-lg/wipe clearance level, when
considering the uncertainty of the two methods.
Table 10 contains the linear regression constants for
SA-5000 measured concentration versus estimated
concentration for the three CLs, average percent
recovery values and standard deviations, and an
estimate of the reported SA-5000 concentration at
the clearance levels for both the ELPAT and UC
samples. The UC samples average recoveries
indicate that the SA-5000 results were more
negatively biased for the 250 and 400 |-lg/wipe
levels than for the 40 |-lg/wipe level. This is also
apparent in the estimated concentration that a user
might require from the SA-5000 in order to be
reasonably confident that the true result is below the
clearance level. For the UC samples, both
DataChem and the SA-5000 results reported as 35
|-lg/wipe correspond to a true concentration of 40
|J,g/wipe, but the reported values at the CLs for the
250 and 400 |-lg/wipe clearance levels are lower for
18
-------
the SA-5000 (189 and 308 |_lg/wipe) than for
DataChem (218 and 355 |_lg/wipe). The SA-5000
ELPAT sample results were negatively biased
relative to the estimated concentrations, but not as
low as the UC samples, so the estimates of a
reported concentration at the CL are slightly higher
for the ELPAT samples. Regardless of analytical
technique, there is some uncertainty in assessing
false positive and false negative error rates around
critical action levels due to "normal" levels of
variability (Song et al, 2001). Analytical values
falling near the level of interest should be
interpreted with care for both fixed-laboratory and
field-based analytical methods.
Table 9. False Positive/False Negative Error Rates for SA-5000 Measurements
Evaluation Parameter
fjp: # samples where SA-5000
reported the result as > CLa of the
# samples where the estimated
concentration was < CL
fn: # samples where SA-5000
reported the result as < CL of the
# samples where the estimated
concentration was > CL
Sample
Source
UC
ELPAT
UC
ELPAT
Number of Samples
40 |_lg/wipe
Oof 14
Oof 4
6 of 6
4 of 12
250 |J,g/wipe
Oof 11
Oof 8
9 of 9
8 of 8
400 |J,g/wipe
Oof 13
OofOb
7 of 7
5 of 8
Total
Oof 38
Oof 12
22 of 22
17 of 28
a CL = clearance level
b Because all eight ELPAT values were above 400 |_ig/wipe, no samples were available to assess fp results at this level.
True Pb Concentration (ug/wipe)
Figure 8. Comparison of the false negative probabilities for the SA-
5000 and DataChem at a target concentration level of 40 |ig/wipe.
19
-------
s.
225 250 275 300 325 350 375 400 425 450
True Pb Concentration (ug/wipe)
Figure 9. Comparison of the false negative probabilities for the SA-5000
and DataChem at a target concentration level of 250 |ig/wipe.
True Pb Concentration (ug/wipe)
Figure 10. Comparison of the false negative probabilities for the SA-
5000 and DataChem at a target concentration of 400 |ig/wipe.
20
-------
Table 10. Summary of the Linear Regression and Recovery Data for the SA-5000 Response versus
the Estimated Concentrations
Evaluation Parameter
n
slope
intercept
correlation coefficient
average % recovery
SD of % recovery
Reported concentration
atCL
40 |_lg/wipe
UC
20
0.652
8.880
0.694
88%
5%
35
l-ig/wipe
ELPAT
16
0.559
13.791
0.904
90%
9%
36
l-ig/wipe
250 |J,g/wipe
UC
20
0.764
-1.898
0.616
76%
5%
189
l-ig/wipe
ELPAT
16
0.570
78.537
0.697
90%
7%
221
l-ig/wipe
400 |J,g/wipe
UC
20
0.883
-44.860
0.780
77%
4%
308
l-ig/wipe
ELPAT
8
1.170
-95.766
0.323
94%
4%
372
l-ig/wipe
Completeness
Completeness is defined as the percentage of
measurements that are judged to be usable (i.e., the
result was not rejected). Valid results were obtained
by the technology for all 160 dust wipe samples.
Therefore, completeness was 100%.
Sample Throughput
Sample throughput is representative of the estimated
amount of time required to prepare and analyze the
samples and perform the data analysis. Operating in
the field, one Palintest analyst accomplished a
sample throughput rate of approximately eighty
samples per day. The analyst prepared and analyzed
all 160 dust wipe samples in two 12-hour days,
completing duplicate analysis on each sample and
operating two instruments simultaneously.
Ease of Use
The technology can be operated by a single person.
Users unfamiliar with the technology may need
approximately one-half day of additional training to
operate the instrument. No particular level of
educational training is required for the operator. The
analyst who operated the instrument during the
verification test was a Palintest expert.
Cost Assessment
The purpose of this economic analysis is to estimate
the range of costs for analysis of lead in dust wipe
samples using the SA-5000 and a conventional
analytical laboratory method. The analysis was
based on the results and experience gained from this
verification test, costs provided by Palintest, and
representative costs provided by the NLLAP
analytical laboratory to analyze the samples. To
account for the variability in cost data and
assumptions, the economic analysis is presented as a
list of cost elements and a range of costs for sample
analysis by the SA-5000 instrument and by the
laboratory.
Several factors affected the cost of analysis. Where
possible, these factors were addressed so that
decision makers can complete a site-specific
economic analysis to suit their needs. The following
categories are considered in the estimate:
• sample shipment costs,
• labor costs, and
• equipment costs.
Each of these cost factors is defined and discussed
and serves as the basis for the estimated cost ranges
presented in Table 11. This analysis assumed that
the individuals performing the analyses were fully
trained to operate the technology. Costs for sample
acquisition and pre-analytical sample preparation,
tasks common to both methods, were not included in
this assessment.
SA-5000 Costs
The costs associated with using the instrument
included labor and equipment costs. No sample
shipment charges were associated with the cost of
operating the instrument because the samples were
analyzed on site.
21
-------
Table 11. Estimated analytical costs for lead dust wipe samples
Analysis method:
Analyst/manufacturer:
Sample throughput:
Cost category
Sample shipment
Labor
Rate
Equipment
Instrument purchase price
Reagents/supplies
Waste Disposal
SA-5000
Palintest
80 samples/day
Cost ($)
0
50-100/h per analyst
3850
7 per sample *
250
Analysis method:
NLLAP Laboratory:
Actual turnaround:
Cost category
Sample shipment
Labor
Overnight shipping
Labor
Rate
Equipment
Waste Disposal
EPASW8466010b
DataChem
18 working days
Cost ($)
100-200
50-100
30 per sample
Included "
Included
""Included" indicates that the cost is included in the labor rate.
* Price per sample when purchased as one 10-test kit. Discounts are given for higher volume purchases.
Labor
Labor costs included on-site labor to perform the
analyses. The cost of the on-site labor was estimated
at a rate of $50-100/h, depending on the required
expertise level of the analyst. This cost element
included the labor involved during the entire
analytical process, comprising sample preparation,
sample management, analysis, and reporting. If the
user would have to travel to the site, the cost of
mobilization and demobilization, travel, and per
diem expenses should also be considered. However,
in a typical application where the SA-5000 might be
used, the analysis would usually be carried out by a
person located on site.
Equipment
Equipment costs included purchase of equipment
and the reagents and other consumable supplies
necessary to complete the analysis.
• Instrument purchase. The instrument can be
purchased for $3,850. This includes the SA-
5000 Scanning Analyzer, deluxe carrying case,
1-mL and 5-mL pipettors, and an ultrasonicator.
• Reagents and supplies. The dust sample
preparation and electrode pack costs $72.45 for
one pack often tests and contains: electrodes,
SoluPrep SP-B tablets, crushing and stirring
rods, and sonicator tubes. Discounts are given
for larger volume purchases.
Laboratory Costs
Sample Shipment
The costs of shipping samples to the laboratory
included overnight shipping charges as well as labor
charges associated with the various organizations
involved in the shipping process.
• Labor. This cost element included all of the
tasks associated with shipping the samples to the
NLLAP laboratory. Tasks included packing the
shipping coolers, completing the chain-of-
custody documentation, and completing the
shipping forms. The estimate to complete this
task ranged from 2 to 4 h, at $50 per hour.
• Overnight shipping. The overnight express
shipping service cost was estimated to be $50 -
100 for two boxes of samples.
Labor, Equipment, and Waste Disposal
The labor quotes from commercial analytical
laboratories that offered to perform the analyses for
this verification test ranged from $20 to $30 per
sample, with turnaround time estimates ranging
from 7 to 14 days. Some laboratories can provide a
1-2 day turnaround, but the quick turnaround was
not necessary for this test. The quotes were
dependent on many factors, including the perceived
difficulty of the sample matrix, the current workload
of the laboratory, data packaging, and the
competitiveness of the market. This rate was a fully
loaded analytical cost that included equipment,
22
-------
labor, waste disposal, and report preparation. The
cost for DataChem to analyze samples for this
verification test was $30 per sample, with a
turnaround time of 18 working days.
Cost Assessment Summary
An overall cost estimate for use of the SA-5000
instrument versus use of the NLLAP- laboratory was
not made because of the extent of variation in the
different cost factors, as outlined in Table 11. The
overall costs for the application of any technology
would be based on the number of samples requiring
analysis, the sample type, and the site location and
characteristics. Decision-making factors, such as
turnaround time for results, must also be weighed
against the cost estimate to determine the value of
the field technology's providing immediate answers
versus the laboratory's provision of reporting data
within 18 days of receipt of samples.
Miscellaneous Factors
The following are general observations regarding
the field operation and performance of the SA-5000
instrument:
The SA-5000 required no electrical power and
worked continuously through a 12-hour
workday without the need for recharging the
battery.
The Palintest analyst was ready for the first set
of samples within 1 h of arriving on site. If the
nitric acid is pre-prepared, and the equipment is
packaged in a carrying case and brought to the
test site, atypical set-up time is 15 min. The set-
up took a little longer for the verification test
because the equipment had to be unpacked from
sealed boxes and the concentrated nitric acid
had to be diluted.
Tests with the SA-5000 generated a 5-gal bucket
of vials containing dilute nitric acid waste,
which cost approximately $250 to dispose by a
commercial vendor. The actual volume of waste
was approximately 2.1 gallons (8 L). The
disposable electrodes have passed leaching tests
for mercury and silver, so they were thrown
away in the regular trash.
The Palintest analyst analyzed all of the samples
in duplicate (i.e., analyzed each sample with two
different electrodes). The duplicate analysis was
performed for Palintest's interests only, so the
result reported was the first result of the
duplicate measurements. Even though this
added some time to the analysis, the analyst
confirmed each of the results and still was able
to analyze all 160 samples in a two working
days because he operated two instruments.
The shelf-life of the electrodes is approximately
18 months.
Solu-Prep tablets are added to each vial prior to
analysis. The main ingredient is potassium
chloride. The tablet is colored red so the analyst
can quickly know that the right pellet is being
used.
On the last day of testing, 15 potential users
attending a nearby conference on lead-safe
housing observed the technology in operation
and completed a survey about its user
friendliness. Most (n=10) thought the system
was user-friendly and commented that the
relatively low cost (<$4,000) was attractive
when considering purchase options. Some of
the participants (n=6) stated they would
consider purchasing or using this instrument
based on their observations and felt a new user
could be trained in 2 to 4 hours. A few observers
reported that the SA-5000 seemed like it should
be operated in a laboratory rather than in a field
setting because of the 45 min digestion period
and the hazardous waste that is generated.
Summary of Performance
A summary of performance is presented in Table 12.
Note that performance is based on the specific
protocols employed for this verification test. If
different testing protocols are used, different
performance results may be obtained. The
verification test found that the SA-5000 instrument
was relatively simple for a trained analyst to operate
in the field, requiring less than an hour for initial
setup. The sample throughput of the SA-5000 was
eighty samples per day with a single operator.
The overall performance of the SA-5000 for the
analysis of lead in dust wipe samples was
characterized as having an acceptable amount of
negative bias, very precise, and has a strong linear
relationship with the NLLAP-laboratory results.
ORNL and ETV remind the reader that, while the
ETV test provides valuable information in the form
of a snapshot of performance, state, tribal, or federal
requirements regarding the use of the technologies
(such as NLLAP recognition for analysis of
clearance samples where required) need to be
followed.
23
-------
Table 12. Performance Summary for the Scanning Analyzer SA-5000 System
Feature/parameter
Precision : average RSD
Accuracy: average % recovery
Positive results on "detectable
blank" samples (< 2 jag/wipe)
False positive results
False negative results
Comparison with slope
laboratory results intercept
l-ig/wipe samples) correlation
coefficient
Overall evaluation
Completeness
Size and Weight
Sample throughput (1 analyst)
Power requirements
Training requirements
Cost
Waste generated
Performance summary
UC Samples
8%
80%
n/a
DataChem SA-5000
Oof 41 Oof 38
DataChem SA-5000
16 of 19 22 of 22
0.839
5.539
1.000
- Statistically significant negative
bias but within the acceptable bias
range
- Very Precise
- Strong linear relationship to the
NLLAP lab results
- No fp results
- Higher number of fn results
ELPAT Samples
5%
91%
0 of 20 samples
DataChem SA-5000
2 of 12 0 of 12
DataChem SA-5000
7 of 28 17 of 28
0.926
6.506
0.995
- Statistically significant negative
bias but within the acceptable bias
range
- Very Precise
- Strong linear relationship to the
NLLAP lab results
- No fp results
- Higher number of fn results
100% of 160 dust wipe samples
6 3/4" x 5 1/8" x 2 1/8"; 13.3 oz
80 samples/12-hr day
battery operated (eight - AA)
One-half day instrument-specific training
Purchase: $3,850
Reagents/Supplies: $72.45 for one pack of 10 tests (discounts for larger
volume purchases)
5 -gal bucket of vials of diluted nitric acid/extract dust wipes
2.1 gallons (8 L) for 160 samples analyzed
24
-------
Section 6 — References
American Society for Testing and Materials. 1996. "Specification E1792-96a: Standard Specification for
Wipe Sampling Materials for Lead in Surface Dust" in ASTM Standards on Lead Hazards Associated with
Buildings. West Conshohocken, PA.
American Society for Testing and Materials. 1998. "Practice El644: Standard Practice for Hot Plate
Digestion of Dust Wipe Samples for the Determination of Lead" in ASTM Standards on Lead Hazards
Associated with Buildings. West Conshohocken, PA.
Ashley, Kevin, Korrie Mapp, and Mark Millson. 1998. "Ultrasonic Extraction and Field-Portable Anodic
Stripping Voltammetry for the Determination of Lead in Workplace Air Samples." AIHA Journal. 59(10),
671-679.
Ashley, K., T.J. Wise, W. Mercado, and D.B. Parry. 2001. "Ultrasonic Extraction and Field-Portable
Anodic Stripping Voltammetric Measurement of Lead in Dust Wipe Samples." Journal of Hazardous
Materials. 83, p 41-50.
Code of Federal Regulations. 2001. "Identification of Dangerous Levels of Lead", Final Rule, 40 CFR Part
745,January.
Draper, N. R., and H. Smith. 1981. Applied Regression Analysis. 2nd ed. John Wiley & Sons, New York.
EPA (U.S. Environmental Protection Agency). 1996. "Method 3050B-1: Acid Digestion of Sediment, Sludge,
and Soils." In Test Methods for Evaluating Solid Waste: Physical/ Chemical Methods, Update II. SW-846.
U.S. Environmental Protection Agency, Washington, D.C., December.
EPA (U.S. Environmental Protection Agency). 1996. "Method 6010B-1: Inductively Coupled Plasma-Atomic
Emission Spectrometry." In Test Methods for Evaluating Solid Waste: Physical/ Chemical Methods, Update
II. SW-846. U.S. Environmental Protection Agency, Washington, D.C., December.
EPA (U.S. Environmental Protection Agency). 1996. "Evaluation of the Performance of Reflectance and
Electrochemical Technologies for the Measurement of Lead in Characterized Paints, Bulk Dusts, and Soils."
EPA 600/R-95/093. U.S. Environmental Protection Agency, Research Triangle Park, NC.
Keith, L.H., G. L. Patton, D.L. Lewis and P.O. Edwards. 1996. Chapter 1: Determining What Kinds of
Samples and How Many Samples to Analyze, pp. 19. In Principles of Environmental Sampling. Second
Edition. Edited by L. H. Keith, ACS Professional Reference Book, American Chemical Society, Washington,
DC.
ORNL (Oak Ridge National Laboratory). 1998. Quality Management Plan for the Environmental
Technology Verification Program 's Site Characterization and Monitoring Technologies Pilot. QMP-X-98-
CASD-001, Rev. 0. Oak Ridge National Laboratory, Oak Ridge, Tenn., November.
ORNL (Oak Ridge National Laboratory). 2001. Technology Verification Test Plan: Evaluation of Field
Portable Measurement Technologies for Lead in Dust Wipes. Chemical Sciences Division, Oak Ridge
National Laboratory, Oak Ridge, Tenn., November.
Song, Ruiguang, Paul C. Schlecht, and Kevin Ashley. 2001. "Field Screening Test Methods: performance
criteria and performance characteristics." Journal of Hazardous Materials. 83, 29-39.
25
-------
Appendix
Palintest's SA-5000 Results Compared with Laboratory Results
Sample
Analysis
Order
138
133
35
12
9
77
100
o
J
112
11
19
59
152
82
40
92
41
48
139
25
34
126
104
74
44
72
5
128
65
7
79
10
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
o
3
4
1
2
3
4
1
2
o
5
4
1
2
3
4
1
2
o
3
4
1
2
o
J
4
1
2
3
4
1
2
3
4
Palintest SA-5000
Result
|ig/wipe
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
<25
30
32
30
28
Estimated
|ig/wipe
o
.J
.3
o
.J
.3
.3
o
.J
.3
o
.J
o
.J
.3
o
.J
.3
.3
o
.J
.3
o
.J
o
.J
.3
o
.J
.3
16.9
16.9
16.9
16.9
17.6
17.6
17.6
17.6
29.8
29.8
29.8
29.8
DataChem
Result
|ig/wipe
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
30
<20
<20
<20
33
26
28
28
Estimated
|ig/wipe
o
.J
.3
o
.J
.3
.3
o
.J
.3
o
.J
o
.J
.3
o
.J
.3
.3
o
.J
.3
o
.J
o
.J
.3
o
.J
.3
16.9
16.9
16.9
16.9
17.6
17.6
17.6
17.6
29.8
29.8
29.8
29.8
26
-------
Sample
Analysis
Order
14
125
143
58
78
4
151
30
105
53
134
109
135
103
123
95
27
158
119
145
149
147
108
1
144
142
45
84
102
38
51
66
57
121
55
94
Source
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
o
6
4
1
2
3
4
1
2
o
J
4
1
2
3
4
1
2
o
J
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
o
J
4
Palintest SA-5000
Result
|ig/wipe
38
32
38
36
36
38
30
32
32
34
36
34
34
38
34
30
36
34
36
30
36
36
40
40
42
38
42
40
38
40
42
45
47
51
44
47
Estimated
|ig/wipe
39.9
35.1
43.3
42.1
38.8
42.8
35.0
36.4
38.5
38.9
38.8
37.6
43.1
41.1
37.3
35.1
44.5
36.4
38.8
39.2
41.3
41.3
41.3
41.3
49.0
49.0
49.0
49.0
49.1
49.1
49.1
49.1
58.6
58.6
58.6
58.6
DataChem
Result
|ig/wipe
33
32
31
29
32
38
37
36
37
37
33
41
32
38
30
35
36
31
34
34
37
42
44
41
43
52
49
48
70
54
48
44
64
55
56
52
Estimated
|ig/wipe
35.4
35.7
38.5
36.4
35.1
40.7
39.4
41.0
41.0
38.8
39.3
44.7
36.0
44.7
39.9
37.5
37.4
36.7
35.8
39.7
41.3
41.3
41.3
41.3
49.0
49.0
49.0
49.0
49.1
49.1
49.1
49.1
58.6
58.6
58.6
58.6
27
-------
Sample
Analysis
Order
62
136
80
56
90
70
6
155
43
68
106
20
111
69
97
115
96
33
75
150
49
37
114
99
91
29
2
76
88
71
130
83
18
64
54
124
156
17
36
67
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
Rep
1
2
o
5
4
1
2
o
5
4
1
2
o
J
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
o
6
4
1
2
3
4
1
2
o
3
4
1
2
3
4
Palintest SA-5000
Result
|ig/wipe
75
71
65
67
96
104
110
100
155
162
157
157
200
186
194
192
225
194
221
231
164
182
168
178
186
176
211
192
225
198
170
166
213
205
196
176
178
194
192
180
Estimated
|ig/wipe
88.0
88.0
88.0
88.0
117.0
117.0
117.0
117.0
162.3
162.3
162.3
162.3
201.6
201.6
201.6
201.6
239.0
239.0
239.0
239.0
249.5
239.0
245.1
225.7
244.0
269.4
258.9
250.0
270.0
263.9
240.1
237.3
255.6
262.8
258.9
225.7
228.5
240.6
255.0
241.2
DataChem
Result
|ig/wipe
82
83
79
100
120
120
120
110
150
160
150
160
200
190
200
220
230
250
250
230
210
250
230
230
200
240
210
210
220
220
230
170
190
210
210
250
220
210
210
220
Estimated
|ig/wipe
88.0
88.0
88.0
88.0
117.0
117.0
117.0
117.0
162.3
162.3
162.3
162.3
201.6
201.6
201.6
201.6
239.0
239.0
239.0
239.0
244.0
274.4
252.8
258.9
241.7
274.9
244.5
236.2
244.0
242.3
260.0
228.5
242.3
267.2
236.2
275.5
262.2
226.3
227.4
243.4
28
-------
Sample
Analysis
Order
46
110
52
116
89
120
85
113
26
23
22
153
86
8
50
73
107
160
127
140
93
117
60
122
148
137
129
32
118
21
39
42
132
98
131
141
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
o
6
4
1
2
o
6
4
1
2
o
6
4
1
2
3
4
1
2
o
5
4
1
2
3
4
1
2
o
6
4
1
2
o
J
4
1
2
3
4
Palintest SA-5000
Result
|ig/wipe
200
207
211
239
225
246
223
246
285
293
303
266
318
311
274
307
274
274
339
278
317
278
299
324
337
274
315
322
409
388
378
355
407
378
403
386
Estimated
|ig/wipe
256.7
256.7
256.7
256.7
260.8
260.8
260.8
260.8
366.2
388.9
401.6
372.3
429.3
391.7
370.1
401.1
376.7
367.9
383.9
385.0
388.3
365.1
385.0
421.5
428.2
367.3
405.5
403.8
408.7
408.7
408.7
408.7
418.1
418.1
418.1
418.1
DataChem
Result
|ig/wipe
290
240
230
250
220
250
210
210
320
360
350
340
350
340
370
340
370
340
370
390
330
320
330
360
340
360
390
330
360
430
410
410
440
410
430
420
Estimated
|ig/wipe
256.7
256.7
256.7
256.7
260.8
260.8
260.8
260.8
377.8
395.0
399.4
385.0
395.5
382.8
413.8
374.0
426.5
378.9
401.1
423.2
372.9
362.9
384.5
411.0
397.2
393.3
437.6
375.1
408.7
408.7
408.7
408.7
418.1
418.1
418.1
418.1
29
-------
Sample
Analysis
Order
63
157
81
101
87
47
15
13
61
159
24
16
154
31
28
146
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
o
5
4
1
2
o
5
4
1
2
o
J
4
1
2
3
4
Palintest SA-5000
Result
|ig/wipe
581
515
551
558
523
521
576
541
809
780
817
841
1257
1322
1236
1416
Estimated
|ig/wipe
561.9
561.9
561.9
561.9
564.7
564.7
564.7
564.7
805.1
805.1
805.1
805.1
1482.6
1482.6
1482.6
1482.6
DataChem
Result
|ig/wipe
580
540
560
540
560
560
570
530
760
770
760
740
1500
1500
1500
1400
Estimated
|ig/wipe
561.9
561.9
561.9
561.9
564.7
564.7
564.7
564.7
805.1
805.1
805.1
805.1
1482.6
1482.6
1482.6
1482.6
30
------- |