United States Office of Research and EPA/600/R-02/058
Environmental Protection Development September 2002
Agency Washington, D.C. 20460
v>EPA Environmental Technology
Verification Report
Lead in Dust Wipe Measurement
Technology
KeyMaster Technologies
X-Ray Fluorescence Instrument
Pb-Test
onvl
Oak Ridge National Laboratory
-------
THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
ET
AM^
V
r/EPA
oml
Oak Ridge National Laboratory
Verification Statement
TECHNOLOGY TYPE:
APPLICATION:
TECHNOLOGY NAME:
COMPANY:
ADDRESS:
WEB SITE:
E-MAIL:
X-RAY FLUORESCENCE
MEASUREMENT OF LEAD IN DUST WIPES
Pb-Test XRF Instrument
KeyMaster Technologies
415 N. Quay
Kennewick, WA 99336
www.keymastertech.com
thowe@keymastertech.com
PHONE: (509) 783-9850
FAX: (509) 735-9696
The U.S. Environmental Protection Agency (EPA) has created the Environmental Technology
Verification Program (ETV) to facilitate the deployment of innovative or improved environmental
technologies through performance verification and dissemination of information. The goal of the ETV
Program is to further environmental protection by substantially accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-quality,
peer-reviewed data on technology performance to those involved in the design, distribution, financing,
permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized standards and testing organizations and stakeholder groups
consisting of regulators, buyers, and vendor organizations, with the full participation of individual
technology developers. The program evaluates the performance of innovative technologies by developing
test plans that are responsive to the needs of stakeholders, conducting field or laboratory tests (as
appropriate), collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are
conducted in accordance with rigorous quality assurance protocols to ensure that data of known and
adequate quality are generated and that the results are defensible.
Oak Ridge National Laboratory (ORNL) is one of the verification organizations operating under the
Advanced Monitoring Systems (AMS) Center. AMS, which is administered by EPA's National Exposure
Research Laboratory (NERL), is one of six technology areas under ETV. In this verification test, ORNL
evaluated the performance of lead in dust wipe measurement technologies. This verification statement
provides a summary of the test results for KeyMaster Technologies' Pb-Test x-ray fluorescence (XRF)
instrument.
EPA-VS-SCM-54
The accompanying notice is an integral part of this verification statement.
September 2002
-------
VERIFICATION TEST DESCRIPTION
This verification test was designed to evaluate technologies that detect and measure lead in dust wipes.
The test was conducted at the Oak Ridge National Laboratory in Oak Ridge, TN, from January 7 through
January 9, 2002. KeyMaster Technologies, a vendor of commercially-available, field portable x-ray
fluorescence (XRF) instruments for lead detection and measurement, blindly analyzed 160 dust wipe
samples containing known amounts of lead, ranging in concentration from <2 to 1,500 |_lg/wipe. The
experimental design was particularly focused on important clearance standards, such as those identified
in 40 CFR Part 745.227(e)(8)(viii) of 40 |_lg/ft2 for floors, 250 |_lg/ft2 for window sills, and 400 |_lg/ft2 for
window troughs. The samples included wipes newly-prepared and archived from the Environmental
Lead Proficiency Analytical Testing Program (ELPAT). These samples were prepared from dust
collected in households in North Carolina and Wisconsin. Also, newly-prepared samples were acquired
from the University of Cincinnati (UC). The (UC) dust wipe samples were prepared from National
Institute of Standards and Technology (NIST) Standard Reference Materials (SRMs). The results of the
lead analyses generated by the technology were compared with results from analyses of similar samples
by conventional laboratory methodology in a laboratory that was recognized as proficient by the
National Lead Laboratory Accreditation Program (NLLAP) for dust testing. Details of the test, including
a data summary and discussion of results, may be found in the report entitled Environmental Technology
Verification Report: Lead in Dust Wipe Detection Technology— KeyMaster Technologies, Pb-Test X-
Ray Fluorescence Instrument, EPA/600/R-02/058.
TECHNOLOGY DESCRIPTION
The Pb-Test is an energy dispersive x-ray fluorescence (EDXRF) spectrometer that uses a sealed, highly
purified Cobalt-57 radioisotope source (<12 mCi) to excite a test sample's constituent elements. The Pb-
Test utilizes the recently developed Cadmium Telluride (CdTe) Schottky diode detectors. The age of the
detector at the time of testing was approximately 4 to 5 months. Each element produces x-rays at a
unique set of energies, allowing one to non-destructively measure the elemental composition of a sample.
These characteristic x-rays are continuously detected, identified, and quantified by the spectrometer
during sample analysis. In other words, the energy of each x-ray identifies a particular element present in
the sample and the rate at which the x-rays of a given energy are emitted allows the analyzer to determine
the quantity of a particular element present in that sample. Signals from the detector are amplified,
digitized, and then quantified via an integrated multichannel analyzer and data processor. Sample test
results are displayed in total micrograms of lead per dust-wipe. KeyMaster did not provide a reporting
limit for the instrument during the verification test.
VERIFICATION OF PERFORMANCE
The following performance characteristics of the Pb-Test XRF were observed:
Precision: Precision, based on the average percent relative standard deviation (RSD), was 18% for the
ELPAT samples and 15% for the UC samples. A technology's performance is considered very precise if
the average RSD is less than 10%, but acceptable as long as the average RSD is less than 20%.
Accuracy: Accuracy was assessed using the estimated concentrations of the ELPAT and UC samples.
Acceptable bias falls in a range of average percent recovery values of 100% ± 25%. The average
percent recovery values for the ELPAT and UC samples (excluding the "detectable blank" samples at
concentrations < 2 |_lg/wipe that are described in more detail below) were 189% and 168%, respectively. If
only those samples with concentrations between 200 and 1,500 |_lg/wipe are considered, the Pb-Test results
were unbiased, with an average percent recovery value of 96% for ELPAT samples and 102% for the UC
EPA-VS-SCM-54 The accompanying notice is an integral part of this verification statement. September 2002
-------
samples. The Pb-Test results for samples at 800, and 1,500 |_lg/wipe were both negatively biased
(78% and 72%, respectively), but there was not enough data to ascertain that the technology was negatively
biased above 800 |_lg/wipe. For the NLLAP laboratory results, the average percent recovery values were
98% and 91%, respectively, for the ELPAT and UC samples. The NLLAP laboratory's negative bias for
both the ELPAT and UC samples was statistically significant.
Comparability: A comparison of the average Pb-Test results and the average NLLAP-recognized
laboratory results was performed for all samples (ELPAT and UC) for estimated concentrations above
and below 200 |_lg/wipe. The correlation coefficient (r) for the < 200 |_lg/wipe data set was 0.967 [slope
(m)= 1.060, intercept = 66]. For the > 200 |_lg/wipe data, the rvalue was 0.989 [slope = 0.662,
intercept = 121). The slopes for both data sets were statistically different from 1.00. The Pb-Test results
above 200 |_lg/wipe indicate fair agreement with the NLLAP laboratory's results, since correlation
coefficient values greater than 0.990 indicate good agreement with the laboratory data.
Detectable blanks: All twenty samples, prepared at concentrations < 2 |_lg/wipe, were reported as
detections by the Pb-Test, with concentrations ranging from 46 to 137 |_lg/wipe.
False positive results: A false positive (fp) result is one in which the technology reports a result that is
above the clearance level when the true (or estimated) concentration is actually below. For the UC
samples, the Pb-Test reported 20 of a possible 38 fp results, while the NLLAP laboratory did not report
any fp results. For the ELPAT samples, the Pb-Test reported 6 of a possible 12 fp results, while the
NLLAP laboratory reported two.
False negative results: A false negative (fn) result is one in which the technology reports a result that is
below the clearance level when the true (or estimated) concentration is actually above. For the UC
samples, the Pb-Test reported 7 of a possible 22 fn results, while the NLLAP laboratory reported 23 of a
possible 30 fn results. For the ELPAT samples, the Pb-Test reported 8 of a possible 28 fn results, while
the NLLAP laboratory reported 7.
Completeness: Completeness is defined as the percentage of measurements that are judged to be usable
(i.e., the result is not rejected). An acceptable completeness rate is 95% or greater. The Pb-Test
instrument generated results for all 160 dust wipes samples for a completeness of 100%.
Sample Throughput: Sample throughput is a measure of the number of samples that can be processed
and reported by a technology in a given period of time. With two analysts, the Key Master team
accomplished a sample throughput rate of approximately eighty samples per 10-hour day. One operator
prepared the samples, while the other performed the analyses. The vendor chose to run the samples on
two instruments and report the average value. The instrument can be operated by a single trained
analyst.
EPA-VS-SCM-54 The accompanying notice is an integral part of this verification statement. September 2002
-------
Overall Evaluation: The overall performance was characterized as having acceptable precision, biased
high for concentrations below 200 |-lg/wipe, and unbiased for concentrations above 200 |_lg/wipe. The
Pb-Test results above 200 |-lg/wipe were also found to be in fair linear agreement with the NLLAP
laboratory's results. The verification team found that the Pb-Test was simple for the trained analyst to
operate in the field, requiring less than one-half hour for initial setup. As with any technology
selection, the user must determine if this technology is appropriate for the application and the project
data quality objectives. Additionally, ORNL and ETV remind the reader that, while the ETV test
provides valuable information in the form of a snapshot of performance, state, tribal, or federal
requirements regarding the use of the technologies (such as NLLAP recognition for analysis of
clearance samples where required) need to be followed. For more information on this and other verified
technologies, visit the ETV web site at http://www.epa. gov/etv.
Gary J. Foley, Ph.D.
Director
National Exposure Research Laboratory
Office of Research and Development
W. Franklin Harris, Ph.D.
Associate Laboratory Director
Biological and Environmental Sciences
Oak Ridge National Laboratory
NOTICE: EPA verifications are based on evaluations of technology performance under specific, predetermined criteria
and appropriate quality assurance procedures. EPA and ORNL make no expressed or implied warranties as to the
performance of the technology and do not certify that a technology will always operate as verified. The end user is solely
responsible for complying with any and all applicable federal, state, and local requirements. Mention of commercial
product names does not imply endorsement or recommendation.
EPA-VS-SCM-54
The accompanying notice is an integral part of this verification statement.
September 2002
-------
EPA/600/R-02/058
September 2002
Environmental Technology
Verification Report
Lead in Dust Wipe Measurement
Technology
KeyMaster Technologies
X-Ray Fluorescence Instrument
Pb-Test
By
Amy B. Dindal
Charles K. Bayne, Ph.D.
Roger A. Jenkins, Ph.D.
Oak Ridge National Laboratory
Oak Ridge, Tennessee 37831-6120
Eric N. Koglin
U.S. Environmental Protection Agency
Environmental Sciences Division
National Exposure Research Laboratory
Las Vegas, Nevada 89193-3478
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and Development (ORD),
funded and managed, through Interagency Agreement No. DW89937854 with Oak Ridge National
Laboratory, the verification effort described herein. This report has been peer and administratively reviewed
and has been approved for publication as an EPA document. Mention of trade names or commercial products
does not constitute endorsement or recommendation for use of a specific product.
-------
Table of Contents
Notice ii
List of Figures v
List of Tables vi
Acknowledgments vii
Abbreviations and Acronyms viii
Section 1 — Introduction 1
Section 2 — Technology Description 2
General Technology Description 2
Sample Preparation 2
Calibration 2
Sample Analysis 2
Section 3 — Verification Test Design 3
Objective 3
Testing Location and Conditions 3
Drivers and Objectives for the Test 3
Summary of the Experimental Design 3
ELPAT and Blank Sample Description 3
University of Cincinnati Sample Description 4
Distribution and Number of Samples 5
Description of Performance Factors 5
Precision 5
Accuracy 6
Comparability 7
Detectable Blanks 7
False Positive/Negative Results 7
Completeness 8
Sample Throughput 8
Ease of Use 8
Cost 8
Miscellaneous Factors 8
Section 4 — Laboratory Analyses 9
Background 9
NLLAP Laboratory Selection 9
Laboratory Method 9
Section 5 — Technology Evaluation 16
Objective and Approach 16
Precision 16
Accuracy 16
Comparability 16
Detectable Blanks 19
iii
-------
False Positive/False Negative Results 19
Completeness 22
Sample Throughput 22
Ease of Use 22
Cost Assessment 22
Pb-Test XRF Costs 23
Labor 23
Equipment 23
Laboratory Costs 24
Labor, Equipment, and Waste Disposal 24
Cost Assessment Summary 24
Miscellaneous Factors 24
Summary of Performance 24
Section 6 — References 27
Appendix 28
IV
-------
List of Figures
1. KeyMaster Technologies Pb-Test instrument 2
2. Pb-Test sample holder 2
3. Distribution of concentration levels 6
4. Plot of DataChem reported values versus estimated values, shown for concentrations less than 500
|J,g/wipe 11
5. False negative probabilities for DataChem average concentrations at a target concentration level of 40
|J,g/wipe 13
6. False negative probabilities for DataChem average concentrations at a target concentration level of
250 |_lg/wipe 14
7. False negative probabilities for DataChem average concentrations at a target concentration level of
400 |_lg/wipe 14
8. Plot of the Pb-Test's average percent recovery values (Pb-Test result/estimated value x 100%) versus
the average estimated values for all concentrations and sample types (n=23) 17
9. Plot of the Pb-Test average concentrations versus DataChem average concentrations for all UC and
ELPAT samples with positive DataChem concentrations less than 200 |_lg/wipe (n=9) 17
10. Plot of the Pb-Test average concentration versus the DataChem average concentration for all UC and
ELPAT samples > 200 |_lg/wipe (n=12) 18
11. Comparison of the false negative probabilities for the Pb-Test and DataChem average concentrations
at a target concentration of 40 |_lg/wipe 20
12. Comparison of the false negative probabilities for the Pb-Test and Data Chem average
concentrations at a target concentration of 250 |_lg/wipe 21
13. Comparison of the false negative probabilities for the Pb-Test and DataChem average concentrations
at a target concentration of 400 |_lg/wipe 21
-------
List of Tables
1. Summary of DataChem Pre-Test Results 10
2. Summary of DataChem Percent Recovery Values by Sample Source 11
3. Summary of DataChem Precision Estimates by Sample Source 11
4. False Positive/False Negative Results for DataChem Measurements of UC Samples 13
5. Summary of the Linear Regression Constants and Recovery Data for DataChem's Measurements
Versus the Estimated Concentrations at the Clearance Levels 15
6. Precision of the Pb-Test Instrument 16
7. Accuracy of Pb-Test Instrument 17
8. False Positive/False Negative Error Rates for Pb-Test Measurements 20
9. Summary of the Linear Regression and Recovery Data for the Pb-Test Response versus the
Estimated Concentrations 22
10. Estimated analytical costs for lead dust wipe samples 23
11. Performance Summary for the Pb-Test Instrument 26
VI
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the verification test,
analyze the data, and prepare this report. In particular, we recognize: Patricia Lindsey, Capitol Community
Technical College, for providing a site for the verification test; Sandy Roda (University of Cincinnati) and
Laura Hodson (Research Triangle Institute) for coordination of sample preparation; the inorganic analytical
laboratory in EPA/Region 1 (North Chelmsford, MA) for the analysis of quality control samples; and the
expertise of the technical advisory panel, including Kenn White (American Industrial Hygiene Association),
John Schwemberger, Dan Reinhart, Oksana Pozda, and Darlene Watford (EPA/Office of Pollution
Prevention and Toxics), Paul Carroll (EPA/Region 1), Sharon Harper (EPA/Research Triangle Park), Peter
Ashley, Warren Friedman, Gene Pinzer, and Emily Williams (U.S. Department of Housing and Urban
Development); Paul Halfmann and Sharon Cameron (Massachusetts Childhood Lead Poisoning Prevention
Program), Kevin Ashley (National Institute for Occupational Safety and Health); Walt Rossiter and Mary
McKnight (National Institute of Standards & Technology); Bill Gutknecht (Research Triangle Institute), and
Bruce Buxton (Battelle Memorial Institute). The authors would specifically like to thank Paul Halfmann and
John Schwemberger for serving as peer reviewers of this report. The authors also acknowledge the
participation of KeyMaster Technologies, in particular, Therese Howe and Steve Price.
For more information on the Lead in Dust Wipe Measurement Technology Verification contact:
Eric N. Koglin Roger A. Jenkins
Project Technical Leader Program Manager
Environmental Protection Agency Oak Ridge National Laboratory
Environmental Sciences Division Chemical Sciences Division
National Exposure Research Laboratory P.O. Box 2008
P.O. Box 93478 Oak Ridge, TN 37831-6120
Las Vegas, Nevada 89193-3478 (865) 574-4871
(702) 798-2332 jenkinsra@ornl.gov
koglin.eric@epa.gov
For more information on KeyMaster's Pb-Test XRF Instrument, contact:
Therese Howe Tony Williams
KeyMaster Technologies ED AX Inc.
415 N. Quay 91 McKee Drive
Kennewick, WA 99336 Mahwah, NJ 07430
509-783-8950 201-529-6118
thowe@kevmastertech.com twilliams@edax.com
www.keymastertech.com www.edax.com
vn
-------
Abbreviations and Acronyms
AIHA American Industrial Hygiene Association
AMS Advanced Monitoring Systems Center, ETV
ASTM American Society for Testing and Materials
CDC Centers for Disease Control and Prevention
CdTe Cadmium Telluride
CFR Code of Federal Regulations
CL clearance level of 40, 250, and 400 |_lg/wipe
EDXRF energy dispersive x-ray fluorescence
ELPAT Environmental Lead Proficiency Analytical Testing program
EPA U. S. Environmental Protection Agency
ETV Environmental Technology Verification Program
ETVR Environmental Technology Verification Report
fn false negative result
fp false positive result
ICP-AES Inductively coupled plasma-atomic emission spectrometry
NERL National Exposure Research Laboratory, U.S. EPA
NIOSH National Institute for Occupational Safety and Health, CDC
NIST National Institute of Standards & Technology
NLLAP National Lead Laboratory Accreditation Program, U.S. EPA
OPPT Office of Pollution Prevention and Toxics, U.S. EPA
ORNL Oak Ridge National Laboratory
QA quality assurance
QC quality control
RSD relative standard deviation
RTI Research Triangle Institute
SD standard deviation
SRM Standard Reference Material
UC University of Cincinnati
XRF x-ray fluorescence instrument
Vlll
-------
Section 1 — Introduction
The U.S. Environmental Protection Agency (EPA)
created the Environmental Technology Verification
Program (ETV) to facilitate the deployment of
innovative or improved environmental technologies
through performance verification and dissemination
of information. The goal of the ETV Program is to
further environmental protection by substantially
accelerating the acceptance and use of improved and
cost-effective technologies. ETV seeks to achieve
this goal by providing high-quality, peer-reviewed
data on technology performance to those involved in
the design, distribution, financing, permitting,
purchase, and use of environmental technologies.
ETV works in partnership with recognized standards
and testing organizations and stakeholder groups
consisting of regulators, buyers, and vendor
organizations, with the full participation of
individual technology developers. The program
evaluates the performance of innovative
technologies by developing verification test plans
that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate),
collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in
accordance with rigorous quality assurance (QA)
protocols to ensure that data of known and adequate
quality are generated and that the results are
defensible.
ETV is a voluntary program that seeks to provide
objective performance information to all of the
participants in the environmental marketplace and to
assist them in making informed technology
decisions. ETV does not rank technologies or
compare their performance, label or list technologies
as acceptable or unacceptable, seek to determine
"best available technology," or approve or
disapprove technologies. The program does not
evaluate technologies at the bench or pilot scale and
does not conduct or support research. Rather, it
conducts and reports on testing designed to describe
the performance of technologies under a range of
environmental conditions and matrices.
The program now operates six centers covering a
broad range of environmental areas. ETV began
with a 5-year pilot phase (1995-2000) to test a wide
range of partner and procedural alternatives in
various technology areas, as well as the true market
demand for and response to such a program. In these
centers, EPA utilizes the expertise of partner
"verification organizations" to design efficient
processes for conducting performance tests of
innovative technologies. These expert partners are
both public and private organizations, including
federal laboratories, states, industry consortia, and
private sector entities. Verification organizations
oversee and report verification activities based on
testing and QA protocols developed with input from
all major stakeholder/customer groups associated
with the technology area. The verification described
in this report was administered by the Advanced
Monitoring Systems (AMS) Center, with Oak Ridge
National Laboratory (ORNL) serving as the
verification organization. (To learn more about
ETV, visit ETV's Web site at
http://www.epa.gov/etv.) The AMS Center is
administered by EPA's National Exposure Research
Laboratory (NERL).
The verification of a field analytical technology for
measurement of lead in dust wipe samples is
described in this report. The verification test was
conducted in Oak Ridge, Tennessee, from January 7
through January 9, 2002. The performance of
KeyMaster Technologies' Pb-Test x-ray
fluorescence (XRF) instrument was determined. The
technology was evaluated by comparing its results to
estimated concentration values and with results
obtained on similar samples using a recognized
laboratory analytical method.
-------
Section 2 — Technology Description
Figure 1. KeyMaster Technologies Pb-Test
instrument.
In this section, the vendor (with minimal editorial changes by ORNL) provides a description of the
technology and the analytical procedure used during the verification testing activities.
General Technology Description
The Pb-Test (see Figure 1) is an energy dispersive
x-ray fluorescence (EDXRF) spectrometer that
uses a sealed, highly purified Cobalt-57
radioisotope source (<12 mCi, 4-5 months old at
the time of the test) to excite a test sample's
constituent elements. Each element produces x-
rays at a unique set of energies, allowing the user
to non-destructively measure the elemental
composition of a sample. These characteristic x-
rays are continuously detected, identified, and
quantified by the spectrometer during sample
analysis. In other words, the energy of each x-ray
identifies a particular element present in the
sample and the rate at which the x-rays of a given
energy are emitted allows the analyzer to
determine the quantity of a particular element
present in that sample. The Pb-Test utilizes the
recently developed Cadmium Telluride (CdTe)
Schottky diode detectors. The use of Schottky
contacts in CdTe detectors reduces the leakage
current, permitting the use of a much higher bias
voltage than in a standard detector. Since the
charge transport properties of the CdTe detector
are much higher, the net result is an improvement
in the usable depth (sensitivity). Additionally,
because hole tailing and electronic noise are
reduced, not only is the sensitivity improved but
resolution is much better. Signals from the
detector are amplified, digitized, and then
quantified via an integrated multichannel analyzer
and data processor. Re-sourcing requirements are
on the order of 15 to 24 months, depending on the
user's needs. Sample test results are displayed in
total micrograms of lead per dust-wipe.
Sample Preparation
For this test, the dust wipe samples had to be cut
to 1/4 of the size of the original wipe. The wipes
were cut such that the majority of the dust was
included in the 1/4 area. The wipe was folded and
placed in the sample holder which is the size of a
small thimble (Figure 2). The wipe was pressed
into the sample holder using a specially-designed
tool that is provided with the instrument. The
sample was then labeled and ready for analysis.
Figure 2. Pb-Test sample holder.
Calibration
The instrument is factory calibrated. KeyMaster
did not perform additional calibration checks
during testing.
Sample Analysis
The Pb-Test should be turned on at least 10
minutes prior to taking the first measurement.
"Confirm" precision was selected which
indicated a 5 minute measurement. After placing
the sample in the sample holder (Figure 2), the
analysis started. When the measurement was
completed, the Pb-Test automatically displayed
the results in |J,g. For the verification test, two
instruments were used, with one analyst analyzing
all samples on both, and the average reading from
the two instruments reported as the final result.
-------
Section 3 — Verification Test Design
Objective
The purpose of this section is to describe the
verification test design. It is a summary of the test
plan (ORNL, 2001).
Testing Location and Conditions
The verification of field analytical technologies for
lead in dust wipes was conducted at Oak Ridge
National Laboratory, in Oak Ridge, TN. The test
was conducted in an office. The temperature and
relative humidity were monitored during field
testing, but remained fairly constant. The average
temperature and relative humidity over the three
days of testing were 70 °F and 24%, respectively.
Drivers and Objectives for the Test
The purpose of this test was to evaluate the
performance of field analytical technologies that are
capable of analyzing dust wipe samples for lead
contamination. This test provides information on the
potential applicability of field technologies to EPA
standards for dust clearance testing. The
experimental design was designed around the three
clearance standards of 40 |-lg/ft2 for floors, 250
|~lg/ft2 for window sills, and 400 |-lg/ft2 for window
troughs that are outlined in 40 CFR Part
745.227(e)(8)(viii) (CFR 2001).
The primary objectives of this verification were to
evaluate the field analytical technologies in the
following areas: (1) how well each performs relative
to a conventional, fixed-site, analytical method for
the analysis of dust wipe samples for lead; (2) how
well each performs relative to results generated in
previously rounds of ELPAT testing (described in
the next section), and (3) the logistical and
economic resources necessary to operate the
technology. Secondary objectives for this
verification were to evaluate the field analytical
technology in terms of its reliability, ruggedness,
cost, range of usefulness, sample throughput, data
quality, and ease of use. Note that this verification
test does not provide an assessment of the selection
of locations for dust samples in a facility or an
assessment of the way that dust samples are
collected. The planning for this verification test
follows the guidelines established in the data quality
objectives process.
Summary of the Experimental
Design
All of the samples analyzed in this verification test
were prepared gravimetrically. At the time of the
test, both of the wipes utilized in the test
(PaceWipe™ and Aramsco LeadWipe™) were on
the list of wipes recommended for lead testing by
the American Society for Testing and Materials
(ASTM, 1996). Initial consideration was given to
conducting the test in a real-world situation, where
the technologies would have been deployed in a
housing unit that had been evacuated due to high
levels of lead contamination. In addition to the
safety concern of potentially subjecting participants
to lead exposure, the spatial variability of adjacent
samples would have been expected to be so great
that it would be much larger than the anticipated
variability of these types of technologies, thereby
making it difficult to separate instrument/method
variability and sampling variability. The availability
of we 11-characterized samples derived from "real-
world" environments made the use of proficiency
testing samples (so-called "ELPAT" samples) and
other prepared samples an attractive alternative.
ELPAT and Blank Sample
Description
In 1992, the American Industrial Hygiene
Association (AIHA) established the Environmental
Lead Proficiency Analytical Testing (ELPAT)
program. The ELPAT Program is a cooperative
effort of the American Industrial Hygiene
Association (AIHA), and researchers at the Centers
for Disease Control and Prevention (CDC), National
Institute for Occupational Safety and Health
(NIOSH), and the EPA Office of Pollution
Prevention and Toxics (OPPT). The ELPAT
program is designed to assist laboratories in
improving their analytical performance, and
therefore, does not specify use of a particular
analytical method. Participating laboratories are
sent samples to analyze on a quarterly basis. The
reported values must fall within a range of
acceptable values in order for the laboratory to be
deemed proficient for that quarter.
Research Triangle Institute (RTI) in Research
Triangle Park, NC, is contracted to prepare and
distribute the lead-containing paint, soil, and dust
-------
wipe ELPAT samples. For the rounds of testing
which have occurred since 1992, archived samples
are available for purchase. Some of these samples
were used in this verification test. Because the
samples have already been tested by over one-
hundred laboratories, a certified concentration value
is supplied with each sample. This certified value
represents a pooled measurement of all of the results
submitted, with outliers excluded from the
calculation.
The following description, taken from an internal
RTI report, briefly outlines how the samples were
prepared. RTI developed a repository of real-world
housedust, collected from multiple homes in the
Raleigh/Durham/Chapel Hill area, as well as from
an intervention project in Wisconsin. After
collection, the dust was sterilized by gamma
irradiation, and sieved to 150 |_lm. A PaceWipe™
was prepared for receiving the dust by opening the
foil pouch, removing the wet folded wipe and
squeezing the excess moisture out by hand over a
trash can. The wipe was then unfolded and briefly
set on a Kimwipe™ to soak up excess moisture.
The PaceWipe was then transferred to a flat plastic
board to await the dust. After weighing a 0.1000 ±
0.0005 g portion of dust on weighing paper, the pre-
weighed dust was gently tapped out onto the
PaceWipe. The wipe was then folded and placed in
a plastic vial, which was then capped. All vials
containing the spiked wipes were stored in a cold
room as a secondary means of retarding mold
growth until shipment.
Before use in the ELPAT program, RTI performed a
series of analyses to confirm that the samples were
prepared within the quality guidelines established
for the program. The data quality requirements for
the ELPAT samples were: 1) the relative standard
deviation of the samples analyzed by RTI must be
10% or less; 2) the measured concentrations must be
within 20% of the target value that RTI was
intending to prepare; and 3) analysis by an
accredited laboratory must yield results within ±
20% of the RTI result. Ten samples were analyzed
by RTI and nine samples were sent to the Wisconsin
Occupational Health Laboratory for independent,
confirmatory analysis. All ELPAT samples used in
this test met the data quality requirements described
above. The estimated concentration for an ELPAT
sample used in this evaluation was the certified
("consensus") value (i.e., an analytically derived
result).
RTI prepared the blank samples using the same
preparation method as the ELPAT samples, but the
concentration of lead was < 2 |_lg/wipe, well below
the expected reporting limits of the participant
technologies.
University of Cincinnati Sample
Description
The ELPAT samples consisted of dust mounded in
the center of a PaceWipe. The University of
Cincinnati (UC) prepared "field QC samples" where
the dust was sprinkled over the wipe, more similar
to how a wipe would look when a dust wipe sample
is collected in the field. The sample was prepared by
weighing, so the concentrations can be estimated. In
a typical scenario, UC sends these control samples
to a laboratory along with actual field-collected
samples as a quality check of the laboratory
operations. Because the samples are visually
indistinguishable from an actual field sample, are
prepared on the same wipe, and are shipped in the
same packaging, the laboratory blindly analyzes the
control samples, which provides the user with an
independent assessment of the quality of the
laboratory's data.
A cluster of twenty UC samples prepared at the key
clearance levels were added to the experimental
design, primarily so that an abundance of data
would exist near the clearance levels, in order to
assess false positive and false negative error rates.
The UC samples were prepared on Aramsco Lead
Wipes™ (Lakeland, FL). The UC wipe samples
were prepared using National Institute of Standards
& Technology (NIST) Standard Reference Materials
(SRMs). NIST SRM 2711 was used to prepare the
40 |_lg/wipe samples, and NIST SRM 2710 was used
to prepare the 250 and 400 |_lg/wipe samples. Both
SRM 2711 and SRM 2710 are Montana Soil
containing trace concentrations of multiple
elements, including lead. Some NIST SRM
materials that are spiked on dust wipes are known to
have low extraction recoveries when prepared by
standard analytical methods (e.g., lead silicates
cannot be extracted unless hydrofluoric acid is used)
(Ashley et al., 1998). These particular SRMs are not
known to contain lead silicates or to give lower lead
recoveries. However, it is important to note the
possibility of such when using NIST SRMs for lead
dust wipe analysis, since similar SRMs (e.g.,
Buffalo river sediment from Wyoming) do show
recoveries in the low 90% range (Ashley et al.,
1998).
-------
Because accurate and precise estimated
concentrations for the UC samples were imperative,
ORNL imposed the following data quality
requirements for the UC-prepared wipe samples: 1)
each estimated concentration had to be within a ±
10% interval of the target clearance level; 2)
additional quality control (QC) samples (at least 5%
of the total samples ordered) were to be prepared
and analyzed by UC as a quality check prior to
shipment of the samples; and 3) the relative standard
deviation of the QC samples had to be < 10%. It is
important to note here the reason why the data
quality requirements between the UC and ELPAT
samples were different. The data quality
requirements for the ELPAT samples (i.e., ± 20% of
the target value) were established by the ELPAT
program. Since archived samples were being used,
those data quality requirements could not be
changed.
As a quality check of the sample preparation
process, UC prepared an additional 24 samples (5%
of the total number ordered). UC extracted and
analyzed the samples following internal procedures
(nitric/hydrochloric acid extraction, followed by
atomic absorption spectrometry - see EPA, 1996 for
Method 3 05 OB and Method 601 OB) and provided
those results to ORNL. For the 24 samples (eight at
each of the three clearance levels), the average
percent recovery (i.e., UC measured
concentration/UC estimated concentration x 100%)
was 97% (median value = 96%, standard deviation =
3%, range = 93% to 102%). Additionally, 42
randomly-selected samples (14 at each of the three
clearance levels) were analyzed by the EPA Region
1 laboratory in North Chelmsford, MA, as an
independent quality control check of the accuracy
and precision of UC's sample preparation procedure
(nitric acid digestion followed by ICP/AES analysis
- EPA, 1996). The average percent recovery (EPA
Region 1 reported concentration/UC estimated
concentration x 100%) was 90% (median 89%,
standard deviation = 2%), with a range of values
from 86% to 93%. The average recovery determined
from the EPA Region 1 analyses (90%) was lower
than that which was determined by UC (102%), but
both values were within the data quality requirement
of 100 ± 10%. Based on these data, ORNL
determined that the UC sample preparation process
met the established data quality criteria and was
deemed acceptable for use in the determination of
false positive/false negative error rates.
Distribution and Number of Samples
A total of 160 samples were analyzed in the
verification test. Figure 3 is a plot containing the
distribution of the sample concentrations that were
analyzed in this study. Twenty samples were
prepared by the University of Cincinnati at +/- 10%
of each of the three clearance levels (3 test levels x
20 samples = 60 samples total). Research Triangle
Institute prepared 20 "blanks" at lead concentrations
< 2 |_lg/wipe. These samples are noted as such in
Figure 3. The remaining samples in Figure 3 are
ELPAT samples. For most of the ELPAT samples,
four samples were analyzed at each concentration
level (16 test levels x 4 samples each = 64 samples
total). There were two concentration levels (at 49
and 565 |_lg/wipe) where eight samples were
analyzed. While the set of samples at each
concentration level were prepared using
homogeneous source materials and an identical
preparation procedure, ELPAT samples cannot be
considered true "replicates" because each sample
was prepared individually. However, these samples
represent four samples prepared similarly at a
specified target concentration, with an estimated
value calculated from more than 100 analyses of
similarly prepared samples.
Sample Randomization
The samples were packaged in 20-mL plastic
scintillation vials and labeled with a sample
identifier. Each participant received the same suite
of samples, but in a randomized order. The samples
were distributed in batches of 16. Completion of
chain-of-custody forms documented sample transfer.
Description of Performance Factors
In Section 5, technology performance is described in
terms of precision, accuracy, completeness, and
comparability, which are indicators of data quality
(EPA, 1996). False positive and negative results,
sample throughput, and ease of use are also
described. Each of these performance characteristics
is defined in this section.
Precision
Precision is the reproducibility of measurements
under a given set of conditions. Standard deviations
estimated at each concentration level can be used to
establish the relationship between the uncertainty
and the average lead concentration. Standard
deviation (SD) and relative standard deviation
(RSD) for replicate results are used to assess
precision, using the following equation:
-------
25 -,
Clearance Levels at 40,250,400 ug/wipe
GO
01
Q.
S 15-
A
0
Number
o 01 o
^
-
?
8
Z>
5
5
S
oo
oo
£
(N
CD
(N
O
CN
(3)
ro
CN
O
f>
CN
CD
CN
o
(3)
O
•
-------
measurements. Bias was quantified by computing
the percent recovery for four similar samples or a
single sample using the equation:
percent recovery = [measured amount(s)/estimated
value] x 100% (Eq. 2)
Accuracy was assessed using both the ELPAT and
UC estimated concentrations. The comparison to the
ELPAT value represents how close the technology
reported results to the consensus value, which
represents the amount of "recoverable" lead in the
sample. Because the UC samples were prepared
gravimetrically from samples of known lead content,
the comparison to the UC samples represents how
close the technology reported results to an absolute
lead value. Comparison to the gravimetric values
reveals any bias imposed by the tested sampling and
analytical method.
The optimum percent recovery value is 100%.
Percent recovery values greater than 125% indicate
results that are biased high, and values less than
75% indicate results that are biased low. A small but
statistically significant bias may be detectable for a
field technology if precision is high (i.e., low
standard deviation). The field technology can still
have acceptable bias with an average percent
recovery in the interval of 75% to 125%. Bias
within the acceptable range can usually be corrected
to 100% by modification of calibration methods.
Comparability
Comparability refers to how well the field tech-
nology and the conventional laboratory data agree.
The difference between accuracy and comparability
is that accuracy is judged relative to a known value,
comparability is judged relative to the results of a
laboratory procedure, which may or may not report
the results accurately. Because true "replicates"
were not available for use in this study, the averages
from similar samples measured by the technology
were compared with corresponding averages
measured by the laboratory for all target
concentration levels.
A correlation coefficient quantifies the linear
relationship between two measurements (Draper and
Smith, 1981). The correlation coefficient is denoted
by the letter r; its value ranges from -1 to +1, where
0 indicates the absence of any linear relationship.
The value r = -1 indicates a perfect negative linear
relation (one measurement decreases as the second
measurement increases); the value r = +1 indicates a
perfect positive linear relation (one measurement
increases as the second measurement increases).
Correlation coefficients above 0.990 indicate good
linear agreement. The slope of the linear regression
line, denoted by the letter m, is related to r. Whereas
r represents the linear association between the
vendor and laboratory concentrations, m quantifies
the amount of change in the vendor's measurements
relative to the laboratory's measurements. A value
of+1 for the slope indicates perfect agreement.
Values greater than 1 indicate that the vendor results
are generally higher than the laboratory, while
values less than 1 indicate that the vendor results are
usually lower than the laboratory.
Detectable Blanks
Twenty samples in the test were prepared at < 2
|J,g/wipe, below the anticipated reporting limits of
both the field technologies and the laboratory. Any
reported lead for these samples is considered a
"detectable blank".
False Positive/Negative Results
A false positive (fp) result is one in which the
technology detects lead in the sample above a
clearance level when the sample actually contains
lead below the clearance level (Keith et al., 1996). A
false negative (fn) result is one in which the
technology indicates that lead concentrations are
less than the clearance level when the sample
actually contains lead above the clearance level. For
example, if the technology reports the sample
concentration to be 35 |_lg/wipe, and the true
concentration of the sample is 45 |_lg/wipe, the
technology's result would be considered a fn at the
40 |_lg/wipe clearance level. Accordingly, if the
technology reports the result as 45 |_lg/wipe and the
true concentration is 35 |_lg/wipe, the technology's
result would be a fp at the 40 |_lg/wipe clearance
level.
A primary objective for this verification test was to
assess the performance of the technology at each of
the three clearance levels of 40, 250, and 400
|J,g/wipe, and estimate the probability of the field
technology reporting a fp or fn result. For each
clearance level, the probabilities of fn were
estimated as curves that depend on a range of
concentrations reported about the clearance level.
These error probability curves were calculated from
the results on the 60 UC samples at concentrations ±
10% of each clearance level. In order to generate
-------
probability curves to model the likelihood of false
negative results, it was assumed that the estimated
concentration provided by UC was the true
concentration. However, this evaluation did not
include the gravimetric preparation uncertainty in
the UC estimated concentration. This error is likely
to be much smaller than other sources of
measurement error (e.g., extraction efficiency and
analytical).
The fp/fn evaluation also included a comparison to
the ELPAT sample results. The "estimated" value
for the UC and ELPAT samples are defined
differently. The UC value is based on weight of the
NIST-traceable material, while the ELPAT
estimated value is the average analytical reported
value from more than 100 accredited laboratories.
The UC sample estimated lead content is determined
gravimetrically, which should be closer to the "true"
concentration than an analytical measurement that
includes preparation and instrumental errors. In
contrast, determining the technology's fp/fn error
rates relative to the ELPAT estimated
concentrations represents a comparison to typical
laboratory values. One limitation of using the
ELPAT sample is that concentrations covered a
wider overall distribution of lead levels. Thus, the
availability of sample concentrations that were
tightly (i.e., +/- 10%) clustered about the clearance
levels was limited. In order to perform a broader
fp/fn analysis, the range of lead levels in the ELPAT
samples that bracketed the pertinent clearance levels
was extended to ± 25% of the target concentration.
Completeness
Completeness is defined as the percentage of
measurements that are judged to be usable (i.e., the
result is not rejected). An acceptable completeness
is 95% or greater.
Sample Throughput
Sample throughput is a measure of the number of
samples that can be processed and reported by a
technology in a given period of time. Sample
throughput is reported in Section 5 as number of
samples per day per number of analysts.
Ease of Use
A significant decision factor in purchasing an
instrument is how easy the technology is to use.
Several factors are evaluated and reported on in
Section 5:
• What is the required operator skill level (e.g.,
technician or advanced degree)?
• How many operators were used during the test?
• Could the technology be run by a single person?
• How much training would be required in order
to run this technology?
• How much subjective decision-making is
required?
Cost
An important factor in the consideration of whether
to purchase a technology is cost. Costs involved
with operating the technology and a typical
laboratory analyses are estimated in Section 5. To
account for the variability in cost data and
assumptions, the economic analysis is presented as a
list of cost elements and a range of costs for sample
analysis. Several factors affect the cost of analysis.
Where possible, these factors are addressed so that
decision makers can independently complete a site-
specific economic analysis to suit their needs.
Miscellaneous Factors
Any other information that might be useful to a
person who is considering purchasing the
technology is documented in Section 5 under
"Observations". Examples of information that might
be useful to a prospective purchaser are the amount
of hazardous waste generated during the analyses,
the ruggedness of the technology, the amount of
electrical or battery power necessary to operate the
technology, and aspects of the technology or method
that make it user-friendly or user-unfriendly.
-------
Section 4 — Laboratory Analyses
Background
EPA regulations (40 CFRPart 745.227(e)(8)(vii))
specify that residences and child occupied facilities
built before 1978 that have undergone an abatement
must pass clearance testing (CFR 2001). These EPA
regulations also state in 40 CFR Part 745.227(f)(2)
that dust samples for clearance must be analyzed by
a laboratory recognized by EPA (CFR 2001). Many
EPA-authorized state and tribal lead programs have
the same or similar requirements. EPA's vehicle for
recognizing laboratory proficiency is the National
Lead Laboratory Accreditation Program (NLLAP).
Although the NLLAP was initially designed to
accredit fixed site laboratories, in August 1996 the
NLLAP was modified so that mobile laboratory
facilities and testing firms operating portable testing
technologies could also apply for accreditation.
Despite this modification, the NLLAP list of
accredited laboratories has almost exclusively
consisted of fixed site laboratories. One possible
outcome of this ETV test is that more mobile
laboratory facilities and testing firms operating
portable testing technologies will apply for NLLAP
accreditation. In order to assess whether the field
portable technologies participating in this
verification test produce results that are comparable
to NLLAP-recognized data, an NLLAP-recognized
laboratory was selected to analyze samples
concurrently with the field testing.
NLLAP Laboratory Selection
NLLAP was established by the EPA Office of
Pollution Prevention and Toxics under the
legislative directive of Title X, the Lead-Based Paint
Hazard Reduction Act of 1992. In order for
laboratories to be recognized under the NLLAP,
they must successfully participate in the ELPAT
Program and undergo a systems audit. The
acceptable range for the ELPAT test samples is
based upon the reported values from participating
laboratories. Acceptable results are within three
standard deviations from the consensus value. A
laboratory's performance is rated as proficient if
either of the following criteria are met: (1) in the last
two rounds, all samples are analyzed and the results
are 100% acceptable; or (2) three-fourths (75%) or
more of the accumulated results over four rounds are
acceptable.
The NLLAP required systems audit must include an
on-site evaluation by a private or public laboratory
accreditation organization recognized by NLLAP.
Some of the areas evaluated in the systems audit
include laboratory personnel qualifications and
training, analytical instrumentation, analytical
methods, quality assurance procedures, and record
keeping procedures.
The list of recognized laboratories is updated
monthly. ORNL obtained the list of accredited
laboratories in July 2001. The list consisted of
approximately 130 laboratories. Those laboratories
which did not accept commercial samples and those
located on the U.S. west coast were automatically
eliminated as potential candidates. ORNL
interviewed at random approximately ten
laboratories and solicited information regarding
cost, typical turnaround time, and data packaging.
Based on these interviews and discussions with
technical panel members who had personal
experience with the potential laboratories, ORNL
selected DataChem (Cincinnati, OH) as the fixed-
site laboratory. As a final qualifying step, DataChem
blindly analyzed 16 samples (8 ELPAT and 8
prepared by UC) in a pre-test study. As shown in
Table 1 below, DataChem passed the pre-test by
reporting concentrations that were within 25% of
the estimated concentration for samples above the
reporting limit.
Laboratory Method
The laboratory method used by DataChem was hot
plate/nitric acid digestion, followed by inductively
coupled plasma-atomic emission spectrometry (ICP-
AES) analysis. The preparation and analytical
procedures, as supplied by DataChem, can be found
in the test plan (ORNL, 2001). To summarize the
procedure, the wipe was digested in 2 mL of nitric
acid, heated in a hotblock for 1 hour at 95 °C,
diluted to 20 mL with distilled water, and analyzed
by ICP-AES. DataChem's procedures are
modifications of Methods 3 05 OB and 601 OB of EPA
SW-846 Method Compendium for the preparation
and analysis of metals in environmental matrices
(EPA, 1996). Other specific references for the
preparation and analysis of dust wipes are available
from the American Society for Testing and
Materials (ASTM, 1998).
-------
Table 1. Summary of DataChem Pre-Test Results
Sample
Type
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
UC
UC
UC
UC
UC
UC
UC
UC
DataChem
Reported Cone
(|J,2/wipe)
<20
<20
41
44
190
210
440
450
<20
<20
25
38
150
200
250
310
Estimated
Cone
(|J,2/wipe)
2.12
2.12
41.3
41.3
201.6
201.6
408.7
408.7
10.3
5.9
29.9
44
172.4
237.5
327.3
379
Percent
Recovery
n/a
n/a
99%
107%
94%
104%
108%
110%
n/a
n/a
84%
86%
87%
84%
76%
82%
Analysis
Order
16
12
6
3
15
9
2
13
4
1
14
10
11
7
5
8
Laboratory Performance
ORNL validated all of the laboratory data according
to the procedure described in the verification test
plan (ORNL, 2001). During the validation, the
following aspects of the data were reviewed:
completeness of the data package, correctness of the
data, correlation between "replicate" sample results,
and evaluation of QC sample results. Each of these
categories is described in detail in the verification
test plan. An evaluation of the performance of the
laboratory results through statistical analysis of the
data was performed and is summarized below. (See
Section 3 for a detailed description of how the
performance factors are defined and the calculations
that are involved.)
In Table 2, DataChem's reported values are
compared to the estimated values to determine
percent recovery (i.e., accuracy of the DataChem
results) for both the ELPAT and the UC samples.
The results are also shown graphically in Figure 3.
The average percent recovery for the ELPAT
samples was 98%, while the average for the UC
samples was 91%. Both Table 2 and Figure 4
indicate that the analytical results from the
University of Cincinnati wipe samples were
generally reported lower than the estimated value,
while the results for the ELPAT samples were closer
to the estimated value. The better agreement with
the ELPAT samples is not unexpected, given that
the ELPAT estimated concentrations represent
analytical consensus values that include typical
extraction inefficiencies and instrumental error.
The negative bias observed with the UC and the
ELPAT samples was statistically significant. The
cause of the negative bias for the UC samples could
be related to: 1) extraction inefficiencies (due to the
use of NIST SRMs that contain lead that is
unrecoverable with the extraction procedure which
was used) and/or, 2) typical analytical variation due
to preparation and measurement errors. Another
indication of accuracy is the number of individual
ELPAT results which were reported within the
acceptance ranges that have been established for
those samples. For the 72 ELPAT samples (> 20
|-lg/wipe), DataChem reported 71 (99%) within the
acceptable ranges of values.
The precision assessment presented in Table 3
indicates that the analyses were very precise. The
average RSD for the ELPAT samples was 7%, while
the average RSD for the UC samples was 8%. The
variability of the UC sample preparation process,
provided for reference of the minimal achievable
RSD for the UC samples, was 6%. A single
estimate of the ELPAT variability was not
determined, since the ELPAT samples were
comprised of 20 different batches of samples.
DataChem reported all 20 detectable blank samples
correctly as < 20 |_lg/wipe. In addition, DataChem
reported seven of the eight samples with estimated
concentrations of either 16.9 |_lg/wipe or 17.6
|-lg/wipe as less than their reporting limit of 20
|-lg/wipe and only one was incorrectly reported as 30
|-lg/wipe.
10
-------
Table 2. Summary of DataChem Percent Recovery Values by Sample Source
Statistic
n a
average % recovery
standard deviation
minimum % recovery
maximum % recovery
ELPAT
72
98
9
81
143
uc
60
91
3
86
102
1 excludes estimated values <20 jig/wipe (n=28)
01]
500 n
400 -
300 -
200 -
100 H
D ELPAT (n=72)
• Univ of Cinci (n=60)
100 200 300 400
Estimated value (ug/wipe)
500
Figure 4. Plot of DataChem reported values versus estimated values, shown for concentrations
less than 500 |ig/wipe.
Table 3. Summary of DataChem Precision Estimates by Sample Source
Sample Source
ELPAT
UC
UC preparation
n
18 a
3b
3C
average RSD
7
8
6
Min RSD
2
6
6
Max RSD
21
9
7
" 4 replicates in each sample set
6 20 replicates in each sample set
c This value represents the variability in the sample preparation process.
11
-------
An important evaluation parameter for the analysis
of dust wipe samples is how the method performs at
the clearance levels and the method's likelihood of
reporting false positive (fp) and false negative (fn)
results. Recall from the experimental design that 20
UC samples were prepared at ± 10% of each
clearance level of 40, 250, and 400 |_lg/wipe, for a
total of 60 UC samples. The ELPAT samples
covered a wider range of concentrations. There was
a total of 40 ELPAT samples that fell within a ±25%
interval of the target values that could be used for the
fp/fn assessment. The number of false negative and
false positive results reported by DataChem relative
to the UC and ELPAT estimated concentrations is
summarized in Table 4. There are a specific number
of possible fp and fn results. For example, if the
estimated lead level on the wipe is less than the
clearance level (CL), then it is not possible to
produce a false negative result; only a false positive
(i.e., > 40) result is possible. For the UC samples, in
every case where the estimated concentration was
less than the CL, DataChem reported a result for that
was also less than the CL, indicating no fp results at
any of the three CL. DataChem reported two fp
results for the ELPAT samples out of a possible 12.
When the estimated concentration was above the
clearance level, however, DataChem sometimes
reported results as less than the clearance level.
DataChem reported a higher rate of fn results for the
UC samples than the ELPAT samples (23 of 30 vs 7
of 28 possible fn results, respectively). This finding
is not surprising, since the results reported above
indicated that DataChem's results were negatively
biased, or reported lower than the estimated values
for the UC samples. As stated in Section 3, it is
important to note that in this evaluation, the
estimated concentration of the UC samples is
assumed to be the "true"concentration, and the
uncertainty in gravimetric preparation for the UC
estimated concentration is not considered in the
evaluation.
Figures 5, 6, and 7 show models of the likelihood of
DataChem reporting a false negative result at each of
the clearance levels versus the true concentrations of
the UC samples. (Note that only the UC samples
must be used in generation of probability curves
because these estimated values are a closer
representation of the true lead concentration than the
ELPAT estimated concentration. See Song et al.,
2001.) These figures indicate that the likelihood of
DataChem reporting false negative results for the UC
samples at the exact clearance level is high, near
100% in all three cases. This means, for example,
that if DataChem reported a value as exactly 250
|-lg/wipe, the probability that the true concentration
is >250 is essentially 100%. Again, this is due to the
negative bias that was observed in the measurement
of the UC samples. The plots also demonstrate that,
due to the relatively high level of precision of results
reported by DataChem, the performance is very
minimally impacted by performing replicate
analyses, as the distribution of false negative
probabilities is very similar whether 1 or 5
measurements (in Figures 5, 6, and 7, delineated as
N = 1, N = 2, etc.) are performed. The interpretation
of these curves for use in a "real-world" situation
can be demonstrated by the following example.
Suppose that a user decides that an acceptable level
of risk for having false negative results is 5%. Using
Figure 5, 5% FN probability (y = 0.05) corresponds
to a "true" lead concentration of 46 |_lg/wipe
(meaning if the true concentration of the sample is
46 |_lg/wipe, there is only a 5% chance/risk that
DataChem will report the value as < 40 |_lg/wipe.)
By plotting DataChem's measured values versus the
estimated concentrations, the equations of the linear
regression lines can be calculated for each of the
three CL. The slope, intercept, and correlation
coefficient for the ELPAT and UC samples are
presented in Table 5. The user might like to know at
what reported value (and at what associated
probability) will DataChem be likely to report a
"clean" sample (i.e., there is a high probability that
the true concentration is < CL). For example, for the
UC samples, we know that a value reported by
DataChem as 39 |_lg/wipe is biased low and will have
a true concentration of > 40 (41.8 |_lg/wipe, using the
linear regression equation in Table 5). A true
concentration of 40 |_lg/wipe for a UC sample would
correspond to a reported value rounded to the nearest
whole number of 37 |_lg/wipe (see Table 5). For an
ELPAT sample, a true concentration of 40 |_lg/wipe
corresponds to a DataChem reported value of 40
|J,g/wipe, because the negative bias was not as large
for the ELPAT samples. Estimates of the reported
concentration at the 250 and 400 |_lg/wipe levels are
reported in Table 5. In both cases, the reported
concentrations for the ELPAT samples are higher
(i.e., closer to the clearance level) than those of the
UC samples.
The user is reminded that the data obtained during
this verification test represent performance at one
12
-------
point in time. The data produced by DataChem at
some other time after the writing of this report may
or may not be similar to what has been produced
here. To understand a method's performance at
critical clearance levels, it is recommended that the
user perform their own assessment of the method's
performance by including samples of known
concentration (at or near the clearance levels) along
with the analysis of "real-world" samples.
Table 4. False Positive/False Negative Results for DataChem Measurements of UC Samples
Evaluation Parameter
fp: # samples where
DataChem reported the result
as > CLa of the # samples
where the estimated
concentration was < CL
fn: # samples where
DataChem reported the result
as < CL of the # samples
where the estimated
concentration was > CL
Sample
Source
UC
ELPAT
UC
ELPAT
Number of Samples
40 |_lg/wipe
Oof 9
Oof 4
5 of 11
lof 12
250 |J,g/wipe
Oof 11
2 of 8
9 of 9
5 of 8
400 |J,g/wipe
Oof 10
OofOb
9 of 10
lof 8
Total
Oof 30
2 of 12
23 of 30
7 of 28
1 CL = clearance level
b Because all eight ELPAT values were above 400 jig/wipe, no samples were available to assess fp results at this level.
1.0 -
True Pb Concentration (ug/wipe)
Figure 5. False negative probabilities for DataChem average concentrations at a target
concentration level of 40 |ig/wipe.
13
-------
250
260
270
280
290
300
310
320
True Pb Concentration (ug/wipe)
Figure 6. False negative probabilities for DataChem average concentrations at a target
concentration level of 250 |_ig/wipe.
0.0 -I
400
410 420 430 440 450 460 470 480 490 500
True Pb Concentration (ug/wipe)
Figure 7. False negative probabilities for DataChem average concentrations at a
target concentration level of 400 |ig/wipe.
14
-------
Table 5. Summary of the Linear Regression Constants and Recovery Data for DataChem's
Measurements Versus the Estimated Concentrations at the Clearance Levels
Evaluation Parameter
n
slope
intercept
correlation coefficient
average % recovery
SD of % recovery
Reported concentration at
theCL
40 |_lg/wipe
UC
20
1.021
-3.673
0.884
93%
4%
37
l-ig/wipe
ELPAT
16
1.612
-6.182
0.840
101%
13%
40
l-ig/wipe
250 |J,g/wipe
UC
20
0.829
18.557
0.879
90%
3%
226
l-ig/wipe
ELPAT
16
0.578
90.826
0.549
96%
9%
234
l-ig/wipe
400 |J,g/wipe
UC
20
0.736
67.649
0.861
91%
3%
362
l-ig/wipe
ELPAT
8
2.394
-575.771
0.492
100%
5%
382
l-ig/wipe
15
-------
Section 5 — Technology Evaluation
Objective and Approach
The purpose of this section is to present a statistical
evaluation of the Pb-Test XRF data and determine
the technology's ability to measure lead in dust wipe
samples. This section includes an evaluation of
comparability through a one-to-one comparison with
NLLAP-recognized laboratory data. Other aspects of
the technology (such as accuracy, precision, cost,
sample throughput, hazardous waste generation, and
logistical operation) are also evaluated in this section.
The Appendix contains the raw data provided by the
vendor during the verification test that were used to
assess the performance of the Pb-Test.
Precision
Precision is the reproducibility of measurements
under a given set of conditions. Precision was
determined by examining the results of blind analyses
for replicate samples with estimated concentrations
greater than 2 |_lg/wipe. For the ELPAT samples,
precision was measured on each set of four samples
from a particular round of archived samples. For the
20 sets of ELPAT samples, the Pb-Test's average
RSD value was 18%, with a range from 7 to 54%,
indicating that the Pb-Test's measurements were, on
average, precise. For the UC samples, 20 samples
were analyzed at three target concentration levels of
40, 250, and 400 |_lg/wipe. The average precision of
the UC sample measurements by the Pb-Test was
15% RSD. The measurements above 200 |_lg/wipe
(average RSD = 14%) were more precision than the
results for the samples with concentrations below 200
|J,g/wipe (average RSD = 21%). With the expectation
that UC was to prepare the samples as close to the
target concentrations as possible, the allowable
variability was 10% RSD. The actual variability of
the UC preparation process was an average of 6%
RSD.
Accuracy
Accuracy represents the closeness of the Pb-Test's
measured concentrations to the estimated content of
spiked samples. A plot of the Pb-Test percent
recovery values versus the average estimated
concentration for both UC and ELPAT samples is
provided in Figure 8. A percent recovery value of
100% indicates that there is no difference between
Table 6. Precision of the Pb-Test Instrument
Source
ELPAT
UC
> 200 |ig/wipe
< 200 |ig/wipe
UC prep c
No. of
sample
sets
20 a
3b
12
11
o
6
% RSD
avg
18
15
14
21
6
min
7
10
7
7
4
max
54
19
21
54
7
" 4 replicates in each sample set
6 20 replicates in each sample set
c precision of UC sample preparation process
the Pb-Test result and the estimated value. Table 7
contains the summary of the percent recovery values
for the ELPAT and UC samples, including an
evaluation of concentrations above and below 200
|J,g/wipe. The Pb-Test results were biased high for
concentrations below 200 |_lg/wipe (average %
recovery = 282% for ELPAT samples and 300% for
UC samples), with the amount of bias consistently
decreasing as concentration increased (Figure 8).
Above 200 |_lg/wipe, the Pb-Test results were
unbiased (average % recovery = 96% for the
ELPAT samples and 102% for the UC samples).
The average % recovery values were not statistically
different than 100%. The results at higher
concentration levels (800 and 1,500 |_lg/wipe)
indicated a negative bias (78% and 72%,
respectively), but too few sample results existed at
those concentrations to definitely say that the
technology was biased low above 800 |_lg/wipe.
Another measure of accuracy is the number of
results for the ELPAT samples that were reported
within the individual acceptance ranges that have
been established for those samples. Of the 80
ELPAT samples, the Pb-Test reported 34 of the
results (43% of the total) within the acceptance
ranges. If only the ELPAT samples above 200
|J,g/wipe are considered (40 samples), 32 samples
(80%) are within the ELPAT acceptance limits.
16
-------
Comparability
Comparability refers to how well the Pb-Test and the
NLLAP-recognized laboratory data agreed. In this
evaluation, the laboratory results are not presumed to
be the "correct" answers. Rather, these results
represent what a typical fixed laboratory would
report for these types of samples. A direct
comparison of the Pb-Test results and the laboratory
results was performed for all ELPAT and UC
samples. In Figures 9 and 10, the Pb-Test average
concentration is plotted versus the DataChem average
concentration in the range from 0 to 200 |_lg/wipe and
200 to 1500 |_lg/wipe, respectively. Note that, for this
Table 7. Accuracy of Pb-Test Instrument
evaluation only, the ELPAT and UC samples are
combined because there are only 3 UC data points.
For concentrations less than 200 |_lg/wipe (Figure 9),
the results agree in a linear fashion (m = 1.060, r =
0.967), but are biased high (intercept = 65.6). The
average Pb-Test measurements were in fairly good
agreement with the average laboratory results
(Figure 10) for concentrations greater than 200
|J,g/wipe (r = 0.989), but the slope was statistically
different than 1.00 (m = 0.662).
Statistic
na
average % recovery
standard deviation
minimum % recovery
maximum % recovery
ELPAT
All
80
189
154
60
753
<200
|-lg/wipe
40
282
173
65
753
>200
|-lg/wipe
40
96
20
60
141
UC
All
60
168
102
78
485
<200
|-lg/wipe
20
300
64
203
485
>200
|-lg/wipe
40
102
18
78
145
1 Excludes "detectable blank" samples at concentrations < 2 |_ig/wipe
800 -,
700 -
o
500 -
400 -
oT 300 -
&B
OS
w 200 H
100
0
200 400 600 800 1000 1200
Average estimated value (ug/wipe)
1400 1600
Figure 8. Plot of the Pb-Test's average percent recovery values (Pb-Test
result/estimated valuer 100%) versus the average estimated values for all
concentrations and sample types (n=23). The dashed line represents 100% recovery -
the value at which the Pb-Test result and the estimated value agree.
17
-------
20
40
60
80
100
120
140
160
180
DataChem Average Pb (ug/wipe)
Figure 9. Plot of the Pb-Test average concentrations versus DataChem average concentrations
for all UC and ELPAT samples with positive DataChem concentrations less than 200 |ig/wipe
(n=9). The equation of the linear regression line is y = 1.060x + 65.586, correlation coefficient r:
0.967.
200 |ig/wipe (n=12). The equation of the linear regression line is
y = 0.662x + 121, r = 0.989.
18
-------
Detectable Blanks
Of the samples that were prepared at < 2 |_lg/wipe,
the Pb-Test reported all 20 samples as positive for
lead, with concentrations ranging from 46 to 137
|-lg/wipe. Key Master did not provide a reporting
limit for the instrument during the verification test.
False Positive/False Negative Results
Similar to the evaluation described and presented in
Section 4 for DataChem, the number of false
negative and false positive results reported by the
Pb-Test relative to the estimated concentrations for
both the ELPAT and UC samples are summarized in
Table 8. In about half of the cases where the
estimated concentration was less than the clearance
level (CL), the Pb-Test reported a result that was
greater than the CL (20 of 38 possible fp results for
the UC samples, and 6 of 12 fp results for the
ELPAT samples at the three CLs). When the
estimated concentration was equal to or greater than
the clearance level, the Pb-Test reported some as less
than the CL, but most of the results as greater than
the CL (7 of 22 possible fn results for UC samples
and 8 of 28 fn results for ELPAT samples). About
half of the fp results were at the 40 |_lg/wipe level,
which is not surprising, based on the findings
reported above that indicated that the Pb-Test results
were positively biased, or reported more than the
estimated values, for concentrations < 200 |_lg/wipe.
Table 9 presents the linear regression constants for
Pb-Test measured concentrations versus estimated
concentrations for the three CLs. The average
recoveries indicate that the Pb-Test results were
significantly positively biased at the 40 |_lg/wipe
clearance level for both ELPAT and UC samples.
The results were more accurate and precise at the
250 and 400 |_lg/wipe clearance levels.
For all UC samples, the slopes are not significantly
different than zero at the 5% significance level. The
non-significant slopes indicate that the false negative
probabilities for the Pb-Test are constant (i.e.,
horizontal lines) over the concentration ranges
examined. In Figures 11, 12 and 13, false negative
probabilities at the three clearance levels are
compared for DataChem and Pb-Test results. In
these figures, the two-sided 90% confidence
intervals (not shown for clarity) are used to
express uncertainty on the false negative curves.
In the three figures, the 90% confidence intervals for
the two methods only overlap for parts of the true
lead concentration ranges (greater than 43 |_lg/wipe
for the 40 |_lg/wipe clearance level; from 257
|-lg/wipe to 334 |_lg/wipe for the 250 |_lg/wipe
clearance level; and from 389 |_lg/wipe to 500
|J,g/wipe for the 400 |_lg/wipe clearance level). Both
the constant probabilities for Pb-Test (i.e., horizontal
lines) and the partial overlapping of the confidence
intervals indicate that the Pb-Test false negative
error rates are not comparable to DataChem's false
negative error rates for all ranges of true lead
concentrations in the three figures.
Once again, the reader is reminded that the fp/fn
evaluation reported herein is based on the
instrument's performance during this verification
test. Results produced under different conditions and
with different samples may or may not be similar.
Regardless of analytical technique, there is some
uncertainty in assessing false positive and false
negative error rates around critical action levels due
to "normal" levels of variability (Song et al., 2001).
Analytical values falling near the level of interest
should be interpreted with care for both fixed-
laboratory and field-based analytical methods.
19
-------
Table 8. False Positive/False Negative Error Rates for Pb-Test Measurements
Evaluation Parameter
fp: # samples where Pb-Test
reported the result as > CLa of
the # samples where the
estimated concentration was <
CL
fn: # samples where Pb-Test
reported the result as < CL of
the # samples where the
estimated concentration was >
CL
Sample
Source
UC
ELPAT
UC
ELPAT
Number of Samples
40 |_lg/wipe
llof 11
3 of 4
Oof 9
Oof 12
250 |J,g/wipe
6 of 11
3 of 8
3 of 9
3 of 8
400 |J,g/wipe
3 of 16
OofOb
4 of 4
5 of 8
Total
20 of 38
6 of 12
7 of 22
8 of 28
a CL = clearance level
b Because all eight ELPAT values were above 400 |_ig/wipe,
no samples were available to assess fp results at this level.
Tme Pb Concentration (tig/wipe)
Figure 11. Comparison of the false negative probabilities for the
Pb-Test and DataChem average concentrations at a target
concentration of 40 |_ig/wipe.
20
-------
Tme Pb Concentration (ugAwipe)
Figure 12. Comparison of the false negative probabilities for the Pb-
Test and Data Chem average concentrations at a target concentration of
250 |ig/wipe.
True Pb Concentration (ugA/vipe)
Figure 13. Comparison of the false negative probabilities for the Pb-
Test and DataChem average concentrations at a target concentration
of 400 |ig/wipe.
-------
Table 9. Summary of the Linear Regression and Recovery Data for the Pb-Test Response versus the
Estimated Concentrations
Evaluation Parameter
n
slope
intercept
correlation coefficient
average % recovery
SD of % recovery
Reported concentration at
theCL
40 |_lg/wipe
UC
20
-0.490
137.304
-0.064
300%
64%
118
l-ig/wipe
ELPAT
16
2.090
15.467
0.564
247%
73%
99
l-ig/wipe
250 |J,g/wipe
UC
20
-0.358
364.570
-0.104
112%
19%
275
l-ig/wipe
ELPAT
16
0.089
231.458
0.060
107%
19%
254
l-ig/wipe
400 |J,g/wipe
UC
20
0.352
224.482
0.159
93%
10%
365
l-ig/wipe
ELPAT
8
11.634
-4405.34
0.640
98%
21%
248
l-ig/wipe
Completeness
Completeness is defined as the percentage of
measurements that are judged to be usable (i.e., the
result was not rejected). Valid results were obtained
by the technology for all 160 dust wipe samples.
Therefore, completeness was 100%.
Sample Throughput
Sample throughput is representative of the estimated
amount of time required to prepare and analyze the
sample and perform the data analysis. Operating in
the field, the two-person KeyMaster team
accomplished a sample throughput rate of
approximately eighty samples per 10-hour day for
the 160 dust wipe analyses. One person did sample
preparation and the other did sample analysis. Two
instruments were used, with one analyst analyzing
all samples on both, and the average reading from
the two instruments reported as the final result.
Ease of Use
Two operators were used for the test because of the
number of samples and the working conditions, but
the technology can be operated by a single person.
Users unfamiliar with the technology may need
approximately one day of training to operate the
instrument. No particular level of educational
training is required for the operator. Both operators
were company experts. All samples were analyzed
on two instruments during the test, with the average
result reported. This was done as a test of variability
between instruments that KeyMaster elected to
perform. A single instrument is intended to be used
in typical applications.
Cost Assessment
The purpose of this economic analysis is to estimate
the range of costs for analysis lead in dust wipe
samples using the Pb-Test XRF and a typical
laboratory method. The analysis was based on the
results and experience gained from this verification
test, costs provided by ED AX, KeyMaster's
distributor, and representative costs provided by the
laboratory to analyze the samples. To account for the
variability in cost data and assumptions, the
economic analysis is presented as a list of cost
elements and a range of costs for sample analysis by
the XRF instrument and by the laboratory. Costs
were prepared at the time this report was written and
are subject to change.
Several factors affected the cost of analysis. Where
possible, these factors were addressed so that
decision makers can complete a site-specific
economic analysis to suit their needs. The following
categories are considered in the estimate:
• sample shipment costs,
• labor costs, and
• equipment costs.
Each of these cost factors is defined and discussed
and serves as the basis for the estimated cost ranges
presented in Table 10. This analysis assumed that the
individuals performing the analyses were fully
trained to operate the technology. Costs for sample
acquisition and pre-analytical sample preparation,
tasks common to both methods, were not included in
this assessment.
22
-------
Table 10. Estimated analytical costs for lead dust wipe samples
Analysis method:
Analyst/manufacturer:
Sample throughput:
Cost category
Sample shipment
Labor
Rate
Pb-Test XRF
KeyMaster Technologies
80 samples/day
Cost ($)
0
50-100/h per analyst
Equipment
Mobilization/demobilization 0
Instalment purchase price
Instrument lease price
Reagents/supplies
Re-sourcing 15-24 months
Waste Disposal
15,130
338 per month
0.70 per sample
2,600
Ob
Analysis method:
NLLAP Laboratory:
Actual turnaround:
Cost category
Sample shipment
Labor
Overnight shipping
Labor
Rate
Equipment
Waste Disposal
EPASW8466010b
DataChem
18 working days
Cost ($)
100-200
50-100
30 per sample
Included a
Included
""Included" indicates that the cost is included in the labor rate.
b There was no cost to dispose of hazardous waste from the verification test because KeyMaster kept the "used" wipes.
Pb-Test XRF Costs
The costs associated with using the instrument
included labor and equipment costs. No sample
shipment charges were associated with the cost of
operating the instrument because the samples were
analyzed on site.
Labor
Labor costs included on-site labor to perform the
analyses. The cost of the on-site labor was estimated
at a rate of $50-100/h, depending on the required
expertise level of the analyst. This cost element
included the labor involved during the entire
analytical process, comprising sample preparation,
sample management, analysis, and reporting. If the
user would have to travel to the site, the cost of
mobilization and demobilization, travel, and per
diem expenses should also be considered. However,
in a typical application where the Pb-Test might be
used, the analysis would usually be carried out by a
person located on site.
Equipment
Equipment costs included mobilization and
demobilization, purchase of equipment, and the
reagents and other consumable supplies necessary to
complete the analysis.
Mobilization and demobilization. This included
the cost of shipping the equipment to the test
site. For this verification test, the cost of
shipping equipment and supplies was estimated
at $150.
Instrument purchase. The instrument can be
purchased from ED AX for $15,130. The
instrument comes standard with an 18-month
factory warranty (excluding the source) to be
free of manufacturer's defects, dust wipe
analysis kit (100 wipes, 500 sample holders,
calibration standard, sample compressor, and
dust wipe block), off-line report generation
software, two rechargeable batteries, charger,
manuals, and carrying case. The instrument can
be leased from ED AX for as low as $338 per
month (with first and last month's payments in
advance), but rates are dependent on customer's
credit. With regards to training, the unit comes
standard with a flowchart showing how to use
the product as well as a detailed user manual.
On-site training by ED AX is available at a fee.
Reagents and supplies. Dust wipe holders are the
only consumable item and can be purchased for
$350 for a set of 500. Lynx dustwipes can be
23
-------
purchased for $288 forlOO wipes. Re-sourcing
cost is about $2,600. Key Master recommends re-
sourcing every 15 to 24 months, depending on
the user's needs.
Laboratory Costs
Sample Shipment
The costs of shipping samples to the laboratory
included overnight shipping charges as well as labor
charges associated with the various organizations
involved in the shipping process.
• Labor. This cost element included all of the
tasks associated with shipping the samples to the
NLLAP laboratory. Tasks included packing the
shipping coolers, completing the chain-of-
custody documentation, and completing the
shipping forms. The estimate to complete this
task ranged from 2 to 4 h, at $50 per hour.
• Overnight shipping. The overnight express
shipping service cost was estimated to be $50 -
100 for two boxes of samples.
Labor, Equipment, and Waste Disposal
The labor quotes from commercial analytical
laboratories that offered to perform the analyses for
this verification test ranged from $20 to $30 per
sample, with turnaround time estimates ranging from
7 to 14 days. Some laboratories can provide a 1-2
day turnaround, but the quick turnaround time was
not necessary for this test. The quotes were
dependent on many factors, including the perceived
difficulty of the sample matrix, the current workload
of the laboratory, data packaging, and the
competitiveness of the market. This rate was a fully
loaded analytical cost that included equipment,
labor, waste disposal, and report preparation. The
cost for DataChem to analyze samples for this
verification test was $30 per sample, with a
turnaround time of 18 working days.
Cost Assessment Summary
An overall cost estimate for use of the Pb-Test
versus use of the NLLAP- laboratory was not made
because of the extent of variation in the different
cost factors, as outlined in Table 10. The overall
costs for the application of any technology would be
based on the number of samples requiring analysis,
the sample type, and the site location and
characteristics. Decision-making factors, such as
turnaround time for results, must also be weighed
against the cost estimate to determine the value of
the field technology's providing immediate answers
versus the laboratory's provision of reporting data
within 18 days of receipt of samples.
Miscellaneous Factors
The following are general observations regarding the
field operation and performance of the Pb-Test XRF
instrument:
KeyMaster Technologies is the manufacturer
and developer of this instrument. ED AX is the
distributor who markets and sells the technology.
The instrument required no electrical power. The
instrument worked continuously through at least
6 hours of each workday before the battery was
changed.
The KeyMaster analyst was ready for the first
set of samples within 30 minutes of arriving on
site.
The KeyMaster analyst took one five-minute
reading for each sample. The sample was
analyzed on two separate instruments and the
results were averaged to produce the final result.
KeyMaster elected to analyze the samples on
two instruments as a study of precision variation
between instruments.
This instrument can be used to detect and
quantify multiple metals, although the
instrument's performance for lead is the only
metal that was verified in this test.
Each wipe had to be cut to one-fourth of its
original size, according to the KeyMaster
method. This may have introduced some error in
the measurements, as some dust might have been
lost in this process, but no systematic errors were
detected in the data that could be attributed to
this practice. With the purchase of an instrument,
KeyMaster/EDAX will provide the appropriately
sized wipe with the instrument, so this step
would be avoided.
The instrument contains a radioactive source.
The source was 4-5 months old at the time of the
test. As the source decays, it may be necessary to
increase the time of the measurement to obtain
the same precision that was obtained with the
source in this evaluation.
Licensing requirements vary by state and ED AX
offers contact information and guidance with
instrument purchase or lease.
This method did not generate any waste, other
than the used dust wipe samples.
It is recommended that KeyMaster/EDAX be
24
-------
contacted with any specific questions (such as
transportation issues) that a user might have.
Summary of Performance
A summary of performance is presented in Table 11.
Note that performance is based on the specific
protocols employed for this verification test. If
different testing protocols are used, different
performance results may be obtained.
The verification test found that the Pb-Test
instrument was relatively simple for a trained analyst
to operate in the field, requiring less than an one-half
hour for initial setup. The sample throughput of the
Pb-Test was eighty samples per day with two
operators and two instruments.
The overall performance of the Pb-Test instrument
for the analysis of lead in dust wipe samples was
characterized as having acceptable precision, biased
high for concentrations below 200 |_lg/wipe, but
unbiased for concentrations greater than 200
|J,g/wipe.
ORNL and ETV remind the reader that, while the
ETV test provides valuable information in the form
of a snapshot of performance, state, tribal, or federal
requirements regarding the use of the technologies
(such as NLLAP recognition for analysis of
clearance samples where required) need to be
followed.
25
-------
Table 11. Performance Summary for the Pb-Test Instrument
Feature/parameter
Precision : average RSD
Accuracy: average %
recovery
Positive results on "detectable
blank" samples (< 2 jag/wipe)
False positive results
False negative results
Comparison
with NLLAP-
recognized
laboratory
results
(excluding < 25
l-ig/wipe
samples)
slope
intercept
correlation
coefficient
Overall evaluation
Completeness
Size and Weight
Sample throughput
(2 analysts and 2 instruments)
Power requirements
Training requirements
Cost
Waste generated
Performance summary
UC Samples
15%
All samples: 168%
> 200 |_ig/wipe only: 102%
n/a
DataChem
Total: Oof 30
0 at 40 |_ig/wipe
0 at 250 |_ig/wipe
0 at 400 |_ig/wipe
DataChem
23 of 30
5 at 40 |_ig/wipe
9 at 250 |_ig/wipe
9 at 400 |_ig/wipe
Pb-Test
Total: 20 of 38
1 1 at 40 |_ig/wipe
6 at 250 |_ig/wipe
3 at 400 |_ig/wipe
Pb-Test
Total: 7 of 22
0 at 40 |_ig/wipe
3 at 250 |_ig/wipe
4 at 400 |_ig/wipe
ELPAT Samples
18%
All samples: 189%
> 200 |_ig/wipe only: 96%
20 of 20 samples
DataChem
2 of 12
0 at 40 |_ig/wipe
2 at 250 |_ig/wipe
0 at 400 |_ig/wipe
DataChem
7 of 28
1 at 40 |_ig/wipe
5 at 250 |_ig/wipe
1 at 400 |_ig/wipe
Pb-Test
Total: 6 of 12
3 at 40 |_ig/wipe
3 at 250 |_ig/wipe
0 at 400 |_ig/wipe
Pb-Test
Total: 8 of 28
0 at 40 |_ig/wipe
3 at 250 |_ig/wipe
5 at 400 |_ig/wipe
< 200 |_ig/wipe samples only: 1.060 (standard error = 0. 105)
> 200 |_ig/wipe samples only: 0.662 (standard error = 0.031)
< 200 |_ig/wipe samples only: 66 (standard error =
> 200 |-ig/wipe samples only: 121 (standard error =
8.416)
17.997)
< 200 |_ig/wipe samples only: 0.967
> 200 |-ig/wipe samples only: 0.989
- Positive bias for concentrations < 200 |_ig/wipe
- Unbiased for concentrations > 200 |_ig/wipe
- Acceptable Precision
- Linear relationship to the NLLAP lab results for > 200 |_ig/wipe
- Some fn results, but higher number of fp results
100% of 160 dust wipe samples
4" x 10" x 8"; 5.61
Ibs
80 samples/10-hr day
rechargeable Nickel-Cadmium battery
on-site provided for a fee (contact ED AX)
Purchase: $15,130
Lease: $338 per month (based on customer's credit)
Reagents/Supplies: $0.70 per sample
Annual re-sourcing: $2,600
none
26
-------
Section 6 — References
American Society for Testing and Materials. 1996. "Specification E1792-96a: Standard Specification for
Wipe Sampling Materials for Lead in Surface Dust" in ASTM Standards on Lead Hazards Associated with
Buildings. ASTM: West Conshohocken, PA.
American Society for Testing and Materials. 1998. "Practice El644: Standard Practice for Hot Plate
Digestion of Dust Wipe Samples for the Determination of Lead" in ASTM Standards on Lead Hazards
Associated with Buildings. ASTM: West Conshohocken, PA.
Ashley, K., K. Mapp, and M. Millson. 1998. "Ultrasonic Extraction and Field-Portable Anodic Stripping
Voltammetry for the Determination of Lead in Workplace Air Samples." AIHA Journal. 59(10), 671-679.
Code of Federal Regulations. 2001. "Identification of Dangerous Levels of Lead", Final Rule, 40 CFR Part
745,January.
Draper, N. R., and H. Smith. 1981. Applied Regression Analysis. 2nd ed. John Wiley & Sons, New York.
EPA (U.S. Environmental Protection Agency). 1996. "Method 3050B-1: Acid Digestion of Sediment, Sludge,
and Soils." In Test Methods for Evaluating Solid Waste: Physical/Chemical Methods, Update II. SW-846.
U.S. Environmental Protection Agency, Washington, D.C., December.
EPA (U.S. Environmental Protection Agency). 1996. "Method 6010B-1: Inductively Coupled Plasma-Atomic
Emission Spectrometry." In Test Methods for Evaluating Solid Waste: Physical/ Chemical Methods, Update
II. SW-846. U.S. Environmental Protection Agency, Washington, D.C., December.
Keith, L.H., G. L. Patton, D.L. Lewis and P.O. Edwards. 1996. Chapter I: Determining What Kinds of
Samples and How Many Samples to Analyze, pp. 19. In Principles of Environmental Sampling. Second
Edition. Edited by L. H. Keith, ACS Professional Reference Book, American Chemical Society, Washington,
DC.
ORNL (Oak Ridge National Laboratory). 1998. Quality Management Plan for the Environmental Technology
Verification Program's Site Characterization and Monitoring Technologies Pilot. QMP-X-98-CASD-001,
Rev. 0. Oak Ridge National Laboratory, Oak Ridge, Tenn., November.
ORNL (Oak Ridge National Laboratory). 2001. Technology Verification Test Plan: Evaluation of Field
Portable Measurement Technologies for Lead in Dust Wipes. Chemical Sciences Division, Oak Ridge
National Laboratory, Oak Ridge, Tenn., November.
Song, R., P. Schlecht, and K. Ashley. 2001. "Field Screening Test Methods: performance criteria and
performance characteristics." Journal of Hazardous Materials. 83, 29-39.
27
-------
Appendix
KeyMaster's Pb-Test XRF Results Compared with Laboratory Results
Sample
Analysis
Order
6
121
110
117
128
145
32
106
113
65
126
133
53
10
157
158
30
82
67
35
62
48
154
105
49
9
141
120
16
47
55
118
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
KeyMaster Pb-Test
Result
l-ig/wipe
136.87
79.85
53.55
69.76
69.46
97.3
60.11
50.04
45.62
72.15
45.51
89.31
46.79
117.38
94.23
107.47
72.36
76.22
98.68
65.85
117.99
89.43
127.25
116.16
109.83
22.05
99.24
81.91
115.89
19.29
82.65
84.52
Estimated
l-ig/wipe
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
16.9
16.9
16.9
16.9
17.6
17.6
17.6
17.6
29.8
29.8
29.8
29.8
DataChem
Result
l-ig/wipe
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
<20
30
<20
<20
<20
33
26
28
28
Estimated
l-ig/wipe
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
1.3
16.9
16.9
16.9
16.9
17.6
17.6
17.6
17.6
29.8
29.8
29.8
29.8
28
-------
Sample
Analysis
Order
71
3
45
57
131
125
18
75
41
40
92
146
155
94
156
50
17
46
151
38
43
26
98
61
112
80
68
51
136
22
86
135
160
33
111
20
Source
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
KeyMaster Pb-Test
Result
|ig/wipe
106.91
124.67
128.72
131.22
141.59
105.27
126.56
115.23
99.57
97.29
111.74
175.12
141.61
79.29
142.92
99.72
111.83
87.65
116.12
115.13
97.69
133.55
130.94
66.83
125.15
120.36
101.17
97.22
119.55
106.95
136.94
123.47
163.99
138.00
148.24
83.29
Estimated
|ig/wipe
40.2
36.7
45.0
41.0
39.3
39.4
44.3
40.7
43.1
37.6
35.4
36.1
36.1
39.0
40.4
37.4
36.6
38.6
40.1
44.4
41.3
41.3
41.3
41.3
49.0
49.0
49.0
49.0
49.1
49.1
49.1
49.1
58.6
58.6
58.6
58.6
DataChem
Result
|ig/wipe
33
32
31
29
32
38
37
36
37
37
33
41
32
38
30
35
36
31
34
34
37
42
44
41
43
52
49
48
70
54
48
44
64
55
56
52
Estimated
|ig/wipe
35.4
35.7
38.5
36.4
35.1
40.7
39.4
41.0
41.0
38.8
39.3
44.7
36.0
44.7
39.9
37.5
37.4
36.7
35.8
39.7
41.3
41.3
41.3
41.3
49.0
49.0
49.0
49.0
49.1
49.1
49.1
49.1
58.6
58.6
58.6
58.6
29
-------
Sample
Analysis
Order
116
149
87
1
89
101
140
104
81
69
56
127
108
90
44
142
138
34
99
109
83
5
102
66
95
14
58
13
31
12
139
152
19
148
63
85
147
15
137
27
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
Rep
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
KeyMaster Pb-Test
Result
|ig/wipe
184.73
206.53
139.49
170.38
214.22
185.92
183.25
177.20
211.38
209.23
215.97
243.62
229.18
249.41
270.85
285.10
233.42
225.00
206.05
252.48
257.36
284.56
216.20
341.82
264.55
235.88
267.48
325.29
215.08
249.01
249.03
232.06
332.81
231.90
287.15
288.76
248.60
330.74
350.48
308.12
Estimated
|ig/wipe
88.0
88.0
88.0
88.0
117.0
117.0
117.0
117.0
162.3
162.3
162.3
162.3
201.6
201.6
201.6
201.6
239.0
239.0
239.0
239.0
240.1
245.6
244.0
252.3
252.3
270.5
252.3
253.4
239.0
270.5
270.0
248.4
245.1
234.6
255.0
234.6
232.9
228.5
253.9
234.0
DataChem
Result
|ig/wipe
82
83
79
100
120
120
120
110
150
160
150
160
200
190
200
220
230
250
250
230
210
250
230
230
200
240
210
210
220
220
230
170
190
210
210
250
220
210
210
220
Estimated
|ig/wipe
88.0
88.0
88.0
88.0
117.0
117.0
117.0
117.0
162.3
162.3
162.3
162.3
201.6
201.6
201.6
201.6
239.0
239.0
239.0
239.0
244.0
274.4
252.8
258.9
241.7
274.9
244.5
236.2
244.0
242.3
260.0
228.5
242.3
267.2
236.2
275.5
262.2
226.3
227.4
243.4
30
-------
Sample
Analysis
Order
93
8
29
153
23
24
122
52
7
60
70
124
100
96
4
64
97
143
129
28
73
76
115
132
150
134
36
130
159
88
119
103
21
114
2
59
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
UCLAB
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
KeyMaster Pb-Test
Result
|ig/wipe
254.43
234.38
192.51
299.75
233.74
262.98
334.93
279.68
351.36
368.37
330.97
338.30
332.97
316.70
398.68
412.32
327.68
419.21
332.68
364.14
399.06
395.33
389.27
351.82
309.40
390.35
309.42
408.52
310.75
394.38
387.10
306.16
421.31
509.41
561.38
343.74
Estimated
|ig/wipe
256.7
256.7
256.7
256.7
260.8
260.8
260.8
260.8
398.3
424.9
404.9
398.9
406.0
398.3
371.8
396.1
376.7
396.6
366.2
380.6
386.7
390.0
373.4
383.4
394.4
416.6
359.0
399.4
408.7
408.7
408.7
408.7
418.1
418.1
418.1
418.1
DataChem
Result
|ig/wipe
290
240
230
250
220
250
210
210
320
360
350
340
350
340
370
340
370
340
370
390
330
320
330
360
340
360
390
330
360
430
410
410
440
410
430
420
Estimated
|ig/wipe
256.7
256.7
256.7
256.7
260.8
260.8
260.8
260.8
377.8
395.0
399.4
385.0
395.5
382.8
413.8
374.0
426.5
378.9
401.1
423.2
372.9
362.9
384.5
411.0
397.2
393.3
437.6
375.1
408.7
408.7
408.7
408.7
418.1
418.1
418.1
418.1
31
-------
Sample
Analysis
Order
79
25
84
107
78
11
54
144
39
123
72
37
42
77
91
74
Source
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
ELPAT
Rep
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
KeyMaster Pb-Test
Result
|ig/wipe
521.26
522.16
598.54
551.72
558.21
514.70
544.80
381.06
649.33
726.67
479.39
649.33
1155.83
901.86
1110.34
1130.73
Estimated
|ig/wipe
561.9
561.9
561.9
561.9
564.7
564.7
564.7
564.7
805.1
805.1
805.1
805.1
1482.6
1482.6
1482.6
1482.6
DataChem
Result
|ig/wipe
580
540
560
540
560
560
570
530
760
770
760
740
1500
1500
1500
1400
Estimated
|ig/wipe
561.9
561.9
561.9
561.9
564.7
564.7
564.7
564.7
805.1
805.1
805.1
805.1
1482.6
1482.6
1482.6
1482.6
32
------- |