United States Office of Research and EPA/600/R-01/064
Environmental Protection Development August 2001
Agency Washington, D.C. 20460
4>EPA Environmental Technology
Verification Report
TNT Detection Technology
Texas Instruments
Spreeta™ Sensor
Environmental Security
Technology Certification
Program
oml
Oak Ridge National Laboratory
ETYET
-------
THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
v>EPA
ornl
I A Ep.lrt.uMriPNiKil.fc I*"? Environmental Security Oak Ridge National Laboratory
Technology Certification
Program
ETV Joint Verification Statement
TECHNOLOGY TYPE: SURFACE PLASMON RESONANCE
APPLICATION: MEASUREMENT OF TNT IN CONTAMINATED SOIL
TECHNOLOGY NAME: Spreeta™ Sensor
COMPANY: Texas Instruments
ADDRESS: 13536 N. Central Expressway PHONE: (972)995-1214
MS-945
Dallas, Texas 75243 FAX: (972) 995-8787
WEB SITE: www.ti.com
EMAIL: elkind@ti.com
The U.S. Environmental Protection Agency (EPA) has created the Environmental Technology
Verification Program (ETV) to facilitate the deployment of innovative or improved environmental
technologies through performance verification and dissemination of information. The goal of the ETV
Program is to further environmental protection by substantially accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high quality, peer-
reviewed data on technology performance to those involved in the design, distribution, financing,
permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized standards and testing organizations and stakeholder groups
consisting of regulators, buyers, and vendor organizations, with the full participation of individual
technology developers. The program evaluates the performance of innovative technologies by developing
test plans that are responsive to the needs of stakeholders, conducting field or laboratory tests (as
appropriate), collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are
conducted in accordance with rigorous quality assurance protocols to ensure that data of known and
adequate quality are generated and that the results are defensible.
The Department of Defense (DoD) has a similar verification program known as the Environmental
Security Technology Certification Program (ESTCP). The purpose of ESTCP is to demonstrate and
validate the most promising innovative technologies that target DoD's most urgent environmental needs
and are projected to pay back the investment within 5 years through cost savings and improved
efficiencies. ESTCP demonstrations are typically conducted under operational field conditions at DoD
facilities. The demonstrations are intended to generate supporting cost and performance data for
acceptance or validation of the technology. The goal is to transition mature environmental science and
technology projects through the demonstration/validation phase, enabling promising technologies to
receive regulatory and end user acceptance in order to be field tested and commercialized more rapidly.
EPA-VS-SCM-49 The accompanying notice is an integral part of this verification statement. August 2001
-------
The Oak Ridge National Laboratory (ORNL) is one of the verification organizations operating under the
Site Characterization and Monitoring Technologies (SCMT) program. SCMT, which is administered by
EPA's National Exposure Research Laboratory, is one of six technology areas under ETV. In this
verification test, ORNL evaluated the performance of explosives detection technologies. This verification
statement provides a summary of the test results for Texas Instruments' (TI's) Spreeta™ Sensor for
2,4,6-trinitrotoluene (TNT) detection. This verification was conducted jointly with the DoD's ESTCP.
VERIFICATION TEST DESCRIPTION
This verification test was designed to evaluate technologies that detect and measure explosives in soil.
The test was conducted at ORNL in Oak Ridge, Tennessee, from August 21 through 30, 2000. Spiked
samples of known concentration were used to assess the accuracy of the technology. Environmentally
contaminated soil samples, collected from DoD sites in California, Louisiana, Iowa, and Tennessee and
ranging in concentration from 0 to approximately 90,000 mg/kg, were used to assess several performance
characteristics. The primary constituents in the samples were 2,4,6-trinitrotoluene (TNT); isomeric
dinitrotoluene (DNT), including both 2,4-dinitrotoluene and 2,6-dinitrotoluene; hexahydro-l,3,5-trinitro-
1,3,5-triazine (RDX); and octahydro-l,3,5,7-tetranitro-l,3,5,7-tetrazocine (HMX). The results of the soil
analyses conducted under field conditions by TI's Spreeta Sensor were compared with results from
reference laboratory analyses of homogenous replicate samples determined using EPA SW-846 Method
8330. (Note that the TI sensor is a bioassay for TNT only.) Details of the test, including a data summary
and discussion of results, may be found in the report entitled Environmental Technology Verification
Report: Explosives Detection Technology—Texas Instruments, Spreeta™ Sensor, EPA/600/R-01/064.
TECHNOLOGY DESCRIPTION
Spreeta is an integrated, miniaturized sensor platform which employs surface plasmon resonance (SPR)
to detect changes in refractive index within a few thousand angstroms of the active gold surface.
Specificity is provided by placing a thin biofilm on the sensor surface. For example, by placing an
antibody to fluoroscein on the sensor surface, the binding of fluorosceinated proteins, seen as a local
increase in refractive index, is simply performed. SPR has been used in this way to study biomolecular
binding events for more than a decade, but Spreeta is the first miniaturized SPR platform. TNT detection
is most efficiently performed by methods other than direct binding. This is because on a molecule-for-
molecule basis, small molecules are much less effective than large molecules at changing refractive
index; thus, any direct SPR assay can detect large molecules at a lower concentration than it can detect
small molecules. For this reason, Texas Instruments has developed a robust inhibition assay in which the
presence of two TNT molecules (228 daltons) effectively inhibits the binding of one antibody molecule
(150,000 daltons). To analyze a sample, 0.5 g of soil is extracted in an aqueous solution. The assay starts
with a conjugate of trinitrobenzene (TNB) and bovine serum albumin on the gold sensing surface. Assays
are then performed by exposing that sensing surface to an anti-TNT antibody solution which may or may
not contain free TNT. When free TNT is present, it binds to anti-TNT antibodies in solution and thereby
keeps them from binding to the surface-bound TNT analog. This inhibited binding is compared to a
reference run where the antibody solution did not contain free TNT. Results from this assay are reported
as interval data (i.e., the concentration of TNT is between 0.3 and 0.9 mg/kg). The lowest reporting
interval was 0 to 0.3 mg/kg.
VERIFICATION OF PERFORMANCE
The following performance characteristics of the Spreeta Sensor were observed.
Precision: Precision was assessed by the percentage of combined sample sets where all four replicates were
reported as the same interval. For all data, 41% of the 27 data sets were reported consistently (i.e., all four
replicates were reported as the same interval). Another 44% had three of four replicates reported consistently,
and the remaining 15% had two of four replicates reported consistently.
Accuracy: Accuracy was assessed using the performance evaluation (PE) soil samples, which were spiked to
nominal TNT concentrations of 0, 10, 50, 100, 250, and 500 mg/kg by an independent laboratory. Accuracy,
defined as the percentage of the Spreeta Sensor interval results that agreed with the nominal (i.e., spiked)
TNT concentration, was 75%. In the remaining samples, 21% of the results were biased low and 4% of the
results were biased high. For each of the samples that were biased low, the upper limit of the reported Spreeta
Sensor interval was within 10% of the nominal concentration (e.g., TI reported the result as 3 to 9 mg/kg, and
EPA-VS-SCM-49 The accompanying notice is an integral part of this verification statement. August 2001
-------
he nominal concentration was 10 mg/kg). Further, when comparing the Spreeta Sensor interval to the
acceptance ranges provided by the preparation laboratory for the PE soils, the agreement was 96%.
False positive/false negative results: Of the 20 blank soil samples, TI reported TNT as 0.3 to 0.9 mg/kg in
,wo samples (10% false positives). False positive and false negative results were also determined by
comparing the Spreeta Sensor result to the reference laboratory result on environmental and spiked samples
e.g., whether the Spreeta Sensor reports a result as a nondetect that the reference laboratory reported as a
detect, and vice versa). For TNT, none of the results were false positives relative to the reference laboratory
•esult. TI reported two samples as 0 to 0.3 mg/kg when the laboratory reported a detection at 0.8 mg/kg; these
•esults were considered false negatives (3% rate).
Completeness: The Spreeta Sensor generated results for all 108 soil samples for a completeness of 100%.
Comparability: Comparability, like accuracy, was defined as the percentage of results that agreed with, was
above, or was below the reference laboratory result. The percentage of samples that agreed with the reference
aboratory results was 65% for all soils (excluding two suspect reference laboratory values). Approximately
3% of the TI results were above the reference laboratory results, but more (32%) were below. One-third of
he TI samples that were below the reference laboratory result were for samples with very high (> 10,000
mg/kg) TNT concentrations. Of the sample results that did not agree with the reference laboratory, 79% were
within ±10 mg/kg of the reference laboratory result.
Sample Throughput: Operating out of a motor home, the TI team accomplished a sample throughput rate of
approximately 12 samples per day for the soil analyses. Two instruments were used for the TNT analyses.
Two operators analyzed samples in tandem to accomplish a higher sample throughput rate, so the technology
;an be run by a single trained operator. A mean of four tests per sample was required to generate a reported
result.
Overall Evaluation: The verification team found that the Spreeta Sensor was relatively simple for the trained
analyst to operate in the field, requiring less than an hour for initial setup. The overall performance of the
Spreeta Sensor for the analysis of soil samples was characterized as precise and unbiased for TNT less than
10,000 mg/kg. As with any technology selection, the user must determine if this technology is appropriate for
he application and for the project data quality objectives. For more information on this and other verified
echnologies, visit the ETV web site at http://www.epa.gov/etv.
Gary J. Foley, Ph.D.
Director
National Exposure Research Laboratory
Office of Research and Development
Jeffrey Marqusee, Ph.D.
Director
Environmental Security Technology Certification Program
U.S. Department of Defense
W. Frank Harris, Ph.D.
Associate Laboratory Director
Biological and Environmental Sciences
Oak Ridge National Laboratory
NOTICE: EPA and ESTCP verifications are based on evaluations of technology performance under specific,
predetermined criteria and appropriate quality assurance procedures. EPA, ESTCP, and ORNL make no expressed
or implied warranties as to the performance of the technology and do not certify that a technology will always
operate as verified. The end user is solely responsible for complying with any and all applicable federal, state, and
local requirements. Mention of commercial product names does not imply endorsement or recommendation.
EPA-VS-SCM-49
The accompanying notice is an integral part of this verification statement.
August 2001
-------
EPA/600/R-01/064
August 2001
Environmental Technology
Verification Report
TNT Detection Technology
Texas Instruments
Spreeta™ Sensor
By
Amy B. Dindal
Charles K. Bayne, Ph.D.
Roger A. Jenkins, Ph.D.
Oak Ridge National Laboratory
Oak Ridge, Tennessee 37831-6120
Eric N. Koglin
U.S. Environmental Protection Agency
Environmental Sciences Division
National Exposure Research Laboratory
Las Vegas, Nevada 89193-3478
This verification test was conducted in cooperation with the
U.S. Department of Defense
Environmental Security Technology Certification Program
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and Development (ORD),
and the U.S. Department of Defense's Environmental Security Technology Certification Program (ESTCP)
Program, funded and managed, through Interagency Agreement No. DW89937854 with Oak Ridge National
Laboratory, the verification effort described herein. This report has been peer and administratively reviewed
and has been approved for publication as an EPA document. Mention of trade names or commercial products
does not constitute endorsement or recommendation for use of a specific product.
-------
Table of Contents
List of Figures v
List of Tables vii
Acknowledgments ix
Abbreviations and Acronyms xi
1. INTRODUCTION 1
2. TECHNOLOGY DESCRIPTION 3
General Technology Description 3
Sample Preparation 3
Calibration and Data Analysis 3
3. VERIFICATION TEST DESIGN 5
Objective 5
Testing Location and Conditions 5
Soil Sample Descriptions 5
Sources of Samples 5
Iowa Army Ammunition Plant 5
Louisiana Army Ammunition Plant 5
Milan Army Ammunition Plant 5
Volunteer Army Ammunition Plant 5
Fort Ord Military Base 5
Performance Evaluation Samples 6
Soil Sample Preparation 6
Sample Randomization 6
Summary of Experimental Design 6
Description of Performance Factors 7
Precision 7
Accuracy 7
False Positive/Negative Results 7
Completeness 7
Comparability 7
Sample Throughput 8
Ease of Use 8
Cost 8
Miscellaneous Factors 8
4. REFERENCE LABORATORY ANALYSES 9
Reference Laboratory Selection 9
Reference Laboratory Method 9
Reference Laboratory Performance 9
5. TECHNOLOGY EVALUATION 11
Objective and Approach 11
Precision 11
Accuracy 11
False Positive/False Negative Results 12
Completeness 12
Comparability 12
Sample Throughput 13
Ease of Use 13
Cost Assessment 13
Spreeta Sensor Costs 14
Labor 14
Equipment 16
Reference Laboratory Costs 16
Sample Shipment 16
Labor, Equipment, and Waste Disposal 16
Cost Assessment Summary 16
iii
-------
Miscellaneous Factors 16
Summary of Performance 17
6. TECHNOLOGY UPDATE 18
Technology Update 18
7. REFERENCES 19
Appendix A — TFs Spreeta Sensor Sample Results Compared with Reference
Laboratory Results 20
Appendix B — Data Quality Objective (DQO) Example 23
IV
-------
List of Figures
1 Spreeta Sensor 3
2 A graphical representation of data used during Spreeta's TNT detection assay 4
3 Upper graph represents the comparison of the Spreeta Sensor's results versus
the reference laboratory for concentrations less than 100 mg/kg, while the lower
graph represents the comparison for higher concentrations 14
4 Absolute difference between Spreeta Sensor result and reference laboratory result 15
-------
-------
List of Tables
1 Summary of Experimental Design 6
2 Classification of Precision Results 7
3 Summary of the Reference Laboratory Performance for Soil Samples 10
4 Summary of the Reference Laboratory Performance on Blank Samples 10
5 Summary of Spreeta Sensor Precision 11
6 Summary of Spreeta Sensor Accuracy: Comparison to Nominal Value 12
7 Number of Spreeta Sensor and Reference Laboratory TNT Results within
Acceptance Ranges for Spiked Soils 12
8 Summary of Spreeta Sensor False Positive Performance on Blank Samples 13
9 Summary of the Spreeta Sensor Detect/Nondetect Performance Relative to
the Reference Laboratory Results 13
10 Summary of Spreeta Sensor Comparability 13
11 Estimated Analytical Costs for Explosives-contaminated Samples 15
12 TNT Performance Summary for the Spreeta Sensor 17
vn
-------
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the verification test,
analyze the data, and prepare this report. In particular, we recognize Dr. Thomas Jenkins (U.S. Army, Cold
Regions Research and Engineering Laboratory) and Dr. Michael Maskarinec (Oak Ridge National
Laboratory), who served as the technical experts for this project. We thank the people who helped us to
obtain the samples from the various sites, including Dr. Jenkins, Danny Harrelson (Waterways Experiment
Station), Kira Lynch (U.S. Army Corps of Engineers, Seattle District), Larry Stewart (Milan Army
Ammunition Plant), and Dick Twitchell and Bob Elmore (Volunteer Army Ammunition Plant). For external
peer review, we thank Dr. C. L. Grant (Professor Emeritus, University of New Hampshire), and Harry Craig
(US EPA, Region 10) for EPA peer review. The authors also acknowledge the participation of Jerry Elkind
and Anita Strong of Texas Instruments, who performed the analyses during the verification test.
For more information on the Explosives Detection Technology Verification contact
Eric Koglin
Project Technical Leader
Environmental Protection Agency
Environmental Sciences Division
National Exposure Research Laboratory
P.O. Box 93478
Las Vegas, Nevada 89193-3478
(702) 798-2432
koglin.eric@epa.gov
Roger Jenkins
Program Manager
Oak Ridge National Laboratory
Chemical and Analytical Sciences Division
P.O. Box 2008
Oak Ridge, TN 37831-6120
(865) 574-4871
j enkinsra@ornl .gov
For more information on Texas Instruments' Spreeta Sensor contact
Jerry Elkind
Texas Instruments
13536 N. Central Expressway, MS 945
Dallas, Texas 75243
(972)995-1214
elkind@ti.com
www.ti.com
-------
Abbreviations and Acronyms
2-Am-DNT 2-amino-4,6-dinitrotoluene
4-Am-DNT 4-amino-2,6-dinitrotoluene
BSA bovine serum albumin
CRREL U.S. Army Cold Regions Research and Engineering Laboratory
2,4-DNT 2,4-dinitrotoluene
2,6-DNT 2,6-dinitrotoluene
DNT isomeric dinitrotoluene (includes both 2,4-DNT and 2,6-DNT)
DSP digital signal processor
DoD U.S. Department of Defense
DQO data quality objective
EPA U.S. Environmental Protection Agency
ERA Environmental Resource Associates
ESTCP Environmental Security Technology Certification Program
ETV Environmental Technology Verification Program
FA false acceptance error rate
fn false negative result
fp false positive result
FR false rejection error rate
HMX octahydro-l,3,5,7-tetranitro-l,3,5,7-tetrazocine
HPLC high-performance liquid chromatograph
LAAAP Louisiana Army Ammunition Plant
MLAAP Milan Army Ammunition Plant
ORNL Oak Ridge National Laboratory
PE performance evaluation sample
QA quality assurance
QC quality control
RDX hexahydro-1,3,5 -trinitro-1,3,5 -triazine
RSD relative standard deviation
SCMT Site Characterization and Monitoring Technologies
SD standard deviation
SPR surface plasmon resonance
TI Texas Instruments
TNB 1,3,5-trinitrobenzene
TNT 2,4,6-trinitrotoluene
-------
-------
Section 1 — Introduction
The U.S. Environmental Protection Agency (EPA)
created the Environmental Technology Verification
Program (ETV) to facilitate the deployment of
innovative or improved environmental technologies
through performance verification and dissemination
of information. The goal of the ETV Program is to
further environmental protection by substantially
accelerating the acceptance and use of improved and
cost-effective technologies. ETV seeks to achieve
this goal by providing high-quality, peer-reviewed
data on technology performance to those involved in
the design, distribution, financing, permitting,
purchase, and use of environmental technologies.
ETV works in partnership with recognized standards
and testing organizations and stakeholder groups
consisting of regulators, buyers, and vendor
organizations, with the full participation of
individual technology developers. The program
evaluates the performance of innovative
technologies by developing verification test plans
that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate),
collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in
accordance with rigorous quality assurance (QA)
protocols to ensure that data of known and adequate
quality are generated and that the results are
defensible.
ETV is a voluntary program that seeks to provide
objective performance information to all of the
participants in the environmental marketplace and to
assist them in making informed technology
decisions. ETV does not rank technologies or
compare their performance, label or list technologies
as acceptable or unacceptable, seek to determine
"best available technology," or approve or
disapprove technologies. The program does not
evaluate technologies at the bench or pilot scale and
does not conduct or support research. Rather, it
conducts and reports on testing designed to describe
the performance of technologies under a range of
environmental conditions and matrices.
The program now operates six centers covering a
broad range of environmental areas. ETV began
with a 5-year pilot phase (1995-2000) to test a wide
range of partner and procedural alternatives in
various technology areas, as well as the true market
demand for and response to such a program. In these
Centers, EPA utilizes the expertise of partner
"verification organizations" to design efficient
processes for conducting performance tests of
innovative technologies. These expert partners are
both public and private organizations, including
federal laboratories, states, industry consortia, and
private sector entities. Verification organizations
oversee and report verification activities based on
testing and QA protocols developed with input from
all major stakeholder/customer groups associated
with the technology area. The verification described
in this report was administered by the Site
Characterization and Monitoring Technologies
(SCMT) Center, with Oak Ridge National
Laboratory (ORNL) serving as the verification
organization. (To learn more about ETV, visit
ETV's web site at http://www.epa.gov/etv.) The
SCMT Center is administered by EPA's National
Exposure Research Laboratory, Environmental
Sciences Division, in Las Vegas, Nevada.
The Department of Defense (DoD) has a similar
verification program known as the Environmental
Security Technology Certification Program
(ESTCP). The purpose of ESTCP is to demonstrate
and validate the most promising innovative
technologies that target DoD's most urgent
environmental needs and are projected to pay back
the investment within 5 years through cost savings
and improved efficiencies. ESTCP responds to:
(1) concern over the slow pace and cost of
remediation of environmentally contaminated sites
on military installations, (2) congressional direction
to conduct demonstrations specifically focused on
new technologies, (3) Executive Order 12856, which
requires federal agencies to place high priority on
obtaining funding and resources needed for the
development of innovative pollution prevention
programs and technologies for installations and in
acquisitions, and (4) the need to improve defense
readiness by reducing the drain on the Department's
operation and maintenance dollars caused by real
world commitments such as environmental
restoration and waste management. ESTCP
demonstrations are typically conducted under
operational field conditions at DoD facilities. The
demonstrations are intended to generate supporting
cost and performance data for acceptance or
validation of the technology. The goal is to
transition mature environmental science and
technology projects through the
demonstration/validation phase, enabling promising
technologies to receive regulatory and end user
acceptance in order to be field tested and
commercialized more rapidly. (To learn more about
ESTCP, visit ESTCP's web site at
http://www.estcp.org.)
EPA's ETV program and DoD's ESTCP program
established a memorandum of agreement in 1999 to
work cooperatively on the verification of
technologies that are used to improve environmental
cleanup and protection at both DoD and non-DoD
sites. The verification of field analytical
technologies for explosives detection described in
this report was conducted jointly by ETV's SCMT
Center and ESTCP. The verification was conducted
-------
at ORNL in Oak Ridge, Tennessee, from August 21
through 30, 2000. The performances of two field
analytical techniques for explosives were
determined under field conditions. Each technology
was independently evaluated by comparing field
analysis results with those obtained using an
approved reference method, EPA SW-846 Method
8330. The verification was designed to evaluate the
field technology's ability to detect and measure
explosives in soil. The primary constituents in the
samples were 2,4,6-trinitrotoluene (TNT); isomeric
dinitrotoluene (DNT), including both 2,4-
dinitrotoluene (2,4-DNT) and 2,6-dinitrotoluene
(2,6-DNT); hexahydro-l,3,5-trinitro-l,3,5-triazine
(RDX); and octahydro-l,3,5,7-tetranitro-l,3,5,7-
tetrazocine (HMX). Naturally contaminated
environmental soil samples, ranging in concen-
tration from 0 to approximately 90,000 mg/kg, were
collected from DoD sites in California, Louisiana,
Iowa, and Tennessee, and were used to assess
several performance characteristics. This report
discusses the performance of the Texas Instruments'
Spreeta™ Sensor for the determination of TNT in
soil samples.
-------
Section 2 — Technology Description
In this section, the vendor (with minimal editorial changes by ORNL) provides a description of the
technology and the analytical procedure used during the verification testing activities.
General Technology Description
Spreeta (see Figure 1) is an integrated, miniaturized
sensor platform, which employs surface plasmon
resonance (SPR) to detect changes in refractive
index within a few thousand angstroms of an active
gold surface. Analyte specificity is provided by a
thin biofilm on the sensor surface. For example, by
placing an antibody to fluorescein on the sensor
surface, the binding of fluoresceinated proteins is
seen as a local increase in refractive index. SPR has
been used in this way to study biomolecular binding
events for more than a decade, but Spreeta is the
first miniaturized SPR platform. TNT and other
small molecules are most efficiently detected by
methods other than direct binding. For this reason,
Texas Instruments (TI) has developed a robust
inhibition assay in which the presence of two TNT
molecules (228 daltons) effectively inhibit the
binding of one antibody molecule (150,000 daltons).
Figure 1. Spreeta Sensor.
The Prototype Spreeta System employed for this test
utilized one miniature peristaltic pump, four
electromagnetic-actuated valves and a digital signal
processor (DSP) driven electronics interface with a
keypad and alphanumeric liquid crystal display.
Running and regeneration buffers were held in IV
bags within the instrument. Waste was also held in a
bag inside the instrument. Bags were replaced at
intervals of approximately every two days.
The assay used here is a standard inhibition
immunoassay. The surface biofilm is a conjugate of
trinitrobenzene (TNB) and bovine serum albumin
(BSA), which has been attached to the gold sensing
surface of the Spreeta sensor. BSA serves both as an
adhesion medium for the TNB groups as well as a
nonspecific binding reduction layer. The assay is
then performed by exposing that sensing surface to
an anti-TNT antibody solution, which may or may
not contain free TNT, and monitoring surface
binding using SPR. When free TNT is present, it
binds to anti-TNT antibodies in solution and thereby
keeps them from binding to the surface-bound TNT
analog. If no TNT is present, the anti-TNT
antibodies are not inhibited from binding to the
surface and, again, this is detected in real time by
SPR. The actual binding for a given sample is
compared to a reference run (where the antibody
solution did not contain free TNT) to determine the
presence or absence of TNT. At the end of any
binding test, the surface is regenerated by a brief
exposure to an aqueous NaOH/Triton X solution
which liberates surface-bound antibody and leaves
the biofilm free for the next test.
Figure 2 shows a plot of the refractive index versus
time for a reference run, a sample run that was
negative, and a sample run that was positive. This
illustrates the data the DSP analyses use to provide
quantitative results.
Sample Preparation
Soil extracts were prepared using a completely
aqueous protocol. Approximately 500 ±1 mg of soil
was suspended in 5 mL of phosphate buffered saline
and 0.1% Triton X-100 (a non-ionic detergent) in a
10-mL glass vial. The mixture was gently shaken for
3-5 minutes and then allowed to settle for a few
minutes. Next, 1.5 mL of the supernatant was
removed by pipette and was mixed with 15 |_iL of
antibody solution. This sample was then analyzed
for TNT content as previously described. TI has
determined that the extraction efficiency of this
protocol is approximately 40%.
Calibration and Data Analysis
Reference runs (with no TNT present) were made
periodically to verify assay fidelity and biofilm
integrity. The antibody used in this assay is
completely cross-reactive with trinitrobenzene
(TNB), is approximately 10% cross-reactive with
dinitrotoluenes (DNT), and is much less cross-
reactive with other nitro-aromatic compounds.
Therefore, we report an "effective" TNT
concentration, which primarily includes
contributions from TNT, TNB, and DNT. RDX and
HMX do not react with this antibody to an
appreciable degree, and therefore their presence is
not a factor in this assay. A negative result (with
sample-run binding less than 65% of the reference-
-------
1.3316
1.3314-
1.3312-
1.331 -
1.3308 -
1.3306 -
1.3304-
1.3302
SPR Competitive Assay
Regeneration
Sample
t
t
200
400 600
Time (seconds)
800
1000
Figure 2. A graphical representation of data used during Spreeta's TNT detection assay.
run binding) with an undiluted sample indicates an
effective TNT concentration of less than 0.3 ppm
(mg/kg) in soil. A positive result (with sample-run
binding greater than 35% of the reference-run
binding) with an undiluted sample calls for a
dilution and retest of the diluted sample until a
negative result occurs yielding a dilution bracket.
For the purposes of this verification test, 3 x
dilutions were used, and this resulted in answers that
were reported such that the central point of the
bracket is approximately 50% above the lower limit
and approximately 50% below the upper limit. For
example, if the test was positive for a 100* dilution
and negative for a 300* dilution, the result was
reported as [10-30] mg/kg.
-------
Section 3 — Verification Test Design
Objective
The purpose of this section is to describe the
verification test design. It is a summary of the test
plan (ORNL 2000).
Testing Location and Conditions
The verification of field analytical technologies
for explosives was conducted at ORNL's Building
5507, in Oak Ridge, Tennessee. TI elected to
operate their technology in a motor home. The
temperature and relative humidity in the motor
home were monitored during testing. During the
warmer portions of the day, the air conditioner
was run. Over the 10 days of testing, the average
temperature in the motor home was 73 °F, and
ranged from 64 to 82°F. The average relative
humidity in the motor home was 53%, and it
ranged from 32 to 81%.
The samples used in this study were shipped to the
testing location for evaluation by the vendors.
Explosives-contaminated soils from Army
ammunition plants in Iowa, Louisiana, Tennessee,
and a former Army base in California (Fort Ord)
were used in this verification. Because samples
were obtained from multiple DoD sites, the
samples represented a reasonable cross section of
the population of explosives-contaminated
matrices, such that the versatility of the field
technology could be evaluated. More specific
details about the samples are presented in the
following sections.
Primary Analytes in Soil Samples
The primary contaminants in the soil samples
were TNT, DNT, RDX, and HMX. The samples
also contained trace amounts of 2-amino-4,6-
dinitrotoluene (2-Am-DNT) and 4-amino-2,6-
dinitrotoluene (4-Am-DNT), which are
degradation products of TNT. The total
concentration of explosives ranged from 0 to
approximately 90,000 mg/kg. The following
sections describe the sites from which the samples
were collected.
Sources of Samples
Iowa Army Ammunition Plant
Currently an active site, the Iowa Army Ammuni-
tion Plant was constructed to load, assemble, and
pack various conventional ammunition and fusing
systems. Current production includes 120-mm
tank rounds, warheads for missiles, and mine
systems. During the early years of operation, the
installation used surface impoundments, landfills,
and sumps for disposal of industrial wastes
containing explosives. The major contaminants in
these samples are TNT, RDX, and HMX.
Louisiana Army Ammunition Plant
The Louisiana Army Ammunition Plant
(LAAAP), near Shreveport, Louisiana, is a
government-owned facility that began production
in 1942. The facility is currently an Army Reserve
plant. Production items at LAAAP have included
metal parts for artillery shells; the plant also loads,
assembles, and packs artillery shells, mines,
rockets, mortar rounds, and demolition blocks. As
a result of these activities and the resulting soil
and groundwater contamination, EPA placed
LAAAP on the National Priorities List of
contaminated sites (Superfund) in 1989. The
major constituents in the samples from this site
are TNT, RDX, and HMX, with trace levels of
1,3,5-trinitrobenzene (TNB), DNT, 2-Am-DNT,
and 4-Am-DNT.
Milan Army Ammunition Plant
Currently active, the Milan Army Ammunition
Plant (MLAAP) in Milan, Tennessee, was
established in late 1940 as part of the pre-World
War II buildup. The facility still has ten ammuni-
tion loading, assembly, and packaging lines.
Munitions-related wastes have resulted in soil
contamination. The primary contaminants in these
soils are RDX and TNT.
Volunteer Army Ammunition Plant
The Volunteer Army Ammunition Plant, in
Chattanooga, Tennessee, was built in 1941 to
manufacture TNT and DNT. All production
ceased in 1977. Past production practices resulted
in significant soil and groundwater contamination.
In the samples from this site, concentrations of
TNT and DNT ranged from 10 to 90,000 mg/kg,
with significantly smaller concentrations of Am-
DNT isomers.
Fort Ord Military Base
Fort Ord, located near Marina, California, was
opened in 1917 as a training and staging facility
for infantry troops and was closed as a military
installation in 1993. Since then, several
nonmilitary uses have been established on the site:
California State University at Monterey Bay has
opened its doors on former Fort Ord property, the
University of California at Santa Cruz has
established a new research center there, the
Monterey Institute of International Studies will
take over the officer's club and several other
buildings, and the post's airfield was turned over
to the city of Marina. The Army still occupies
several buildings.
An Army study conducted in 1994 revealed that
the impact areas at the inland firing ranges of Fort
Ord were contaminated with residues of high
explosives (Jenkins, Walsh, and Thome 1998).
-------
Fort Ord is on the National Priorities List of
contaminated sites (Superfund), requiring the
installation to be characterized and remediated to
a condition that does not pose unacceptable risks
to public health or the environment. The
contaminant present at the highest concentration
(as much as 300 mg/kg) was HMX; much lower
concentrations of RDX, TNT, 2-Am-DNT, and
4-Am-DNT are present.
Performance Evaluation Samples
Spiked soil samples were obtained from
Environmental Resource Associates (ERA,
Arvada, Colorado). The soil was prepared using
ERA's semivolatile blank soil matrix. This matrix
was a 40% clay topsoil that had been dried,
sieved, and homogenized. Particle size was 60
mesh and smaller. The samples, also referred to as
performance evaluation (PE) samples, contained
known levels of TNT and RDX. The
concentrations that were evaluated nominally
contained 10, 50, 100, 250, and 500 mg/kg of each
analyte. Prior to the verification test, ORNL
analyzed the spiked samples to confirm the
concentrations were within the performance
acceptance limits established by the preparation
laboratory. The method used was a modified
Method 8330, similar to the reference laboratory
method described in Section 4. For the verification
test, four replicates were prepared at each
concentration level.
Blank soil samples were evaluated to determine
the technology's ability to identify samples with
no contamination (i.e., to ascertain the false
positive error rate). The soil was collected in
Monroe County, Tennessee, and was certified by
ORNL to be free of contamination prior to
verification testing. A reasonable number of
blanks (N = 20) was chosen to balance the
uncertainty for estimating the false positive error
rate and the required number of blank samples to
be measured.
Soil Sample Preparation
All of the soil samples were shipped in plastic
bags at ambient temperature to ORNL. The
samples were stored frozen (<0°C) prior to
preparation. To ensure that the vendors and the
reference laboratory analyzed comparable
samples, the soils were homogenized prior to
sample splitting. The process was as follows. The
sample was kneaded in the Ziplock plastic bag to
break up large clumps. Approximately 1500 g of
soil was poured into a Pyrex pan, and debris was
removed. The sample was then air dried overnight.
The sample was sieved using a 10-mesh (2-mm
particle size) screen and placed in a 1-L wide-
mouthed jar. After thorough mixing with a metal
spatula, the sample was quartered. After mixing
each quarter, approximately 250 g from each
quarter was placed back in the 1-L widemouthed
jar, for a total sample amount of approximately
1000 g. Analysis by the ORNL method confirmed
sample homogeneity (variability of 20% relative
standard deviation or less for replicate
measurements). The sample was then split into
subsamples for analysis during the verification
test. Each 4-oz sample jar contained
approximately 20 g of soil. Four replicate splits of
each soil sample were prepared for each
participant. The design included a one-to-one
pairing of the replicates, such that the vendor and
reference lab samples could be directly matched.
To ensure that degradation did not occur, the soil
samples were frozen (<0°C) until analysis
(Maskarinec et al. 1991).
Sample Randomization
The samples were randomized in two stages. First,
the order in which the filled jars were distributed
was randomized so that the same vendor did not
always receive the first jar filled for a given
sample set. Second, the order of analysis was
randomized so that each participant analyzed the
same set of samples, but in a different order. Each
jar was labeled with a sample number. Replicate
samples were assigned unique (but not sequential)
sample numbers. Spiked materials and blanks
were labeled in the same manner, such that these
quality control samples were indistinguishable
from other samples. All samples were analyzed
blindly by both the vendor and the reference
laboratory.
Summary of Experimental Design
The distribution of samples from the various sites
is described in Table 1. A total of 108 soil samples
were analyzed, with approximately 60% of the
samples being naturally contaminated environ-
mental soils, and the remaining 40% being spikes
and blanks. Four replicates were analyzed for each
sample type. For example, 4 replicate splits of
each of 3 Fort Ord soils were analyzed, for a total
of 12 individual Fort Ord samples.
Table 1. Summary of Experimental Design
Sample source or type
Fort Ord
Iowa
LAAAP
MLAAP
Volunteer
Spiked
Blank
Total
No. of samples
12
4
16
20
12
24
20
108
-------
Description of Performance Factors
In Section 5, technology performance is described
in terms of precision, accuracy, completeness, and
comparability, which are indicators of data quality
(EPA 1998). False positive and negative results,
sample throughput, and ease of use are also
described. Each of these performance
characteristics is defined in this section.
Precision
Precision is the reproducibility of measurements
under a given set of conditions. Standard deviation
(SD) and relative standard deviation (RSD) are
generally used to assess precision for quantitative
data. For this evaluation of interval data, the
frequency with which the same interval was
reported within a set of replicates was used to
quantify precision. Examples of how the precision
was classified are presented in Table 2. Reporting
a higher number of replicates in the same interval
for a given replicate set indicates higher precision.
In other words, reporting all four replicate results
as the same interval indicates the highest possible
precision.
Accuracy
Accuracy represents the closeness of the tech-
nology's measured concentrations to known (in
this case, spiked/PE) values. For quantitative data,
accuracy is usually assessed in terms of
percentage recovery. For this evaluation of
interval data, accuracy was evaluated in terms of
the percentage of samples that agreed with, were
above (i.e., biased high), and were below (i.e.,
biased low) the certified value.
False Positive/Negative Results
A false positive (fp) result is one in which the
technology detects explosives in the sample when
there actually are none (Berger, McCarty, and
Smith 1996). A false negative (fn) result is one in
which the technology indicates that no explosives
are present in the sample, when there actually are
(Berger, McCarty, and Smith 1996). The
evaluation of fp and fn results is influenced by the
actual concentration in the sample and includes an
assessment of the reporting limits of the
technology. False positive results are assessed in
two ways. First, the results are assessed relative to
the blanks (i.e., the technology reports a detected
value when the sample is a blank). Second, the
results are assessed on environmental and spiked
samples where the analyte was not detected by the
reference laboratory (i.e., the reference laboratory
reports a nondetect and the field technology
reports a detection). False negative results, also
assessed for environmental and spiked samples,
indicate the frequency that the technology
reported a nondetect (i.e., [0, 0.3] ppm) and the
reference laboratory reported a detection. Note
that the reference laboratory results were validated
by ORNL so that fp/fn assessment would not be
influenced by faulty laboratory data. The reporting
limit is considered in the evaluation. For example,
if the reference laboratory reported a result as
0.8 mg/kg, and the technology's paired result was
reported as [0.3, 0.9] mg/kg, the technology's
result was considered correct and not a false
negative result.
Completeness
Completeness is defined as the percentage of mea-
surements that are judged to be usable (i.e., the
result is not rejected). The acceptable complete-
ness is 95% or greater.
Comparability
Comparability refers to how well the field tech-
nology and reference laboratory data agree. The
difference between accuracy and comparability is
that accuracy is judged relative to a known value,
and comparability is judged relative to the results
of a standard or reference procedure, which may
or may not report the results accurately. Note that
the reference laboratory result is not assumed to
be the "correct" result. This evaluation is
performed for comparison of the field analytical
technology result with what a typical fixed
analytical laboratory might report for the same
sample. A one-to-one sample comparison of the
technology results and the reference laboratory
results is performed in Section 5. As with
accuracy, it is reported as the percentage of
samples that agree with, are above, and are below
the reference result.
Table 2. Classification of Precision Results
If the replicate results are. . .
[0, 0.3], [0, 0.3], [0, 0.3], [0, 0.3]
[0, 0.3], [0, 0.3], [0, 0.3], [0.3, 0.9]
[0, 0.3], [0, 0.3], [0.3, 0.9], [0.3, 0.9]
[0, 0.3], [0.3, 0.9], [0.9, 30], [30, 90]
. . .then the number reported in
identical intervals is. . .
4
3
2
1
. . .And the precision
classification is. . .
High
Medium
Low
None
-------
Sample Throughput
Sample throughput is a measure of the number of
samples that can be processed and reported by a
technology in a given period of time. This is
reported in Section 5 as the number of samples per
hour or day times the number of analysts.
Ease of Use
A significant factor in purchasing an instrument or
a test kit is how easy the technology is to use.
Several factors are evaluated and reported on in
Section 5.
What is the required operator skill level (e.g.,
technician or advanced degree)?
How many operators were used during the
test? Could the technology be run by a single
person?
How much training would be required in order
to run this technology?
How much subjective decision-making is
required?
Cost
An important factor in the consideration of
whether to purchase a technology is cost. Costs
involved with operating the technology and the
standard reference analyses are estimated in
Section 5. To account for the variability in cost
data and assumptions, the economic analysis is
presented as a list of cost elements and a range of
costs for sample analysis. Several factors affect
the cost of analysis. Where possible, these factors
are addressed so that decision makers can
independently complete a site-specific economic
analysis to suit their needs.
Miscellaneous Factors
Any other information that might be useful to a
person who is considering purchasing the
technology is documented in Section 5. Examples
of information that might be useful to a
prospective purchaser are the amount of hazardous
waste generated during the analyses, the
ruggedness of the technology, the amount of
electrical or battery power necessary to operate
the technology, and aspects of the technology or
method that make it easy to use.
-------
Section 4 — Reference Laboratory Analyses
Reference Laboratory Selection
The verification process is based on the presence of
a statistically validated data set against which the
performance of the technology may be compared.
The choice of an appropriate reference method and
reference laboratory are critical to the success of the
verification test. To assess the performance of the
explosives field analytical technologies, the data
obtained from verification test participants were
compared to data obtained using conventional
analytical methods.
The first evaluation of explosives-detection
technologies under the ETV program occurred in
1999. Specialized Assays Inc. (SAI), now known as
TestAmerica, Inc., of Nashville, Tennessee, was
selected as the reference laboratory for that study. A
sample holding time study performed by ORNL in
May 2000 indicated that the concentration of
explosives in the samples had not changed
significantly. Therefore, archived soil samples and
the reference laboratory data generated in 1999 were
used for comparison with the vendor results.
The following describes how SAI was chosen to
perform the 1999 analyses. Specialized Assays, Inc.
was selected to perform the analyses based on
ORNL's experience with laboratories capable of
performing explosives analyses using EPA SW-846
Method 8330. ORNL reviewed Specialized Assays'
record of laboratory validation performed by the
U.S. Army Corps of Engineers (Omaha, Nebraska).
EPA and ORNL decided that, based on the
credibility of the Army Corps program and ORNL's
prior experience with the laboratory, Specialized
Assays would be selected to perform the reference
analyses.
ORNL conducted an audit of Specialized Assays'
laboratory operations on May 4, 1999. This
evaluation focused specifically on the procedures
that would be used for the analysis of the
verification test samples. Results from this audit
indicated that Specialized Assays was proficient in
several areas, including quality management,
document/record control, sample control, and
information management. Specialized Assays was
found to be compliant with implementation of
Method 8330 analytical procedures. The company
provided a copy of its QA plan, which details all of
the QA and quality control (QC) procedures for all
laboratory operations (Specialized Assays 1999).
The audit team noted that Specialized Assays had
excellent procedures in place for data backup,
retrievability, and long-term storage. ORNL
conducted a second audit at Specialized Assays
while the analyses were being performed. Since the
initial qualification visit, management of this
laboratory had changed because Specialized Assays
became part of TestAmerica. The visit included
tours of the laboratory, interviews with key
personnel, and review of data packages. Overall, no
major deviations from procedures were observed,
and laboratory practices appeared to meet the QA
requirements of the technology verification test plan
(ORNL 1999).
Reference Laboratory Method
The reference laboratory's analytical method,
presented in the technology test plan, followed the
guidelines established in EPA SW-846 Method 8330
(EPA 1994). According to Specialized Assays'
procedures, soil samples were prepared by
extracting 2-g samples of soil in acetonitrile by
sonication for approximately 16 h. An aliquot of the
extract was then combined with a calcium chloride
solution to precipitate out suspended particulates.
After the solution was filtered, the filtrate was ready
for analysis. The analytes were identified and
quantified using a high-performance liquid
chromatograph (HPLC) with a 254-nm UV detector.
The primary analytical column was a C-18 reversed-
phase column with confirmation by a secondary
cyano column. The practical quantitation limit for
soil was 0.5 mg/kg.
Reference Laboratory Performance
ORNL validated all of the reference laboratory data
according to the procedure described in the test plan
(ORNL 2000). During the validation, the following
aspects of the data were reviewed: completeness of
the data package, adherence to holding time
requirements, correctness of the data, correlation
between replicate sample results, evaluation of QC
sample results, and evaluation of spiked sample
results. Each of these categories is described in
detail in the test plan. The reference laboratory
reported valid results for all samples, so com-
pleteness was 100%. Preanalytical holding time
requirements (14 days to extract; 40 days to analyze)
were met. A few errors were found in a small
portion of the data (~4%). Those data were
corrected for transcription and calculation errors
that were identified during the validation. One data
point, a replicate Iowa soil sample, was identified as
suspect. The result for this sample was 0.8 mg/kg;
the results from the other three replicates averaged
27,400 mg/kg. This data point was excluded from
the evaluation of comparability with the field
technology (reported in Section 5) because it was an
obvious suspect value. The reference laboratory
results for QC samples were flagged when the
results were outside the QC acceptance limits. The
reference laboratory results were evaluated by a
statistical analysis of the data. Due to the limited
results reported for the other Method 8330 analytes,
only the results for the major constituents in the
-------
samples (total DNT, TNT, RDX, and HMX) are
evaluated in this report.
The accuracy and precision of the reference
laboratory results are summarized in Table 3.
Accuracy was assessed using the PE (spiked)
samples, while precision was assessed using the
results from both spiked and environmental samples.
The reference laboratory results were unbiased
(accurate), as mean percentage recovery values were
near 100%. The reference laboratory results were
precise; all but one of the mean RSDs were less than
30%. The one mean RSD that was greater than 30%
(DNT, 56%) was for a limited data set of three.
Table 4 presents the laboratory results for blank
samples. A false positive result is identified as any
detected result on a known blank. For the soil
samples, one false positive detection appeared to be
a preparation error because the concentration was
near 70,000 mg/kg. Overall, it was concluded that
the reference laboratory results were unbiased,
precise, and acceptable for comparison with the
field analytical technology.
Table 3. Summary of the Reference Laboratory Performance for Soil Samples
Statistic
Mean (SD)d
Median
Range
Accuracy
(% recovery)
RDX
N = 20
102 (17)
99
84-141
TNT
N = 20
100 (23)
96
76-174
Precision"
(% RSD)
DNT*
Nr = 3c
56
32
14-123
HMX
Nr=13
29
30
12-63
RDX
Nr=13
25
21
4-63
TNT
Nr=18
29
25
2-72
"Calculated from those samples where all four replicates were reported as a detect.
6DNT represents total concentration of 2,4-DNT and 2,6-DNT.
CNR represents the number of replicate sets; N represents the number of individual samples
d (SD) = standard deviation calculated for the accuracy measurements only. The mean RSD may not be the best
representation of precision, but it is reported for convenient reference.
Table 4. Summary of the Reference Laboratory
Performance on Blank Samples
Statistic
Number of data points
Number of detects
% of fp results
Soil
DNT
20
0
0
HMX
20
0
0
RDX
20
0
0
TNT
20
2
10
10
-------
Section 5 — Technology Evaluation
Objective and Approach
The purpose of this section is to present a statistical
evaluation of the Spreeta Sensor data and determine
the technology's ability to measure TNT in
contaminated soil samples. The technology's
performance verification includes an evaluation of
comparability with SW-846 Method 8330 reference
laboratory data. Other aspects of the technology
(such as cost, sample throughput, hazardous waste
generation, and logistical operation) are also
evaluated in this section. Appendix A contains the
raw data provided by the vendor during verification
testing that were used to assess the performance of
the Spreeta Sensor. Appendix B contains a data
quality objective (DQO) example which uses the
performance information generated in this report.
This example illustrates the use of the Spreeta
Sensor in a real world application.
Precision
Precision is the reproducibility of measurements
under a given set of conditions. Precision was
determined for this technology by examining the
results of blind analyses for four replicates of a
sample and evaluating the frequency of all four
replicates being reported as the same interval. For
example, NR = 11 (11 sets of four replicates)
represents a total of 44 individual sample analyses.
A summary of the overall precision of the sensor for
the soil sample results is presented in Table 5. Some
inconsistencies occurred because TI reported
intervals that overlapped. In some cases where three
of the four intervals were reported consistently, the
fourth interval was different, but overlapped the
other three (e.g., the four replicates were reported as
3.0 to 9.0, 3.0 to 9.0, 3.0 to 9.0, and 4.0 to 14.0).
Overall, 85% of the analyses were precise, as either
all four or three of four replicates were reported
consistently.
Accuracy
Accuracy represents the closeness of the Spreeta
Sensor's measured concentrations to the known
content of spiked samples. A summary of the
Spreeta Sensor's overall accuracy relative to the
nominal spike concentration for the PE soils is
presented in Table 6. Note that the PE samples were
spiked with both TNT and RDX, but since this is a
sensor for TNT, accuracy was only evaluated for
that analyte. Of the 24 PE samples, the Spreeta
Sensor accurately reported an interval that included
the nominal spike concentration for 18 samples
(75% agreement). For the remaining samples, most
of the intervals were slightly below the nominal
concentration (21%), and only one sample had an
interval that was reported above (4% of total).
Performance acceptance ranges for the TNT-spiked
samples are shown in Table 7. These are the
guidelines established by the provider of the spiked
materials to gauge acceptable analytical results.
Because there is uncertainty in the true
concentration of TNT in the samples based on the
variability of the preparation method, these
acceptance ranges represent a window of results that
closely approximate the 95% confidence interval
about the nominal value. TI's reported intervals and
the reference laboratory results were compared with
these acceptance ranges. For all of those PE samples
with detectable levels of TNT, TI reported intervals
that overlapped with the acceptance ranges, where
the reference laboratory reported two samples
outside the acceptance ranges. For the four PE
samples which contained no spiked TNT, TI
reported three samples as 0 to 0.3 mg/kg
(acceptable), and one as 0.3 to 0.9 mg/kg. This was
the one sample listed in Table 6 as "above."For each
of the samples that were biased low, the upper limit
of the reported Spreeta Sensor interval was within
10% of the nominal concentration (e.g., TI reported
the result as 3 to 9 mg/kg, and the nominal
concentration was 10 mg/kg). Further, when
comparing the Spreeta Sensor interval to the
Table 5. Summary of Spreeta Sensor Precision
Precision
Frequency of replicate sets where all 4 were reported as
same interval
Frequency of replicate sets where 3 of 4 were reported as
same interval
Frequency of replicate sets where 2 of 4 were reported as
same interval
Frequency of replicate sets where none were reported as
same interval
N "
!>IR
11
12
4
0
Total NR
27
27
27
27
%
41
44
15
0
°NR represents the number of replicate sets.
11
-------
Table 6. Summary of Spreeta Sensor Accuracy: Comparison to
Nominal Value
Statistic
Agreement with nominal
Spreeta interval above nominal
Spreeta interval below nominal
No. of samples
18
1
5
Percentage
75
4
21
Table 7. Number of Spreeta Sensor and Reference Laboratory TNT
Results within Acceptance Ranges for Spiked Soils
TNT nominal
concentration
(mg/kg)
0°
10
50
100
250
500
Acceptance
range
(mg/kg)
Nondetect value "
7-13
35-63
70-126
174-315
348-630
No. of Spreeta
intervals that
overlapped range
3 of 4°
4 of 4
4 of 4
4 of 4
4 of 4
4 of 4
No. of reference
laboratory results
in range
4 of 4
4 of 4
4 of 4
4 of 4
3 of 4
3 of 4
" No TNT was spiked in this sample, so only a nondetect value was acceptable. TI reported one of
the four samples as 0.3 to 0.9 mg/kg.
acceptance ranges provided by the preparation
laboratory for the PE soils, the agreement was 96%.
False Positive/False Negative Results
Table 8 shows the Spreeta Sensor performance for
false positive (fp) results for blank samples. Of the
20 blank soils, TI reported TNT in two samples
(10% fp), as did the reference laboratory.
Table 9 summarizes the Spreeta Sensor's fp and fn
results relative to the reference laboratory results.
(See Section 3 for a more detailed discussion of this
evaluation.) For the environmental and spiked soils,
none of the TNT results were reported as false
positives relative to the reference laboratory results
(i.e., the laboratory reported the analyte as a
nondetect when TI reported it as a detect). In the
case where the laboratory reported a detection and
TI reported a nondetect (i.e., false negative), two of
the TNT results (3%) were false negatives. The two
false negative results were reported on replicate Fort
Ord samples that TI reported the sample results as 0
to 0.5 mg/kg, and the reference laboratory reported
each as 0.8 mg/kg. It is interesting to note that the
other two replicates were reported as 0 to 0.3 mg/kg
by TI and <0.5 mg/kg by the reference laboratory.
TI reported all four replicates consistently as
nondetects, and the reference laboratory reported
two of the replicates as slightly over the reporting
limits, which accounts for TI having two "false
negative" results.
Completeness
Completeness is defined as the percentage of
measurements that are judged to be usable (i.e., the
result was not rejected). Valid results were obtained
by the technology for all 108 soil samples.
Therefore, completeness was 100%.
Comparability
Comparability refers to how well the Spreeta Sensor
and reference laboratory data agreed. In this
evaluation, the laboratory results are not presumed
to be the "correct" answers. Rather, these results
represent what a typical fixed laboratory would
report for these types of samples. A one-to-one
sample comparison of the Spreeta Sensor results and
the reference laboratory results was performed for
all environmental and spiked samples that were
reported as a detection. (Please refer to Appendix A
to review the raw data. See Section 4 for a complete
evaluation of the reference laboratory performance.
Recall from Section 4 that the reference laboratory's
overall precision was mean RSD = 29% and overall
accuracy was mean recovery = 100%.)
12
-------
Table 8. Summary of Spreeta Sensor False
Positive Performance on Blank
Samples
Statistic
No. of data points
No. of fp results
% of total results that were fp
TNT reported
20
2
10%
Table 9. Summary of the Spreeta Sensor
Detect/Nondetect Performance
Relative to the Reference
Laboratory Results
Statistic
No. of results lab reported as non-detects
No. of fp results by Spreeta
% of total results that were fp
No. of results lab reported as detects
No. of fn results by Spreeta
% of total results that were fn
TNT
reported
14
0
0
74
2
3°
" See False Positive/False Negative Results section for details.
As shown in Table 10, most of the TI results (65%)
agreed with the reference laboratory, and the
majority of the remaining results (32%) were below.
Figure 3 represents graphically the comparison of
the Spreeta Sensor and reference laboratory results.
TFs results are plotted as the intervals reported. The
straight line represents the corresponding reference
laboratory results plotted against itself (slope =
1.00). As shown in Figure 3, TFs reported interval
generally (65% of the time) included the reference
laboratory result. For visual clarity, excluded from
Table 10. Summary of Spreeta Sensor
Comparability a
Statistic
Agreement
Spreeta interval above
Spreeta interval below
No. of
samples
69
3
34
Percentage
65
3
32
the graphs were the higher concentration (> 10,000
mg/kg) TNT samples; TI under reported the
concentrations of all twelve of these samples.
Figure 4 represents the absolute difference between
the Tl-reported interval and the reference laboratory
result. This graph includes data from 106 samples,
excluding two reference laboratory suspect values
(see Section 4 for more information). Figure 4
shows the absolute direction of disagreement that
indicates most (79%) of the Spreeta Sensor's
measurements were within ±10 mg/kg of the
reference laboratory result.
Sample Throughput
Sample throughput is representative of the estimated
amount of time required to prepare and analyze the
sample and perform the data analysis. Operating out
of a motor home, the two-person TI team
accomplished a sample throughput rate of
approximately 12 samples per day for thelOS soil
analyses. In order to isolate the reporting interval,
several tests had to be run per sample. TI averaged
four tests per sample to generate a final result.
Ease of Use
Two operators were used for the test because of the
number of samples and working conditions, but the
technology can be operated by a single person. The
Spreeta instrument does not inherently require any
particular skill level. Sample preparations and
dilutions do require use of a pipettor.
Cost Assessment
The purpose of this economic analysis is to estimate
the range of costs for analysis of explosives-
contaminated soil samples using the Spreeta Sensor
and a conventional analytical reference laboratory
method. The analysis is based on the results and
experience gained from this verification test, costs
are provided by TI, and representative costs are
provided by the reference analytical laboratories that
offered to analyze these samples. To account for the
variability in cost data and assumptions, the
economic analysis is presented as a list of cost
elements and a range of costs for sample analysis by
the Spreeta Sensor instrument and by the reference
laboratory.
Several factors affected the cost of analysis. Where
possible, these factors were addressed so that
decision makers can complete a site-specific
economic analysis to suit their needs. The following
categories are considered in the estimate:
sample shipment costs,
labor costs, and
equipment costs.
" Excludes two reference laboratory suspect result (total
N= 106).
13
-------
10 20 30 40 50 60 70 80 90
Reference laboratory TNT result (mg/kg)
100
100
Figure 3.
200
300
400
500
600
700
Reference laboratory TNT result (mg/kg)
Upper graph represents the comparison of the Spreeta
Sensor's results versus the reference laboratory for
concentrations <100 mg/kg, while the lower graph
represents the comparison for higher concentrations.
An extra tick mark in an interval (e.g., at reference
laboratory value of 400 mg/kg) indicates TI reported
the same lower limit in the interval, but two different
upper limits.
Each of these cost factors is defined and discussed
and serves as the basis for the estimated cost ranges
presented in Table 11. This analysis assumed that
the individuals performing the analyses were fully
trained to operate the equipment. Costs for sample
acquisition and preanalytical sample preparation,
which are tasks common to both methods, were not
included in this assessment.
Spreeta Sensor Costs
The costs associated with using the Spreeta Sensor
instrument included labor, equipment, and waste
disposal costs. No sample shipment charges were
associated with the cost of operating the instrument
because the samples were analyzed on site.
Labor
Labor costs included mobilization/demobilization,
travel, per diem expenses and on-site labor.
• Mobilization/demobilization. This cost element
included the time for one person to prepare for
and travel to each site. This estimate ranged
from zero (if the person is on site) to 5 h, at a
rateof$50/h.
• Travel. This element was the cost for the
analyst(s) to travel to the site. If the analyst is
14
-------
(0
3
(0 90
O
to
80 -
70-
0)
(/) 60
re
"S 50
0)
Q. 40
M- 30 -
O
% 20
.Q
E 10
0
Figure 4.
I
I
0 to 10
11 to 25
26 to 50
Greater Than 50
Distance from reference lab value (mg/kg)
Absolute difference between Spreeta Sensor result and reference laboratory result.
Table 11. Estimated Analytical Costs for Explosives-Contaminated Samples
Analysis method:
Analyst/manufacturer:
Sample throughput:
Spreeta Sensor
Texas Instruments
12 samples/day
Analysis method:
Analyst/manufacturer:
Typical turnaround:
EPA SW-486 Method 8330
Reference laboratory
21 working days
Cost category
Cost ($)
Cost category
Cost ($)
Sample shipment
0
Sample shipment
Labor
Overnight shipping
100-200
50-150
Labor
Mobilization/demobilization 0-250
Travel 0-1,000 per analyst
Per diem expenses 0-150/day per analyst
Rate 30-75/h per analyst
Equipment
Mobilization/demobilization 0-150
Instrument purchase price to be determined
Reagents/supplies <$1 per sample
(expected)
Labor
Mobilization/demobilization Included"
Travel Included
Per diem expenses Included
Rate 150-188 per sample
Equipment
Included
" "Included" indicates that the cost is included in the labor rate.
15
-------
located at the site, travel cost to the site would
be zero. The estimated cost of an analyst
traveling to the site for this verification test
($1000) included the cost of airline travel and
rental car fees.
• Per diem expenses. This cost element included
food, lodging, and incidental expenses. The
estimate ranged from zero (for a local site) to
$150/day for each analyst.
Rate. The cost of the on-site labor was estimated
at a rate of $30-$75/h, depending on the
required expertise level of the analyst. This cost
element included the labor involved with the
entire analytical process, comprising sample
preparation, sample management, analysis, and
reporting.
Equipment
Equipment costs included mobilization/
demobilization, rental fees or purchase of
equipment, and the reagents and other consumable
supplies necessary to complete the analysis.
• Mobilization/demobilization. This included the
cost of shipping the equipment to the test site. If
the site is local, the cost would be zero. For this
verification test, the cost of shipping equipment
and supplies was estimated at $150.
• Instrument purchase. The current version of this
instrument applicable to TNT detection is at the
pre-commercial stage, and TI has not
determined the retail price.
• Reagents/supplies. These items are consumable
and are purchased on a per sample basis. TI
estimates that this will be less than $1 per
sample, once the instrument is available
commercially.
Reference Laboratory Costs
Sample Shipment
Sample shipment costs to the reference laboratory
included the overnight shipping charges, as well as
labor charges associated with the various
organizations involved in the shipping process.
• Labor. This cost element included all of the
tasks associated with the shipment of the
samples to the reference laboratory. Tasks
included packing the shipping coolers,
completing the chain-of-custody documentation,
and completing the shipping forms. The
estimate to complete this task ranged from 2 to
4 h at $50/h.
• Overnight shipping. The overnight express
shipping service cost was estimated to be $50
for one 50-lb cooler of samples.
Labor, Equipment, and Waste Disposal
The labor bids from commercial analytical reference
laboratories that offered to perform the reference
analysis for this verification test ranged from $150
to $188 per sample. The bid was dependent on many
factors, including the perceived difficulty of the
sample matrix, the current workload of the
laboratory, and the competitiveness of the market.
This rate was a fully loaded analytical cost that
included equipment, labor, waste disposal, and
report preparation.
Cost Assessment Summary
An overall cost estimate for use of the Spreeta
Sensor instrument versus use of the reference
laboratory was not made because of the extent of
variation in the different cost factors, as outlined in
Table 11. The overall costs for the application of
any technology would be based on the number of
samples requiring analysis, the sample type, and the
site location and characteristics. Decision-making
factors, such as turnaround time for results, must
also be weighed against the cost estimate to
determine the value of the field technology's
providing immediate answers versus the reference
laboratory's provision of reporting data within
30 days of receipt of samples.
Miscellaneous Factors
The following are general observations regarding
the field operation and performance of the Spreeta
Sensor instrument:
The system, which weighs approximately 2 Ib,
was easily transported to the field.
The technology could have been operated
outdoors, as there was no AC power
requirement (a lantern battery was used), but TI
elected to work out of a motor home to simulate
a mobile laboratory environment.
No organic solvents were used for soil
extraction, only buffered deionized water.
Waste generated during the analyses was rather
innocuous.
An extraction efficiency correction (40%) was
applied by TI to all Spreeta results. This
extraction efficiency was determined on one soil
sample prior to the verification test. The
extraction efficiency most likely varies from
soil-to-soil and may have effected the results.
TI had to regenerate their sensors after arriving
on-site, as they learned that the sensor surface
was changed after shipment by airplane. The
problem appeared to be easily corrected.
TI used approximately 25 sensors to analyze the
108 samples. Over 500 tests were performed
during the verification test, and the Spreeta
Sensor was replaced after about every 20 tests.
Although this particular application for TNT
detection has not been commercially released,
the Spreeta Sensor is currently available in the
form of an evaluation kit from TI
(www.ti.com.spreeta). In addition, a life
sciences R&D instrument, based on Spreeta, is
due to be released during the third quarter of
2001 from Prolinx, Inc. (www.plinx.com).
16
-------
Other nitroaromatic compounds (such as
trinitrobenzene and dinitrotoluene) will respond
to the sensor and be quantified as TNT.
Some scatter in the TI results may have been
attributed to the use of a small sample size (0.5
g).
Early on in the verification test, TI elected to
reduce the amount of time that the sensor was
rinsed before and after sample exposure,
therefore reducing the amount of analysis time
for each test from 14 min to 7 min. This did not
appear to affect the results.
Waste generated during the test consisted of
12 L of nonregulated aqueous buffered solutions
(i.e., no hazardous waste generated).
Summary of Performance
A summary of performance is presented in Table 12.
Precision defined as the frequency that TI reported
replicate sets consistently. In 85% of the replicates
sets, TI reported either all four as the same interval
or three of four as the same interval. Accuracy,
defined as the percentage of the Spreeta Sensor
results which agreed with the spiked concentration,
was 75%, indicating that the soil results were
unbiased. Of the 20 blank soils, TI reported TNT in
two samples (10% false positives). Additionally,
false positive and false negative results were
determined by comparing the Spreeta Sensor result
to the reference laboratory result for the
environmental and spiked samples. None of the
TNT results were reported as false positives relative
to the reference laboratory results, but 3% of the
results were false negatives.
The verification test found that the Spreeta Sensor
instrument was relatively simple for a trained
analyst to operate in the field, requiring less than an
hour for initial setup. The sample throughput of the
Spreeta Sensor was twelve samples per day. Two
operators analyzed samples during the verification
test, but the technology can be run by a single
trained operator. The overall performance of the
Spreeta Sensor for the analysis of TNT was
characterized as unbiased for low concentration
(< 10,000 mg/kg) samples and precise for soil
analyses.
Table 12. TNT Performance Summary for the Spreeta Sensor
Feature/Parameter
Precision
Accuracy
False positive results on blank
samples
False positive results relative to
reference laboratory results
False negative results relative to
reference laboratory results
Comparison with reference
laboratory results (all data,
excluding suspect values)
Completeness
Weight
Sample throughput (2 operators)
Power requirements
Training requirements
Cost
Performance summary
Frequency of replicate sets where all 4 were reported as same interval:
Frequency of replicate sets where 3 of 4 were reported as same interval:
Frequency of replicate sets where 2 of 4 were reported as same interval:
Frequency of replicate sets where 0 of 4 were reported as same interval:
41%
44%
15%
0%
% agreement with nominal concentration: 75%
% Spreeta interval below nominal concentration: 21%
% Spreeta interval above nominal concentration: 4%
10%
0%
3%
% agreement with laboratory result: 65%
% Spreeta interval below laboratory result: 32%
% Spreeta interval above laboratory result: 3%
100% of 108 soil samples
21b
12 samples per day
250 mA at 6V (A lantern battery was used in the verification test.)
One-half day technology-specific training
To be determined after commercially-available
17
-------
Section 6 — Technology Update
In this section, the vendor (with minimal editorial changes by ORNL) provides information regarding new
developments with its technology since the verification activities. In addition, the vendor provides a list of
representative applications in which its technology has been used.
Technology Update
Spreeta Sensor technology is a low-cost
immunoassay platform that can be applied to
essentially any biosensing application (Melendez et
al. 1996, Melendez et al. 1997, Elkind et al. 1999,
Strong et al. 1999). It can be made available for
license by suitable equipment manufacturers. The
TNT assay demonstrated here could, in principle, be
replicated for any small molecule for which
antibodies can be generated. Such developments are
currently under way.
During this verification test, dilutions of >3000x
were inadvertently not performed, and so the
reported concentration of TNT in samples over
3000 ppm was accidentally underestimated. This
procedural problem adversely affected the accuracy
of 12 out of the 108 samples tested here.
The Spreeta Sensor as well as the sample
preparation protocols used here were development
prototypes and, therefore, not optimized for speed.
In the commercial version, the cycle time for this
test should be <4 min, including system clean-out
and sensor regeneration steps. The number of tests
needed to quantify TNT for each soil sample will
vary with dilution strategy, but with 3 x dilution
steps and with a 0.3 tolOO,000 ppm TNT
concentration dynamic range, the number of tests
will average between four and five.
18
-------
Section 7 — References
ASTM (American Society for Testing and Materials). 1997a. Standard Practice for Generation of
Environmental Data Related to Waste Management Activities: Quality Assurance and Quality Control
Planning and Implementation, D5283-92.
ASTM (American Society for Testing and Materials). 1997b. Standard Practice for Generation of
Environmental Data Related to Waste Management Activities: Development of Data Quality Objectives,
D5792-95.
Berger, W., H. McCarty, and R-K. Smith. 1996. Environmental Laboratory Data Evaluation. Genium
Publishing Corp., Schenectady, N.Y.
Elkind, J., D. I. Stimpson, Anita A. Strong, D. U. Bartholomew, and J. L. Melendez. 1999. "Integrated
Analytical Sensors: The Use of the TISPR-1 as a Biosensor," Sensors and Actuators-B 54,182-90.
EPA (U.S. Environmental Protection Agency). 1994. "Method 8330: Nitroaromatics and Nitramines by High
Performance Liquid Chromatography (HPLC)." Test Methods for Evaluating Solid Waste: Physical/
Chemical Methods, Update II. SW-846. U.S. Environmental Protection Agency, Washington, B.C.,
September.
EPA (U.S. Environmental Protection Agency). 1996. Guidance for Data Quality Assessment, EPA QA/G-9,
EPA/600/R-96/084, Washington, D.C., July.
Jenkins, T. F., M. E. Walsh, and P. G. Thorne. 1998. Site Characterization for Explosives Contamination at a
Military Firing Range Impact Area. Special Report 98-9. U.S. Army Cold Regions Research and Engineering
Laboratory, Hanover, N.H. Available at http://www.crrel.usace.army.mil .
Maskarinec, M. P., C. K. Bayne, L. H. Johnson, S. K. Holladay, R. A. Jenkins, and B. A. Tomkins. 1991.
Stability of Explosives in Environmental Water and Soil Samples. ORNL/TM-11770. Oak Ridge National
Laboratory, Oak Ridge, Tenn., January.
Melendez, J., R. Carr, D. U. Bartholomew, K. Kukanskis, J. Elkind, S. Yee, C. Furlong, and R. Woodbury.
1996. "A Commercial Solution for Surface Plasmon Sensing," Sensors andActuators-B 35, 1-5.
Melendez, J., R. Carr, D. U. Bartholomew, H. Taneja, S. Yee, C. Jung, and C. Furlong. 1997. "Development
of a Surface Plasmon Resonance Sensor for Commercial Applications," Sensors and Actuators-B 38-39,
375-79.
ORNL (Oak Ridge National Laboratory). 1999. Technology Demonstration Plan: Evaluation of Explosives
Field Analytical Techniques. Oak Ridge National Laboratory, Oak Ridge, Tenn., August.
ORNL (Oak Ridge National Laboratory). 2000. Technology Demonstration Plan: Evaluation of Explosives
Field Analytical Techniques. Oak Ridge National Laboratory, Oak Ridge, Tenn., August.
Sachs, Lothar. 1984. Applied Statistics: A Handbook of Techniques, 2nd ed., Springer-Verlag, New York.
Specialized Assays, Inc. 1999. Comprehensive Quality Assurance Plan. SAL-QC-Rec 5.0, January 6.
Strong, A., D. I. Stimpson, D. U. Bartholomew, T. F. Jenkins, and J. L. Elkind. 1999. "Detection of
Trinitrotoluene (TNT) Extracted from Soil Using a Surface Plasmon Resonance (SPR)-Based Sensor
Platform, SPIE—The International Society for Optical Engineering, Detection and Remediation
Technologies for Mines and Mine-like Targets IV, 3710, 362-72.
19
-------
Appendix A
TI's Spreeta Sensor Sample Results Compared
with Reference Laboratory Results
Sample site
or type
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Blank
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Fort Ord
Sample
no.
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
1
1
1
1
2
2
2
2
3
3
3
3
Sample
replicate
1
2
3
4
1
2
o
J
4
1
2
o
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
o
J
4
1
2
o
J
4
TNT concentration (mg/kg)
TI lower
interval
0.0
0.0
0.0
0.0
0.3
0.3
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.4
0.4
0.3
0.4
0.0
0.0
0.0
0.0
TI upper
interval
0.3
0.3
0.5
0.3
0.9
0.9
0.5
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.5
0.5
0.3
1.4
1.4
0.9
1.4
0.3
0.3
0.5
0.3
Reference
laboratory
<0.5
0.5
0.5
O.5
0.5
70900.0
O.5
O.5
0.5
O.5
O.5
0.5
0.9
O.5
0.5
0.5
O.5
0.5
0.5
0.5
0.5
0.8
0.8
O.5
0.8
2.1
0.8
0.8
0.5
O.5
O.5
0.5
TI
analysis
order"
1057
1010
1072
1044
1030
1066
1048
1065
1076
1101
1040
1089
1051
1053
1063
1009
1059
1029
1054
1058
1074
1094
1005
1020
1039
1037
1069
1016
1023
1013
1034
1007
20
-------
Sample site
or type
Iowa
Iowa
Iowa
Iowa
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Louisiana
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Milan
Sample
no.
1
1
1
1
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
Sample
replicate
1
2
3
4
1
2
o
J
4
1
2
o
5
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
o
J
4
1
2
o
J
4
1
2
3
4
1
2
3
4
TNT concentration (mg/kg)
TI lower
interval
900.0
900.0
1500.0
900.0
30.0
30.0
30.0
30.0
30.0
90.0
15.0
15.0
90.0
90.0
90.0
90.0
40.0
90.0
40.0
40.0
0.9
0.9
0.9
0.9
0.0
0.3
0.0
0.3
90.0
90.0
90.0
90.0
30.0
30.0
30.0
30.0
3.0
3.0
3.0
3.0
TI upper
interval
3000.0
3000.0
4500.0
3000.0
90.0
90.0
90.0
90.0
90.0
300.0
45.0
45.0
300.0
300.0
300.0
300.0
140.0
300.0
140.0
140.0
3.0
3.0
3.0
3.0
0.5
0.9
0.5
0.9
300.0
300.0
300.0
300.0
90.0
90.0
90.0
90.0
9.0
9.0
9.0
9.0
Reference
laboratory
20400.0
0.8
33400.0
28300.0
109.0
120.0
111.0
125.0
50.0
51.0
51.0
10.6
205.0
170.0
300.0
400.0
89.0
78.0
81.5
67.5
2.7
1.1
1.4
1.7
0.5
0.5
O.5
O.5
190.0
270.0
320.0
273.0
220.0
260.0
80.0
162.0
11.5
10.2
11.3
10.6
TI
analysis
order"
1041
1022
1071
1004
1032
1061
1019
1095
1067
1006
1064
1060
1015
1104
1068
1108
1049
1021
1090
1056
1045
1018
1075
1087
1096
1036
1028
1002
1014
1106
1077
1105
1031
1008
1085
1078
1046
1098
1027
1035
21
-------
Sample site
or type
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Spike/PE
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Volunteer
Sample
no.
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
6
6
6
6
1
1
1
1
2
2
2
2
3
3
3
3
Sample
replicate
1
2
3
4
1
2
o
J
4
1
2
o
J
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
o
J
4
1
2
o
J
4
1
2
3
4
TNT concentration (mg/kg)
TI lower
interval
40.0
40.0
30.0
40.0
0.0
0.3
0.0
0.0
3.0
3.0
4.0
3.0
15.0
30.0
30.0
30.0
150.0
90.0
150.0
150.0
300.0
300.0
300.0
300.0
1500.0
1500.0
900.0
1500.0
1500.0
900.0
900.0
900.0
3.0
3.0
3.0
3.0
TI upper
interval
140.0
140.0
90.0
140.0
0.3
0.9
0.3
0.3
9.0
9.0
14.0
9.0
45.0
90.0
90.0
90.0
450.0
300.0
450.0
450.0
900.0
900.0
900.0
900.0
4500.0
4500.0
3000.0
4500.0
4500.0
3000.0
3000.0
3000.0
9.0
9.0
9.0
9.0
Reference
laboratory
81.8
104.0
90.0
124.0
0.5
0.5
O.5
O.5
8.4
7.6
10.0
8.5
47.5
48.5
48.5
47.0
230.0
205.0
435.0
205.0
535.0
505.0
675.0
510.0
108000.0
75500.0
117000.0
61000.0
11300.0
12600.0
26200.0
8920.0
12.0
10.3
13.8
10.4
TI
analysis
order"
1079
1091
1011
1038
1083
1047
1024
1050
1017
1042
1033
1003
1073
1055
1097
1107
1099
1093
1052
1084
1062
1026
1082
1001
1043
1103
1025
1080
1102
1081
1100
1070
1012
1092
1088
1086
"These are the sample numbers from which the analysis order can be discerned. For example, 1001 was analyzed first,
then 1002, etc.
22
-------
Appendix B
Data Quality Objective (DQO) Example
Disclaimer
The following hypothetical example serves to demonstrate how the information provided in this report may
be used in the data quality objectives (DQO) process. This example serves to illustrate the application of
quantitative DQOs to a decision process, but it cannot attempt to provide a thorough education in this topic.
Please refer to other educational or technical resources for further details. Additionally, because the focus of
this report is on the analytical technology, this example makes simplifying assumptions (such as the sample is
homogeneous and the reference laboratory results represent the true concentration) in the example that may
not be valid in the real world.
Background and Problem Statement
An Army Ammunition Plant that produced TNT was recently decommissioned. Past practices had resulted in
contamination of four areas around the plant. Soils at each site were mixtures of clay, silt, and organic matter
with initial concentrations of about 1500 mg/kg of TNT. Forty cubic yards (40 yd3) of TNT-contaminated
soil were loaded into a bioreactor. After three months of processing, the soil mixture was dewatered and put
into drums. The simplifying assumption was made that the soil in each drum was homogeneous based on
process knowledge. In agreement with regulators, the treatment goal established for the site was to reduce
the soil concentration to < 15 mg/kg of TNT. Soil with < 15 mg/kg of TNT would be returned to the four
areas around the plant. Those drums containing soil with TNT concentrations > 15 mg/kg would be stored
for additional processing.
The company's DQO team considered using Texas Instruments' Spreeta Sensor to measure the TNT
concentration in each drum, based on the data generated in the ETV study. The plan was to randomly select
soil samples from each drum and determine the TNT concentration with the TI Spreeta Sensor. In the ETV
test, the TNT concentrations measured by the TI Spreeta Sensor were reported in variety of different intervals
some of which overlapped. The maximum concentration in seven intervals was < 15 mg/kg, and eight
intervals reported a minimum concentration as > 15 mg/kg. The DQO team decided that a drum would be
sent to storage if any of the results from the TI Spreeta Sensor indicated a concentration > 15 mg/kg.
General Decision Rule
If all of the TNT analyses indicate concentrations of < 15 mg/kg then return the soil to the plant
areas.
If any of the TNT analyses indicate > 15 mg/kg then send the soil to a storage warehouse.
DQO Goals
The DQO team's primary goal was to calculate how many samples would need to be analyzed by the Spreeta
Sensor in order to confidently make a decision about remediating the processed soil, given the uncertainties
of the technology's results. Because the team decided that inadvertently returning soil that exceeded 15
mg/kg of TNT was the worst possible mistake, the number of samples measured is primarily related to this
false-rejection decision error rate. A secondary decision error would be to unnecessarily store a drum that
contained TNT concentrations < 15 mg/kg which would be a false-acceptance decision error. Consideration
of both the false-rejection decision error and the false-acceptance decision error was used to determine the
final sampling plan.
EPA required that a sufficient number of samples be measured from each drum so that the false-rejection
error rate (FR) for the decision rule was 0.05 or less if the true drum concentration was 15 mg/kg or greater.
This DQO goal represents a 5% chance of returning a drum containing 15 mg/kg or more of TNT to the plant
area.
The DQO team did not want to store and reprocess an excessive number of drums if a drum's TNT
concentration was < 15 mg/kg because of the expense. Therefore, the DQO team recommended that the
false-acceptance error rate (FA) for the decision rule be 0.10 if the true drum concentration was < 15 mg/kg.
That is, there would be a 10% chance of storing and reprocessing a drum if the true TNT concentration for a
23
-------
drum was < 15 mg/kg.
Determining the Number of Samples
The number of samples needed to satisfy the FR and FA requirements depends on the misclassification error
rates of the Spreeta Sensor. Two types of misclassifications have to be considered:
1. Underestimating the TNT concentration (Py) — classifying a sample concentration to be < 15 mg/kg
when the true TNT concentration is > 15 mg/kg.
2. Overestimating the TNT concentration (P0) — classifying a sample concentration to be > 15 mg/kg when
the true TNT concentration is < 15 mg/kg.
The probabilities PU and P0 are relative to the target value (15 mg/kg) and depend on the true TNT
concentration of the sample. These probabilities will decrease with distance of the true concentration from
the target value. Ideally, a project-specific experiment should be run with replicate samples having TNT
concentrations near the target value but have TNT concentrations that are both below and above the target
value. The ETV verification results do not provide sufficient information to make good estimates of PU and
P0. However, we will use the ETV results to illustrate this example even though the data clearly
underestimates the misclassification errors.
The ETV verification results for 108 analyses of performance evaluation soil samples and environmental soil
samples will be used to estimate the error rates for the two types of misclassifications as follows. For the 5 1
samples where the reference values were > 15 mg/kg, Spreeta underestimated the TNT concentration one
time, or 1/51, for an estimated probability of underestimation Py = 0.020. Note many of the TNT
concentrations were much higher than 15 mg/kg so Pv is most likely too small. For the 57 samples where the
reference values < 15 mg/kg, Spreeta overestimated the TNT concentration two times, for an estimated
probability of overestimation P0 = 2/57 = 0.035.
The probability distribution of classifying the number of soil samples in different concentration intervals
follows a binomial probability distribution (Sachs 1984). This probability distribution and the requirements
for FR and FA can be used to determine the number of samples to meet the DQO goals. The FR for the
decision rule is related to Pv by
FR = Pr[ All Spreeta results < 15 mg/kg for TNT * 15 mg/kg ] = (PV)N (Eq. B-l)
The FR error rate decreases as the sample size increases. The sample size is solved as
,r log(Ffl)
=
-------
FA = Pr[ Some of Spreeta results > ISmg/kgfor TNT < I5mg/kg ] = !-(!- PQ)N (Eq. B-3)
The error rate of a FA decision (sending a drum to storage and reprocessing) actually increases with
increasing sample size because the chance that the Spreeta Sensor will overestimate a concentration increases
with continued testing. The sample size required to meet the FA requirement is
A, log( I - FA)
"
where
N = number of samples from a drum to be measured
FA = false-acceptance decision error rate (e.g., FA = 0.10)
P0 = probability of overestimating a TNT concentration
N= log(l - 0.10) = -0.046 =
log(l - 0.035) -0.016
The sample size must be rounded up to N = 3 (fractions of a sample analysis are not possible). When N = 3,
the value of FA percentage is 10.2% which is only slightly higher than the DQO team's goal of 10% . By
taking three samples from the drum, the probability with regard to false-rejection results improves. That is,
FR percentage decreases to 0.0008%.
Therefore, the DQO team in this example decided that the sampling procedure would be to randomly select
three soil samples from each drum and analyze the sample with the Spreeta Sensor. The DQO team would
return a soil drum to the excavated area if all TNT concentrations were < 15 mg/kg, and store the soil drum
for reprocessing if any of the TNT concentrations were > 15 mg/kg. The DQO team's goals of a 5% for the
FR percentage and 10% for the FA percentage would be met by this sampling plan.
Decision Rule for 5% FR Percentage and 10% FA Percentage
If three randomly selected soil sample has a Spreeta Sensor result reported in an interval < 15 mg/kg
then return the soil drum to the excavated area.
If one or more of the three randomly selected soil samples has a Spreeta Sensor result in the interval
> 15mg/kg then store the soil drum for additional processing.
Worst Case Pv and P0 Estimates
The DQO team used all 108 samples in the EPA ETV verification test to estimate the Pv and P0 because the
number of performance evaluation samples were not sufficient in the range of interest (15 mg/kg). Because
this analysis is based on the reference laboratory data and not the actual true concentration, the determination
for the required number of samples was recalculated using the upper 95% confidence limits (i.e., higher
possible values considering the uncertainties on the Pv and P0 values), representing a "worst case" scenario.
The values of the upper 95% confidence limits for the two probabilities were 0.090 for Pv and 0.106 for P0.
Calculations with these upper limit values determine the number of samples to be 2, a FR percentage of
0.8%, and a FA percentage of 20.1%. These results show that the regulatory DQO would still be met, but the
chance of storing "good" drums increases. If we keep the original number of samples (N =3), the calculated
error rates using the upper 95% confidence limits on Pv and Pq would give a FR percentage of 0.07% and FA
percentage of 28.5%. Considering the uncertainties in Pv and in Pp, the regulatory DQO will also be met
with the original sampling plan but the probability uncertainties will have the greater affect on the FA error
rate.
25
------- |