September 2006
Environmental Technology
Verification Report
PROTEIN-BIOSENSOR
OP-Stick Sensor
Prepared by
Battelle
Battelle
I he Business of Innovation
Under a cooperative agreement with
V8* trr\ U.S. Environmental Protection Agency
ET1/ET1/ET1/
-------
September 2006
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Protein-Biosensor
OP-Stick Sensor
By
Raj Mangaraj
Stephanie Buehler
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at http://www.epa.gov/
etv/centers/centerl .html.
in
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. Many thanks go to Battelle's
Hazardous Materials Research Center for providing the facilities for and personnel capable of
working with chemical warfare agents. We sincerely appreciate the contribution of drinking
water samples from the Metropolitan Water District of Southern California (Paul Rochelle and
Melinda Stalvey), the New York Department of Environmental Protection (Virginia Murray),
and Orange County Utilities, Orlando, Florida (Theresa Slifko and Liza Robles). We would also
like to thank Armah de la Cruz (U.S. EPA, National Exposure Research Laboratory), Ricardo
DeLeon (Metropolitan Water District of Southern California), Yves Mikol (New York City
Department of Environmental Protection), and Helen Schurz Rogers (Centers for Disease
Control and Prevention National Center for Environmental Health) for their careful review of the
test/QA plan and this verification report.
IV
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations vii
Chapter 1 Background 1
Chapter 2 Technology Description 2
Chapter 3 Test Design 4
3.1 Introduction 4
3.2 Test Samples 4
3.2.1 PT Samples 5
3.2.2 DW Samples 6
3.2.3 QC Samples 7
3.2.4 Operational Factors 7
3.3 Verification Schedule 8
3.4 Test Procedure 8
3.4.1 Test Sample Preparation and Storage 8
3.4.2 Test Sample Analysis Procedure 8
3.4.3 Drinking Water Characterization 9
Chapter 4 Quality Assurance/Quality Control 11
4.1 Sample Chain-of Custody Procedures 11
4.2 QC Samples 11
4.3 Equipment/Calibration 12
4.4 Characterization of Stock Solutions 13
4.5 Audits 13
4.5.1 Performance Evaluation Audit 13
4.5.2 Technical Systems Audit 14
4.5.3 Audit of Data Quality 15
4.6 QA/QC Reporting 15
4.7 Data Review 15
Chapter 5 Statistical Methods and Reported Parameters 17
5.1 Accuracy 17
5.2 False Positive/False Negative Rates 17
5.3 Precision 18
5.4 Potential Matrix and Interferent Effects 18
5.5 Operational Factors 18
Chapter 6 Test Results 19
6.1 Accuracy 19
6.2 False Positive/False Negative Rates 20
6.3 Precision 27
-------
6.4 Potential Matrix and Interferent Effects 27
6.4.1 Interferent PT Samples 27
6.4.2 DW Samples 28
6.5 Operational Factors 28
6.5.1 Technical Operators 28
6.5.2 Non-Technical Operator 29
Chapter 7 Performance Summary 31
Chapters References 38
Figures
Figure 2-1. OP-Stick Sensor Results Analysis 2
Figure 6-1. Side View of PPE Worn by Non-Technical Operator 30
Figure 6-2. Testing the OP-Stick Sensor with the Non-Technical Operator Wearing PPE 30
Tables
Table 3-1. Lethal Dose of Target Contaminants 5
Table 3-2. Performance Test Samples 6
Table 3-3. Drinking Water Samples 7
Table 3-4. ATEL Water Quality Characterization of Drinking Water Samples 10
Table 4-1. Reference Methods for Target Contaminants and Interferents 12
Table 4-2. Performance Evaluation Samples and Percent Difference 14
Table 4-3. Summary of Data Recording Process 16
Table 6-1. Contaminant-Only PT Sample Results 20
Table 6-2a. VX False Positive/Negative Results 22
Table 6-2b. GB False Positive/False Negative Results 23
Table 6-2c. GD False Positive/False Negative Results 24
Table 6-2d. Aldicarb False Positive/False Negative Results 25
Table 6-2e. Dicrotophos False Positive/False Negative Results 26
Table 7-la. VX Summary Table 32
Table 7-lb. GB Summary Table 33
Table 7-lc. GD Summary Table 34
Table 7-Id. Aldicarb Summary 35
Table 7-le. Dicrotophos Summary Table 36
VI
-------
List of Abbreviations
AChe
AMS
ASTM
ATEL
Ca
CWA
DI
DPD
DW
BCD
EPA
ETV
GB
GC
GD
HAZWOPER
HOPE
HMRC
ICP
kg
L
LC
LD50
LOD
LRB
MB
Mg
Hg/L
mg/L
mL
MS
uMHO
acetylcholinesterase
Advanced Monitoring Systems
American Society for Testing and Materials
Aqua Tech Environmental Laboratories
calcium
chemical warfare agent
deionized
diethyl-p-phenylene diamine
drinking water
electron capture detection
U.S. Environmental Protection Agency
Environmental Technology Verification
sarin
gas chromatography
soman
Hazardous Waste Operations and Emergency Response
high density polyethylene
Hazardous Materials Research Facility
inductively coupled plasma
kilogram
liter
liquid chromatography
lethal dose for half of test subjects
limit of detection
laboratory record book
method blank
magnesium
microgram per liter
milligram per liter
milliliter
mass spectrometry
micromho
vn
-------
NDR negative differential resistance
NTU nephelometric turbidity unit
OP organophosphate
PE performance evaluation
PPE personal protective equipment
PT performance test
QA quality assurance
QC quality control
QMP quality management plan
SCBA self-contained breathing apparatus
SM standard method
SOP standard operating procedure
ISA technical systems audit
Vlll
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
recently evaluated the performance of the Protein-Biosensor OP-Stick Sensor in detecting
chemical agents, carbamate pesticides, and organophosphate (OP) pesticides in drinking water.
Enzymatic test kits were identified as a priority technology category for verification through the
AMS Center stakeholder process.
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This verification report provides
results for testing the OP-Stick Sensor. Following is a description of the OP-Stick Sensor, based
on information provided by the vendor. The information provided below was not verified in this
test.
The OP-Stick Sensor is an enzymatic colorimetric assay designed for detecting organophosphate
(including thiophosphate) and carbamate (OP/C) pesticide residues in water, soil, and food. This
technology had not been used to test for chemical warfare agents (CWA) prior to this verification
test. This assay is a field diagnostic test that measures acetylcholinesterase (AChE) activity and
is based on an enzyme engineered for increased sensitivity to OP and C pesticides.
When not in presence of inhibiting pesticides, AChE hydrolyzes acetylthiocholine to thiocholine,
which reacts with a colorimetric substrate on a test stick (Figure 2-1) to produce a brown color.
In the presence of OP/Cs (which are oxidized during the test to an "oxon" form), AChE is
irreversibly inhibited and color formation is reduced or absent depending on the pesticide
concentration. The intensity of the brown color is inversely proportional to OP/C concentration.
Lower spot indicates the presence of insecticide by
Upper spot (reference) contrast with the reference.
Presence of insecticide in the
solution tested
No insecticide in the solution
tested
Stick before any test
Figure 2-1. OP-Stick Sensor Results Analysis
Detection limits for the various
OP/Cs differ depending on their
ability to inhibit the enzyme.
Combinations of various OP/Cs
will have an additive effect on the
inhibition assay. The test allows
screening without any laboratory.
Positive tests would need
confirmation by further analysis
for qualitative and quantitative
assay.
-------
One OP-Stick Sensor kit is composed of three tubes each labeled with a colored sticker and one
test stick. Tube 1 (labeled yellow) contains an oxidizing agent for phosphorothioate activation in
an "oxon" form. Tube 2 (labeled blue) contains a neutralizing agent to avoid denaturation of
AChE by the reagent from Tube 1. Tube 3 (labeled red) contains the chromogen reagent. The
OP-Stick Sensor kit is 10 by 5 by 2 centimeters. The price of the kit, which can be used for one
test, is approximately $20.
-------
Chapter 3
Test Design
3.1 Introduction
Enzymatic test kits, generally designed to be handheld and portable, detect the presence of
chemical agents, carbamate pesticides, and/or OP pesticides by relying on the reaction of the
cholinesterase enzyme. Under normal conditions, the enzyme reacts as expected with other
chemicals present in the test kit. The activity of the enzyme is inhibited, however, by chemical
agents, carbamate pesticides, and OP pesticides. The effects of this inhibition will then generally
lead to a color change, indicating the presence or absence of these compounds.
The objective of this verification test was to evaluate the ability of the OP-Stick Sensor to detect
chemical agents, carbamate pesticides, and OP pesticides in drinking water. This verification test
assessed the performance of the OP-Stick Sensor relative to
• Accuracy
• False positive and negative rates
• Precision
• Potential matrix and interference effects
• Operational factors (operator observations, ease of use, and sample throughput).
3.2 Test Samples
This test evaluated the ability of the OP-Stick Sensor to detect VX, sarin (GB), soman (GD)
(chemical agents); aldicarb (carbamate pesticide); and dicrotophos (OP pesticide) in performance
test (PT) and drinking water (DW) samples. Quality Control (QC) samples were also included as
part of the test matrix to ensure the integrity of the test. Contaminants were tested individually,
and stock solutions of each contaminant were prepared separately in American Society for
Testing and Materials (ASTM) Type II deionized (DI) water. Samples were prepared in the
appropriate matrix using these stock solutions and analyzed on the same day. To minimize the
loss of analytes to hydrolysis, contaminant stock solutions prepared in DI water were made on a
daily basis. Chemical agent stock solutions were prepared twice daily, once in the morning and
once in the afternoon. Aliquots of each stock solution were diluted to the appropriate
concentration using volumetric glassware and volumetric or calibrated pipettes. In some cases,
reference solutions were prepared in ASTM Type II DI water using the stock solutions used to
-------
prepare the test samples. In other cases, the actual stock solutions were submitted for
concentration confirmation by the respective reference analysis (Table 4-1). Aqua Tech
Environmental Laboratories, Inc. (ATEL) of Marion, OH performed the physiochemical
characterization for each type of DW sample along with reference analyses of the interferent
solutions. All other reference analyses were performed at Battelle.
3.2.1 PT Samples
PT samples were prepared separately in ASTM Type IIDI water for each contaminant. The first
type of PT samples consisted of ASTM Type II DI water spiked with the contaminant at five
different concentrations: the lethal dose concentration given in Table 3-1 for each contaminant,
along with dilutions at approximately 10, 100, 1,000, and 10,000 times less than the lethal dose.
The contaminants were added individually to each spiked sample. The lethal dose of each
contaminant was determined by calculating the concentration at which 250 milliliters (mL) of
water is likely to cause the death of a 70-kilogram (kg) person based on human oral LD50 (lethal
dose for half of the test subjects) data.(1>2) Human oral LD50 data were not available for aldicarb,
so rat oral LDso data were used instead.(3) Each concentration level for the PT samples was
analyzed in triplicate.
In addition to the contaminant-only PT samples described above, a second type of PT sample
was a potential interferent sample. Three replicates of each interferent PT sample were analyzed
to determine the susceptibility of the OP-Stick Sensor to these commonly found interferents in
DW. One interferent PT sample contained calcium (Ca) and magnesium (Mg) from carbonates
spiked into ASTM Type II DI water, and the other contained humic and fulvic acids isolated
from the Elliot River (obtained from the International Humic Substances Society) spiked into
ASTM Type II DI water. Each interferent mixture was prepared at two concentration levels: near
the upper limit of what would be expected in drinking water (250 mg/L total concentration for
Ca and Mg, 5 mg/L total concentration for humic and fulvic acids) and at a mid-low range of
what would be expected (50 mg/L total concentration for Ca and Mg, 1 mg/L total concentration
for humic and fulvic acids). These spiked interferent levels were confirmed through analysis of
aliquots by ATEL. Also, each contaminant, with the exception of aldicarb, was added to these
samples, along with the potential interferent, at a concentration consistent with a lOx dilution of
the lethal dose, and the resulting samples were analyzed in triplicate. Table 3-2 lists the PT
samples analyzed in this verification test for each contaminant. The vendor provided a limit of
detection (LOD) of > 100 mg/L for aldicarb, therefore interferent PT samples for aldicarb were
fortified at the lethal dose concentration.
Table 3-1. Lethal Dose of Target Contaminants
Contaminant
(common name)
VX
GB (sarin)
GD (soman)
aldicarb
dicrotophos
Oral Lethal Dose
Concentration
2.1 milligrams/liter (mg/L)
20 mg/L
1.4 mg/L
260 mg/L
1400 mg/L
Contaminant Class
Chemical agent
Chemical agent
Chemical agent
Carbamate pesticide
Organophosphate pesticide
-------
Table 3-2. Performance Test Samples
Type of PT
Sample
Sample Characteristics
Concentrations
Contaminant-
only
Contaminants in DI water
VX: 2.1 to 0.00021 mg/L
GB:20to0.002mg/L
GD: 1.4 to 0.00014 mg/L
aldicarb: 260 to 0.026 mg/L
dicrotophos: 1400 to 0.14 mg/L
Contaminants in 1 mg/L humic
and fulvic acids
Interferent
Contaminants in 5 mg/L humic
and fulvic acids
Contaminants in 50 mg/L Ca
and Mg
Contaminants in 250 mg/L Ca
andMg
VX: 0.21 mg/L
GB: 2.0 mg/L
GD: 0.14 mg/L
aldicarb: 260 mg/L
dicrotophos: 140 mg/L
3.2.2 DW'Samples
Table 3-3 lists the DW samples analyzed for each contaminant in this test. DW samples were
collected from four geographically distributed municipal sources (Ohio, New York, California,
and Florida) to evaluate the performance of the OP-Stick Sensor with various DW matrices.
These samples varied in their source, treatment, and disinfection process. All samples had
undergone either chlorination or chloramination disinfection prior to receipt. Samples were
collected from water utility systems with the following treatment and source characteristics:
• Chlorinated filtered surface water source
• Chlorinated unfiltered surface water source
• Chlorinated filtered groundwater source
• Chloraminated filtered surface water source
Approximately 175 liters (L) of each of the DW samples were collected in pre-cleaned,
translucent, low-density polyethylene containers. . After sample collection, an aliquot of each
DW sample was sent to ATEL to determine the following water quality parameters:
concentration of trihalomethanes, haloacetic acids, total organic halides, Ca and Mg, pH,
conductivity, alkalinity, turbidity, organic carbon, and hardness. All DW samples were
dechlorinated prior to their use with sodium thiosulfate pentahydrate to prevent the degradation
of the target contaminants by chlorine. The dechlorination of the DW was qualitatively
confirmed by adding a diethyl-p-phenylene diamine (DPD) tablet to an aliquot of DW. If the
water did not turn pink, the dechlorination process was successful. If the water did turn pink,
additional dechlorinating reagent was added and the dechlorination confirmation procedure
repeated. Each DW sample was analyzed before addition of contaminant, as well as after
fortification with each individual contaminant at a single concentration level (lOx dilution of the
-------
lethal dose, with the exception of aldicarb which was spiked at the lethal dose). Aliquots of each
contaminant stock solution were diluted with DW samples to the appropriate concentration. Each
sample was tested in triplicate.
Table 3-3. Drinking Water Samples
Drinking Water Sample Description
Water
Utility
Columbus, Ohio
(OH DW)
New York City, New
York (NY DW)
Orlando, Florida
(FL DW)
Metropolitan Water
District of Southern
California (CA DW)
Water
Treatment
chlorinated
filtered
chlorinated
unfiltered
chlorinated
filtered
chloraminated
filtered
Source
Type
surface
surface
ground
surface
Contaminant Concentrations
VX: 0.21 mg/L
GB: 2.0 mg/L
GD: 0.14 mg/L
aldicarb: 260 mg/L
dicrotophos: 140 mg/L
3.2.3 QCSamples
QC samples included method blank (MB) samples consisting of ASTM Type IIDI water. All
MB QC samples were exposed to sample preparation and analysis procedures identical to the test
samples. The MB samples were used to ensure that no sources of contamination were introduced
in the sample handling and analysis procedures. At least 10% of the test samples (seven samples
for each contaminant) were MB samples. All of the test samples and MB samples were analyzed
blindly by the operator in that the samples used for analysis were prepared by someone other
than the operator and were marked with non-identifying numbers.
3.2.4 Operational Factors
3.2.4.1 Technical Operator
All of the test samples were analyzed by a technical operator who was trained by the vendor.
Operational factors such as ease of use and sample throughput were evaluated based on
observations recorded by the technical operator and the Verification Test Coordinator.
Operational factors were noted during the laboratory portions of the verification test. These
observations were summarized to describe the operational performance of the OP-Stick Sensor in
this verification.
3.2.4.2 Non-Technical Operator
A subset of the samples was also tested by a non-technical operator using the OP-Stick Sensor.
The non-technical operator was someone with little to no laboratory experience who would be
representative of a first responder. For this test, the non-technical operator was a State of Ohio
certified firefighter with Hazardous Waste Operations and Emergency Response (HAZWOPER)
training. The non-technical operator was trained in the use of the OP-Stick Sensor by another
Battelle staff person who was trained by the vendor. Since many of the contaminants being tested
-------
are highly toxic and unsafe to be handled outside of a special facility, only MB samples were
analyzed as part of the operational factors assessment. Because no samples spiked with the
contaminants of interest were used, only the operational aspects of the OP-Stick Sensor were
evaluated with the non-technical operator. As the OP-Stick Sensor may be used by first-
responders, its performance was evaluated under simulated first-response conditions by having
the operator don a Level B protective suit, neoprene latex gloves, boots, and a self-contained
breathing apparatus (SCBA). The operator had prior experience working in personal protective
equipment (PPE). One set of MB samples was also tested without the use of PPE. Ease of use
from the perspective of the operator was documented both with and without the PPE.
3.3 Verification Schedule
The verification test of the OP-Stick Sensor took place from November 2005 through February
2006 at Battelle facilities in Columbus and West Jefferson, Ohio.
3.4 Test Procedure
3.4.1 Test Sample Preparation and Storage
All testing for this verification test was conducted within Battelle laboratories. Aldicarb and
dicrotophos samples were tested at Battelle Columbus laboratories, while VX, GB, and GD
samples were tested at Battelle's Hazardous Materials Research Center (HMRC) facility in West
Jefferson, OH. Appropriate safety guidelines associated with each laboratory were followed
throughout the verification test. Samples were prepared fresh each day from stock solutions in
either DI water, an interferent matrix, or a DW matrix. Sample solutions were prepared to the
specified concentration based on the concentration of the stock solution, which was confirmed
through reference analysis. Test solutions were prepared in 100-mL quantities. Appropriate
aliquots of this sample preparation were used for each test sample. Triplicate samples of 10 mL
each were taken from the same sample preparation. Each sample was placed in its own container
and labeled only with a sample identification number that was also recorded in a laboratory
record book (LRB) along with details of the sample preparation.
3.4.2 Test Sample Analysis Procedure
The OP-Stick Sensor kit is composed of one plastic stick (with two spots at one end that contain
the detecting agent) and three reagent containing tubes. Three test samples were analyzed in
parallel. For each test sample, the operator used a pipette to introduce 10 mL of the test sample
into a "tube 1" that is labeled with a yellow dot, which contains the activating agent. The
operator carefully shook the tube to ensure that the pellet at the bottom of the tube was dissolved
into solution. The sample in the tube was then incubated for 15 minutes. The operator recorded
the time, kept by a stopwatch (timer), on the data sheet for each incubation step.
After the 15 minute incubation period, the operator transferred the solution from tube 1 into
"tube 2," labeled with a blue dot. After shaking the tube to dissolve the powder at the bottom of
this tube, the operator inserted the plastic stick into tube 2, without removing the adhesive tape
-------
covering the upper spot. The stick was incubated for 1 hour for each sample. The vendor
indicates that the longer incubation period will results in lower detection limits, though this was
not verified during this test.
After the 1 hour incubation, the operator introduced 10 mL of contaminant-free water into "tube
3," labeled with a red or green dot. After shaking the tube to ensure that the pellet at the bottom
of the tube was dissolved, the operator removed the plastic stick from tube 2. The operator then
used a pair of tweezers to remove the adhesive tape covering the upper spot on the stick and
dipped the stick into tube 3. The plastic stick was incubated in this solution for 15 minutes.
This upper spot is a reference for the OP-Stick Sensor. After the incubation period, the operator
removed the stick from tube 3 and visually inspected the color of the two spots. Though the
instructions provided with the kit indicate that the operator should observe a black or white color,
which respectively indicates the absence or presence of a contaminant, the colors observed
during testing were mostly not black or white, but also included shades of grey, green, and
yellow. The operator compared the lower spot to the reference for the result. If the lower spot
had the same color as the upper spot (black or dark), then no contaminant was detected by the
OP-Stick Sensor. If the lower spot had a reduced color or white, then the test sample was
considered to be positive for the presence of a contaminant.
3.4.3 Drinking Water Characterization
An aliquot of each DW sample, collected as described in Section 3.2.2, was sent to ATEL to
determine the following water quality parameters: turbidity; concentration of dissolved and total
organic carbon; conductivity; alkalinity; pH; concentration of Ca and Mg; hardness; and
concentration of total organic halides, trihalomethanes, and haloacetic acids. Table 3-4 lists the
characterization data from the four water sample types used in this verification test. Water
samples were collected and water quality parameters were measured by ATEL in June 2005,
while verification testing was tested with the DW between November 2005 and February 2006.
The time delay between collection and testing was due to the fact that the water samples were
collected for use during a separate ETV test conducted prior to this one. Because of this, an
aliquot of each DW was tested by ATEL again in January 2006 to verify some of the parameters
with the most potential to change over time. Note that dissolved organic carbon was not retested
as this result was verified by the total organic carbon results, additionally the total organic
halides and calcium and magnesium were not verified as there was no reason to expect a change
in these parameters. The concentrations of most water quality parameters were similar; however,
there was a decrease in levels of volatile compounds such as trihalomethanes and haloacetic
acids over this time-period.
-------
Table 3-4. ATEL Water Quality Characterization of Drinking Water Samples
Parameter
Turbidity
Dissolved
Organic Carbon
Total Organic
Carbon
Specific
Conductivity
Alkalinity
PH
Calcium
Magnesium
Hardness
Total Organic
Halides
Trihalomethanes
Haloacetic Acids
Unit
NTU(a)
mg/L
mg/L
uMHO(c)
mg/L
mg/L
mg/L
mg/L
ug/L
ug/L/
analyte
ug/L/
analyte
Method
EPA180.1(4)
SM5310(5)
SM5310(5)
SM2510(5)
SM 2320(5)
EPA 150.1(6)
EPA200.8(7)
EPA200.8(7)
EPA 130.2(8)
SM 5320(5)
EPA 524.2(9)
EPA 552.2(10)
Columbus,
OH
(OH DW)
2005
0.1
2.1
2.1
572
40
7.6
33
7.7
118
220
74.9
32.8
2006
0.6
NA
2.3
602
44
7.4
NA
NA
107
NA
16.6
<6.0
New York
City, NY
(NY DW)
2005
1.1
1.1
1.6
84
14
6.9
5.6
1.3
20
82
39.0
39.0
2006
1.3
NA
4.1
78
12
6.8
NA
NA
26
NA
23.1
<6.0
Orlando, FL
(FL DW)
2005
0.5
1.6
1.7
322
142
8.5
8.8
43
143
300
56.4
34.6
2006
0.1
NA
2.1
325
125
7.6
NA
NA
130
NA
41.8
<6.0
MWD (b), CA
(CA DW)
2005
0.1
2.9
2.5
807
71
8.0
45
20
192
170
39.2
17.4
2006
0.2
NA
2.7
812
97
7.9
NA
NA
182
NA
24.1
<6.0
(a) NTU = Nephelometric turbidity unit.
(b) MWD = Metropolitan Water District of Southern California
(iMHO = micromho
10
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed in accordance with the quality management plan (QMP) for
the AMS Center(11) and the test/QA plan(12) for this verification test.
QC procedures as noted in the reference methods or laboratory's operating procedures were
followed in confirming analyses of stock or reference solutions of contaminants and interfering
compounds and in characterizing the DW. The reference methods for this verification test are
listed in Table 4-1. A summary of the QC samples and acceptance criteria associated with each
method is presented in Table 7 in the test/QA plan.(12)
4.1 Sample Chain-of Custody Procedures
Sample custody was documented throughout collection, shipping, and analysis of the samples.
Sample chain-of-custody procedures were in accordance with ASAT.I-009-DRAFT, Standard
Operating Procedure for Sample Chain of Custody. The chain-of-custody forms summarized the
samples collected and analyses requested and were signed by the person relinquishing samples
once that person had verified that the custody forms were accurate. The original sample custody
forms accompanied the samples; the shipper kept a copy. Upon receipt at the sample destination,
sample custody forms were signed by the person receiving the samples once that person had
verified that all samples identified on the custody forms were present in the shipping container.
4.2 QC Samples
The QC measures for the reference methods included the analysis of a MB sample with the
analyses of the reference or stock solution. MB samples were analyzed to ensure that no sources
of contamination were present. If the analysis of an MB sample indicated a concentration above
the minimum detection limit for the confirmatory instrument, contamination was suspected. Any
contamination source(s) were corrected, and proper blank readings were achieved, before
proceeding with the analyses. In general, a matrix spike or laboratory fortified spike sample was
also analyzed. Average acceptable recoveries for these samples were between 70 and 150%.
Samples outside of the acceptable range were generally flagged and rerun once the QC
acceptance criteria had been met. QC samples were run with every batch of 1 to 20 samples.
Specific QC samples and acceptance criteria associated with each method can be found in the
appropriate reference (Table 4-1).
11
-------
Table 4-1. Reference Methods for Target Contaminants and Interferents
Target
Analyte/Interferent
VX
GB (sarin)
GD (soman)
aldicarb
dicrotophos
calcium (Ca)
magnesium (Mg)
Humic and fulvic
acids
Reference Method
(Instrumentation)
Battelle Internally
Developed Method (LC-MS)
HMRC-IV-1 18-05 (13)
(GC-MS)
HMRC-IV-1 18-05 (13)
(GC-MS)
SOP for Analysis of Water
Sample Extracts for Type 1
Analytes by Liquid
Chromatography/Mass
Spectrometry (14) (LC-MS)
SOP for Extracting and
Preparing Water Samples for
Analysis of Dicrotophos,
Mevinphos, and
Dichlorovos (15) (GC-MS)
EPA 200.8 (7)(ICP-MS)
EPA 200.8 (7)(ICP-MS)
Standard Method 53 10 (5)
Combustion Infrared NDR
Number of
Observations
10
4
4
2
2
4
1
1
1
1
Expected
Concentrations
(mg/L)
2.1
20.0
1.4
26.0
260
140
1400
125
125
1.0
Average
Measured
Concentration
(mg/L) ± SD
2.1±0.1
17.0 ±1.4
1.7 ±0.05
34
303
157 ± 24
1326
140
130
0.9
Recovery
(%R) ± SD
101 ±5
85 ±7
121 ±4
123 ±7 (a)
108 ± 17 (a)
112
104
90
> Average of two concentration levels
MB samples were run as part of the verification test. Of the 70 method blank samples analyzed,
1 detect, 8 inconclusive, and 61 non-detect results were observed.
4.3 Equipment/Calibration
The instruments used for the reference analyses were calibrated per the standard reference
methods being used to make each measurement or the standard operating procedures (SOPs) of
12
-------
the analysis laboratory. Instruments used in the reference analyses for this test included gas
chromatography-mass spectrometry (GC-MS), liquid chromatography-mass spectrometry (LC-
MS), pH electrodes, inductively coupled plasma/mass spectrometry (ICP-MS), and gas
chromatography with electron capture detector (GC-ECD). All calibrations were documented by
Battelle in the project LRB. Calibration of mass spectrometers involved a 4- to 8-point
calibration curve covering the range of concentrations of the reference solutions to be analyzed.
Calibration of each reference instrument was performed as frequently as required by the
reference method guidelines.
Pipettes used during solution preparation were maintained and calibrated as required by Battelle
SOPs (i.e., minimum of every 6 months). Pipettes were checked and either recalibrated or
replaced if they were found out of calibration over the course of testing.
4.4 Characterization of Stock Solutions
During testing, aliquots of the stock solutions used for sample preparation were submitted for
concentration confirmation via the respective methods. The results, along with the reference
methods, are listed in Table 4-1. Averages and associated standard deviations are given in cases
where more than two samples were tested. The %R, listed in Table 4-1, represents the average of
the %R across both concentration levels for those compounds. Recovery (%R) is calculated by
the following equation:
%R = — xlOO (1)
where C is the measured concentration (or average measured concentration if more than one
sample was tested) and^4 is the expected concentration of the contaminant or interferent in
solution. For aldicarb and dicrotophos, aliquots at two different concentration levels were
confirmed through reference analysis. The %R, listed in Table 4-1, represents the average of the
%R across both concentration levels for those compounds. Table 4-1 shows that %R values
ranged from 85% to 123% across all analytes and interferents.
Contaminant stock solutions were prepared and tested individually. Interferent stock solutions
contained multiple analytes in the same solution (e.g., Ca and Mg or humic and fulvic acids
together). Up to four aliquots of each stock solution were analyzed over the course of the
verification test. In the case of VX, extra aliquots were analyzed and all are reported in
Table 4-1. Aliquots were preserved or extracted on the day of preparation and stored as
prescribed by the standard method.
4.5 Audits
4.5.1 Performance Evaluation Audit
The concentration of the standards used to prepare the samples fortified with contaminants and
potential interfering compounds was confirmed by analyzing standards prepared in ASTM Type
IIDI water from two separate commercial vendors using the reference methods noted in
13
-------
Table 4-1. The standards from one vendor were used during the verification test, while the
standards from the second vendor were used exclusively to confirm the accuracy of the standards
from the first vendor.
Given the security requirements and lack of alternate sources for the chemical agents (VX, GB,
GD) used in this verification test, PE audits were not performed for these contaminants. PE
audits were done for all remaining compounds when more than one source of the contaminant or
potential interfering compounds was available. PE audits were performed only on compounds
used to prepare test samples and not on any solutions supplied as part of the OP-Stick Sensor.
Agreement of the standards within 25% (percent difference) was required for the measurements
to be considered acceptable. The percent difference (%D) between the measured concentration of
the PE sample and the nominal concentration of that sample was calculated using the following
equation:
%D = — xlOO
A
(2)
where Mis the absolute value of the difference between the measured and the expected
concentration, and^4 is the expected concentration. The results of the PE samples are given in
Table 4-2. All %D values were within the 25% acceptable tolerance.
Table 4-2. Performance Evaluation Samples and Percent Difference
Contaminant
aldicarb
dicrotophos
Ca
Mg
Expected
Concentration
(mg/L)
50
1000
1000
1000
Measured
Concentration
(mg/L)
57
1103
890
990
Percent
Difference
(%)
14
10
11
1
4.5.2 Technical Systems A udit
The Battelle Quality Manager conducted technical systems audits (TSAs) in November 2005
(11/01, 11/11, 11/16, 11/18), December 2005 (12/01, 12/29) and January 2006 (01/30) to ensure
that the verification test was performed in accordance with the AMS Center QMP,(11) the
test/QA plan,(12) published reference methods, and any SOPs used by Battelle. As part of the
audit, the Battelle Quality Manager reviewed the reference methods, compared actual test
procedures to those specified or referenced in the test/QA plan, and reviewed data acquisition
and handling procedures. The Battelle Quality Manager also observed testing in progress and the
reference method sample preparation and analysis, inspected documentation, and reviewed the
LRBs used to record testing results. The Battelle Quality Manager also checked calibration
certifications and conferred with Battelle staff. Observations and findings from this audit were
documented and submitted to the Battelle Verification Test Coordinator for response. No major
findings were reported from the audits. The records concerning the TSA are permanently stored
with the Battelle Quality Manager.
14
-------
4.5.3 Audit of Data Quality
At least 10% of the data acquired during the verification test was audited. The Battelle Quality
Manager traced the data from initial acquisition, through reduction and statistical comparisons, to
final reporting. All calculations performed on the data undergoing the audit were checked.
4.6 QA/QC Reporting
Each assessment and audit was documented in accordance with Section 3.3.4 of the AMS Center
QMP.(11) Once the assessment report was prepared, the Battelle Verification Test Coordinator
responded to each potential problem and implemented any necessary follow-up corrective action.
The Battelle Quality Manager ensured that follow-up corrective action was taken. The results of
the TSA were sent to the EPA.
4.7 Data Review
Records generated in the verification test were reviewed before they were used to calculate,
evaluate, or report verification results. Table 4-3 summarizes the types of data recorded. The
review was performed by a technical staff member involved in the verification test but not the
staff member who originally generated the record. The person performing the review added
his/her initials and the date to a hard copy of the record being reviewed.
15
-------
Table 4-3. Summary of Data Recording Process
Data to Be Recorded
Dates, times, and
details of test events
Sample preparation
(dates, concentrations,
etc.)
Enzymatic test kit
procedures and sample
results
Reference method
sample preparation
Reference method
procedures,
calibrations, QA, etc.
Reference method
analysis results
Responsible
Party
Battelle
Battelle
Battelle
Battelle
Battelle or
subcontract
laboratory
Battelle or
subcontract
laboratory
Where
Recorded
ETV laboratory
record book or
data recording
forms
ETV laboratory
record books
ETV data sheets
and laboratory
record book
ETV laboratory
record book
Laboratory
record book or
data recording
forms
Electronically
from reference
analytical method
How Often
Recorded
Start/end of test
procedure, and at
each change of a
test parameter
When each
solution was
prepared
Throughout test
duration
Throughout
sample
preparation
Throughout
sampling and
analysis
processes
Every sample
analysis
Disposition
of Data
Used to organize and
check test results and
manually incorporated
into data spreadsheets
as necessary
Used to confirm the
concentration and
integrity of the
samples analyzed
Manually incorporated
into data spreadsheets
for statistical analysis
and comparisons
Used to demonstrate
validity of samples
submitted for
reference
measurements
Retained as
documentation of
reference method
performance
Converted to
spreadsheets for
calculations
16
-------
Chapter 5
Statistical Methods and Reported Parameters
The OP-Stick Sensor was evaluated for qualitative results (i.e., positive/negative responses to
samples). All data analyses were based on these qualitative results. QC and MB samples were
not included in any of the analyses.
5.1 Accuracy
Accuracy was assessed by evaluating how often the OP-Stick Sensor result is positive in the
presence of a concentration above the limit of detection (LOD). Contaminant-only PT samples
were used for this analysis. An overall percent agreement was determined by dividing the
number of positive responses by the overall number of analyses of contaminant-only PT samples
greater than the OP-Stick Sensor's LOD (see Equation 3). If the LOD was not known or
available, then all analyzed contaminant-only PT samples greater than the concentration level
where consistent negative results were obtained were used.
Accuracy (% Agreement) = # of positive contaminant only PT samples x 100 (3)
total # of contaminant only PT samples
5.2 False Positive/False Negative Rates
A false positive response was defined as a response indicating the presence of a contaminant
when the ASTM Type IIDI water (including interferent samples) or DW sample was not spiked
with a contaminant. A false positive rate was reported as the number of false positive results out
of the total number of unspiked samples (Equation 4). A false negative response was defined
as a response indicating the absence of a contaminant when the sample was spiked with a
contaminant at a concentration greater than the OP-Stick Sensor's LOD as defined above. Spiked
PT (contaminant and interferent) samples and spiked DW samples were included in the analysis.
Contaminant-only PT samples above the OP-Stick Sensor's LOD or the level at which consistent
negative responses are obtained (when the LOD was not known) were included in the analysis. A
false negative rate was evaluated as the number of false negative results out of the total number
of spiked samples for a particular contaminant (Equation 5). Inconclusive results were not
considered positive or negative (so the total number of unspiked or spiked samples was
decreased accordingly)."
17
-------
False Positive Rate = # of positive results (4)
total # of unspiked samples
False Negative Rate = # of negative results (5)
total # of spiked samples
5.3 Precision
Precision measures the repeatability and reproducibility of the OP-Stick Sensor's responses. The
precision of three replicates of each sample set was assessed. Responses were considered
inconsistent if one or more of the three replicates differed from the response of the other samples
in the replicate set. The precision for the OP-Stick Sensor was assessed by calculating the overall
number of consistent responses for all the sample sets. The results are reported as the percentage
of consistent responses out of all replicate sets (Equation 6).
Precision (% Consistent results) = # of consistent responses of replicate sets x 100 (6)
total # of replicate sets
5.4 Potential Matrix and Interferent Effects
The potential effect of the DW matrix on the OP-Stick Sensor's performance was evaluated
qualitatively by comparing the results for the spiked and unspiked DW samples to those for the
PT samples spiked with the contaminant at 10 times less than the lethal dose. Similarly, the
potential effect of interferent PT samples was evaluated. The results indicating the correct or
incorrect reporting of the presence of a contaminant were evaluated. The findings are reported
and discussed in Section 6.4.
5.5 Operational Factors
Operational aspects of the OP-Stick Sensor's performance such as ease of use and sample
throughput were evaluated through observations made during testing. Also addressed are the
qualitative observations of the verification staff pertaining to the performance of the OP-Stick
Sensor from both the technical and non-technical operators' perspectives.
18
-------
Chapter 6
Test Results
The results for the OP-Stick Sensor are discussed in the following sections. It is important to
note that the ability of the operator to discern and compare the colors that appear on the stick at
the end of the test is integral to the outcome of the test. As described in the Technology
Description (Chapter 2), the degree to which the indicator spots change color depends on the
concentration of the contaminant in a water sample. It was observed during the testing that the
colors of the upper spot (reference) were often yellow and green (and not only degrees of brown
or black as the instructions indicate). Comparison of the lower spot to assess reduced or absent
color formation was necessary to conclude a test result. This led occasionally to inconclusive
results, which could not be categorized by the operator as either positive or negative indications
from the OP-Stick Sensor. Additionally, the same outcomes could have been interpreted by two
operators in different ways, potentially leading to different test results.
6.1 Accuracy
Accuracy was determined using contaminant-only PT samples that were equal to or above the
vendor-provided LOD. If no LOD was known, only those concentration levels above which
consistent negative results were obtained were used to determine accuracy. Results are provided
in Table 6-1. No LODs were provided by the vendor for any of the target contaminants with the
exception of aldicarb, for which an LOD of > 100 mg/L for aldicarb was provided by the vendor.
For this reason, accuracy was determined using the three PT samples at 260 mg/L aldicarb. For
VX, GB, and GD, all contaminant-only PT samples were included in the calculation of accuracy
since consistent negative responses were not established above the lowest set of PT samples.
However, several inconclusive results were observed with these contaminants, primarily at the
lower concentrations. Consistent negative results were obtained at and below 1.4 mg/L
dicrotophos. Therefore, accuracy is determined for all replicates above this concentration.
Six inconclusive results were observed among the nine replicates at the 0.021 mg/L VX
concentration level and below, though two positive results were also detected at the lowest
concentration level for VX (0.00021 mg/L) to yield an overall accuracy of 33% for VX.
Inconsistent results were also observed for GB and GD including two occurrences at the highest
concentration level of 1.4 mg/L for GD. Overall accuracy for GB and GD was 60% and 27%,
respectively. Accuracy for both aldicarb and dicrotophos was 100%, in the relatively high
concentration levels for which accuracy was calculated for these contaminants. No inconclusive
results were obtained for aldicarb and dicrotophos.
19
-------
Table 6-1. Contaminant-Only PT Sample Results
Contaminant
VX
GB
GD
aldicarb
dicrotophos
Concentration
(mg/L)
2.1(a)
0.21
0.021
0.0021
0.00021
20(a)
2.0
0.20
0.020
0.0020
L4(a)
0.14
0.014
0.0014
0.00014
260 (a)
26 ^
2.6 ^
0.26 (b)
0.026 (b)
1400 (a)
140
14
1.4°°
0.014
Positive Results
Out of
Total Replicates
3/3
0/3
0/3
0/3
2/3
3/3
3/3
3/3
0/3
0/3
1/3
3/3
0/3
0/3
0/3
3/3
0/3
0/3
0/3
0/3
3/3
3/3
3/3
0/3
0/3
Inconclusive Results Out of
Total Replicates
0/3
0/3
2/3
3/3
1/3
0/3
0/3
0/3
3/3
1/3
2/3
0/3
3/3
3/3
1/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
Accuracy
33%
(5/15)
60%
(9/15)
27%
(4/15)
100%
(3/3)
100%
(9/9)
^Lethal dose
(b) Vendor provided LOD of >100 mg/L; therefore concentrations below this limit were not used to calculate accuracy.
(c) Concentration at or below which consistent negative responses were observed
6.2 False Positive/False Negative Rates
Contaminant-only PT samples, interferent PT samples, and DW samples were evaluated to
determine false positive and false negative results for the Protein Biosensor OP-Stick Sensor. A
false positive response was defined as a positive result when the contaminant was not spiked into
the sample. A false negative response was defined as a negative result when the sample was
spiked with a contaminant at a concentration greater than the maximum level where consistent
20
-------
negative responses were obtained (see Section 6.1). Tables 6-2a through 6-2e present the false
positive and false negative responses for VX, GB, GD, aldicarb, and dicrotophos, respectively.
The number of positive samples out of the total replicates analyzed is presented in each table.
For VX, GB, and GD, only one set of unspiked DW and PT interferent samples were run for all
three chemical agents. Thus, the unspiked DW and PT-interferent sample results shown for
these unspiked samples in Tables 6-2a through 6-2c are the same and from only one set of
triplicate samples. For aldicarb and dicrotophos, sets of unspiked DW and PT interferent
samples were run separately for each pesticide.
As shown in Table 6-2a, seven false negative responses were observed for VX, in one replicate
of the 0.021 mg/L VX in DI water PT sample, and three replicates each of the 0.21 mg/L VX in
DI water PT sample and the 0.21 mg/L VX in 1 mg/L humic and fulvic acid solution interferent
sample. No false positive results were observed for VX. For GB, two of 39 samples gave false
negative results. These samples were the lowest concentration of GB only PT samples, fortified
at 0.002 mg/L, as shown in Table 6-2b. No false positive results were observed for GB. For GD
as well, two of 39 samples gave false negative results. These samples were also the lowest
concentration of contaminant-only PT samples for GD: 0.00014 mg/L GD as shown in
Table 6-2c. No false positives were observed for GD.
For aldicarb, the vendor-provided LOD is stated as < 100 ng/mL aldicarb in water. Therefore,
no samples with lower concentrations of aldicarb were included in the calculations for false
negatives rates, though these concentrations were included in the false positive calculation. One
false positive result for aldicarb was observed for the Protein Biosensor OP-Stick Sensor with the
250 mg/L total Ca and Mg unfortified solution as shown in Table 6-2d. No false negatives were
observed for aldicarb. Neither false negatives nor false positives were observed for dicrotophos
as shown in Table 6-2e.
21
-------
Table 6-2a. VX False Positive/Negative Results
Sample Type
Contaminant-
only PT
samples
Interferent
samples (c)
DW samples (c)
Matrix
DI water
DI water
DI water
DI water
DI water
1 mg/L humic and
fulvic acids
1 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
50 mg/L Ca + Mg
50 mg/L Ca + Mg
250 mg/L Ca + Mg
250 mg/L Ca + Mg
OHDW
OHDW
NYDW
NYDW
FLOW
FLOW
CADW
CADW
False Positive Rate
False Negative Rate
Concentration
(mg/L)
2.1 <">
0.21
0.021
0.0021
0.00021
Blank
0.21
Blank
0.21
Blank
0.21
Blank
0.21
Blank
0.21
Blank
0.21
Blank
0.21
Blank
0.21
Negative
Results
0
3
1
0
0
3
3
2
0
0
0
3
0
1
0
3
0
3
0
1
0
Inconclusive
Results
0
0
2
3
1
0
0
1
0
3
1
0
0
2
1
0
0
0
1
2
0
Positive
Results out of
Total
Replicates
3/3
0/3
0/3
0/3
2/3
0/3
0/3
0/3
3/3
0/3
2/3
0/3
3/3
0/3
2/3
0/3
3/3
0/3
2/3
0/3
3/3
0/24
7/39
(a) Shaded results indicate false negative observations
(b) Lethal dose
(c) Only one set of unspiked DW and PT interferent samples were run for VX, GB, and GD.
22
-------
Table 6-2b. GB False Positive/False Negative Results
Sample Type
Contaminant-
only PT
samples
Interferent
samples (c)
DW samples (c)
A/r , . Concentration
Matrix , /T .
(mg/L)
DI water
DI water
DI water
DI water
DI water
1 mg/L humic and
fulvic acids
1 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
50 mg/L Ca + Mg
50 mg/L Ca + Mg
250 mg/L Ca + Mg
250 mg/L Ca + Mg
OHDW
OHDW
NYDW
NYDW
FLOW
FLOW
CADW
CADW
False Positive Rate
False Negative Rate
20(b)
2.0
0.2
0.02
0.002
Blank
2.0
Blank
2.0
Blank
2.0
Blank
2.0
Blank
2.0
Blank
2.0
Blank
2.0
Blank
2.0
Negative
Results
0
0
0
0
2
3
0
2
0
0
0
3
0
1
0
3
0
3
0
1
0
Inconclusive
Results
0
0
0
3
1
0
0
1
0
3
2
0
0
2
2
0
0
0
0
2
0
Positive
Results out of
Total
Replicates
3/3
3/3
3/3
0/3
0/3
0/3
3/3
0/3
3/3
0/3
1/3
0/3
3/3
0/3
1/3
0/3
3/3
0/3
3/3
0/3
3/3
0/24
2/30
(a) Shaded results indicate false negative observations
(b) Lethal dose
(c) Only one set of unspiked DW and PT interferent samples were run for VX, GB, and GD.
23
-------
Table 6-2c. GD False Positive/False Negative Results
Sample Type
Contaminant-
only PT
samples
Interferent
samples (c)
DW samples (c)
Matrix
DI water
DI water
DI water
DI water
DI water
1 mg/L humic and
fulvic acids
1 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
50 mg/L Ca + Mg
50 mg/L Ca + Mg
250 mg/L Ca + Mg
250 mg/L Ca + Mg
OHDW
OHDW
NYDW
NYDW
FLOW
FLOW
CADW
CADW
False Positive Rate
False Negative Rate
Concentration
(mg/L)
1.4(a)
0.14
0.014
0.0014
0.00014
Blank
0.14
Blank
0.14
Blank
0.14
Blank
0.14
Blank
0.14
Blank
0.14
Blank
0.14
Blank
0.14
Negative
Results
0
0
0
0
2
3
0
2
0
0
0
3
0
1
0
3
0
3
0
1
0
Inconclusive
Results
2
0
3
3
1
0
2
1
0
3
0
0
0
2
1
0
0
0
2
2
1
Positive
Results out of
Total
Replicates
1/3
3/3
0/3
0/3
0/3
0/3
1/3
0/3
3/3
0/3
3/3
0/3
3/3
0/3
2/3
0/3
3/3
0/3
1/3
0/3
2/3
0/24
2/39
(a) Shaded results indicate false negative observations
(b) Lethal dose
(c) Only one set of unspiked DW and PT interferent samples were run for VX, GB, and GD.
24
-------
Table 6-2d. Aldicarb False Positive/False Negative Results
c , T, A/r , . Concentration Negative Inconclusive
Sample Type Matrix , /T . "
F JF (mg/L) Results Results
DI water
Contaminant- DI water
only PT DI water
samples DI water
DI water
1 mg/L humic and
fulvic acids
1 mg/L humic and
fulvic acids
5 mg/L humic and
Interferent fulvic acids
samples 5 mg/L humic and
fulvic acids
50 mg/L Ca + Mg
50 mg/L Ca + Mg
250 mg/L Ca + Mg
250 mg/L Ca + Mg
OHDW
OHDW
NYDW
NYDW
™ i FL DW
DW samples pL DW
CADW
CADW
False Positive Rate
False Negative Rate
(a) Boxed results indicate respectively false negative
(b) Lethal dose
260 (b)
26
2.6
0.26
0.026
Blank
260
Blank
260
Blank
260
Blank
260
Blank
260
Blank
260
Blank
260
Blank
260
or false
0
o
J
o
J
3
o
J
o
J
0
3
0
3
0
2
0
o
J
0
o
J
0
3
0
3
0
positive observations
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Positive
Results out of
Total
Replicates
3/3
0/3
0/3
0/3
0/3
0/3
3/3
0/3
3/3
0/3
3/3
1/3
3/3
0/3
3/3
0/3
3/3
0/3
3/3
0/3
3/3
1/24
0/27
25
-------
Table 6-2e. Dicrotophos False Positive/False Negative Results
Sample Type
Contaminant-
only PT
samples
Interferent
samples
DW samples
A/r , . Concentration
Matrix , _ ,
(mg/L)
DI water
DI water
DI water
DI water
DI water
1 mg/L humic and
fulvic acids
1 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
5 mg/L humic and
fulvic acids
50 mg/L Ca + Mg
50 mg/L Ca + Mg
250 mg/L Ca + Mg
250 mg/L Ca + Mg
OHDW
OHDW
NYDW
NYDW
FLOW
FLOW
CADW
CADW
False Positive Rate
False Negative Rate
1400 (a)
140
14
1.4
0.14
Blank
140
Blank
140
Blank
140
Blank
140
Blank
140
Blank
140
Blank
140
Blank
140
Negative
Results
0
0
0
3
3
3
0
o
5
0
o
3
0
o
J
0
3
0
3
0
o
5
0
o
6
0
Inconclusive
Results
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Positive
Results out of
Total
Replicates
3/3
3/3
3/3
0/3
0/3
0/3
3/3
0/3
3/3
0/3
3/3
0/3
3/3
0/3
3/3
0/3
3/3
0/3
3/3
0/3
3/3
0/24
0/33
(a)
Lethal dose
26
-------
6.3 Precision
During testing with VX, the Protein Biosensor OP-Stick Sensor gave inconsistent results. Eight
of the 21 sample sets, each consisting of three replicates, had at least one replicate that differed
from the other two replicates, resulting in consistent results in 13 out of 21 sets, or a precision of
62%. For GB, 15 of 21 sample sets yielded consistent results, giving a precision of 71%. For
GD, 12 of 21 samples sets yielded consistent results, giving a precision of 57%. Note that only
one set of unspiked interferent samples were tested for VX, GB, and GD. These sample sets
were shared among the three contaminants. Three of these 8 sample sets had at least one
replicate that differed from the other two replicates.
For aldicarb, 20 of the 21 sample sets had consistent results, yielding a precision of 95%. One
unspiked interferent sample (250 mg/L total Ca and Mg) gave a false positive result. No
inconsistent results were observed during testing for dicrotophos, yielding a precision of 100%.
6.4 Potential Matrix and Interferent Effects
The Protein Biosensor OP-Stick Sensor was able to consistently detect GB, GD, and dicrotophos
at 10 times less than the respective LD50 concentrations in DI water, so testing of matrix and
interferent effects for these contaminants was conducted at these respective concentration levels.
For aldicarb, one-tenth of its LD50 was below the vendor provided detection limit of the
technology, so the test of potential matrix and interferent effects for aldicarb was performed at
the LD50 concentration (260 mg/L) in water. For VX, testing was conducted at 10 times less
than the LD50 concentration (i.e., at 0.21 mg/L) even though the contaminant-only PT samples at
this concentration yielded consistent negative results.
6.4.1 Interferent PT Samples
For VX (Table 6-2a), testing with the 1 mg/L humic and fulvic acid matrix yielded consistent
negative responses when the matrix was spiked with the contaminant at 10 times less than the
LDso (0.2 mg/L). One inconclusive result was obtained when VX was spiked into the 50 mg/L
Ca + Mg solution, though the unspiked matrix gave three inconclusive results. With these
exceptions, the remaining eight fortified interferent samples yielded positive results for VX,
indicating that other matrix effects were minimal if present. For GB (Table 6-2b), one
inconclusive result was observed in the unfortified 5 mg/L humic and fulvic acid. The 50 mg/L
Ca + Mg solution produced three inconclusive results while the samples with a higher
concentration of Ca + Mg (250 mg/L) indicated no interferent effects were present for GB. Two
inconclusive results were also observed with the 50 mg/L Ca + Mg matrix spiked with 2.0 mg/L
of GB. With these exceptions, the other ten fortified interferent samples yielded positive results
for GB. For GD (Table 6-2c), two inconclusive results were observed for 0.14 mg/L GD in
1 mg/L humic and fulvic acid solution. One inconclusive result was observed for the unfortified
5 mg/L humic and fulvic acid solution, and three were observed for the unfortified 50 mg/L Ca +
Mg solution. With these exceptions, the other ten fortified samples for GD yielded positive
results. For dicrotophos, all fortified samples yielded positive results while unfortified samples
yielded negative results, indicating that no matrix effects are present for these contaminants in
the interferent solutions tested. For aldicarb, all fortified samples also yielded positive results
27
-------
though one unfortified sample, 250 mg/L Ca + Mg, gave a positive response (as shown in
Table 6-2d). All of the remaining unfortified samples yielded negative results for aldicarb.
6.4.2 DWSamples
For VX, GB, and GD, two inconclusive results were reported for both unfortified OH DW and
unfortified CA DW samples. For VX, one inconclusive result was also reported for the fortified
(0.21 mg/L) test samples in each of the OH and FL DW matrices. With these exceptions, ten of
the 12 remaining fortified samples yielded positive results, indicating minimal matrix effects for
VX in these DW samples. For GB, two inconclusive results were also observed for the 2 mg/L
GB in OH DW. With these exceptions, 10 of 12 fortified samples yielded positive results for
GB. One inconclusive result was also observed for GD in each of the fortified OH DW, FL DW,
and CA DW matrices. With these exceptions, nine of 12 fortified samples yielded positive
results for GD. For aldicarb and dicrotophos, all fortified samples yielded positive results while
blank samples yielded negative results, indicating that no matrix effects were present for these
contaminants in the DW matrices tested.
6.5 Operational Factors
6.5.1 Technical Operators
The Protein Biosensor OP-Stick Sensor was operated by one Battelle technical operator
throughout testing with the pesticides and by a different Battelle technical operator throughout
testing with CWA. The technical operators were trained by the vendor in the operation of the
test kit. The half-day of face-to-face training was provided by a vendor representative. Both
technical operators had extensive laboratory experience.
The operators commonly observed that the tape on the bottom of the sticks is extremely difficult
to remove. Since the test samples may be potentially hazardous, it may not be acceptable to
remove the tape by hand. Therefore, tweezers of some kind must be used and this requires
dexterity that may not be achievable while gloved.
With the first lot of OP-Stick Sensors used, the spots were not black or white as the instructions
indicated they should be. More often they were various shades of yellow, grey or green. This
made it very difficult to discern the result for a particular sample, leading to inconclusive results.
It appeared that the various lots used during the testing differed in the reactivity to the
contaminants used during testing. The second lot of OP-Stick Sensors that were used toward the
end of testing was much more reactive. The reference spot on these tubes showed a deep black
color and the indicator spot was either a deep black or plain white. These results were less
subjective and much easier to read.
Some kind of rack should be included with the test kit to hold the several tubes. The tubes
should be more clearly labeled as they were only labeled with a colored dot which would
occasionally fall off of the bag. Nothing is included in the kit to deliver the 10 mL of test sample
needed in tube 1 or the 10 mL of water needed in tube 3. It would be helpful to include some
28
-------
kind of pipette with the kit for this purpose. Tweezers also should be included with the kit for
retrieving the OP-Stick Sensor from tube 2 and transferring it to tube 3. Sample throughput
varied with the operator as multiple samples can be analyzed simultaneously. Physical
accommodations (i.e., hood space or table space) and operator preference for sample size may
affect sample throughput. Instructions that include a diagram of the various steps required of the
test are included with the kit. Samples did not require any storage considerations and were kept
at room temperature.
6.5.2 Non-Technical Operator
Unspiked DI water samples were tested on the Protein Biosensor OP-Stick Sensor by a non-
technical operator both with and without PPE (see Section 3.2.4). The samples were analyzed
while wearing full PPE, consisting of a Level B suit, neoprene latex gloves, boots and SCBA as
shown in Figures 6-1 and 6-2. The SCBA was worn throughout the entire testing procedure by
the non-technical operator (only during the tests in which PPE was to be donned) to represent the
physical burden borne by a similarly outfitted first responder. However, the operator ran the air
from the SCBA only part of the time during testing to conserve the tank.
Including set up and operation, the time required for a test is approximately 1.5 to 2 hours for
each sample though multiple samples may be analyzed simultaneously. An operator equipped
with SCBA would have to obtain a new tank of air for the duration of the test. A gloved operator
would also have trouble removing the tape on the OP-Stick Sensor. The operator had to use
tweezers to remove the tape.
Samples were also analyzed without PPE. All samples were negative for the unspiked DI water
samples. While the individual test tubes containing the reagents (three separate kinds are
required for a single test) and OP-stick Sensors are portable, the time required for incubation of
the samples makes it necessary for the operator to have sufficient working space in which the test
tubes may be placed for the duration of the test (which is over 1 hour). Adequate lighting is also
required to read the color changes which may preclude the use of the test in environments with
low lighting. A test tube rack and a pipetter are also recommended for use, so these items should
be considered for field use of the Protein Biosensor OP-Stick Sensor.
29
-------
Figure 6-1. Side View of PPE Worn by Non-
Technical Operator
Figure 6-2. Testing of the OP-Stick Sensor with the
Non-Technical Operator Wearing PPE
30
-------
Chapter 7
Performance Summary
The Protein Biosensor OP-Stick Sensor results from this verification test for samples containing
VX, GB, GD, aldicarb, and dicrotophos are presented in Tables 7-la-e, respectively. Qualitative
responses for each set of sample replicates as well as accuracy, false negatives and positives, and
precision are presented in each table. A summary of the other performance factors associated
with the Protein Biosensor OP-Stick Sensor is presented at the end of this chapter. These
performance factors apply across all contaminants.
31
-------
Table 7-la. VX Summary Table
Parameter
Qualitative
Results
Contaminant-
Only PT
Samples
Interferent PT
Samples
DW Samples
Matrix
DI Water
Humic and Fulvic
Ca and Mg
DW
VX
Concentration
2.1mg/L(a)
0.21 mg/L
0.021 mg/L
0.0021 mg/L
0.00021 mg/L
0.21 mg/L
0.21 mg/L
0.21 mg/L
Number
Detected/Number
of Samples
3/3
0/3
0/3
0/3
2/3
3/6
5/6
10/12
Accuracy
33% (5 out of 15) of the contaminant-only PT samples gave
positive results during testing at concentrations levels of
0.00021 to 2.1 mg/L VX. Six inconclusive results were
observed in the nine replicates of the contaminant-only PT
samples at and below the concentration level of 0.021 mg/L
VX
False Positive Rate
No false positive results (0 out of 24) were observed during
the testing with VX.
False Negative Rate
Seven false negative results out of 39 samples were observed
during testing with VX: one replicate of the 0.021 mg/L VX
in DI water PT sample, and three replicates each of the
0.21 mg/L VX in DI water PT sample and the 0.21 mg/L VX
in 1 mg/L humic and fulvic acid solution interferent sample.
Precision
62% (13 out of 21) of the sample sets showed consistent
results among the individual replicates within each set during
testing with VX.
(S)
Lethal dose
32
-------
Table 7-lb. GB Summary Table
Parameter
Qualitative
Results
Contaminant-
Only PT
Samples
Interferent PT
Samples
DW Samples
Matrix
DI Water
Humic and Fulvic
Ca and Mg
DW
Concentration
20 mg/L (a)
2.0 mg/L
0.2 mg/L
0.02 mg/L
0.002 mg/L
2.0 mg/L
2.0 mg/L
2.0 mg/L
Number
Detected/Number
of Samples
3/3
3/3
3/3
0/3
0/3
6/6
4/6
10/12
Accuracy
60% (9 out of 15) of the contaminant-only PT samples gave
positive results during testing with GB. Four inconclusive
results were observed at the 0.02 and 0.002 mg/L GB
concentration levels, with two negative results observed at the
0.002 mg/L GB concentration level.
False Positive Rate
No false positive results (0 out of 24) were observed during
testing with GB.
False Negative Rate
Two false negative results out of 39 samples were observed
during testing with GB. These samples were at the lowest
concentration of the contaminant-only PT samples, fortified at
0.002 mg/L.
Precision
71% (15 out of 21) of the sample sets showed consistent
results among the individual replicates with each set during
testing with GB.
Lethal dose
33
-------
Table 7-lc. GD Summary Table
Parameter
Qualitative
Results
Contaminant-
Only PT
Samples
Interferent PT
Samples
DW Samples
Matrix
DI Water
Humic and Fulvic
Ca and Mg
DW
Concentration
1.4 mg/L (a)
0.14 mg/L
0.0 14 mg/L
0.0014 mg/L
0.000 14 mg/L
0.14 mg/L
0.14 mg/L
0.14 mg/L
Number
Detected/Number
of Samples
1/3
3/3
0/3
0/3
0/3
4/6
6/6
8/12
Accuracy
27% (4 out of 15) of the contaminant-only PT samples gave
positive results during testing at concentrations of 0.00014 to
1.4 mg/L GD. Seven inconclusive results were observed at
the concentration level of 0.014 mg/L GD and below. Two
negative results were observed at the lowest concentration
level tested, 0.00014 mg/L GD.
False Positive Rate
No false positive results (0 out of 24) were observed during
testing with GD.
False Negative Rate
Two false negative results (2 out of 39) were observed during
testing with GD. These results were observed at the lowest
concentration level tested, 0.00014 mg/L GD.
Precision
57% (12 out of 21) of the sample sets showed consistent
results among the individual replicates within that set during
testing with GD.
Lethal dose
34
-------
Table 7-ld. Aldicarb Summary Table
Parameter
Qualitative
Results
Contaminant-
Only PT
Samples
Interferent PT
Samples
DW Samples
Matrix
DI Water
Humic and Fulvic
Ca and Mg
DW
Aldicarb
Concentration
260 mg/L (a)
26 mg/L
2.6 mg/L
0.26 mg/L
0.026 mg/L
260 mg/L
260 mg/L
260 mg/L
Number
Detected/Number
of Samples
3/3
0/3
0/3
0/3
0/3
6/6
6/6
12/12
Accuracy
100% (3 out of 3) of the contaminant-only PT samples at
260 mg/L gave positive results during testing with aldicarb.
The vendor provided an LOD of > 100 mg/L, therefore none
of the other concentration levels were included in the
calculation of accuracy.
False Positive Rate
One false positive result (out of 24 results) was observed
during testing with aldicarb. This positive result was
observed in a 250 mg/L Ca and Mg solution into which no
aldicarb was spiked. The other two results for this sample set
were two negative results.
False Negative Rate
No false negative results (0 out of 27) were observed during
testing with aldicarb.
Precision
95% (20 out of 21) of the sample sets showed consistent
results among the individual replicates with each set during
testing with aldicarb. The one set which did not have
consistent results was the unfortified 250 mg/L Ca and Mg
solution described above under False Positive Rate.
Lethal dose
35
-------
Table 7-le. Dicrotophos Summary Table
Parameter
Qualitative
Results
Contaminant-
Only PT
Samples
Interferent PT
Samples
DW Samples
Matrix
DI Water
Humic and Fulvic
Ca and Mg
DW
Dicrotophos
Concentration
1400 mg/L (a)
140 mg/L
14 mg/L
1.4 mg/L
0.14 mg/L
140 mg/L
140 mg/L
140 mg/L
Number
Detected/Number
of Samples
3/3
3/3
3/3
0/3
0/3
6/6
6/6
12/12
Accuracy
100% (9 out of 9) of the contaminant-only PT samples gave
positive results during testing with dicrotophos. Consistent
negative results were observed at and below the concentration
level of 1.4 mg/L dicrotophos, therefore only concentrations
above this level were used to calculate accuracy.
False Positive Rate
False Negative Rate
Precision
No false positive results (0 out of 24) were observed during
testing with dicrotophos.
No false negative results (0 out of 30) were observed during
testing with dicrotophos.
100% (21 out of 21) of the sample sets showed consistent
results among the individual replicates within each set during
the testing of dicrotophos.
Lethal dose
36
-------
Operational Factors:
Technical Operators
The Protein Biosensor OP-Stick Sensor was operated by one Battelle technical operator
throughout testing with the pesticides and by a different Battelle technical operator throughout
testing with chemical warfare agents. The technical operators were trained by the vendor in the
operation of the test kit. Both technical operators had extensive laboratory experience. The
operators commonly observed that the tape on the bottom of the sticks is extremely difficult to
remove. Since the test samples may be potentially hazardous, it may not be acceptable to
remove the tape by hand.
Some variability within the production lots of kits was observed. The first lot of OP-Stick
Sensors showed spots that were various shades of yellow, grey, or green, not black or white as
the instructions indicated they should be. This made it very difficult to discern the result for a
particular sample, leading to inconclusive results. The second lot of OP-Stick Sensors that were
used toward the end of testing was much more reactive. The reference spot on these tubes
showed a deep black color, and the indicator spot was either a deep black or plain white. These
results were less subjective and much easier to read. Sample throughput varied with the operator
as multiple samples can be analyzed simultaneously. Physical accommodations (i.e., hood space
or table space) and operator preference for sample size may affect sample throughput.
Non-Technical Operator
Unspiked DI water samples were tested on the Protein Biosensor OP-Stick Sensor by a non-
technical operator both with and without PPE. During testing with the PPE on, the samples were
analyzed while the operator wore a full PPE, consisting of a Level B suit, neoprene latex gloves,
boots and SCB A. Including set up and operation, the time required for a test was approximately
1.5 to 2 hours; an operator equipped with a SCB A would have to obtain a new tank of air for the
duration of the test. A gloved operator would also have trouble removing the tape on the OP-
Stick Sensor. The operator had to use tweezers to remove the tape. The length of time for the
test and the need to manipulate the OP-Stick Sensor make its use difficult for users wearing PPE,
such as first responders.
37
-------
Chapter 8
References
1. U.S. Army Center for Health Promotion and Preventative Medicine, USACHPPM
Technical Guide 230, Chemical Exposure Guidelines for Deployed Military Personnel.,
January 2002.
2. Gosselin et al., Clinical Toxicology of Commercial Products. 5th edition, Baltimore, MD,
1984.
3. World Health Organization, The WHO Recommended Classification of Pesticides by
Hazard and Guidelines to Classification: 2004, 2005.
4. EPA-600-R-93/100. EPA Method 180.1. Turbidity (Nephelometric), Methods for the
Determination of Inorganic Substances in Environmental Samples. 1993.
5. American Public Health Association, et al. Standard Methods for Examination of Water
andWastewater. 19th Edition. 1997. Washington D.C.
6. EPA 600/4-79/020 Method 150.1. pH, Electrometric Method.. 1982.
7. EPA 600/R-94/111 Method 200.8. Determination of Trace Metals by Inductively Coupled
Plasma - Mass Spectrometry. 1994.
8. EPA 600/4-79/020 Method 130.2. Hardness, Total (mg/L as CaCO3) Titrimetric, EDTA.
1982.
9. EPA 600/R-95/131. EPA Method 524.2. Purgeable Organic Compounds by Capillary
Column GC/Mass Spectrometry. Methods for Determination of Organic Compounds in
Drinking Water, Supplement III. 1995.
10. EPA 600/R-95/131. EPA Method 552.2. Haloacetic Acids andDalapon by Liquid-Liquid
Extraction, Derivatization and GC with Electron Capture Detector. Methods for the
Determination of Organic Compounds in Drinking Water, Supplement III. 1995.
38
-------
11. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center.,
Version 5.0, U.S. EPA Environmental Technology Verification Program, Battelle,
Columbus, Ohio, March 2004.
12. Test/QA Plan for Verification of Enzymatic Test Kits, Battelle, Columbus, Ohio,
September 2005.
13. Battelle, SOP HMRC-IV-118-05: Standard Operating Procedure for the Determination of
CA in Wastewater.
14. Battelle, Standard Operating Procedure for Analysis of Water Extracts for Type I Analytes
by Liquid Chromatography/Mass Spectrometry, Version 1, January 2004.
15. Battelle, Standard Operating Procedure for Extracting and Preparing Water Samples for
Analysis of Dicrotophos, Mevinphos, andDichlorovos, Version 3, March 2005.
39
------- |