August 2003
Environmental Technology
Verification Report
Industrial Test Systems, Inc.
Quick™ Low Range II
Test Kit
Prepared by
Baiteiie
Putting Technology To Work
Battel le
Under a cooperative agreement with
&EPA U.S. Environmental Protection Agency
ETV ElV ElV
-------
August 2003
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Industrial Test Systems, Inc.
Quick™ Low Range II
Test Kit
by
Tim Kaufman
Patty White
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency and recommended for public release.
Mention of trade names or commercial products does not constitute endorsement or
recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of seven environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. In 1997, through a competitive cooperative agreement, Battelle was awarded EPA
funding and support to plan, coordinate, and conduct such verification tests for "Advanced
Monitoring Systems for Air, Water, and Soil" and report the results to the community at large.
Information concerning this specific environmental technology area can be found on the Internet
at http://www.epa.gov/etv/centers/centerl .html.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. In particular we would like to thank
Rosanna Buhl, Adam Abbgy, and Bea Weaver of Battelle, and Mike Madigan and Rick Linde of
the Ayer, Massachusetts Department of Public Works Water Division. We also acknowledge the
assistance of Jeff Adams of EPA and AMS Center stakeholders Vito Minei and Marty Link, who
reviewed the verification reports.
iv
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations viii
1 Background 1
2 Technology Description 2
3 Test Design and Procedures 4
3.1 Introduction 4
3.2 Test Design 4
3.3 Test Samples 5
3.3.1 QC Samples 5
3.3.2 PT Samples 7
3.3.3 Environmental Samples 7
3.4 Reference Analysis 8
3.5 Verification Schedule 8
4 Quality Assurance/Quality Control 9
4.1 Laboratory QC for Reference Method 9
4.2 Audits 12
4.2.1 Performance Evaluation Audit 12
4.2.2 Technical Systems Audit 12
4.2.3 Data Quality Audit 13
4.3 QA/QC Reporting 13
4.4 Data Review 13
5 Statistical Methods 15
5.1 Accuracy 15
5.2 Precision 16
5.3 Linearity 16
5.4 Method Detection Limit 16
5.5 Matrix Interference Effects 17
5.6 Operator Bias 17
5.7 Inter-Unit Reproducibility 17
5.8 Rate of False Positives/False Negatives 17
v
-------
6 Test Results 18
6.1 QC Samples 18
6.2 PT and Environmental Samples 22
6.2.1 Accuracy 22
6.2.2 Precision 29
6.2.3 Linearity 29
6.2.4 Method Detection Limit 34
6.2.5 Matrix Interference Effects 34
6.2.6 Operator Bias 35
6.2.7 Inter-Unit Reproducibility 35
6.2.8 Rate of False Positives/False Negatives 37
6.3 Other Factors 37
6.3.1 Ease of Use 37
6.3.2 Analysis Time 41
6.3.3 Reliability 41
6.3.4 Waste Material 41
6.3.5 Cost 41
7 Performance Summary 42
8 References 45
Figures
Figure 2-1. Industrial Test Systems, Inc., Quick™ Low Range II Test Kit 2
Figure 2-2. Quick™ Low Range II Color Chart 3
Figure 6-1. Linearity of Quick™ Low Range II Color Chart Results 33
Figure 6-2. Linearity of Quick™ Low Range II Quick™ Arsenic Scan Results 33
Figure 6-3. Linearity of Quick™ Low Range II Compu-Scan Results 34
Figure 6-4. Comparison of Quick™ Low Range II Test Results for Technical and
Non-Technical Operators 36
Figure 6-5. Comparison of Quick™ Low Range II Test Results for Two
Arsenic Quick™ Scan and Compu-Scan Units 36
vi
-------
Tables
Table 3-1. Test Samples for Verification of the Quick™ Low Range II Test Kit 6
Table 3-2. Schedule of Verification Test Days 8
Table 4-1. Reference Method QCS Analysis Results 10
Table 4-2. Reference Method LFM Sample Results 11
Table 4-3. Reference Method Duplicate Analysis Results 11
Table 4-4. Reference Method PE Audit Results 12
Table 4-5. Summary of Data Recording Process 14
Table 6-la. RB Sample Results for the Technical Operator 19
Table 6-lb. RB Sample Results for the Non-Technical Operator 19
Table 6-2a. QCS Results for the Technical Operator 20
Table 6-2b. QCS Results for the Non-Technical Operator 21
Table 6-3a. LFM Sample Results for the Technical Operator 23
Table 6-3b. LFM Sample Results for the Non-Technical Operator 24
Table 6-4. Quick™ Low Range II Test Kit and Reference Sample Results 25
Table 6-5. Quantitative Evaluation of Accuracy for Quick™ Low Range II Test Kits 28
Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range II Test Kits 30
Table 6-7. Precision Results for Quick™ Low Range II Test Kits 32
Table 6-8. Detection Limit Results for Quick™ Low Range II Test Kit 35
Table 6-9. Rate of False Positives for Quick™ Low Range II Test Kits 38
Table 6-10. Rate of False Negatives for Quick™ Low Range II Test Kits 40
Table 7-1. Summary of Linear Regression Equations for Test Kit and Reference Results 43
vii
-------
List of Abbreviations
AMS
Advanced Monitoring Systems
EPA
U.S. Environmental Protection Agency
ETV
Environmental Technology Verification
HDPE
high-density polyethylene
ICPMS
inductively coupled plasma mass spectrometry
LFM
lab oratory-fortified matrix
MDL
method detection limit
MSDS
Material Safety Data Sheet
NIST
National Institute of Standards and Technology
ppb
parts per billion
ppm
parts per million
PE
performance evaluation
PT
performance test
QA
quality assurance
QA/QC
quality assurance/quality control
QC
quality control
QCS
quality control standard
QMP
Quality Management Plan
RB
reagent blank
RPD
relative percent difference
RSD
relative standard deviation
TSA
technical systems audit
viii
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders, con-
ducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
recently evaluated the performance of the Industrial Test Systems, Inc., Quick™ Low Range II
test kit for measuring arsenic in water.
1
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This verification report provides
results for the verification testing of the Quick™ Low Range II test kit for arsenic in water
(Figure 2-1). The following is a description of the test kit, based on information provided by the
vendor. The information provided below was not verified in this test.
To perform arsenic analyses with the Quick™ Low
Range II test kit, the water sample to be tested is
mixed in the supplied reaction vessel with reagent #1
(tartaric acid with rate enhancers) to acidify the water
sample. Reagent #2, an oxidizer (potassium
peroxymonosulfate), is added to remove hydrogen
sulfide interference. The test tolerates up to 2 parts per
million (ppm) hydrogen sulfide without interference.
Zinc powder, reagent #3, is added to reduce inorganic
arsenic compounds (As 3 and As+5) to arsine gas. As
arsine gas is generated and comes in contact with the
Figure 2-1. Industrial Test Systems, test striP'the mercuric bromide indicator on the test
Inc., Quick™ Low Range II Test Kit stnP chanSes color from whlte t0 shades of yellow or
brown. Material Safety Data Sheets (MSDS) for all
reagents and test strips are provided with each test kit.
The MSDSs include information on how to safely handle the reagents and test strips, including
instructions for exposure controls and personal protection.
Once the reaction is completed, the test strip is removed and visually compared to a color chart
to obtain a semi-quantitative measure of the arsenic concentration in the tested sample.
The color chart consists of a series of color blocks (Figure 2-2). The color blocks correspond to
concentrations ranging from 0.6 parts per billion (ppb) to >100 ppb. If the color on the test strip
is between two color blocks, then the operator may estimate the concentration as between the
values associated with the color blocks on either side.
The test strip may also be read with the Quick™ Arsenic Scan hand-held instrument, which
operates on the same principle as a colorimeter and provides a quantitative result. The Quick™
Arsenic Scan is calibrated weekly using a calibration card provided by the manufacturer.
Quantitative results may also be obtained from the test strip with a portable Compu-Scan scanner
2
-------
Ii9,'l0,6ppb 1 1.3 1.7 2 2,5 3 4
5
Pl/l 7 ppb 9 10 12 15 25 30 >30 >100
Ou/cjif™ Low Range II Calibration Valid 24° to 30' C BDL r Bek« DeMKlon Lt«l 01/10112003
Figure 2-2. Quick™ Low Range II Color Chart
and laptop system. The scanned test strip image is converted to an arsenic concentration using
the Home Port Computer System Arsenic Program Revision 5b software. The scanner is
calibrated by the manufacturer. The Quick™ Arsenic Scan and Compu-Scan are not provided
with the Quick™ Low Range II test kit as a standard feature. The standard test kit with the color
chart was the subject of this verification test; however, results for the Quick™ Arsenic Scan and
Compu-Scan are also provided.
The optimal detection range for the Quick™ Low Range II test kit is below 15 ppb. Dilution
instructions are provided for samples with arsenic levels above 8 ppb. The recommended
temperature range for sample analysis is 24°C to 30°C. A modified testing protocol that specifies
longer reaction times (up to 30 minutes longer for samples between 5°C and 15°C) is available
for sample temperatures below this range.
The kits are available in sets of 50 tests with test strips provided in individually sealed packets.
The typical shelf life of the kits is 24 months.
3
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Portable Analyzers^ The verification was based on comparing the arsenic
results from the Quick™ Low Range II test kit to those from a laboratory-based reference
method. The reference method for arsenic analysis was inductively coupled plasma mass
spectrometry (ICPMS) performed according to EPA Method 200.The Quick™ Low Range
II test kit relies on comparisons to a color chart provided with the test kit to allow semi-
quantitative measurements of arsenic concentrations. Quantitative results were also obtained
from a Quick™ Arsenic Scan instrument and a Compu-Scan system. The test kit performance
was verified by analyzing laboratory-prepared performance test samples, treated and untreated
drinking water, and fresh surface water. All samples were tested using both the test kit and the
reference method. Both semi-quantitative and quantitative analyses were performed by the
technical and non-technical operators. The test design and procedures are described further
below.
3.2 Test Design
The Quick™ Low Range II test kit was verified by evaluating the following parameters:
¦ Accuracy
¦ Precision
¦ Linearity
¦ Method detection limit (MDL)
¦ Matrix interference effects
¦ Operator bias
¦ Inter-unit reproducibility
¦ Rate of false positives/false negatives.
All sample preparation and analyses were performed according to the manufacturer's
recommended procedures. All samples were warmed to 24°C prior to analysis using a hot water
bath, which is at the lower end of the optimal temperature range listed in the test kit instructions.
Color chart, Quick™ Arsenic Scan, and Compu-Scan results were recorded manually. The
results from the Quick™ Low Range II test kits were compared to those from the reference
method to assess accuracy and linearity. Multiple aliquots of performance test samples, drinking
water samples, and surface water samples were analyzed to assess precision. Multiple aliquots of
a low-level performance test sample were analyzed to assess the detection limit of the method.
4
-------
Potential matrix interference effects were assessed by challenging the test kit with performance
test samples of known arsenic concentrations that contained both low levels and high levels of
interfering substances.
Identical sets of samples were analyzed independently by a technical and a non-technical
operator. The technical operator was a technician at Battelle with three years of field and
laboratory experience and a B.A. degree. The non-technical operator was a part-time temporary
helper enrolled in undergraduate studies. Because the reagents of the Quick™ Low Range II test
kits were consumed in use, it was not feasible for the two operators to use the same kits; how-
ever, each operator used multiple kits in order to analyze all the samples and it is assumed that
kit-to-kit variability was similar for both operators. Results of all analyses were statistically
compared to evaluate operator bias. The technical operator analyzed all samples using two
Quick™ Arsenic Scan units and two Compu-Scan systems to assess inter-unit reproducibility.
The rate of false positive and false negative results were evaluated relative to the 10-ppb
maximum contaminant level for arsenic in drinking water.(4) Other factors that were quali-
tatively assessed during the test included time required for sample analysis, ease of use, and
reliability.
3.3 Test Samples
Three types of samples were analyzed in the verification test, as shown in Table 3-1: quality
control (QC) samples, performance test (PT) samples, and environmental water samples. The QC
and PT samples were prepared from National Institute of Standards and Technology (NIST)
traceable standards purchased from a commercial supplier and subject only to dilution as
appropriate. Under the Safe Drinking Water Act, the EPA lowered the maximum contaminant
level for arsenic from 50 ppb to 10 ppb in January 2001; public water supply systems must
comply with this standard by January 2006.(4) Therefore, the QC sample concentrations targeted
the 10 ppb arsenic level. The PT samples ranged from 10% to 1,000% of the 10 ppb level (i.e.,
from 1 ppb to 100 ppb). The environmental water samples were collected from various drinking
water and surface freshwater sources.
Each sample was assigned a unique sample identification number when prepared in the
laboratory or collected in the field. The PT and environmental samples were submitted blind to
the technical and non-technical operators and were analyzed randomly to the degree possible.
3.3.1 QC Samples
QC samples included laboratory reagent blank (RB) samples, quality control samples (QCS), and
laboratory-fortified matrix (LFM) samples (Table 3-1). The RB samples consisted of the same
ASTM Type I water used to prepare all other samples and were subjected to the same handling
and analysis procedures as the other samples. The RB samples were used to verify that no
arsenic contamination was introduced during sample handling and analysis. RB samples were
analyzed at a frequency of 10%.
5
-------
Table 3-1. Test Samples for Verification of the Quick™ Low Range II Test Kit
Type of
Sample
Sample Characteristics
Arsenic
Concentration (a)
No. of
Replicates
Quality Control
Reagent Blank (RB)
~ 0 ppb
10% of all
Quality Control Sample (QCS)
10 ppb
10% of all
Laboratory Fortified Matrix (LFM)
10 ppb above native
level
1 per site
Performance
Test
Prepared arsenic solution
Prepared arsenic solution
1 ppb
3 ppb
4
4
Prepared arsenic solution
10 ppb
4
Prepared arsenic solution
30 ppb
4
Prepared arsenic solution
100 ppb
4
Prepared arsenic solution for detection limit
determination
3 ppb
7
Prepared arsenic solution spiked
with low levels of interfering substances
10 ppb
4
Prepared arsenic solution spiked
with high levels of interfering substances
10 ppb
4
Environmental
Battelle drinking water
<0.5 ppb
4
Ayer untreated water
64.8 ppb
4
Ayer treated water
1.39 ppb
4
Falmouth Pond water
<0.5 ppb
4
Taunton River water
1.31 ppb
4
Performance Test sample concentrations are target levels; environmental sample concentrations are actual
(average of four replicate measurements).
The QCS consisted of ASTM Type I water spiked in the lab to a concentration of 10 ppb arsenic
with a NIST-traceable standard. QCS were used as calibration checks to verify that the Quick™
Low Range II test kit was operating properly. QCS were analyzed at the beginning and end of
each testing period, as well as after every tenth sample. Because the test kit utilized a color chart
that could not be calibrated, no performance criteria were specified for the QCS.
The LFM samples consisted of aliquots of environmental samples that were spiked in the field to
increase the arsenic concentration by 10 ppb. The spike solution used for the LFM samples was
prepared in the laboratory and brought to the field site. One LFM sample was prepared from
each environmental sample.
6
-------
3.3.2 PT Samples
Three types of PT samples used in this verification test (Table 3-1): spiked samples ranging
from 1 ppb to 100 ppb arsenic, a low-level spiked sample for evaluation of the test kit's detection
limit, and matrix interference samples that were spiked with potential interfering substances. All
PT samples were prepared in the laboratory using ASTM Type I water and NIST-traceable
standards.
Five PT samples containing arsenic at concentrations from 1 ppb to 100 ppb were prepared to
evaluate Quick™ Low Range II test kit accuracy and linearity. Four aliquots of each of these
samples were analyzed to assess precision.
To determine the detection limit of the Quick™ Low Range II test kit, a PT sample was prepared
with an arsenic concentration five times the lowest concentration shown on the test kit color
chart (i.e., 0.6 ppb X 5 = 3 ppb). Seven non-consecutive replicates of this 3 ppb arsenic sample
were analyzed to provide precision data with which to estimate the method detection limit
(MDL).
The matrix interference samples were spiked with 10 ppb arsenic as well as potentially
interfering substances commonly found in natural water samples. One sample contained low
levels of interfering substances that consisted of 1 ppm iron, 3 ppm sodium chloride, and 0.1
ppm sulfide. The second sample contained high levels of interfering compounds at the following
concentrations: 10 ppm iron, 30 ppm sodium chloride, and 1.0 ppm sulfide. Four replicates of
each of these samples were analyzed.
3.3.3 Environmental Samples
The environmental samples listed in Table 3-1 included three drinking water samples and two
surface water samples. All environmental samples were collected in 20-L high density poly-
ethylene (HDPE) carboys. The Battelle drinking water sample was collected directly from a tap
without purging. Untreated and treated groundwater samples from the Ayer, Massachusetts
Department of Public Works Water Treatment Plant were collected directly from spigots, also
without purging. Four aliquots of each sample were analyzed using the Quick™ Low Range II
test kit in the Battelle laboratory as soon as possible after collection. One aliquot of each sample
was preserved with nitric acid and submitted to the reference laboratory for reference analysis.
One surface water sample was collected from a pond in Falmouth, Massachusetts and another
was collected from the Taunton River near Bridgewater, Massachusetts. These samples were
collected near the shoreline by submerging a 2-L HDPE sample container no more than one inch
below the surface of the water, and decanting the water into a 20-L HDPE carboy until full. Each
water body was sampled at one accessible location. These samples could not be analyzed at the
field location as planned because of persistent, severe winter weather conditions. Therefore, the
samples were returned to a storage shed at the Battelle laboratory, which was heated but not
serviced by running water. The storage shed was intended to simulate realistic field conditions
under which the test kits might be used. Four aliquots of each surface water sample were
7
-------
analyzed in the storage shed as soon as possible after collection. One aliquot of each sample was
preserved with nitric acid and submitted to the reference laboratory for reference analysis.
3.4 Reference Analysis
The reference arsenic analyses were performed in a Battelle laboratory using a Perkin Elmer
Sciex Elan 6000 ICPMS according to EPA Method 200.8, Revision 5.5.^ The sample was
introduced through a peristaltic pump by pneumatic nebulization into a radiofrequency plasma
where energy transfer processes caused desolvation, atomization, and ionization. The ions were
extracted from the plasma through a pumped vacuum interface and separated on the basis of their
mass-to-charge ratio by a quadrupole mass spectrometer. The ions transmitted through the
quadrupole were registered by a continuous dynode electron multiplier, and the ion information
was processed by a data handling system.
The ICPMS was tuned, optimized, and calibrated daily. The calibration was performed using a
minimum of five calibration standards at concentrations ranging between 0.5 and 250 ppb, and a
required correlation coefficient of a minimum of 0.999. Internal standards were used to correct
for instrument drift and physical interferences. These standards were introduced in line through
the peristaltic pump and analyzed with all blanks, standards, and samples.
3.5 Verification Schedule
The verification test took place from January 29 through February 24, 2003. Table 3-2 shows the
daily activities that were conducted during this period. The reference analyses were performed
on March 7 and March 14, 2003, five to six weeks after sample collection.
Table 3-2. Schedule of Verification Test Days
Sample Analysis Date
Sample
Collection Date
Tech. Op.
Non-tech.
Op.
Testing
Location
Activity
1/29/03-2/11/03
1/29/03-
2/11/03
1/29/03-
2/11/03
Battelle
Laboratory
Preparation and analysis of PT and associated
QC samples.
2/12/03
2/14/03
2/13/03
Battelle
Laboratory
Collection and analysis of Ayer untreated and
treated water and associated QC samples.
2/17/03
2/17/03
2/19/03
Battelle
Laboratory
Collection and analysis of Battelle drinking
water and associated QC samples.
2/21/03
2/21/03
2/21/03
Battelle
Storage Shed
Collection and analysis of Falmouth Pond
water and associated QC samples.
2/23/03
2/24/03
2/24/03
Battelle
Storage Shed
Collection and analysis of Taunton River
water and associated QC samples.
8
-------
Chapter 4
Quality Assurance/Quality Control
Quality assurance/quality control (QA/QC) procedures were performed in accordance with the
quality management plan (QMP) for the AMS Center(3) and the test/QA plan for this verification
test.(1) QA/QC procedures and results are described below.
4.1 Laboratory QC for Reference Method
Reference analyses were conducted on March 7 and March 14, 2003. Laboratory QC for the
reference method included the analysis of RB, QCS, LFM and analytical duplicate samples.
Laboratory RB samples were analyzed to ensure that no contamination was introduced by the
sample preparation and analysis process. The test/QA plan stated that if arsenic was detected in a
RB sample above the MDL for the reference instrument, then the contamination source would be
identified and removed and proper blank readings achieved before proceeding with the reference
analyses. All of the laboratory RB samples analyzed were below the reporting limit for arsenic
(i.e., below the concentration of the lowest calibration standard) except for several blanks that
were analyzed at the end of the day on March 7. Three of the six test samples that were
associated with these RB samples were re-analyzed on March 14, with acceptable RB sample
results. The other three test samples had arsenic concentrations that were approximately twenty
times higher than the RB sample concentrations; therefore, no action was taken.
On March 7, the instrument used for the reference method was calibrated using nine calibration
standards, with concentrations ranging from 0.5 to 250 ppb arsenic. On March 14, it was
calibrated using eight standards ranging in concentration from 0.1 to 25 ppb arsenic for more
accurate analysis of low level samples. The accuracy of the calibration was verified after the
analysis of every 10 samples by analyzing a QCS of a known concentration. The percent
recovery of the QCS was calculated from the following equation:
C m
R = —^-x 100 v ;
s
where Cs is the measured concentration of the QCS and 5 is the spike concentration. If the QCS
analysis differed by more than 10% from the true value of the standard, the instrument was
recalibrated before continuing the test. As shown in Table 4-1, all QCS analyses were within the
required range.
9
-------
Table 4-1. Reference Method QCS Analysis Results
Measured Percent
Sample ID Analysis Date (ppb) Actual (ppb) Recovery
CCV25
3/7/2003
24.96
25.00
100%
QCS 25
3/7/2003
26.81
25.00
107%
CCV25
3/7/2003
24.50
25.00
98%
CCV25
3/7/2003
25.39
25.00
102%
CCV25
3/7/2003
25.73
25.00
103%
CCV25
3/7/2003
25.81
25.00
103%
CCV25
3/7/2003
25.64
25.00
103%
CCV25
3/7/2003
25.30
25.00
101%
CCV25
3/7/2003
24.90
25.00
100%
CCV25
3/7/2003
22.67
25.00
91%
QCS 25
3/14/2003
24.90
25.00
100%
CCV 2.5
3/14/2003
2.74
2.50
110%
QCS 2.5
3/14/2003
2.70
2.50
108%
CCV 2.5
3/14/2003
2.58
2.50
103%
CCV 2.5
3/14/2003
2.65
2.50
106%
CCV 2.5
3/14/2003
2.66
2.50
106%
CCV 2.5
3/14/2003
2.61
2.50
104%
CCV 2.5
3/14/2003
2.60
2.50
104%
LFM samples were analyzed to assess whether matrix effects influenced the reference method
results. The LFM percent recovery (R) was calculated from the following equation:
R= °s xlOO (2)
s
where Cs is the measured concentration of the spiked sample, C is the measured concentration of
the unspiked sample, and 5 is the spike concentration. If the percent recovery of an LFM sample
fell outside the range from 85% to 115%, a matrix effect was suspected. As shown in Table 4-2,
all of the LFM sample results were within this range.
Duplicate samples were analyzed to assess the precision of the reference analysis. The relative
percent difference (RPD) of the duplicate sample analysis was calculated from the following
equation:
RPD= {C ~ C"} x 100 (3)
(C + CD)/2
Where C is the concentration of the sample analysis, and Cd is the concentration of the duplicate
sample analysis. If the RPD was greater than 10%, the instrument was recalibrated before con-
tinuing the test. As shown in Table 4-3, the RPDs for the duplicate analyses were all less than
10
-------
Table 4-2. Reference Method LFM Sample Results
Amount
Sample ID
Matrix
Analysis Date
Unspiked
(PPb)
Spiked
(PPb)
Spiked
(PPb)
Percent
Recovery
CAA-22
ASTM Type I
water
3/7/2003
11.02
37.20
25.00
105%
CAA-25 R4
ASTM Type I
water
3/7/2003
0.95
22.76
25.00
87%
CAA-28 R2
ASTM Type I
water
3/7/2003
3.45
30.64
25.00
109%
CAA-29 R4
ASTM Type I
water
3/7/2003
34.98
60.37
25.00
102%
CAA-37 R4
Drinking water
3/7/2003
0.52
28.20
25.00
111%
CAA-41 R4
Drinking water
3/7/2003
1.24
28.88
25.00
111%
CAA-48
Surface water
3/7/2003
12.26
39.40
25.00
109%
CAA-47 R4
Surface water
3/7/2003
1.07
28.41
25.00
109%
CAA-27 R1
ASTM Type I
water
3/14/2003
2.56
4.73
2.50
87%
CAA-37 R3
Drinking water
3/14/2003
0.45
3.11
2.50
107%
CAA-47 R1
Surface water
3/14/2003
1.36
4.16
2.50
112%
CAA-88 R3
Drinking water
3/14/2003
0.43
3.16
2.50
109%
CAA-88 R4
Drinking water
3/14/2003
0.42
3.18
2.50
111%
Table 4-3. Reference Method Duplicate Analysis Results
Sample ID
Analysis
Date
Sample
concentration
(PPb)
Duplicate
concentration
(PPb)
Relative
Percent
Difference
CAA-4
3/7/2003
9.33
9.20
1.4%
CAA-70
3/7/2003
10.93
10.82
1.0%
CAA-26 R1
3/7/2003
1.14
1.13
1.4%
CAA-28 R3
3/7/2003
3.49
3.45
1.1%
CAA-31 R1
3/7/2003
111.89
112.20
0.3%
CAA-38
3/7/2003
11.96
11.90
0.5%
CAA-42
3/7/2003
13.02
13.06
0.3%
CAA-48
3/7/2003
12.26
12.22
0.4%
CAA-23
3/14/2003
3.03
2.99
1.3%
CAA-27 R2
3/14/2003
2.64
2.61
0.9%
CAA-37 R4
3/14/2003
0.44
0.43
2.3%
CAA-47 R2
3/14/2003
1.31
1.32
0.2%
CAA-88 R4
3/14/2003
0.42
0.38
9.5%
11
-------
10%. The RPD for one duplicate pair was 9.5%; however, the reported concentrations were
below the reporting limit for the reference method (i.e., below the concentration of the lowest
calibration standard).
4.2 Audits
Three types of audits were performed during the verification test: a performance evaluation (PE)
audit of the reference method, a technical systems audit of the verification test performance, and
a data quality audit. Audit procedures are described further below.
4.2.1 Performance Evaluation Audit
A PE audit was conducted to assess the quality of the reference measurements made in this
verification test. For the PE audit, an independent, NIST-traceable, reference material was
obtained from a different commercial supplier than the calibration standards and the standard
used to prepare the PT and field QCS samples. Accuracy of the reference method was verified by
comparing the arsenic concentration measured using the calibration standards to that obtained
using the independently-certified PE standard. Relative percent difference as calculated by
Equation 3 was used to quantify the accuracy of the results. Agreement of the standard within
10%) was required for the measurements to be considered acceptable. As shown in Table 4-4, the
PE sample analysis was within the required range.
Table 4-4. Reference Method PE Audit Results
Measured
Arsenic
Actual Arsenic
Date of
Concentration
Concentration
Percent
Sample ID
Analysis
(PPb)
(PPb)
Difference
PE-1
3/24/03
9.63
10.0
4
4.2.2 Technical Systems Audit
An independent Battelle quality staff conducted a technical systems audit (TSA) from February 3
to 6 to ensure that the verification test was being conducted in accordance with the test/QA
plann' and the AMS Center QMP/3' A TSA of the reference method performance was con-
ducted by the Battelle Quality Manager on March 5, 2003, when the reference analyses were
initiated. As part of the TSA, test procedures were compared to those specified in the test/QA
plan, data acquisition and handling procedures were reviewed, and the reference standards and
method were reviewed. Observations and findings from the TSA were documented and sub-
mitted to the Battelle Verification Test Coordinator for response. None of the TSA findings
required corrective action. TSA records are permanently stored with the Battelle Quality
Manager.
12
-------
4.2.3 Data Quality Audit
At least 10% of the data acquired during the verification test were audited. The Battelle Quality
Manager traced the data from the initial acquisition, through reduction and statistical analysis, to
final reporting to ensure the integrity of the reported results. All calculations performed on the
data undergoing the audit were checked.
4.3 QA/QC Reporting
Each audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the ETV
AMS Center.(3) Once the audit reports were prepared, the Battelle Verification Test Coordinator
ensured that a response was provided for each adverse finding or potential problem and imple-
mented any necessary follow-up corrective action. The Battelle Quality Manager ensured that
follow-up corrective action was taken. The results of the TSA and the data quality audit were
submitted to the EPA.
4.4 Data Review
Records generated in the verification test received a review before these records were used to
calculate, evaluate, or report verification results. Table 4-5 summarizes the types of data
recorded and reviewed. All data were recorded by Battelle staff. Data were reviewed by a
Battelle technical staff member involved in the verification test, but not the staff member that
originally generated the record. The person performing the review added his/her initials and the
date to a hard copy of the record being reviewed.
13
-------
Table 4-5. Summary of Data Recording Process
Data to be Recorded Where Recorded How Often Recorded Disposition of Data(a)
Dates, times of test
events
Test parameters
(temperature, analyte/
interferant identities,
and all Quick™ Low
Range II test kit results
for color chart, Quick™
Arsenic Scan and
Compu-Scan)
Reference method
sample analysis, chain
of custody, and results
ETV field data
sheets
ETV field data
sheets
Laboratory record
books, data sheets,
or data acquisition
system, as
appropriate
Start/end of test event
When set or changed, or as
needed to document test
Throughout sample
handling and analysis
process
Used to organize/check test
results; manually
incorporated in data
spreadsheets as necessary
Used to organize/check test
results, manually
incorporated in data
spreadsheets as necessary
Transferred to spreadsheets
(a) All activities subsequent to data recording were carried out by Battelle.
14
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the performance factors listed in Section 3.2 are
presented in this chapter. Qualitative observations were also used to evaluate verification test
data.
5.1 Accuracy
All samples were analyzed by both the Quick™ Low Range II test kit and reference methods.
For each sample, accuracy was expressed in terms of a relative bias (B) as calculated from the
following equation:
B = iLxlOO (4)
where d is the average difference between the reading from the Quick™ Low Range II test kit
and those from the reference method, and CR is the average of the reference measurements. An
additional assessment of accuracy was conducted for the color chart results because of the semi-
quantitative nature of the visual comparisons. Performance was assessed by determining whether
the result falls within the expected concentration range as measured by the reference analysis.
Overall agreement was assessed by calculating the percent of results that fell within the correct
range, calculated from the following equation:
Y (5)
A = — xlOO K)
n
where A is the percent of measurements in agreement, 7is the number of measurements within
the expected color range, and n is the total number of measurements. Readings below the
vendor-stated detection limit of the test kit (e.g., <0.6 ppb) were judged to be in agreement with
the reference result if the reference value was in the specified "less than" range.
15
-------
5.2 Precision
When possible, the standard deviation (S) of the results for the replicate samples at each
concentration was calculated and used as a measure of Quick™ Low Range II test kit precision.
Standard deviation was calculated from the following equation:
S =
n -
TI (C,-
1 k=1
iX
cy
(6)
where n is the number of replicate samples, C/, is the concentration measured for the kth sample,
and C is the average concentration of the replicate samples. Precision was reported in terms of
the relative standard deviation (RSD) as follows:
RSD =
S_
c
x 100
(7)
5.3 Linearity
Linearity was assessed by performing a linear regression of Quick™ Low Range II test kit
results against the reference results, with linearity characterized by the slope, intercept, and
correlation coefficient (R). Linearity was tested using the five PT samples over the range 1 ppb
to 100 ppb arsenic. Samples with results below the vendor-stated test kit detection limit were not
included in the analysis. Color chart, Quick™ Arsenic Scan and Compu-Scan results were
plotted against the corresponding reference concentrations and separate regressions were
performed.
5.4 Method Detection Limit
The MDL for the Quick™ Low Range II test kit was assessed using results from all three
detection methods (color chart, Quick™ Arsenic Scan, and Compu-Scan) for seven replicate
analyses of a sample spiked with 3 ppb arsenic. The standard deviation of the seven replicate
samples was calculated using Equation 6. The MDL was calculated using the following equation:
MDL = txS (8)
where t is the Student's t value for a 99% confidence level and S is the standard deviation of the
seven replicate samples.
16
-------
5.5 Matrix Interference Effects
The potential effect of interfering substances on the sensitivity of the Quick™ Low Range II test
kit was evaluated by the calculating accuracy (expressed as bias) using Equation 4. These results
were qualitatively compared with accuracy results for PT samples containing only arsenic to
assess whether there was a positive or negative effect due to matrix interferences.
5.6 Operator Bias
Potential operator bias for the Quick™ Low Range II test kit was assessed by performing a linear
regression of sample results above the detection limit generated by the technical and non-
technical operator. Color chart, Quick™ Arsenic Scan, and Compu-Scan results were evaluated.
The slope, intercept, and correlation coefficient were used to evaluate the degree of operator
bias. A paired t-test was also conducted to evaluate whether the two sets of sample results were
significantly different at a 95% confidence level.
5.7 Inter-Unit Reproducibility
Inter-unit reproducibility for the Quick™ Arsenic Scan and Compu-Scan was assessed by
performing a linear regression of sample results generated by the two units of each device for the
technical operator. The slope, intercept, and correlation coefficient were used to evaluate the
degree of inter-unit reproducibility. A paired t-test was also conducted to evaluate whether the
two sets of sample results were significantly different at a 95% confidence level.
5.8 Rate of False Positives/False Negatives
The rates of false positives and false negatives produced by the Quick™ Low Range II test kit
were assessed relative to the 10-ppb target arsenic level. A false positive result is defined as any
result reported to be greater than the guidance level (10 ppb) and greater than 125% of the
reference value, when the reference value is less than or equal to the guidance level. Similarly, a
false negative result is defined as any result reported below or equal to the guidance level and
less than 75% of the reference value, when the reference value is greater than the guidance level.
The rates of false positives and false negatives were expressed as a percentage of total samples
analyzed for each type of sample.
17
-------
Chapter 6
Test Results
The results of the verification test of the Quick™ Low Range II test kits are presented in this
section.
6.1 QC Samples
As described in Section 3.3.1, the QC samples analyzed with the Quick™ Low Range II test kit
included RB, QCS, and LFM samples (these QC samples were different than those analyzed in
conjunction with the reference method). The RB samples were analyzed at a frequency of 10%
and results were used to verify that no arsenic contamination was introduced during sample
handling and analysis. QCS were analyzed at the beginning and end of each test period, and after
every tenth sample. The QCS results were used to verify that the test kit was operating properly.
One LFM sample was prepared from each environmental sample to evaluate potential matrix
interferences. Acceptance criteria for test kit QC samples were not specified in the test/QA plan
because modifications to the technology would not be made during testing.
RB sample results for the technical and non-technical operators are presented in Tables 6-la and
6-lb, respectively. Unique sample identification codes were assigned to each container of ASTM
Type I water that was used. The RB samples were analyzed at the required frequency. The
technical and non-technical operators reported all RB samples as below the detection limit for
the color chart and the Quick™ Arsenic Scan. The Compu-Scan units always returned a detected
value for the RB samples because no detection limit was provided by the vendor. Because all
color chart and Quick™ Arsenic Scan results for the RB samples were below detection, it
appeared that arsenic contamination resulting from sample handling and analysis did not occur.
QCS results for the technical and non-technical operators are presented in Tables 6-2a and 6-2b,
respectively. The QCS were analyzed at the required frequency except on the first day of testing,
when the technical operator inadvertently omitted these samples. The percent recovery of the
QCS was calculated using Equation 1 (Section 4.1). The QCS percent recoveries for the tech-
nical operator ranged from 25% to 100% for the color chart, from 12% to 72% for the Quick™
Arsenic Scan, and from 8% to 120% for the Compu-Scan. The QCS percent recoveries for the
non-technical operator ranged from 40% to 100% for the color chart, from 15% to 100% for the
Quick™ Arsenic Scan, and from 2% to 89% for the Compu-Scan. In general, the QCS recoveries
were lower than expected for the color chart, Quick™ Arsenic Scan, and the Compu-Scan.
18
-------
Table 6-la. RB Sample Results for the Technical Operator
Quick™
Quick™
Arsenic Scan
Arsenic Scan Compu-Scan
Compu-Scan
Color
#1
#2
#1
#2
Sample ID
Replicate
Analysis Date Chart (ppb)
(ppb)
(ppb)
(ppb)
(ppb)
CAA-3
1
1/29/2003 <0.6
<0.3
<0.3
0.5
0.7
CAA-6
1
1/31/2003 <0.6
<0.3
<0.3
0.6
0.7
CAA-11
1
2/4/2003 <0.6
<0.3
<0.3
0.2
1.6
CAA-50
1
2/7/2003 <0.6
<0.3
<0.3
0.7
1
CAA-52
1
2/11/2003 <0.6
<0.3
<0.3
1.2
1
CAA-53
1
2/11/2003 <0.6
<0.3
<0.3
0.6
0.7
CAA-55
1
2/14/2003 <0.6
<0.3
<0.3
0.5
0.9
CAA-55
2
2/14/2003 <0.6
<0.3
<0.3
0.7
1
CAA-56
1
2/17/2003 <0.6
<0.3
<0.3
0.4
1.5
CAA-58
1
2/21/2003 <0.6
<0.3
<0.3
0.7
0.8
CAA-59
1
2/24/2003 <0.6
<0.3
<0.3
0.8
0.4
Table 6-lb. RB Sample Results for the Non-Technical Operator
Quick™ Arsenic
Color Chart
Scan #1 Compu-Scan #1
Sample ID
Replicate
Analysis Date
(PPb)
(ppb)
(PPb)
CAA-6
1
1/30/2003
<0.6
<0.3
0.3
CAA-6
2
1/31/2003
<0.6
<0.3
0.4
CAA-12
1
2/4/2003
<0.6
<0.3
0.07
CAA-50
1
2/7/2003
<0.6
<0.3
0.5
CAA-52
1
2/11/2003
<0.6
<0.3
1
CAA-53
1
2/11/2003
<0.6
<0.3
0.8
CAA-54
1
2/13/2003
<0.6
<0.3
0.9
CAA-54
2
2/13/2003
<0.6
<0.3
0.07
CAA-57
1
2/19/2003
<0.6
<0.3
0.7
CAA-58
1
2/21/2003
<0.6
<0.3
1.4
CAA-59
1
2/24/2003
<0.6
<0.3
1
-------
Table 6-2a. QCS Results for the Technical Operator(a)
Sample ID
Replicate
Analysis
Date
Color
Chart
(PPb)
Quick™
Arsenic
Scan #1
(PPb)
Quick™
Arsenic
Scan #2
(PPb)
Compu-
Scan #1
(PPb)
Compu-
Scan #2
(PPb)
Amount
Spiked
(PPb)
Percent
Recovery
Color
Chart
Percent
Recovery
Quick™
Arsenic
Scan #1
Percent
Recovery
Quick™
Arsenic
Scan #2
Percent
Recovery
Compu-
Scan #1
Percent
Recovery
Compu-
Scan #2
CAA-15
1
1/31/2003
10(b)
7.2 (b)
6.6 <">
5.4 ^
12 ^
10.0
100%
72%
66%
54%
120%
CAA-15
2
1/31/2003
8(h)
7.2 (b)
5.4 ^
10(b)
8.4 (b)
10.0
80%
72%
54%
100%
84%
CAA-16
1
2/4/2003
6.8 (c)
5.6
4.4 (c:i
3.2 (c)
5.2 (c)
10.0
68%
56%
44%
32%
52%
CAA-16
2
2/4/2003
6.8 (c)
4.4 (c-1
4.4 (c-1
2 (c)
4 (<0
10.0
68%
44%
44%
20%
40%
CAA-18
1
2/7/2003
4(b)
2(b)
2(b)
1.6 <">
4.6 <">
10.0
40%
20%
20%
16%
46%
CAA-18
2
2/7/2003
6 (h)
4.2 (b)
2.8 ^
2.6 ^
3.4 ^
10.0
60%
42%
28%
26%
34%
CAA-19
1
2/11/2003
7
4.5
4.5
6.1
6.6
10.0
70%
45%
45%
61%
66%
CAA-19
2
2/11/2003
4
2.3
2.5
1.6
3.4
10.0
40%
23%
25%
16%
34%
CAA-20
1
2/14/2003
2.6 ^
1.2 ^
1.6 <">
0.8 0»
1.4 0»
10.0
26%
12%
16%
8%
14%
CAA-20
2
2/14/2003
2.5(b)
l.9(b)
2.1 ^
5.7 ^
2.2 (b)
10.0
25%
19%
21%
57%
22%
CAA-21
1
2/17/2003
4
2.4
2.4
3.5
3.6
10.0
40%
24%
24%
35%
36%
CAA-21
2
2/17/2003
4
2.4
2.1
3.7
5.7
10.0
40%
24%
21%
37%
57%
CAA-70
1
2/21/2003
5
3.3
3.6
2.5
3.3
10.0
50%
33%
36%
25%
33%
CAA-70
2
2/21/2003
5
2
2.3
2.4
3
10.0
50%
20%
23%
24%
30%
CAA-72
1
2/24/2003
5 (b)
4(b)
3.8 ^
7 (b)
4(b)
10.0
50%
40%
38%
70%
40%
CAA-72
2
2/24/2003
5 (b)
5.6 <">
5.6 <">
7 4 (b)
5.2 ^
10.0
50%
56%
56%
74%
52%
(a) = The technical operator inadvertently omitted the QCS samples on the first day of testing (1/29/03);
= diluted 1:2;{c> = diluted 1:4.
-------
Table 6-2b. QCS Results for the Non-Technical Operator
Percent
Quick™ Recovery Percent
Color
Arsenic
Compu-
Amount
Percent
Quick™
Recovery
Analysis
Chart
Scan #1
Scan #1
Spiked
Recovery
Arsenic
Compu-
Sample ID
Replicate
Date
(ppb)
(ppb)
(ppb)
(ppb)
Color Chart
Scan #1
Scan #1
CAA-14
1
1/30/2003
9
7.2
0.2
10.0
90%
72%
2%
CAA-14
2
1/30/2003
5
3.5
3
10.0
50%
35%
30%
CAA-14
3
1/31/2003
7
5.8
7.7
10.0
70%
58%
77%
CAA-15
1
1/31/2003
7
6
5.7
10.0
70%
60%
57%
CAA-16
1
2/4/2003
5
7.6
8.2
10.0
50%
76%
82%
CAA-16
2
2/4/2003
7
9.5
7.5
10.0
70%
95%
75%
CAA-18
1
2/7/2003
6 (a)
5 (a)
5.2 (a)
10.0
60%
50%
52%
CAA-18
2
2/7/2003
10(a)
10(a)
y(a)
10.0
100%
100%
70%
CAA-19
1
2/11/2003
4
3.8
4.6
10.0
40%
38%
46%
CAA-19
2
2/11/2003
5
4
4.8
10.0
50%
40%
48%
CAA-20
1
2/13/2003
5
4.5
3
10.0
50%
45%
30%
CAA-20
2
2/13/2003
7
7.2
7.3
10.0
70%
72%
73%
CAA-22
1
2/19/2003
5
4.7
4.3
10.0
50%
47%
43%
CAA-22
2
2/19/2003
7
7.6
8.9
10.0
70%
76%
89%
CAA-70
1
2/21/2003
5
4.8
7.2
10.0
50%
48%
72%
CAA-70
2
2/21/2003
5
2.8
5.1
10.0
50%
28%
51%
CAA-72
1
2/24/2003
4
1.5
3.2
10.0
40%
15%
32%
CAA-72
2
2/24/2003
5
6.6
8.2
10.0
50%
66%
82%
(a) = diluted 1:2.
-------
The LFM sample results for the technical and non-technical operators are presented in Tables
6-3a and 6-3b. The percent recovery associated with each LFM sample was calculated using
Equation 2 (Section 4.1). Reference method results are also provided for comparison. The lowest
recoveries measured by both operators were associated with the Ayer untreated water LFM
sample. The reference method results also indicated a low recovery. The low recoveries indicate
that a matrix interference may be adversely affecting the detection of arsenic. Consequently, test
kit results for the Ayer untreated water sample may be biased low. A matrix effect also may be
affecting the Compu-Scan results for the Falmouth Pond water LFM sample; percent recoveries
for the Compu-Scan are more than three times higher than the color chart or Quick™ Arsenic
Scan results for the same sample. These results suggest that Compu-Scan measurements for the
Falmouth Pond water sample could be biased high compared with the color chart and Quick™
Arsenic Scan results.
6.2 PT and Environmental Samples
Table 6-4 presents the sample results for the PT and environmental samples. The table includes
the Quick™ Low Range II test kit results and the reference method results. The Quick™ Low
Range II test kit results are shown for both the technical and non-technical operators, the
Quick™ Arsenic Scan Units #1 and #2, and the Compu-Scan Units #1 and #2. Some Quick™
Low Range II test kit results were below the detection limit and were assigned a value of
<0.6 ppb for the color chart and <0.3 ppb for the Quick™ Arsenic Scan. No detection level was
provided by the vendor for the Compu-Scan. The reporting limit for the reference analyses was
0.5 ppb, which corresponds to the lowest calibration standard used. Results for each performance
factor are presented below.
6.2.1 Accuracy
Table 6-5 presents the accuracy results for the Quick™ Low Range II test kit, expressed as
percent bias as calculated by Equation 4 (Section 5.1). Percent bias was not calculated for results
below the detection limit. The four replicate analyses for each sample were averaged in the
calculation of bias. The relative bias for the color chart ranged from -92% to -8% for the tech-
nical operator and -74% to 74% for the non-technical operator. The relative bias for the Quick™
Arsenic Scan ranged from -98% to -27% for the technical operator and -76% to 9% for the non-
technical operator. The relative bias for the Compu-Scan ranged from -93% to 104% for the
technical operator, and from -67% to 81% for the non-technical operator. The high negative
biases for the Ayer untreated water sample confirm the apparent matrix effect observed in the
associated LFM sample (see Section 6.1). Biases could not be calculated for the Falmouth Pond
water sample because the reference method results were below the reporting limit. However, the
Compu-Scan results for this sample were two to ten times higher than the color chart and
Quick™ Arsenic Scan results (Table 6-4), which confirms the apparent matrix effect observed in
the associated LFM sample.
22
-------
Table 6-3a. LFM Sample Results for the Technical Operator
Amount
Unspiked(a)
Spiked
spiked
Percent
Description
(PPb)
(PPb)
(ppb)
Recovery
Battelle drinking water LFM
Color Chart
<0.6
5
10
50%
Quick™ Arsenic Scan # 1
<0.3
4
10
40%
Quick™ Arsenic Scan #2
<0.3
3.6
10
36%
Compu-Scan #1
0.7
5
10
43%
Compu-Scan #2
0.7
4.6
10
39%
Reference
<0.5
11.96
10
120%
Ayer untreated water LFM
Color Chart
5
6.5
10
15%
Quick™ Arsenic Scan # 1
3.5
1.5
10
-20%
Quick™ Arsenic Scan #2
3.7
3.0
10
-7%
Compu-Scan #1
6.6
5
10
-16%
Compu-Scan #2
7.2
6.5
10
-7%
Reference
64.82
69.74
10
49%
Ayer treated water LFM
Color Chart
<0.6
6
10
60%
Quick™ Arsenic Scan # 1
<0.3
3.6
10
36%
Quick™ Arsenic Scan #2
<0.3
3.2
10
32%
Compu-Scan #1
0.65
6.6
10
60%
Compu-Scan #2
0.85
6.6
10
58%
Reference
1.39
13.02
10
116%
Falmouth Pond water LFM
Color Chart
<0.6
4
10
40%
Quick™ Arsenic Scan # 1
<0.3
3.3
10
33%
Quick™ Arsenic Scan #2
0.3
3.3
10
30%
Compu-Scan #1
1.7
14.9
10
132%
Compu-Scan #2
2.6
23.5
10
210%
Reference
<0.5
11.50
10
115%
Taunton River water LFM
Color Chart
<0.6
5
10
50%
Quick™ Arsenic Scan # 1
<0.3
3.8
10
38%
Quick™ Arsenic Scan #2
<0.3
3.8
10
38%
Compu-Scan #1
1.4
3.2
10
18%
Compu-Scan #2
1.45
5
10
36%
Reference
1.31
12.26
10
109%
w Average of four replicates. Non-detects were assigned a value of zero.
23
-------
Table 6-3b. LFM Sample Results for the Non-Technical Operator
Amount
Unspiked(a)
Spiked
spiked
Percent
Description
(ppb)
(ppb)
(ppb)
Recovery
Battelle drinking water LFM
Color Chart
<0.6
7
10
70%
Quick™ Arsenic Scan # 1
<0.3
7.9
10
79%
Compu-Scan #1
1.1
7.8
10
67%
Reference
<0.5
11.96
10
120%
Ayer untreated water LFM
Color Chart
32.5
30
10
-25%
Quick™ Arsenic Scan # 1
27.8
27
10
-8%
Compu-Scan #1
42.3
23
10
-193%
Reference
64.82
69.74
10
49%
Ayer treated water LFM
Color Chart
<0.6
9
10
90%
Quick™ Arsenic Scan # 1
<0.3
7.9
10
79%
Compu-Scan #1
0.75
8.6
10
79%
Reference
1.39
13.02
10
116%
Falmouth Pond water LFM
Color Chart
<0.6
3
10
30%
Quick™ Arsenic Scan # 1
0.2
2.8
10
26%
Compu-Scan #1
1.4
15.8
10
144%
Reference
<0.5
11.50
10
115%
Taunton River water LFM
Color Chart
<0.6
10
10
100%
Quick™ Arsenic Scan # 1
<0.3
8.4
10
84%
Compu-Scan #1
2.2
12.8
10
106%
Reference
1.31
12.26
10
109%
w Average of four replicates. Non-detects were assigned a value of zero.
24
-------
Table 6-4. Quick™ Low Range II Test Kit and Reference Sample Results
Non-
Technical
Technical
Non-
Technical
Non-
Technical
Operator
Operator
Technical
Technical
Technical
Operator
Technical
Operator
Quick™
Quick™
Operator
Operator
Operator
Quick™
Operator
Color
Arsenic
Arsenic
Compu-
Compu-
Color
Arsenic
Compu-
Chart
Scan #1
Scan #2
Scan #1
Scan #2
Chart
Scan #1
Scan #1
Reference
Description
Sample ID
Replicate
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
PT -1 ppb As
CAA-25
1
0.6
0.4
0.3
0.3
0.3
1
1
0.9
0.91
CAA-25
2
0.6
0.3
<0.3
0.5
0.9
1
0.4
0.6
0.86
CAA-25
3
1
0.6
0.3
0.4
0.6
1
0.3
0.9
0.90
CAA-25
4
1
1
0.8
0.9
1.3
1
0.3
0.7
0.86
PT - 3 ppb As
CAA-27
1
1
0.4
0.4
1
0.9
4
3.1
3.4
2.56
CAA-27
2
2.5
2
1.6
1.4
3.6
4
2.9
2.7
2.64
CAA-27
3
1
0.4
0.4
0.5
0.8
4
2.5
2
2.50
CAA-27
4
1
0.8
0.8
0.5
0.8
4
2.8
1.6
2.71
PT -10 ppb As
CAA-1
1
9
6.6
5.8
7.2
10
5
7.6
7.8
9.09
CAA-1
2
9
7.6
6.9
7.7
10.9
5
10.4
9.6
8.95
CAA-1
3
5
5
4.7
3.9
8
5
2.7
4.6
8.83
CAA-1
4
10
6.9
6.6
11.2
15.7
5
7.2
5.6
8.99
PT - 30 ppb As
CAA-29
1
30 w
13.5 w
13.5 w
17 w
25 w
60 ie>
38 w
18 e
33.96
CAA-29
2
25 (c)
13.5 (c)
12 (c)
9.5 (c)
19 (o)
60 (e)
40 (e)
26 e
34.39
CAA-29
3
25 (c)
10(c)
10.5 (c)
7 (c)
9.5 (c)
60 (e)
30 (e)
32 e
34.51
CAA-29
4
25 (c)
12 (c)
11.5 (0)
7.5
19.5 (c)
60 (e)
32 (e)
36 e
34.98
PT - 100 ppb As
CAA-31
1
80(e)
38 Ce)
38 {e)
24 w
30Ce)
80Ce)
aT
O
OO
56 e
111.89
CAA-31
2
80 (e)
48 (e)
48 (e)
32 (e)
70 (e)
80 (e)
84 (e)
102 e
115.57
CAA-31
3
100 (e)
42 (e)
42 (e)
60 (e)
36
80 (e)
70 (e)
44 e
114.65
CAA-31
4
100 (e)
66^
58 (e)
38 (e)
54(e)
80 (e)
90 (e)
104 e
113.83
Detection limit
CAA-23
1
2
0.8
0.8
1.2
1.5
1.7
1.2
1.4
CAA-23
2
1.7
0.6
1
1.2
1.3
3
1.5
4.1
CAA-23
3
1.7
<0.3
0.3
1.2
2.4
3
2.4
4
CAA-23
4
1.3
0.3
0.6
1.1
1.4
3
1.9
2.8
3.03
CAA-23
5
1.3
0.4
0.4
1.1
1.6
3
1.6
4.7
CAA-23
6
1
0.8
0.3
0.8
1.2
3
1.9
3.3
CAA-23
7
1
<0.3
<0.3
1
0.8
3
0.3
1.7
-------
Table 6-4. Quick™ Low Range II Test Kit and Reference Sample Results (continued)
Non-
Technical
Technical
Non-
Technical
Non-
Technical
Operator
Operator
Technical
Technical
Technical
Operator
Technical
Operator
Quick™
Quick™
Operator
Operator
Operator
Quick™
Operator
Color
Arsenic
Arsenic
Compu-
Compu-
Color
Arsenic
Compu-
Chart
Scan #1
Scan #2
Scan #1
Scan #2
Chart
Scan #1
Scan #1
Reference
Description
Sample ID
Replicate
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
PT - 10ppbAs +
CAA-33
1
5 la)
3.8 w
3.8 w
3.2 w
9
6.4 {b>
6(W
8(a»
8.2 w
10.2 a
high level
CAA-35
2
6.8 <">
3.2 ^
4.8 ^
10.8 ^
12.8 (b)
8 (a)
4.2 (a)
14.2 a
11.59
interferents
CAA-35
3
6.8 (b)
4(b)
6.4 <">
6.4 <">
8 (h)
8 (a)
8 (a)
42 a
CAA-35
4
6.8 (b)
4(b)
4.4 (b)
7.6 ^
7.6 ^
8 (a)
5.6 W
17.6 a
Battelle drinking
CAA-37
1
<0.6
<0.3
<0.3
0.7
0.6
<0.6
<0.3
0.7
<0.5
water
CAA-37
2
<0.6
<0.3
<0.3
0.7
1
<0.6
<0.3
0.7
<0.5
CAA-37
3
<0.6
<0.3
<0.3
0.7
0.7
<0.6
<0.3
1.4
<0.5
CAA-37
4
<0.6
<0.3
<0.3
0.6
0.6
<0.6
<0.3
1.5
<0.5
Battelle drinking
water LFM
CAA-38
1
5
4
3.6
5
4.6
7
7.9
7.8
11.96
Ayer untreated
CAA-39
1
5
5
5
7.9
8.8
30 d
25 d
70 d
65.61
water
CAA-39
2
5
4.1
4.2
5.3
5.9
30 d
31 d
30 d
62.73
CAA-39
3
5 (c)
<0.3 (c)
<0.3 (c)
3.5
4 0
30 d
20 d
21 d
67.47
CAA-39
4
5
5
5.5
9.8
10
40 d
35 d
48 d
63.48
Ayer untreated
water LFM
CAA-40
1
6.5 (c)
1.5 (c)
3 0
5 0
6.5 (c)
30 d
27 d
23 d
69.74
Ayer treated
CAA-41
1
<0.6
<0.3
<0.3
0.2
0.7
<0.6
<0.3
0.7
1.36
water
CAA-41
2
<0.6
<0.3
<0.3
0.7
0.7
<0.6
<0.3
0.9
1.45
CAA-41
3
<0.6
<0.3
<0.3
1
0.5
<0.6
<0.3
0.6
1.44
CAA-41
4
<0.6
<0.3
<0.3
0.7
1.5
<0.6
<0.3
0.8
1.32
Ayer treated
water LFM
CAA-42
1
6 (a)
3.6 (a)
3.2 (a)
6.6 (a)
6.6 (a)
9
7.9
8.6
13.02
Falmouth Pond
CAA-43
1
<0.6
0.3
0.4
1.9
2.5
<0.6
0.4
2.1
<0.5
water
CAA-43
2
<0.6
<0.3
<0.3
1.8
2.4
<0.6
<0.3
0.9
<0.5
CAA-43
3
<0.6
0.3
0.4
2.1
2.4
<0.6
0.3
2.1
<0.5
CAA-43
4
<0.6
<0.3
0.4
1.1
2.9
<0.6
<0.3
0.6
<0.5
Falmouth Pond
water LFM
CAA-46
1
4
3.3
3.3
14.9
23.5
3
2.8
15.8
11.50
-------
Table 6-4. Quick™ Low Range II Test Kit and Reference Sample Results (continued)
Description
Sample ID
Replicate
Technical
Operator
Color
Chart
(PPb)
Technical
Operator
Quick™
Arsenic
Scan #1
(PPb)
Technical
Operator
Quick™
Arsenic
Scan #2
(PPb)
Technical
Operator
Compu-
Scan #1
(PPb)
Technical
Operator
Compu-
Scan #2
(PPb)
Non-
Technical
Operator
Color
Chart
(PPb)
Non-
Technical
Operator
Quick™
Arsenic
Scan #1
(PPb)
Non-
Technical
Operator
Compu-
Scan #1
(PPb)
Reference
(PPb)
Taunton River
CAA-47
1
<0.6
<0.3
<0.3
1.4
1.3
<0.6
<0.3
1.9
1.36
water
CAA-47
2
<0.6
<0.3
<0.3
1.5
1.8
<0.6
<0.3
2.2
1.31
CAA-47
3
<0.6
<0.3
<0.3
1.7
1.7
<0.6
<0.3
2
1.31
CAA-47
4
<0.6
<0.3
<0.3
1.1
1
<0.6
0.6
2.6
1.26
Taunton River
water LFM
CAA-48
1
5 W
3.8 (a)
3.8 (a)
3.2 (a)
5 W
10 w
8.4 (a)
12.8 (a)
12.26
w = 1:2 dilution; {b> = 1:4 dilution; {0) = 1:5 dilution; w =1:10 dilution; {e> = 1:20 dilution.
to
-------
Table 6-5. Quantitative Evaluation of Accuracy for Quick™ Low Range II Test Kits
Percent Bias
Non-
Technical
Technical
Non-
Technical
Non-
Technical
Operator
Operator
Technical
Technical
Technical
Operator
Technical
Operator
Quick™
Quick™
Operator
Operator
Operator
Quick™
Operator
Description
Color
Arsenic
Arsenic
Compu-
Compu-
Color
Arsenic
Compu-
Chart
Scan #1
Scan #2
Scan #1
Scan #2
Chart
Scan #1
Scan #1
Performance Test Samples
1 ppb As
-9%
NA
NA
-41%
-12%
13%
-43%
-12%
3 ppb As
-47%
-65%
-69%
-67%
-41%
54%
9%
-7%
10 ppb As
-8%
-27%
-33%
-16%
24%
-44%
-22%
-23%
30 ppb As
-24%
-64%
-66%
-70%
-47%
74%
2%
-19%
100 ppb As
-21%
-57%
-59%
-66%
-58%
-30%
-29%
-33%
10 ppb As + low level interferents
-42%
-60%
-57%
-58%
-35%
-19%
-33%
-27%
10 ppb As + high level interferents
-39%
-67%
-59%
-33%
-26%
-31%
-44%
81%
Environmental Samples
Battelle drinking water
NA
NA
NA
NA
NA
NA
NA
NA
Battelle drinking water LFM
-58%
-67%
-70%
-58%
-62%
-41%
-34%
-35%
Ayer untreated water
-92%
NA
NA
-90%
-89%
-50%
-57%
-35%
Ayer untreated water LFM
-91%
-98%
-96%
-93%
-91%
-57%
-61%
-67%
Ayer treated water
NA
NA
NA
-53%
-39%
NA
NA
-46%
Ayer treated water LFM
-54%
-72%
-75%
-49%
-49%
-31%
-39%
-34%
Falmouth Pond water
NA
NA
NA
NA
NA
NA
NA
NA
Falmouth Pond water LFM
-65%
-71%
-71%
30%
104%
-74%
-76%
37%
Taunton River water
NA
NA
NA
9%
11%
NA
NA
66%
Taunton River water LFM
-59%
-69%
-69%
-74%
-59%
-18%
-32%
4%
Percent bias calculated according to Equation 4, Section 5.1.
NA: one or more replicates below detection limit
-------
Table 6-6 presents accuracy results for each PT and environmental replicate sample according to
whether the color chart result agreed with the reference value for that sample. Each color block
on the color chart represents a range of concentrations. The reference sample result was assigned
to the correct corresponding color block. A test kit result was considered to be in agreement with
the reference method result if it fell within the range of plus or minus one color block (i.e., the
concentration range spanning three adjacent color blocks). If the color chart test result for a given
sample was within this range, then a "Y" was reported in Table 6-6. If the color chart result was
outside this range, then an "N" was reported. Overall agreement was determined by calculating
the total percent of results in agreement for the technical and non-technical operators. The total
percent agreement using this method was 45% for the technical operator and 38% for the non-
technical operator.
6.2.2 Precision
Precision results for the Quick™ Low Range II test kit are presented in Table 6-7. The RSD was
determined according to Equation 7 (Section 5.2). The RSD was not calculated if any of the
results for a set of replicates were below the detection limit (i.e., <0.6 ppb for the color chart or
<0.3 ppb for the Quick™ Arsenic Scan). For the technical operator, RSDs ranged from 0% to
55% for the color chart, from 0% to 84% for the Quick™ Arsenic Scan, and from 7% to 91% for
the Compu-Scan. For the non-technical operator, RSDs ranged from 0% to 15% for the color
chart, from 9% to 67% for the Quick™ Arsenic Scan, and from 14% to 55% for the Compu-
Scan. For the reference measurements, RSDs were a maximum of 4%.
6.2.3 Linearity
The linearity of the Quick™ Low Range II test kit results was assessed by performing a linear
regression of the test kit results against the reference method results for the five PT samples
ranging from 1 ppb to 100 ppb arsenic. In these regressions, results reported as below detection
limit by the test kit were not used. Figures 6-1 through 6-3 present the results of the linear
regressions for the color chart, Quick™ Arsenic Scan, and Compu-Scan results, respectively.
The slope, intercept and correlation coefficient for each regression equation are shown on the
charts. For the color chart, the results for the technical operator were more linear than the results
for the non-technical operator and showed a closer correspondence to the reference method
results. For the Quick™ Arsenic Scan and Compu-Scan, the results for the non-technical
operator were more linear and showed a closer correspondence to the reference method results.
29
-------
Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range II Test Kits
Within
Within
Range (Y/N)
Technical
Range (Y/N)
Non-
Technical
Description
Sample ID
Replicate
Operator
Color Chart
Operator
Color Chart
Performance Test Samples
1 ppb As
CAA-25
1
Y
Y
CAA-25
2
Y
Y
CAA-25
3
Y
Y
CAA-25
4
Y
Y
3 ppb As
CAA-27
1
N
N
CAA-27
2
Y
N
CAA-27
3
N
N
CAA-27
4
N
N
10 ppb As
CAA-1
1
Y
N
CAA-1
2
Y
N
CAA-1
3
N
N
CAA-1
4
Y
N
30 ppb As
CAA-29
1
Y
N
CAA-29
2
Y
N
CAA-29
3
Y
N
CAA-29
4
Y
N
100 ppb As
CAA-31
1
Y
Y
CAA-31
2
Y
Y
CAA-31
3
Y
Y
CAA-31
4
Y
Y
10 ppb As +
CAA-33
1
N
N
low level
CAA-33
2
N
N
interferents
CAA-33
3
N
Y
CAA-33
4
N
Y
10 ppb As +
CAA-35
1
N
N
high level
CAA-35
2
N
N
interferents
CAA-35
3
N
N
CAA-35
4
N
N
Environmental Samples
Battelle drinking water
CAA-37
1
Y
Y
CAA-37
2
Y
Y
CAA-37
3
Y
Y
CAA-37
4
Y
Y
Battelle drinking water LFM
CAA-38
1
N
N
Ayer untreated water
CAA-39
1
N
N
CAA-39
2
N
N
CAA-39
3
N
N
CAA-39
4
N
Y
Ayer untreated water LFM
CAA-40
1
N
N
30
-------
Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range II Test Kits
(continued)
Within
Within
Range (Y/N)
Range (Y/N)
Non-
Technical
Technical
Operator
Operator
Description
Sample ID
Replicate
Color Chart
Color Chart
Ayer treated water
CAA-41
1
N
N
CAA-41
2
N
N
CAA-41
3
N
N
CAA-41
4
N
N
Ayer treated water LFM
CAA-42
1
N
N
Falmouth Pond water
CAA-43
1
Y
Y
CAA-43
2
Y
Y
CAA-43
3
Y
Y
CAA-43
4
Y
Y
Falmouth Pond water LFM
CAA-46
1
N
N
Taunton River water
CAA-47
1
N
N
CAA-47
2
N
N
CAA-47
3
N
N
CAA-47
4
N
N
Taunton River water LFM
CAA-48
1
N
Y
Percent Agreement
45%
38%
31
-------
Table 6-7. Precision Results for Quick™ Low Range II Test Kits
RSD
Non-
Technical
Technical
Non-
Technical
Non-
Technical
Operator
Operator
Technical
Technical
Technical
Operator
Technical
Operator
Quick™
Quick™
Operator
Operator
Operator
Quick™
Operator
Description
Color
Arsenic
Arsenic
Compu-
Compu-
Color
Arsenic
Compu-
Reference
Chart
Scan #1
Scan #2
Scan #1
Scan #2
Chart
Scan #1
Scan #1
Method
Performance Test Samples
1 ppb As
29%
54%
62%
50%
55%
0%
67%
19%
3%
3 ppb As
55%
84%
71%
51%
91%
0%
9%
33%
4%
10 ppb As
27%
17%
16%
40%
29%
0%
46%
32%
1%
30 ppb As
10%
14%
11%
45%
35%
0%
14%
28%
1%
100 ppb As
13%
26%
19%
40%
38%
0%
10%
41%
1%
Environmental Samples
Battelle drinking water
NA
NA
NA
7%
26%
NA
NA
40%
NA
Ayer untreated water
0%
11%
13%
42%
38%
15%
24%
51%
3%
Ayer treated water
NA
NA
NA
51%
52%
NA
NA
17%
4%
Falmouth Pond water
NA
0%
0%
25%
9%
NA
20%
55%
NA
Taunton River water
NA
NA
NA
18%
25%
NA
NA
14%
3%
NA: one or more replicates below detection limit.
-------
120
1:1 line
100
y = 0.71x + 7.29
R = 0.9038
40
y = 0.79x - 0.03
R = 0.9904
20
0
20
40
60
80
100
120
140
Reference concentration (ppb)
~ Technical Operator
¦ Non-technical Operator
¦ " Linear (Technical Operator)
" "Linear (Non-technical Operator)
Figure 6-1. Linearity of Quick™ Low Range II Color Chart Results
Reference concentration (ppb)
~ Technical Operator Unit #1
a Technical Operator Unit #2
¦ Non-technical Operator Unit #1
" " " Linear (Technical Operator Unit #1)
Linear (Technical Operator Unit #2)
— —Linear (Non-technical Operator Unit #1)
Figure 6-2. Linearity of Quick™ Low Range II Quick™ Arsenic Scan Results
33
-------
Reference concentration (ppb)
~ Technical Operator Unit #1
A Technical Operator Unit #2
¦ Non-technical Operator Unit #1
¦ ¦ ¦ Linear (Technical Operator Unit #1)
Linear (Technical Operator Unit #2)
~ Linear (Non-technical Operator Unit #1)
Figure 6-3. Linearity of Quick™ Low Range II Compu-Scan Results
6.2.4 Method Detection Limit
The MDLs for the color chart, Quick™ Arsenic Scan and Compu-Scan were assessed by
analyzing seven replicates of a sample spiked at approximately five times the lowest
concentration shown on the Quick™ Low Range II test kit color chart (i.e., 0.6 ppb X 5 = 3 ppb
arsenic). Table 6-8 provides the standard deviations for the seven replicate samples for the color
chart, Quick ™ Arsenic Scan, and Compu-Scan results, and the calculated MDLs.
6.2.5 Matrix Interference Effects
Matrix interference effects were assessed by comparing the calculated bias for the samples
containing low-level and high-level concentrations of interfering substances with the bias
reported for the other PT and environmental samples (Table 6-5). The biases for the samples
with low and high levels of interfering compounds were similar to those for the PT samples that
contained arsenic only, indicating no apparent effect due to the presence of interfering
substances.
34
-------
Table 6-8. Detection Limit Results for Quick™ Low Range II Test Kit
Non-
Technical
Technical
Non-
Technical
Non-
Technical
Operator
Operator
Technical
Technical
Technical
Operator
Technical
Operator
Quick™
Quick™
Operator
Operator
Operator
Quick™
Operator
Color
Arsenic
Arsenic
Compu-
Compu-
Color
Arsenic
Compu-
Chart
Scan #1
Scan #2
Scan #1
Scan #2
Chart
Scan #1
Scan #1
Sample ID
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
(PPb)
CAA-23 Rep 1
2
0.8
0.8
1.2
1.5
1.7
1.2
1.4
CAA-23 Rep 2
1.7
0.6
1
1.2
1.3
3
1.5
4.1
CAA-23 Rep 3
1.7
<0.3
0.3
1.2
2.4
3
2.4
4
CAA-23 Rep 4
1.3
0.3
0.6
1.1
1.4
3
1.9
2.8
CAA-23 Rep 5
1.3
0.4
0.4
1.1
1.6
3
1.6
4.7
CAA-23 Rep 6
1
0.8
0.3
0.8
1.2
3
1.9
3.3
CAA-23 Rep 7
1
<0.3
<0.3
1
0.8
3
0.3
1.7
Standard Deviation
0.38
0.23
0.29
0.15
0.49
0.49
0.67
1.25
Method Detection
Limit (ppb)
1.2
0.7
0.9
0.5
1.5
1.5
2.1
3.9
6.2.6 Operator Bias
Operator bias was evaluated by comparing the color chart, Quick™ Arsenic Scan Unit #1, and
Compu-Scan Unit #1 results above the detection limit for all PT and environmental samples
generated by the technical and non-technical operators (the non-technical operator did not use
the Quick™ Arsenic Scan Unit #2 or Compu-Scan Unit #2). Linear regression results for the
three sets of data are shown in Figure 6-4. The plots indicate that the results for the non-technical
operator tended to be higher than those produced by the technical operator. A paired t-test for
each set of data indicated that the results for the technical and non-technical operators were
significantly different at a 5% significance level.
6.2.7 Inter- Unit Reproducibility
Inter-unit reproducibility was evaluated by comparing the data for the two Quick™ Arsenic Scan
units and two Compu-Scan units used by the technical operator. Detected results for PT and
environmental samples were included in the analysis. Linear regressions of the two sets of data
for each instrument are shown in Figure 6-5. The regressions indicate that the Quick™ Arsenic
Scan units provided more reproducible results than the Compu-Scan units. Paired t-tests of the
two sets of data indicated that the Quick™ Arsenic Scan results were not significantly different
at a 5% significance level. The Compu-Scan results were significantly different at a 5%
significance level.
35
-------
Technical Operator
~
Color Chart
a
Quick Arsenic Scan #1
Compu-Scan #1
- -
¦ Linear (Color Chart)
"Linear (Quick Arsenic Scan #1)
—
— Linear (Compu-Scan #1)
Figure 6-4. Comparison of Quick™ Low Range II Test Results for Technical and
Non-Technical Operators
80
1:1 line
70
R = 0.8574
60
50
40
y = 0.94x + 0.35
R = 0.9964
30
20
10
0
0
10
20
30
40
50
60
70
Unit #1
~ Quick Arsenic Scan
¦ Compu-Scan
¦ ¦ ¦ Linear (Quick Arsenic Scan)
— ~ Linear (Compu-Scan)
Figure 6-5. Comparison of Quick™ Low Range II Test Results for Two Arsenic Quick™
Scan and Compu-Scan Units
36
-------
6.2.8 Rate of False Positives/False Negatives
Tables 6-9 and 6-10 show the data and results for the rates of false positives and false negatives,
respectively, obtained from the Quick™ Low Range II test kit. All PT and environmental
samples were included in this evaluation.
As shown in Table 6-9, 32 samples had an arsenic concentration at or below 10 ppb as measured
by the reference analysis. For these samples, none of the color chart or Quick™ Arsenic Scan
results were >10 ppb and greater than 125% of the reference measurement, yielding false
positive rates of 0% for both the technical and non-technical operators. The rates of false
positives for the Compu-Scan units were 0% and 3% for the technical operator (Units #1 and #2)
and 0% for the non-technical operator (Unit #1).
Twenty-one samples had arsenic concentrations above 10 ppb as measured by the reference
analysis (Table 6-10). For these samples, color chart results were <10 ppb and less than 75% of
the reference measurement for thirteen samples for the technical operator and seven samples for
the non-technical operator, yielding false negative rates of 62% and 33%, respectively. The rates
of false negatives for the Quick™ Arsenic Scan units were 62% for the technical operator (Units
#1 and #2) and 38% for the non-technical operator (Unit #1). The rates of false negatives for the
Compu-Scan units were 67% and 52% for the technical operator (Units #1 and #2, respectively)
and 9.5%) for the non-technical operator (Unit #1).
6.3 Other Factors
During testing activities, the technical and non-technical operators were instructed to keep a
record of their comments on ease of use, reliability, portability, and generation of waste
materials. This section summarizes these observations and other comments pertaining to any
problems encountered during testing. Cost information is also presented.
6.3.1 Ease of Use
The technical and non-technical operators both reported that the Quick™ Low Range II test kit
was very easy to use. The test kit instructions were clear and easy to follow. Although the
manufacturer provided instructions for diluting samples above the 8 ppb arsenic level, the non-
technical operator sometimes had difficulty successfully performing dilutions and correctly
converting the results to a final concentration. Dilution of samples with arsenic concentrations
exceeding the optimal detection range may be a source of error and reduce the accuracy and
precision of the associated results because of the difficulty in performing accurate dilution in a
field setting. The three scoops used to sequentially add reagents were color coordinated, which
facilitated the efficient operation of the test kit. The sample bottles were of moderate size and
were relatively easy to handle, and reagents could easily be added without spillage. Extra care
had to be taken to ensure that the caps to the reaction vessels were completely dry before
proceeding with further analyses. The test kit materials were readily transported to the Battelle
storage shed where environmental samples were tested.
37
-------
Table 6-9. Rate of False Positives for Quick™ Low Range II Test Kits
False Positive (Y/N)
Description
Sample
ID
Replicate
Technical
Operator
Color
Chart
Technical
Operator
Quick™
Arsenic
Scan #1
Technical
Operator
Quick™
Arsenic
Scan #2
Technical
Operator
Compu-
Scan #1
Technical
Operator
Compu-
Scan #2
Non-
Technical
Operator
Color Chart
Non-
Technical
Operator
Quick™
Arsenic
Scan #1
Non-
Technical
Operator
Compu-
Scan #1
1 ppb As
CAA-25
1
N
N
N
N
N
N
N
N
CAA-25
2
N
N
N
N
N
N
N
N
CAA-25
3
N
N
N
N
N
N
N
N
CAA-25
4
N
N
N
N
N
N
N
N
3 ppb As
CAA-27
1
N
N
N
N
N
N
N
N
CAA-27
2
N
N
N
N
N
N
N
N
CAA-27
3
N
N
N
N
N
N
N
N
CAA-27
4
N
N
N
N
N
N
N
N
10 ppb As
CAA-1
1
N
N
N
N
N
N
N
N
CAA-1
2
N
N
N
N
N
N
N
N
CAA-1
3
N
N
N
N
N
N
N
N
CAA-1
4
N
N
N
N
Y
N
N
N
Battelle drinking
CAA-37
1
N
N
N
N
N
N
N
N
water
CAA-37
2
N
N
N
N
N
N
N
N
CAA-37
3
N
N
N
N
N
N
N
N
CAA-37
4
N
N
N
N
N
N
N
N
Ayer treated water
CAA-41
1
N
N
N
N
N
N
N
N
CAA-41
2
N
N
N
N
N
N
N
N
CAA-41
3
N
N
N
N
N
N
N
N
CAA-41
4
N
N
N
N
N
N
N
N
Falmouth Pond
CAA-43
1
N
N
N
N
N
N
N
N
water
CAA-43
2
N
N
N
N
N
N
N
N
CAA-43
3
N
N
N
N
N
N
N
N
CAA-43
4
N
N
N
N
N
N
N
N
-------
Table 6-9. Rate of False Positives for Quick™ Low Range II Test Kits (continued)
False Positive (Y/N)
Non-
Technical
Technical
Technical
Non-
Technical
Operator
Operator
Technical
Technical
Non-
Operator
Technical
Sample
Operator
Quick™
Quick™
Operator
Operator
Technical
Quick™
Operator
Description
Replicate
Color
Arsenic
Arsenic
Compu-
Compu-
Operator
Arsenic
Compu-
ID
Chart
Scan #1
Scan #2
Scan #1
Scan #2
Color Chart
Scan #1
Scan #1
Taunton River
CAA-47
1
N
N
N
N
N
N
N
N
water
CAA-47
2
N
N
N
N
N
N
N
N
CAA-47
3
N
N
N
N
N
N
N
N
CAA-47
4
N
N
N
N
N
N
N
N
10ppbAs +
CAA-33
1
N
N
N
N
N
N
N
N
low level
CAA-33
2
N
N
N
N
N
N
N
N
interferents
CAA-33
3
N
N
N
N
N
N
N
N
CAA-33
4
N
N
N
N
N
N
N
N
Total number of samples
32
32
32
32
32
32
32
32
Total number of false positives
0
0
0
0
1
0
0
0
Percent false positives
0%
0%
0%
0%
3%
0%
0%
0%
-------
Table 6-10. Rate of False Negatives for Quick™ Low Range II Test Kits
False Negative (Y/N)
Description
Sample ID
Replicate
Technical
Operator
Color
Chart
Technical
Operator
Quick™
Arsenic
Scan #1
Technical
Operator
Quick™
Arsenic Scan
#2
Technical
Operator
Compu-
Scan #1
Technical
Operator
Compu-
Scan #2
Non-
Technical
Operator
Color
Chart
Non-
Technical
Operator
Quick™
Arsenic Scan
#1
Non-
Technical
Operator
Compu-
Scan #1
PT - 30 ppb As
CAA-29
1
N
N
N
N
N
N
N
N
CAA-29
2
N
N
N
Y
N
N
N
N
CAA-29
3
N
N
N
Y
Y
N
N
N
CAA-29
4
N
N
N
Y
N
N
N
N
PT - 100 ppb As
CAA-31
1
N
N
N
N
N
N
N
N
CAA-31
2
N
N
N
N
N
N
N
N
CAA-31
3
N
N
N
N
N
N
N
N
CAA-31
4
N
N
N
N
N
N
N
N
Battelle drinking water
LFM
CAA-38
1
Y
Y
Y
Y
Y
Y
Y
Y
Ayer untreated water
CAA-39
1
Y
Y
Y
Y
Y
N
N
N
CAA-39
2
Y
Y
Y
Y
Y
N
N
N
CAA-39
3
Y
Y
Y
Y
Y
N
N
N
CAA-39
4
Y
Y
Y
Y
N
N
N
N
Ayer untreated water
LFM
CAA-40
1
Y
Y
Y
Y
Y
N
N
N
Ayer treated water LFM
CAA-42
1
Y
Y
Y
Y
Y
Y
Y
Y
Falmouth Pond water
LFM
CAA-46
1
Y
Y
Y
N
N
Y
Y
N
Taunton River water
LFM
CAA-48
1
Y
Y
Y
Y
Y
N
Y
N
10 ppb As+
high level
Interferents
CAA-35
CAA-35
CAA-35
1
2
3
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
N
Y
Y
N
Y
Y
Y
Y
Y
Y
Y
N
N
N
CAA-35
4
Y
Y
Y
Y
Y
Y
Y
N
Total number of samples
21
21
21
21
21
21
21
21
Total number of false negatives
13
13
13
14
11
7
8
2
Percent false negatives
62%
62%
62%
67%
52%
33%
38%
9.5%
-------
6.3.2 Analysis Time
The average total analysis time for a sample was about fifteen minutes at a sample temperature
of 24°C. The manufacturer provided a modified protocol that specified increased reaction times
for samples below 24°C. The test kit enabled two samples to be run concurrently without any
confusion.
6.3.3 Reliability
The Quick™ Low Range II test kits operated reliably throughout the period of the test.
6.3.4 Waste Material
The waste generated by the Quick™ Low Range II test kit was manageable. The vendor's
instructions provide a warning that hydrogen and arsine are generated during the test and
recommend that testing be conducted in a well-ventilated area away from open flames and other
sources. MSDSs should be reviewed before handling any chemicals. Instructions for the disposal
of residual materials were clear and complete. The residual liquid in the reaction vessel was
allowed to settle before disposal in order to let particulates accumulate on the bottom. A dilute
hydrochloric acid solution was used to clean the reaction vessel prior to subsequent analyses.
Disposal of this waste in an appropriate manner must be taken into consideration.
6.3.5 Cost
The listed price for a Quick™ Low Range II test kit for analysis of 50 samples is $349.99.
Replacement reagents and supplies are not available; kits are provided as a complete set because
reagents, test strips, and color charts are made to perform optimally with each other. The
Quick™ Arsenic Scan and Compu-Scan are available as options for an additional cost of
$1,599.99 each.
41
-------
Chapter 7
Performance Summary
The Quick™ Low Range II test kit was verified by evaluating the following parameters:
¦ Accuracy
¦ Precision
¦ Linearity
¦ MDL
¦ Matrix interference effects
¦ Operator bias
¦ Inter-unit reproducibility
¦ Rate of false positives/negatives.
The quantitative assessment of accuracy indicated that the relative bias for the color chart ranged
from -92% to -8% for the technical operator and -74% to 74% for the non-technical operator.
The relative bias for the Quick™ Arsenic Scan ranged from -98% to -27% for the technical
operator and -76 to 9% for the non-technical operator. The relative bias for the Compu-Scan
ranged from -93% to 104% for the technical operator and from -67% to 81% for the non-
technical operator. The overall agreement for the color chart results based on an assessment of
whether the result was assigned to the correct color block indicated that the total percent
agreement was 45% for the technical operator and 38% for the non-technical operator.
Precision was assessed by analyzing four replicates of each sample. For the technical operator,
RSDs ranged from 0% to 55% for the color chart, 0% to 84% for the Quick™ Arsenic Scan, and
7%> to 91% for the Compu-Scan. For the non-technical operator, RSDs ranged from 0% to 15%
for the color chart, from 9% to 67% for the Quick™ Arsenic Scan, and from 14% to 55% for the
Compu-Scan.
The linearity of response was evaluated by plotting the test kit results against the reference
analysis results for the PT samples. The equations for the linear regressions that were performed
to evaluate linearity are summarized in Table 7-1. The slope, y-intercept, and correlation
coefficient corresponding to a linear response that exactly matched reference concentrations
would be 1, 0, and 1, respectively.
The MDL was assessed by analyzing seven replicates of a sample spiked at a level approxi-
mately five times the manufacturer's estimated detection limit for the color chart (i.e., 0.6 ppb
X 5 = 3 ppb spiked sample). The MDLs calculated using the precision data from these replicates
ranged from 1.2 ppb to 1.5 ppb for the color charts, from 0.7 ppb to 2.1 ppb for the Quick™
Arsenic Scan, and from 0.5 ppb to 3.9 ppb for the Compu-Scan.
42
-------
Table 7-1. Summary of Linear Regression Equations for Test Kit and Reference Results
Correlation
Coefficient
Description
Slope
Intercept
(R)
Color chart, technical operator
0.79
-0.03
0.9904
Color chart, non-technical operator
0.71
7.29
0.9038
Quick™ Arsenic Scan #1, technical operator
0.42
0.29
0.9619
Quick™ Arsenic Scan #2, technical operator
0.40
0.19
0.9776
Quick™ Arsenic Scan #1, non-technical operator
0.71
2.45
0.9826
Compu-Scan #1, technical operator
0.33
1.03
0.9043
Compu-Scan #2, technical operator
0.40
3.05
0.9063
Compu-Scan #1, non-technical operator
0.67
1.39
0.9174
Results for samples containing low and high levels of interfering compounds indicated that low
and high levels of interferents did not appear to affect the detection of arsenic. Biases for these
samples were similar to those calculated for PT samples containing arsenic only.
An evaluation of Quick™ Low Range II test kit results for the technical and non-technical
operators suggested that measurements for the color chart, Quick™ Arsenic Scan, and Compu-
Scan done by the non-technical operator tended to be higher than for the technical operator. A
paired t-test of the two sets of data indicated that the results were significantly different at a 5%
significance level. The regression equations were as follows:
Color chart y = 0.88x + 7.54, R = 0.8646
Quick™ Arsenic Scan y = 1.59x + 3.40, R = 0.9281
Compu-Scan y = 1.62x + 4.33, R = 0.7275
where x is the technical operator andj' is the non-technical operator.
Inter-unit reproducibility was evaluated by comparing the data for the two Quick™ Arsenic Scan
units and two Compu-Scan units used by the technical operator. A linear regression of the two
sets of data indicated that the Quick™ Arsenic Scan results were more reproducible than the
Compu-Scan results. Paired t-tests of the two sets of data indicated that the Quick™ Arsenic
Scan results were not significantly different at a 5% significance level; however, the Compu-
Scan results were significantly different. The regression equations were as follows:
Quick™ Arsenic Scan y = 0.94x + 0.35, R = 0.9964
Compu-Scan y = 1.08x + 1.50, R = 0.8574
where x is Unit #1 and^ is Unit #2.
A false positive was defined as a test kit result that was greater than 10 ppb and greater than
125% of the reference concentration, when the reference concentration is less than or equal to
10 ppb. The rates of false positives for the technical and non-technical operators using the color
charts and Quick™ Arsenic Scan units were 0%. The rates of false positives for the Compu-Scan
43
-------
were 0% for both operators, except for Compu-Scan Unit #2, with a false positive rate of 3%. A
false negative was defined as a test kit result that was below or equal to 10 ppb and less than
75% of the reference concentration, when the reference concentration was greater than 10 ppb.
The false negative rates for the technical and non-technical operators using the color charts were
62% and 33%, respectively. The rates of false negatives for the Quick™ Arsenic Scan units were
62%) for the technical operator (Units #1 and #2) and 38% for the non-technical operator (Unit
#1). The rates of false negatives for the Compu-Scan were 67% and 52% for the technical
operator (Units #1 and #2, respectively) and 9.5% for the non-technical operator (Unit #1).
The Quick™ Low Range II test kits were easy to use and readily transportable to the field. The
sample bottles were of moderate size and were relatively easy to handle. The test kits were
reliable. Analysis of diluted samples was sometimes difficult for the non-technical operator.
Dilution of samples with arsenic concentrations exceeding the optimal detection range may be a
source of error and reduce the accuracy and precision of the associated results because of the
difficulty in performing accurate dilution in a field setting. The time to analyze one sample is
approximately 15 minutes at a temperature range of 24°C to 30°C; longer reaction times are
required for samples below this range. Two samples can be run concurrently without difficulty.
The cost for a 50-sample test kit with a color chart is listed as $349.99. Replacement reagents
and supplies are not available; kits are provided as a complete set because reagents, test strips,
and color charts are made to perform optimally with each other, according to the vendor. The
Quick™ Arsenic Scan and Compu-Scan are available as options for an additional cost of
$1,599.99 each.
44
-------
Chapter 8
References
Test/QA Plan for Verification of Portable Analyzers, Battelle, Columbus, Ohio, Version 2.
December 8, 2000.
U.S. EPA Method 200.8, Determination of Trace Elements in Waters and Wastes by
Inductively Coupled Plasma Mass Spectrometry, Revision 5.5, October 1999.
Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Pilot, Version
4, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio,
December, 2002.
U.S. Code of Federal Regulations, Title 40, Part 136, Appendix B.
45
------- |