August 2003
Environmental Technology
Verification Report
Industrial Test Systems, Inc.
Quick™ Low Range Test Kit
Prepared by
w
Baffelle
. . . Putting Technology To Work
Battel le
Under a cooperative agreement with
f/EPA U.S. Environmental Protection Agency
ETV EtY ElV
-------
August 2003
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Industrial Test Systems, Inc.
Quick™ Low Range
Test Kit
by
Tim Kaufman
Patty White
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency and recommended for public release.
Mention of trade names or commercial products does not constitute endorsement or
recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of seven environmental technology centers
and two pilot programs. Information about each of these centers can be found on the Internet at
http ://www. epa. gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. In 1997, through a competitive cooperative agreement, Battelle was awarded EPA
funding and support to plan, coordinate, and conduct such verification tests for "Advanced
Monitoring Systems for Air, Water, and Soil" and report the results to the community at large.
Information concerning this specific environmental technology area can be found on the Internet
at http://www.epa.gov/etv/centers/centerl .html.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. In particular we would like to thank
Rosanna Buhl, Adam Abbgy, and Bea Weaver of Battelle, and Mike Madigan and Rick Linde of
the Ayer, Massachusetts Department of Public Works Water Division. We also acknowledge the
assistance of Jeff Adams of EPA and AMS Center stakeholders Vito Minei and Marty Link, who
reviewed the verification reports.
iv
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of abbreviations viii
1 Background 1
2 Technology Description 2
3 Test Design and Procedures 4
3.1 Introduction 4
3.2 Test Design 4
3.3 Test Samples 5
3.3.1 QC Samples 5
3.3.2 PT Samples 7
3.3.3 Environmental Samples 7
3.4 Reference Analysis 8
3.5 Verification Schedule 8
4 Quality Assurance/Quality Control 9
4.1 Laboratory QC for Reference Method 9
4.2 Audits 11
4.2.1 Performance Evaluation Audit 11
4.2.2 Techni cal Sy stem s Audit 12
4.2.3 Data Quality Audit 12
4.3 QA/QC Reporting 13
4.4 Data Review 13
5 Statistical Methods 14
5.1 Accuracy 14
5.2 Precision 15
5.3 Linearity 15
5.4 Method Detection Limit 15
5.5 Matrix Interference Effects 15
5.6 Operator Bias 16
5.7 Inter-Unit Reproducibility 16
5.8 Rate of False Positives/False Negatives 16
v
-------
6 Test Results 17
6.1 QC Samples 17
6.2 PT and Environmental Samples 22
6.2.1 Accuracy 22
6.2.2 Precision 27
6.2.3 Linearity 27
6.2.4 Method Detection Limit 31
6.2.5 Matrix Interference Effects 32
6.2.6 Operator Bias 32
6.2.7 Inter-Unit Reproducibility 33
6.2.8 Rate of False Positives/False Negatives 33
6.3 Other F actors 36
6.3.1 Ease of Use 37
6.3.2 Analysis Time 37
6.3.3 Reliability 37
6.3.4 Waste Material 37
6.3.5 Cost 37
7 Performance Summary 38
8 References 41
Figures
Figure 2-1. Industrial Test Systems, Inc., Quick™ Low Range Test Kit 2
Figure 2-2. Quick™Low Range Color Chart 3
Figure 6-1. Linearity of Quick™ Low Range Color Chart Results 30
Figure 6-2. Linearity of Quick™ Low Range Quick™ Arsenic Scan Results 31
Figure 6-3. Comparison of Quick™ Low Range Test Results for Technical and
Non-Technical Operators 33
Figure 6-4. Comparison of Two Quick™ Arsenic Scan Units 34
Tables
Table 3-1. Test Samples for Verification of the Quick™ Low Range Test Kit 6
Table 3-2. Schedule of Verification Test Days 8
Table 4-1. Reference Method QCS Analysis Results 10
Table 4-2. Reference Method LFM Sample Results 11
vi
-------
Table 4-3. Reference Method Duplicate Analysis Results 12
Table 4-4. Reference Method PE Audit Results 12
Table 4-5. Summary of Data Recording Process 13
Table 6-la. RB Sample Results for the Technical Operator 18
Table 6-lb. RB Sample Results for the Non-Technical Operator 18
Table 6-2a. QCS Results for the Technical Operator 19
Table 6-2b. QCS Results for the Non-Technical Operator 20
Table 6-3a. LFM Sample Results for the Technical Operator 21
Table 6-3b. LFM Sample Results for the Non-Technical Operator 22
Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results 23
Table 6-5. Quantitative Evaluation of Accuracy for Quick™ Low Range Test Kits 26
Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range Test Kits 28
Table 6-7. Precision Results for Quick™ Low Range Test Kits 30
Table 6-8. Detection Limit Results for Quick™ Low Range Test Kit 32
Table 6-9. Rate of False Positives for Quick™ Low Range Test Kits 35
Table 6-10. Rate of False Negatives for Quick™ Low Range Test Kits 36
Table 7-1. Summary of Linear Regression Equations for Test Kit and Reference Results 39
vii
-------
List of Abbreviations
AMS
Advanced Monitoring Systems
EPA
U.S. Environmental Protection Agency
ETV
Environmental Technology Verification
HDPE
high-density polyethylene
ICPMS
inductively coupled plasma mass spectrometry
LFM
lab oratory-fortified matrix
MDL
method detection limit
MSDS
Material Safety Data Sheet
NIST
National Institute of Standards and Technology
ppb
parts per billion
ppm
parts per million
PE
performance evaluation
PT
performance test
QA
quality assurance
QA/QC
quality assurance/quality control
QC
quality control
QCS
quality control standard
QMP
Quality Management Plan
RB
reagent blank
RPD
relative percent difference
RSD
relative standard deviation
TSA
technical systems audit
viii
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders, con-
ducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
recently evaluated the performance of the Industrial Test Systems, Inc., Quick™ Low Range test
kit for measuring arsenic in water.
1
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of environ-
mental monitoring technologies for air, water, and soil. This verification report provides results
for the verification testing of the Quick™ Low Range test kit for arsenic in water (Figure 2-1).
The following is a description of the test kit, based on information provided by the vendor. The
information provided below was not verified in this test.
To perform arsenic analyses with the Quick™
Low Range test kit, the water sample to be
tested is mixed in the supplied reaction vessel
with reagent #1 (tartaric acid with rate
enhancers) to acidify the water sample. Reagent
#2, an oxidizer (potassium peroxymonosulfate),
is added to remove hydrogen sulfide inter-
ference. The test tolerates up to 2 parts per
million (ppm) hydrogen sulfide without
interference. Zinc powder, reagent #3, is added
to reduce inorganic arsenic compounds (As13
and As 5) to arsine gas. As arsine gas is
generated and comes in contact with the test
strip, the mercuric bromide indicator on the test
strip changes color from white to shades of
yellow or brown. Material Safety Data Sheets
(MSDSs) for all reagents and test strips are
provided with each test kit. The MSDSs include information on how to safely handle the
reagents and test strips, including instructions for exposure controls and personal protection.
Figure 2-1. Industrial Test Systems, Inc.,
Quick™ Low Range Test Kit
Once the reaction is completed, the test strip is removed and visually compared to a color chart
to obtain a semi-quantitative measure of the arsenic concentration in the tested sample.
The color chart consists of a series of color blocks (Figure 2-2). The color blocks correspond to
concentrations ranging from 3 parts per billion (ppb) to >80 ppb. If the color on the test strip is
between two color blocks, then the operator may estimate the concentration as between the
values associated with the color blocks on either side. The test strip may also be read with the
Quick™ Arsenic Scan hand-held instrument, which operates on the same principle as a
colorimeter and provides a quantitative result. The Quick™ Arsenic Scan is calibrated weekly
using a calibration card provided by the manufacturer. The Quick™ Arsenic Scan is not provided
2
-------
BDL iigi'L 3 ppb 6
a 10 12 15 19 24
kig.'L 30 ppb 40 SO 60 70 SO >S0 >80 >80
Quick™ Low Range Calibration Valid 24" to sDL»Bei™D«*-i
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Portable Analyzers^ The verification was based on comparing the arsenic
results from the Quick™ Low Range test kit to those from a laboratory-based reference method.
The reference method for arsenic analysis was inductively coupled plasma mass spectrometry
(ICPMS) performed according to EPA Method 200.8(2) Semi-quantitative results for arsenic
were obtained from the Quick™ Low Range test kit by comparisons of test strips to a color chart
provided with the test kit. Quantitative results also were obtained using a Quick™ Arsenic Scan
instrument. The test kit performance was verified by analyzing laboratory-prepared performance
test samples, treated and untreated drinking water, and fresh surface water. All samples were
tested using both the test kit and the reference method. Both semi-quantitative and quantitative
analyses were performed by the technical and non-technical operators. The test design and
procedures are described below.
3.2 Test Design
The Quick™ Low Range test kit was verified by evaluating the following parameters:
¦ Accuracy
¦ Precision
¦ Linearity
¦ Method detection limit (MDL)
¦ Matrix interference effects
¦ Operator bias
¦ Inter-unit reproducibility
¦ Rate of false positives/false negatives.
All sample preparation and analyses were performed according to the manufacturer's
recommended procedures. All samples were warmed to 24°C prior to analysis using a hot water
bath, which is at the lower end of the optimal temperature range listed in the test kit instructions.
Color chart and Quick™ Arsenic Scan results were recorded manually. The results from the
Quick™ Low Range test kits were compared to those from the reference method to assess
accuracy and linearity. Multiple aliquots of performance test samples, drinking water samples,
and surface water samples were analyzed to assess precision. Multiple aliquots of a low-level
performance test sample were analyzed to assess the detection limit of the method. Potential
4
-------
matrix interference effects were assessed by challenging the test kit with performance test
samples of known arsenic concentrations that contained both low levels and high levels of
interfering substances.
Identical sets of samples were analyzed independently by a technical and a non-technical
operator. The technical operator was a technician at Battelle with three years of field and
laboratory experience and a Bachelors degree. The non-technical operator was a part-time
temporary helper enrolled in undergraduate studies. Because the reagents of the Quick™ Low
Range test kits were consumed in use, it was not feasible for the two operators to use the same
kits; however, each operator used multiple kits in order to analyze all the samples and it is
assumed that kit-to-kit variability was similar for both operators. Results of all analyses were
statistically compared to evaluate operator bias. The technical operator analyzed all samples
using two different Quick™ Arsenic Scan units to assess inter-unit reproducibility.
The rate of false positive and false negative results were evaluated relative to the 10-ppb
maximum contaminant level for arsenic in drinking water.(4) Other factors that were quali-
tatively assessed during the test included time required for sample analysis, ease of use, and
reliability.
3.3 Test Samples
Three types of samples were analyzed in the verification test, as shown in Table 3-1: quality
control (QC) samples, performance test (PT) samples, and environmental water samples. The QC
and PT samples were prepared from National Institute of Standards and Technology (NIST)
traceable standards purchased from a commercial supplier and subject only to dilution as
appropriate. Under the Safe Drinking Water Act, the EPA lowered the maximum contaminant
level for arsenic from 50 ppb to 10 ppb in January 2001; public water supply systems must
comply with this standard by January 2006.(4) Therefore, the QC sample concentrations targeted
the 10 ppb arsenic level. The PT samples ranged from 10% to 1,000% of the 10 ppb level (i.e.,
from 1 ppb to 100 ppb). The environmental water samples were collected from various drinking
water and surface freshwater sources.
Each sample was assigned a unique sample identification number when prepared in the
laboratory or collected in the field. The PT and environmental samples were submitted blind to
the technical and non-technical operators and were analyzed randomly to the degree possible.
3.3.1 QC Samples
QC samples included laboratory reagent blank samples (RB), quality control samples (QCS), and
laboratory-fortified matrix (LFM) samples (Table 3-1). The RB samples consisted of the same
ASTM Type I water used to prepare all other samples and were subjected to the same handling
and analysis procedures as the other samples. The RB samples were used to verify that no
arsenic contamination was introduced during sample handling and analysis. RB samples were
analyzed at a frequency of 10%.
5
-------
Table 3-1. Test Samples for Verification of the Quick™ Low Range Test Kit
Type of
Sample
Sample Characteristics
Arsenic
Concentration (a)
No. of
Replicates
Quality Control
Reagent Blank (RB)
~ 0 ppb
10% of all
Quality Control Sample (QCS)
10 ppb
10% of all
Laboratory Fortified Matrix (LFM)
10 ppb above native
level
1 per site
Performance Test
Prepared arsenic solution
1 ppb
4
Prepared arsenic solution
3 ppb
4
Prepared arsenic solution
10 ppb
4
Prepared arsenic solution
30 ppb
4
Prepared arsenic solution
100 ppb
4
Prepared arsenic solution for detection limit
determination
15 ppb
7
Prepared arsenic solution spikedwith low levels
of interfering substances
10 ppb
4
Prepared arsenic solution spiked with high levels
of interfering substances
10 ppb
4
Environmental
Battelle drinking water
<0.5 ppb
4
Ayer untreated water
64.8 ppb
4
Ayer treated water
1.39 ppb
4
Falmouth Pond water
<0.5 ppb
4
Taunton River water
1.31 ppb
4
Performance Test sample concentrations are target levels; environmental sample concentrations are actual
(average of four replicate measurements).
The QCS consisted of ASTM Type I water spiked in the lab to a concentration of 10 ppb arsenic
with a NIST-traceable standard. QCS were used as calibration checks to verify that the Quick™
Low Range test kit was operating properly. QCS were analyzed at the beginning and end of each
testing period, as well as after every tenth sample. Because the test kit utilized a color chart that
could not be calibrated, no performance criteria were specified for the QCS.
The LFM samples consisted of aliquots of environmental samples that were spiked in the field to
increase the arsenic concentration by 10 ppb. The spike solution used for the LFM samples was
prepared in the laboratory and brought to the field site. One LFM sample was prepared from
each environmental sample.
6
-------
3.3.2 PT Samples
Three types of PT samples used in this verification test (Table 3-1): spiked samples ranging
from 1 ppb to 100 ppb arsenic, a low-level spiked sample for evaluation of the test kit's detection
limit, and matrix interference samples that were spiked with potential interfering substances. All
PT samples were prepared in the laboratory using ASTM Type I water and NIST-traceable
standards.
Five PT samples containing arsenic at concentrations from 1 ppb to 100 ppb were prepared to
evaluate Quick™ Low Range test kit accuracy and linearity. Four aliquots of each of these
samples were analyzed to assess precision.
To determine the detection limit of the Quick™ Low Range test kit, a sample with an arsenic
concentration five times the vendor's estimated detection limit was prepared. Seven non-
consecutive replicates of this 15 ppb arsenic sample were analyzed to provide precision data with
which to estimate the MDL.
The matrix interference samples were spiked with 10 ppb arsenic as well as potentially inter-
fering species commonly found in natural water samples. One sample contained low levels of
interfering substances that consisted of 1 ppm iron, 3 ppm sodium chloride, and 0.1 ppm sulfide.
The second sample contained high levels of interfering compounds at the following
concentrations: 10 ppm iron, 30 ppm sodium chloride, and 1.0 ppm sulfide. Four replicates of
each of these samples were analyzed.
3.3.3 Environmental Samples
The environmental samples listed in Table 3-1 included three drinking water samples and two
surface water samples. All environmental samples were collected in 20-L high density poly-
ethylene (HDPE) containers. The Battelle groundwater sample was collected directly from a tap
without purging. Untreated and treated groundwater samples from the Ayer, Massachusetts
Department of Public Works Water Treatment Plant were collected directly from spigots, also
without purging. Four aliquots of each sample were analyzed using the Quick™ Low Range test
kit in the Battelle laboratory as soon as possible after collection. One aliquot of each sample was
preserved with nitric acid and submitted to the reference laboratory for reference analysis.
One surface water sample was collected from a pond in Falmouth, Massachusetts and another
was collected from the Taunton River near Bridgewater, Massachusetts. These samples were
collected near the shoreline by submerging a 2-L HDPE sample container no more than one inch
below the surface of the water, and decanting the water into a 20-L HDPE carboy until full. Each
water body was sampled at one accessible location. These samples could not be analyzed at the
field location as planned because of persistent, severe winter weather conditions. Therefore, the
samples were returned to a storage shed at the Battelle laboratory, which was heated but was not
serviced by running water. The storage shed was intended to simulate realistic field conditions
under which the test kits might be used. Four aliquots of each surface water sample were
analyzed in the storage shed as soon as possible after collection. One aliquot of each sample was
preserved with nitric acid and submitted to the reference laboratory.
7
-------
3.4 Reference Analysis
The reference arsenic analyses were performed in a Battelle laboratory using a Perkin Elmer
Sciex Elan 6000 ICPMS according to EPA Method 200.8, Revision 5.5.^ The sample was
introduced through a peristaltic pump by pneumatic nebulization into a radiofrequency plasma
where energy transfer processes caused desolvation, atomization, and ionization. The ions were
extracted from the plasma through a pumped vacuum interface and separated on the basis of their
mass-to-charge ratio by a quadrupole mass spectrometer. The ions transmitted through the
quadrupole were registered by a continuous dynode electron multiplier, and the ion information
was processed by a data handling system.
The ICPMS was tuned, optimized, and calibrated daily. The calibration was performed using a
minimum of five calibration standards at concentrations ranging between 0.5 and 250 ppb, and a
required correlation coefficient of a minimum of 0.999. Internal standards were used to correct
for instrument drift and physical interferences. These standards were introduced in line through
the peristaltic pump and analyzed with all blanks, standards, and samples.
3.5 Verification Schedule
The verification test took place from January 29 through February 24, 2003. Table 3-2 shows the
daily activities that were conducted during this period. The reference analyses were performed
on March 7 and March 14, 2003, five to six weeks after sample collection.
Table 3-2. Schedule of Verification Test Days
Sample
Collection
Date
Sample Analysis
Date
Tech.
Op.
Non-
Tech. Op.
Testing
Location
Activity
1/29/03-
2/12/03
1/29/03-
2/12/03
1/29/03-
2/12/03
Battelle
Laboratory
Preparation and analysis of PT and
associated QC samples.
2/12/03
2/13/03
2/14/03
Battelle
Laboratory
Collection and analysis of Ayer untreated
and treated water and associated QC
samples.
2/17/03
2/18/03
2/17/03
Battelle
Laboratory
Collection and analysis of Battelle drinking
water and associated QC samples.
2/21/03
2/21/03
2/21/03
Battelle
Storage Shed
Collection and analysis of Falmouth Pond
water and associated QC samples.
2/23/03
2/24/03
2/24/03
Battelle
Storage Shed
Collection and analysis of Taunton River
water and associated QC samples.
8
-------
Chapter 4
Quality Assurance/Quality Control
Quality assurance/quality control (QA/QC) procedures were performed in accordance with the
quality management plan (QMP) for the AMS Center(3) and the test/QA plan for this verification
test.(1) QA/QC procedures and results are described below.
4.1 Laboratory QC for Reference Method
Reference analyses were conducted on March 7 and March 14, 2003. Laboratory QC for the
reference method included the analysis of RB, QCS, LFM and analytical duplicate samples.
Laboratory RB samples were analyzed to ensure that no contamination was introduced by the
sample preparation and analysis process. The test/QA plan stated that if arsenic was detected in a
RB sample above the MDL for the reference instrument, then the contamination source would be
identified and removed and proper blank readings achieved before proceeding with the reference
analyses. All of the laboratory RB samples analyzed were below the reporting limit for arsenic
(i.e., below the concentration of the lowest calibration standard) except for several blanks that
were analyzed at the end of the day on March 7. Three of the six test samples that were
associated with these RB samples were re-analyzed on March 14, with acceptable RB sample
results. The other three test samples had arsenic concentrations that were approximately twenty
times higher than the RB sample concentrations; therefore, no action was taken.
On March 7, the instrument used for the reference method was calibrated using nine calibration
standards, with concentrations ranging from 0.5 to 250 ppb arsenic. On March 14, it was
calibrated using eight standards ranging in concentration from 0.1 to 25 ppb arsenic for more
accurate analysis of low level samples. The accuracy of the calibration was verified after the
analysis of every 10 samples by analyzing a QCS of a known concentration. The percent
recovery of the QCS was calculated from the following equation:
R = —x 100 ^
s
where Cs is the measured concentration of the QCS and 5 is the spike concentration. If the QCS
analysis differed by more than 10% from the true value of the standard, the instrument was
recalibrated before continuing the test. As shown in Table 4-1, all QCS analyses were within the
required range.
9
-------
Table 4-1. Reference Method QCS Analysis Results
Measured Percent
Sample ID Analysis Date (ppb) Actual (ppb) Recovery
CCV25
3/7/2003
24.96
25.00
100%
QCS 25
3/7/2003
26.81
25.00
107%
CCV 25
3/7/2003
24.50
25.00
98%
CCV25
3/7/2003
25.39
25.00
102%
CCV 25
3/7/2003
25.73
25.00
103%
CCV 25
3/7/2003
25.81
25.00
103%
CCV 25
3/7/2003
25.64
25.00
103%
CCV 25
3/7/2003
25.30
25.00
101%
CCV 25
3/7/2003
24.90
25.00
100%
CCV 25
3/7/2003
22.67
25.00
91%
QCS 25
3/14/2003
24.90
25.00
100%
CCV 2.5
3/14/2003
2.74
2.50
110%
QCS 2.5
3/14/2003
2.70
2.50
108%
CCV 2.5
3/14/2003
2.58
2.50
103%
CCV 2.5
3/14/2003
2.65
2.50
106%
CCV 2.5
3/14/2003
2.66
2.50
106%
CCV 2.5
3/14/2003
2.61
2.50
104%
CCV 2.5
3/14/2003
2.60
2.50
104%
LFM samples were analyzed to assess whether matrix effects influenced the reference method
results. The LFM percent recovery (R) was calculated from the following equation:
R= °s xlOO (2)
s
where Cs is the measured concentration of the spiked sample, C is the measured concentration of
the unspiked sample, and 5 is the spike concentration. If the percent recovery of an LFM sample
fell outside the range from 85% to 115%, a matrix effect was suspected. As shown in Table 4-2,
all of the LFM sample results were within this range.
Duplicate samples were analyzed to assess the precision of the reference analysis. The relative
percent difference (RPD) of the duplicate sample analysis was calculated from the following
equation:
RPD= {C ~ C"} x 100 (3)
(C + CD)/2
where C is the concentration of the sample analysis, and Cd is the concentration of the duplicate
sample analysis. If the RPD was greater than 10%, the instrument was recalibrated before
10
-------
Table 4-2. Reference Method LFM Sample Results
Amount
Unspiked Spiked Spiked Percent
Sample ID
Matrix
Analysis Date
(PPb)
(PPb)
(PPb)
Recovery
ASTM Type I
CAA-22
water
3/7/2003
11.02
37.20
25.00
105%
ASTM Type I
CAA-25 R4
water
3/7/2003
0.95
22.76
25.00
87%
ASTM Type I
CAA-28 R2
water
3/7/2003
3.45
30.64
25.00
109%
ASTM Type I
CAA-29 R4
water
3/7/2003
34.98
60.37
25.00
102%
CAA-37 R4
Drinking water
3/7/2003
0.52
28.20
25.00
111%
CAA-41 R4
Drinking water
3/7/2003
1.24
28.88
25.00
111%
CAA-48
Surface water
3/7/2003
12.26
39.40
25.00
109%
CAA-47 R4
Surface water
3/7/2003
1.07
28.41
25.00
109%
ASTM Type I
CAA-27 R1
water
3/14/2003
2.56
4.73
2.50
87%
CAA-37 R3
Drinking water
3/14/2003
0.45
3.11
2.50
107%
CAA-47 R1
Surface water
3/14/2003
1.36
4.16
2.50
112%
CAA-88 R3
Drinking water
3/14/2003
0.43
3.16
2.50
109%
CAA-88 R4
Drinking water
3/14/2003
0.42
3.18
2.50
111%
continuing the test. As shown in Table 4-3, the RPDs for the duplicate analyses were all less than
10%. The RPD for one duplicate pair was 9.5%; however, the sample concentrations were below
the reporting limit for the reference method (i.e., below the concentration of the lowest
calibration standard).
4.2 Audits
Three types of audits were performed during the verification test: a performance evaluation (PE)
audit of the reference method, a technical systems audit of the verification test performance, and
a data quality audit. Audit procedures are described further below.
4.2.1 Performance Evaluation Audit
A PE audit was conducted to assess the quality of the reference measurements made in this
verification test. For the PE audit, an independent, NIST-traceable, reference material was
obtained from a different commercial supplier than the calibration standards and the standard
used to prepare the PT and field QCS samples. Accuracy of the reference method was verified by
comparing the arsenic concentration measured using the calibration standards to that obtained
using the independently-certified PE standard. Relative percent difference as calculated by
Equation 3 was used to quantify the accuracy of the results. Agreement of the standard within
10%) was required for the measurements to be considered acceptable. As shown in Table 4-4, the
PE sample analysis was within the required range.
11
-------
Table 4-3. Reference Method Duplicate Analysis Results
Sample ID
Analysis
Date
Sample
concentration
(PPb)
Duplicate
concentration
(PPb)
Relative
Percent
Difference
CAA-4
3/7/2003
9.33
9.20
1.4%
CAA-70
3/7/2003
10.93
10.82
1.0%
CAA-26 R1
3/7/2003
1.14
1.13
1.4%
CAA-28 R3
3/7/2003
3.49
3.45
1.1%
CAA-31 R1
3/7/2003
111.89
112.20
0.3%
CAA-38
3/7/2003
11.96
11.90
0.5%
CAA-42
3/7/2003
13.02
13.06
0.3%
CAA-4 8
3/7/2003
12.26
12.22
0.4%
CAA-23
3/14/2003
3.03
2.99
1.3%
CAA-27 R2
3/14/2003
2.64
2.61
0.9%
CAA-37 R4
3/14/2003
0.44
0.43
2.3%
CAA-47 R2
3/14/2003
1.31
1.32
0.2%
CAA-88 R4
3/14/2003
0.42
0.38
9.5%
Table 4-4. Reference Method PE Audit Results
Measured
Arsenic
Actual Arsenic
Date of
Concentration
Concentration
Percent
Sample ID
Analysis
(PPb)
(PPb)
Difference
PE-1
3/24/03
9.63
10.0
4
4.2.2 Technical Systems Audit
An independent Battelle quality staff conducted a technical systems audit (TSA) from February 3
to 6 to ensure that the verification test was being conducted in accordance with the test/QA
plann' and the AMS Center QMP/3' A TSA of the reference method performance was con-
ducted by the Battelle Quality Manager on March 5, 2003, when the reference analyses were
initiated. As part of the TSA, test procedures were compared to those specified in the test/QA
plan, data acquisition and handling procedures were reviewed, and the reference standards and
method were reviewed. Observations and findings from the TSA were documented and sub-
mitted to the Battelle Verification Test Coordinator for response. None of the TSA findings
required corrective action. TSA records are permanently stored with the Battelle Quality
Manager.
4.2.3 Data Quality Audit
At least 10% of the data acquired during the verification test was audited. Battelle's Quality
Manager traced the data from the initial acquisition, through reduction and statistical analysis, to
12
-------
final reporting to ensure the integrity of the reported results. All calculations performed on the
data undergoing the audit were checked.
4.3 QA/QC Reporting
Each audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the ETV
AMS Center.(3) Once the audit reports were prepared, the Battelle Verification Test Coordinator
ensured that a response was provided for each adverse finding or potential problem and imple-
mented any necessary follow-up corrective action. The Battelle Quality Manager ensured that
follow-up corrective action was taken. The results of the TSA and the data quality audit were
submitted to the EPA.
4.4 Data Review
Records generated in the verification test received a review before these records were used to
calculate, evaluate, or report verification results. Table 4-5 summarizes the types of data
recorded and reviewed. All data were recorded by Battelle staff. Data were reviewed by a
Battelle technical staff member involved in the verification test, but not the staff member that
originally generated the record. The person performing the review added his/her initials and the
date to a hard copy of the record being reviewed. Review of some of the test data sheets occurred
outside of the two week period specified in the test/QA plan.
Table 4-5. Summary of Data Recording Process
Data to be Recorded Where Recorded How Often Recorded Disposition of Data(a)
Dates, times of test
events
Test parameters
(temperature, analyte/
interferant identities,
and Quick™ Low
Range test kit and
Quick™ Arsenic Scan
results)
Reference method
sample analysis, chain
of custody, and results
ETV field data
sheets
ETV field data
sheets
Start/end of test event
When set or changed, or
as needed to document
test
Laboratory record
books, data sheets,
or data acquisition
system, as
appropriate
Throughout sample
handling and analysis
process
Used to organize/check
test results; manually
incorporated in data
spreadsheets as necessary
Used to organize/check
test results, manually
incorporated in data
spreadsheets as necessary
Transferred to
spreadsheets
All activities
subsequent to data recording are carried out by Battelle.
13
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the performance factors listed in Section 3.2 are
presented in this chapter. Qualitative observations were also used to evaluate verification test
data.
5.1 Accuracy
All samples were analyzed by both the Quick™ Low Range test kit and reference methods. For
each sample, accuracy was expressed in terms of a relative bias (B) as calculated from the
following equation:
where d is the average difference between the reading from the Quick™ Low Range test kit and
those from the reference method, and CR is the average of the reference measurements. An
additional assessment of accuracy was conducted for the color chart results because of the semi-
quantitative nature of the visual comparisons. Each color in the chart represents a concentration
range. Performance was assessed by determining whether the result falls within the expected
concentration range as measured by the reference analysis. Overall agreement was assessed by
calculating the percent of results that fell within the correct range, calculated from the following
equation:
where A is the percent of measurements in agreement, 7is the number of measurements within
the expected color range, and n is the total number of measurements. Readings below the
vendor-stated detection limit of the test kit (e.g., <3 ppb) were judged to be in agreement with the
reference result if the reference value was in the specified "less than" range.
5 = ^x100
(4)
(5)
n
14
-------
5.2 Precision
When possible, the standard deviation (S) of the results for the replicate samples was calculated
and used as a measure of Quick™ Low Range test kit precision at each concentration. Standard
deviation was calculated from the following equation:
S =
1
n -
TI (C,-
1 k=1
iX
cy
(6)
where n is the number of replicate samples, Ci, is the concentration measured for the kth sample,
and C is the average concentration of the replicate samples. Precision was reported in terms of
the relative standard deviation (RSD) as follows:
RSD =
x 100
(7)
5.3 Linearity
Linearity was assessed by performing a linear regression of Quick™ Low Range test kit results
against the reference results, with linearity characterized by the slope, intercept, and correlation
coefficient (R). Linearity was tested using the five PT samples over the range of 1 ppb to 100
ppb arsenic. Samples with results below the vendor-stated detection limit were not included.
Both color chart results and Arsenic Quick™ Scan results were plotted against the corresponding
mean reference concentrations and separate regressions were performed.
5.4 Method Detection Limit
The MDL for the Quick™ Low Range test kit was assessed using results from both detection
methods (color chart and Quick™ Arsenic Scan) for seven replicate analyses of a sample spiked
with approximately 15 ppb arsenic. The standard deviation of the seven replicate samples was
calculated using Equation (6). The MDL was calculated using the following equation:
MDL = txS (8)
where t is the Student's t value for a 99% confidence level and S is the standard deviation of the
seven replicate samples.
5.5 Matrix Interference Effects
The potential effect of interfering substances on the sensitivity of the Quick™ Low Range test
kit was evaluated by the calculating accuracy (expressed as bias) using Equation 4. These results
15
-------
were qualitatively compared with accuracy results for PT samples containing only arsenic to
assess whether there was a positive or negative effect due to matrix interferences.
5.6 Operator Bias
Potential operator bias for the Quick™ Low Range test kit was assessed by performing a linear
regression of sample results above the detection limit generated by the technical and non-
technical operator. Color chart and Quick™ Arsenic Scan results were evaluated. The slope,
intercept, and correlation coefficient were used to evaluate the degree of operator bias. A paired
t-test was also conducted to evaluate whether the two sets of sample results were significantly
different at a 95% confidence level.
5.7 Inter-Unit Reproducibility
Inter-unit reproducibility for the Quick™ Low Range test kit with the Quick™ Arsenic Scan
device was assessed by performing a linear regression of sample results generated by the two
units that were used by the technical operator. The slope, intercept, and correlation coefficient
were used to evaluate the degree of inter-unit reproducibility. A paired t-test was also conducted
to evaluate whether the two sets of sample results were significantly different at a 95%
confidence level.
5.8 Rate of False Positives/False Negatives
The rates of false positives and false negatives produced by the Quick™ Low Range test kit were
assessed relative to the 10-ppb target arsenic level. A false positive result is defined as any result
reported to be greater than or equal to the guidance level (10 ppb) and greater than 125% of the
reference value, when the reference value is less than that guidance level. Similarly, a false
negative result is defined as any result reported below the guidance level and less than 75% of
the reference value, when the reference value is greater than the guidance level. The rates of false
positives and false negatives were expressed as a percentage of total samples analyzed for each
type of sample.
16
-------
Chapter 6
Test Results
The results of the verification test of the Quick™ Low-Range Arsenic test kits are presented in
this section.
6.1 QC Samples
As described in Section 3.3.1, the QC samples analyzed with the Quick™ Low Range test kit
included RB samples, QCS, and LFM samples (these QC samples were different than those
analyzed in conjunction with the reference analyses). The RB samples were analyzed at a
frequency of 10% and results were used to verify that no arsenic contamination was introduced
during sample handling and analysis. QCS were analyzed at the beginning and end of each test
period, and after every tenth sample. The QCS results were used to verify that the test kit was
operating properly. One LFM sample was prepared from each environmental sample to evaluate
potential matrix interferences. Acceptance criteria for test kit QC samples were not specified in
the test/QA plan because modifications to the technology would not be made during testing.
RB sample results for the technical and non-technical operators are presented in Tables 6-la and
6-lb, respectively. Unique sample identification codes were assigned to each container of ASTM
Type I water that was used. The RB samples were analyzed at the required frequency except on
the first day of testing, when the RB sample was inadvertently omitted by the technical operator.
The technical operator recorded one detected value for a RB sample using the color chart; how-
ever, all Quick™ Arsenic Scan readings were below detection. The non-technical operator
reported detected values for two RB samples using the color chart on the first two days of
testing; however, all Quick™ Arsenic Scan readings were below detection. The RB samples
were not analyzed by the reference laboratory. Because the color chart detections for several RB
samples were not confirmed by the Quick™ Arsenic Scan, it was concluded that arsenic
contamination resulting from sample handling and analysis did not occur.
QCS results for the technical and non-technical operators are presented in Tables 6-2a and 6-2b,
respectively. The QCS were analyzed at the required frequency except on the first day of testing,
when the technical operator inadvertently omitted these samples. The percent recovery of the
QCS was calculated using Equation 1 (Section 4.1). The QCS percent recovery for the technical
operator ranged from 100% to 130% for the color chart and from 64% to 144% for the Quick™
Arsenic Scan. The QCS percent recovery for the non-technical operator ranged from 18% to
150% for the color chart and from 40% to 90% for the Quick™ Arsenic Scan. Overall the test
kits and Quick™ Arsenic Scan appeared to be operating as expected.
17
-------
Table 6-la. RB Sample Results for the Technical Operator(a)
Sample ID
Replicate
Analysis Date
Color chart
(ppb)
Quick™ Arsenic Scan #1
(ppb)
Quick™ Arsenic Scan #2
(ppb)
CAA-6
1
1/30/2003
<3
<1
<1
CAA-12
1
2/4/2003
<3
<1
<1
CAA-13
1
2/6/2003
3
<1
<1
CAA-49
1
2/6/2003
<3
<1
<1
CAA-52
1
2/10/2003
<3
<1
<1
CAA-54
1
2/13/2003
<3
<1
<1
CAA-54
2
2/13/2003
<3
<1
<1
CAA-57
1
2/18/2003
<3
<1
<1
CAA-58
1
2/21/2003
<3
<1
<1
CAA-59
1
2/24/2003
<3
<1
<1
fa)
The technical operator inadvertently omitted the RB sample on the first day of testing (1/29/03).
Table 6-lb. RB Sample Results for the Non-Technical Operator
Color Chart Quick™ Arsenic Scan #1
Sample ID
Replicate
Analysis Date
(ppb)
(ppb)
CAA-3
1
1/29/2003
8
<1
CAA-6
1
1/30/2003
6
<1
CAA-10
1
2/3/2003
<3
<1
CAA-13
1
2/6/2003
<3
<1
CAA-49
1
2/6/2003
<3
<1
CAA-53
1
2/11/2003
<3
<1
CAA-55
1
2/14/2003
<3
<1
CAA-55
2
2/14/2003
<3
<1
CAA-56
1
2/17/2003
<3
<1
CAA-58
1
2/21/2003
<3
<1
CAA-59
1
2/24/2003
<3
<1
-------
Table 6-2a. QCS Results for the Technical Operator(a)
Sample ID
Replicate
Analysis Date
Color chart
(PPb)
Quick™
Arsenic Scan
#1
(PPb)
Quick™
Arsenic Scan
#2
(PPb)
Amount
Spiked
(PPb)
Percent
Recovery
Color Chart
Percent
Recovery
Quick™
Arsenic Scan
#1
Percent
Recovery
Quick™
Arsenic Scan
#2
CAA-14
1
1/30/2003
11
10.4
10.4
10.0
110%
104%
104%
CAA-14
2
1/30/2003
10
10.4
9.7
10.0
100%
104%
97%
CAA-16
1
2/4/2003
11
10.4
10.4
10.0
110%
104%
104%
CAA-16
2
2/4/2003
12
11.2
14.4
10.0
120%
112%
144%
CAA-17
1
2/6/2003
11
8.0
9
10.0
110%
80%
90%
CAA-17
2
2/6/2003
11
9.7
10.4
10.0
110%
97%
104%
CAA-19
1
2/10/2003
11
8.0
9.7
10.0
110%
80%
97%
CAA-19
2
2/10/2003
11
6.4
7.3
10.0
110%
64%
73%
CAA-20
1
2/13/2003
12
7.3
9
10.0
120%
73%
90%
CAA-20
2
2/13/2003
12
7.3
9
10.0
120%
73%
90%
CAA-21
1
2/18/2003
10
9.7
12.8
10.0
100%
97%
128%
CAA-21
2
2/18/2003
13
9.7
10.4
10.0
130%
97%
104%
CAA-71
1
2/21/2003
12
9.7
10.4
10.0
120%
97%
104%
CAA-71
2
2/21/2003
13
9.0
9
10.0
130%
90%
90%
CAA-72
1
2/24/2003
11
11.2
11.2
10.0
110%
112%
112%
CAA-72
2
2/24/2003
12
9.0
9
10.0
120%
90%
90%
The technical operator inadvertently omitted the QCS on the first day of testing (1/29/03),
-------
Table 6-2b. QCS Results for the Non-Technical Operator
Sample ID
Replicate
Analysis Date
Color Chart
(PPb)
Quick™ Arsenic Scan #1
(PPb)
Amount Spiked
(PPb)
Percent Recovery
Color Chart
Percent Recovery
Quick™ Arsenic Scan #1
CAA-4
1
1/29/2003
15
5.6
10.0
150%
56%
CAA-4
2
1/29/2003
15
5.6
10.0
150%
56%
CAA-14
1
1/30/2003
15
7.3
10.0
150%
73%
CAA-16
1
2/3/2003
15
9
10.0
150%
90%
CAA-16
2
2/3/2003
15
9
10.0
150%
90%
CAA-17
1
2/6/2003
10
7.3
10.0
100%
73%
CAA-17
2
2/6/2003
10
4.8
10.0
100%
48%
CAA-19
1
2/11/2003
10
5.6
10.0
100%
56%
CAA-19
2
2/11/2003
10
6.4
10.0
100%
64%
CAA-20
1
2/14/2003
10
4
10.0
100%
40%
CAA-20
2
2/14/2003
10
5.6
10.0
100%
56%
CAA-21
1
2/17/2003
8
4
10.0
80%
40%
CAA-21
2
2/17/2003
8
5.6
10.0
80%
56%
CAA-70
1
2/21/2003
1.8
6.4
10.0
18%
64%
CAA-70
2
2/21/2003
1.8
4
10.0
18%
40%
CAA-72
1
2/24/2003
10
8
10.0
100%
80%
CAA-72
2
2/24/2003
10
4.8
10.0
100%
48%
-------
The LFM sample results for the technical and non-technical operators are presented in
Tables 6-3a and 6-3b. The percent recovery associated with each LFM sample was calculated
using Equation 2 (Section 4.1). Reference method results are also provided for comparison. The
lowest recoveries measured by both operators were associated with the Ayer untreated water
LFM sample. The reference method results also indicated a low recovery. The low recoveries
indicate that a matrix interference may be adversely affecting the detection of arsenic.
Consequently, test kit results for this sample may be biased low.
Table 6-3a. LFM Sample Results for the Technical Operator
Amount
Unspiked(a)
Spiked
spiked
Percent
Description
(ppb)
(ppb)
(ppb)
Recovery
Battelle drinking water LFM
Color Chart
<3
12
10
120%
Quick™ Arsenic Scan # 1
<1
7.3
10
73%
Quick™ Arsenic Scan #2
<1
8
10
80%
Reference
<0.5
11.96
10
120%
Ayer untreated water LFM
Color Chart
40
50
10
100%
Quick™ Arsenic Scan # 1
10
5
10
-50%
Quick™ Arsenic Scan #2
14.5
10
10
-45%
Reference
64.82
69.74
10
49%
Ayer treated water LFM
Color Chart
<3
10
10
100%
Quick™ Arsenic Scan # 1
<1
4
10
40%
Quick™ Arsenic Scan #2
<1
4
10
40%
Reference
1.39
13.02
10
116%
Falmouth Pond LFM
Color Chart
<3
11
10
110%
Quick™ Arsenic Scan # 1
<1
9.7
10
97%
Quick™ Arsenic Scan #2
<1
10.4
10
104%
Reference
<0.5
11.50
10
115%
Taunton River LFM
Color Chart
<3
12
10
120%
Quick™ Arsenic Scan # 1
1.3
10.4
10
92%
Quick™ Arsenic Scan #2
2.0
9.7
10
78%
Reference
1.31
12.26
10
109%
w Average of four replicates. Non-detects were assigned a value of zero.
21
-------
Table 6-3b. LFM Sample Results for the Non-Technical Operator
Amount
Unspiked(a)
Spiked
Spiked
Percent
Description
(PPb)
(PPb)
(ppb)
Recovery
Battelle drinking water LFM
Color Chart
<3
8
10
80%
Quick™ Arsenic Scan # 1
<1
4
10
40%
Reference
<0.5
11.96
10
120%
Ayer untreated water LFM
Color Chart
80
80
10
0%
Quick™ Arsenic Scan # 1
10
10
10
0%
Reference
64.82
69.74
10
49%
Ayer treated water LFM
Color Chart
<3
8
10
80%
Quick™ Arsenic Scan # 1
<1
2
10
20%
Reference
1.39
13.02
10
116%
Falmouth Pond LFM
Color Chart
<3
2.2
10
22%
Quick™ Arsenic Scan # 1
2.3
16
10
137%
Reference
<0.5
11.50
10
115%
Taunton River LFM
Color Chart
<3
10
10
100%
Quick™ Arsenic Scan # 1
<1
10.4
10
104%
Reference
1.31
12.26
10
109%
1Average of four replicates. Non-detects were assigned a value of zero.
6.2 PT and Environmental Samples
Table 6-4 presents the sample results for the PT and environmental samples. The table includes
the Low Range Quick™ test kit results and the reference method results. The test kit results are
shown for both the technical and non-technical operators, and the Quick™ Arsenic Scan Units
#1 and #2. Sample results that were obtained from diluted samples are noted. Test kit results
below the detection limit were assigned a value of <3 ppb for the color chart and <1 ppb for the
Quick™ Arsenic Scan. The reporting limit for the reference analyses was 0.5 ppb, which
corresponds to the lowest calibration standard used. Results for each performance factor are
presented below.
6.2.1 Accuracy
Table 6-5 presents the accuracy results for the Quick™ Low Range test kit, expressed as percent
bias as calculated by Equation 4 (Section 5.1). Percent bias was not calculated for results below
the detection limit. The four replicate analyses for each sample were averaged in the calculation
of bias. The relative bias for the color chart ranged from -38% to 239% for the technical operator
and -81%) to 579%> for the non-technical operator. The relative bias for the Quick™ Arsenic Scan
22
-------
Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results
Technical
Technical
Operator
Technical
Non-Technical
Non-Technical
Operator
Quick™
Operator
Operator
Operator Quick™
Sample
Color
Arsenic Scan #1
Quick™ Arsenic
Color Chart
Arsenic Scan #1
Reference
Description
ID
Replicate
Chart (ppb)
(PPb)
Scan #2 (ppb)
(PPb)
(PPb)
(PPb)
PT - 1 ppb As
CAA-25
1
3
<1
1
6
1
0.91
CAA-25
2
3
<1
<1
6
1
0.86
CAA-25
3
3
<1
<1
6
1
0.90
CAA-25
4
3
1
<1
6
1
0.86
PT - 3 ppb As
CAA-27
1
6
2
1
6
2
2.56
CAA-27
2
6
2
1
6
2
2.64
CAA-27
3
6
3
1
6
2
2.50
CAA-27
4
6
3
2
6
3
2.71
PT - 10 ppb As
CAA-1
1
10
9.7
8
15
8
9.09
CAA-1
2
10
10.4
9.7
12
5.6
8.95
CAA-1
3
10
11.2
9.7
19
12.8
8.83
CAA-1
4
10
8
7.3
12
5.6
8.99
PT - 30 ppb As
CAA-29
1
30 (a)
24.4 (a)
35.2 (a)
24
22.4
33.96
CAA-29
2
30(a)
35.2 (a)
28.8 (a)
24
24.1
34.39
CAA-29
3
30 (a)
29.4 (a)
33.6 (a)
24
24.9
34.51
CAA-29
4
30 (a)
27.2 (a)
28.8 (a)
24
26.6
34.98
PT - 100 ppb As
CAA-31
1
95 (b)
91.5 (b)
96(b)
120 (b)
96(b)
111.89
CAA-31
2
95 ^
104 (b)
91.5 (b)
120 (b)
76. 5 (b)
115.57
CAA-31
3
100 (b)
96 (W
112 (b)
95 w
00
00
114.65
CAA-31
4
100 (b)
99.5 (b)
96®
95 ^
52®
113.83
Detection limit
CAA-24
1
12
9.70
9.70
8
4.8
CAA-24
2
13
12.80
12.8
8
3
CAA-24
3
13
10.40
11.2
8
4.8
CAA-24
4
12
11.20
12.20
8
7.3
14.18
CAA-24
5
13
10.40
11.2
12
9
CAA-24
6
13
12.80
13.6
12
9
CAA-24
7
15
12.20
12.8
12
7.3
-------
Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results (continued)
Technical
Non-Technical
Operator
Technical
Non-
Operator
Technical
Quick™
Operator
Technical
Quick™
Operator
Arsenic Scan
Quick™
Operator
Arsenic Scan
Sample
Color Chart
#1
Arsenic Scan
Color Chart
#1
Reference
Description
ID
Replicate
(PPb)
(PPb)
#2 (ppb)
(PPb)
(PPb)
(PPb)
PT -10 ppb
CAA-33
1
13
10.4
10.4
3
1
As +
CAA-33
2
11
9
9
8
9.7
9.90
low level
CAA-33
3
11
9
9.7
10
4.8
interferents
CAA-33
4
11
10.4
11.2
10
4.8
PT -10 ppb
CAA-35
1
24
19.2
27.4
24
17.6
As +
CAA-35
2
22
20.8
22.4
24
19.2
11.59
high level
CAA-35
3
24
19.9
24.9
24
22.4
interferents
CAA-35
4
22
17.6
17.6
24
17.6
Battelle
CAA-37
1
<3
<1
<1
<3
<1
<0.5
drinking
CAA-37
2
<3
<1
<1
<3
<1
<0.5
water
CAA-37
3
<3
<1
<1
<3
<1
<0.5
CAA-37
4
<3
<1
<1
<3
<1
<0.5
Battelle
drinking
CAA-38
1
12
7.3
8
8
4
11.96
water LFM
Ayer
CAA-39
1
40 (b)
10(b)
15(b)
80 (c)
io w
65.61
untreated
CAA-39
2
45 ^
20 ^
28 ^
80 (c)
10(c)
62.73
water
CAA-39
3
40 ^
10®
15®
80 (c)
10(c)
67.47
CAA-39
4
35 ^
<1®
<1®
80 (c)
10(c)
63.48
Ayer
untreated
CAA-40
1
50
5
10
80
10
69.74
water LFM
Ayer treated
CAA-41
1
<3
<1
<1
<3
<1
1.36
water
CAA-41
2
<3
<1
<1
<3
<1
1.45
CAA-41
3
<3
<1
<1
<3
<1
1.44
CAA-41
4
<3
<1
<1
<3
<1
1.32
Ayer treated
CAA-42
1
10
4
4
8
2
13.02
water LFM
-------
Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results (continued)
Technical
Operator
Technical
Technical
Quick™
Operator
Non-Technical
Operator
Arsenic Scan
Quick™
Non-Technical
Operator Quick™
Sample
Color Chart
#1
Arsenic Scan #2
Operator Color
Arsenic Scan #1
Reference
Description
ID
Replicate
(PPb)
(PPb)
(PPb)
Chart (ppb)
(PPb)
(PPb)
Falmouth
CAA-43
1
<3
<1
<1
<3
<1
<0.5
Pond water
CAA-43
2
<3
<1
<1
<3
1
<0.5
CAA-43
3
<3
<1
<1
<3
7.3
<0.5
CAA-43
4
<3
1
<1
<3
1
<0.5
Falmouth
Pond water
CAA-46
1
11
9.7
10.4
2.2
16
11.50
LFM
Taunton
CAA-47
1
<3
1
1
<3
1
1.36
River water
CAA-47
2
<3
<1
2
<3
1
1.31
CAA-47
3
<3
4
4.8
<3
1
1.31
CAA-47
4
<3
<1
<1
<3
<1
1.26
Taunton
River water
CAA-48
1
12
10.4
9.7
10
10.4
12.26
LFM
Note: w = diluted 1:2;(b-1 = diluted 1:5;{c> = diluted 1:10.
-------
Table 6-5. Quantitative Evaluation of Accuracy for Quick™ Low Range Test Kits(a)
Bias
Technical
Technical
Non-Technical
Non-Technical
Description
Technical Operator
Operator Quick™
Operator Quick™
Operator Color
Operator Quick™
Color Chart
Arsenic Scan #1
Arsenic Scan #2
Chart
Arsenic Scan #1
Performance Test Samples
1 ppb As
239%
NA
NA
579%
13.1%
3 ppb As
131%
-3.9%
-51.9%
131%
-13.5%
10 ppb As
11.6%
9.6%
-3.2%
61.8%
-10.7%
30 ppb As
-12.9%
-15.7%
-8.3%
-30.4%
-28.9%
100 ppb As
-14.5%
-14.2%
-13.3%
-5.7%
-31.5%
10 ppb As + low level interferents
16.2%
-2.0%
1.8%
-21.7%
-48.7%
10 ppb As + high level interferents
98.4%
67.1%
99.1%
107%
65.6%
Environmental Samples
Battelle drinking water
NA
NA
NA
NA
NA
Battelle drinking water LFM
0.31%
-39.0%
-33.1%
-33.1%
-66.6%
Ayer untreated water
-38.3%
NA
NA
23.4%
-84.6%
Ayer untreated water LFM
-28.3%
-92.8%
-85.7%
14.7%
-85.7%
Ayer treated water
NA
NA
NA
NA
NA
Ayer treated water LFM
-23.2%
-69.3%
-69.3%
-38.5%
-84.6%
Falmouth Pond water
NA
NA
NA
NA
NA
Falmouth Pond water LFM
-4.4%
-15.7%
-9.6%
-80.9%
39.1%
Taunton River water
NA
NA
NA
NA
NA
Taunton River water LFM
-2.1%
-15.2%
-20.9%
-18.5%
-15.2%
w Percent bias calculated according to Equation 4, Section 5.1.
NA: one or more replicates below detection limit
-------
ranged from -93% to 99% for the technical operator and -86% to 66% for the non-technical
operator. Negative biases for the Ayer untreated water sample (with the exception of the color
chart result for the non-technical operator) confirm the apparent matrix effect observed in the
associated LFM sample (see Section 6.1).
Table 6-6 presents accuracy results for each PT and environmental replicate sample according to
whether the color chart result agreed with the reference value for that sample. The reference
sample result was assigned to the correct corresponding color block. A test kit result was
considered to be in agreement with the reference method result if it fell within the range of plus
or minus one color block (i.e., the concentration range spanning three adjacent color blocks). If
the color chart test result for a given sample was within this range, then a "Y" was reported in
Table 6-6. If the color chart result was outside this range, then an "N" was reported. Overall
agreement was determined by calculating the total percent of results in agreement for the
technical and non-technical operators. The total percent agreement using this method was 81%
for the technical operator and 74% for the non-technical operator.
6.2.2 Precision
Precision results for the Quick™ Low Range test kit are presented in Table 6-7. The RSD was
determined according to Equation 7 (Section 5.2). The RSD was not calculated if any of the
results for a set of replicates were below the detection limit (i.e., <3 ppb for the color chart or
<1 ppb for the Quick™ Arsenic Scan). For the technical operator, RSDs ranged from 0% to 10%
for the color chart and 5% to 23% for the Quick™ Arsenic Scan. For the non-technical operator,
RSDs ranged from 0% to 23% for the color chart and 0% to 42% for the Quick™ Arsenic Scan.
6.2.3 Linearity
The linearity of the Quick™ Low Range test kit readings was assessed by performing a linear
regression of the test kit results against the reference method results for the five PT samples
ranging from 1 ppb to 100 ppb arsenic. In these regressions, results reported as below the
detection limit by the test kit were not used. Figures 6-1 and 6-2 present the results of the linear
regressions for the color chart and Quick™ Arsenic Scan results, respectively. The slope,
intercept and correlation coefficient for each equation are shown on the charts. For the color
chart, the results for the technical operator were more linear than the results for the non-technical
operator; however, the non-technical operator results showed an overall closer correspondence
with the reference method results. For the Quick™ Arsenic Scan, the results for the technical
operator using Unit #1 showed the greatest degree of linearity. The technical operator results for
Unit #2 showed the closest overall correspondence with the reference method results.
27
-------
Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range Test Kits
Within Range
Within Range
(Y/N)
Technical
(Y/N)
Non-Technical
Description
Sample ID
Replicate
Operator Color
Chart
Operator Color
Chart
Performance Test Samples
1 ppb As
CAA-25
1
Y
N
CAA-25
2
Y
N
CAA-25
3
Y
N
CAA-25
4
Y
N
3 ppb As
CAA-27
1
Y
Y
CAA-27
2
Y
Y
CAA-27
3
Y
Y
CAA-27
4
Y
Y
10 ppb As
CAA-1
1
Y
N
CAA-1
2
Y
Y
CAA-1
3
Y
N
CAA-1
4
Y
Y
30 ppb As
CAA-29
1
Y
Y
CAA-29
2
Y
Y
CAA-29
3
Y
Y
CAA-29
4
Y
Y
100 ppb As
CAA-31
1
Y
Y
CAA-31
2
Y
Y
CAA-31
3
Y
Y
CAA-31
4
Y
Y
10 ppb As +
CAA-33
1
N
N
low level
CAA-33
2
Y
Y
interferents
CAA-33
3
Y
Y
CAA-33
4
Y
Y
10 ppb As +
CAA-35
1
N
N
high level
CAA-35
2
N
N
interferents
CAA-35
3
N
N
CAA-35
4
N
N
Environmental Samples
Battelle drinking water
CAA-37
1
Y
Y
CAA-37
2
Y
Y
CAA-37
3
Y
Y
CAA-37
4
Y
Y
Battelle drinking water LFM
CAA-38
1
Y
N
Ayer untreated water
CAA-39
1
N
Y
CAA-39
2
N
Y
CAA-39
3
N
Y
CAA-39
4
N
Y
Ayer untreated water LFM
CAA-40
1
N
Y
28
-------
Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range Test Kits
(continued)
Within Range
(Y/N)
Technical
Within Range
(Y/N)
Non-Technical
Description
Sample ID
Replicate
Operator Color
Chart
Operator Color
Chart
Ayer treated water
CAA-41
1
Y
Y
CAA-41
2
Y
Y
CAA-41
3
Y
Y
CAA-41
4
Y
Y
Ayer treated water LFM
CAA-42
1
Y
N
Falmouth Pond water
CAA-43
1
Y
Y
CAA-43
2
Y
Y
CAA-43
3
Y
Y
CAA-43
4
Y
Y
Falmouth Pond water LFM
CAA-46
1
Y
N
Taunton River water
CAA-47
1
Y
Y
CAA-47
2
Y
Y
CAA-47
3
Y
Y
CAA-47
4
Y
Y
Taunton River water LFM
CAA-48
1
Y
Y
Percent Agreement
81%
74%
w Percent of sample "Y", within range, divided by total number of samples.
29
-------
Table 6-7. Precision Results for Quick™ Low Range Test Kits
RSD
Non-
Technical
Technical
Non-
Technical
Technical
Operator
Operator
Technical
Operator
Operator
Quick™
Quick™
Operator
Quick™
Color
Arsenic
Arsenic
Color
Arsenic
Reference
Description
Chart
Scan #1
Scan #2
Chart
Scan #1
Method
Performance Test Samples
1 ppb As
0%
NA
NA
0%
0%
3%
3 ppb As
0%
23%
40%
0%
22%
4%
10 ppb As
0%
14%
14%
23%
42%
1%
30 ppb As
0%
16%
10%
0%
7%
1%
100 ppb As
3%
5%
9%
13%
25%
1%
Environmental Samples
Battelle drinking water
NA
NA
NA
NA
NA
NA
Ayer untreated water
10%
NA
NA
0%
0%
3%
Ayer treated water
NA
NA
NA
NA
NA
4%
Falmouth Pond water
NA
NA
NA
NA
NA
NA
Taunton River water
NA
NA
NA
NA
0%
3%
NA: one or more replicates below detection limit
140
1:1 line
120
100
y = 0.90X + 2.78
R = 0.9805
80
y = 0.83x + 2.61
R = 0.9992
60
40
20
0
0
20
40
60
80
100
120
140
Reference concentration (ppb)
~ Technical Operator
¦ Non-technical Operator
¦ " " Linear (Technical Operator)
— — Li near (Non-techn i cal Op erator)
Figure 6-1. Linearity of Quick™ Low Range Color Chart Results
30
-------
Reference concentration (ppb)
~ Techn i cal Operator U nit #1
A Techn i cal Operator U nit #2
¦ Non-techncial operator Unit #1
¦ " " Linear (Technical Operator Unit #1)
Linear (Technical Operator Unit #2)
— — Li near (Non-tech n ci al operator Un it # 1)
Figure 6-2. Linearity of Quick™ Low Range Quick™ Arsenic Scan Results
6.2.4 Method Detection Limit
The MDL was assessed by analyzing seven replicates of a sample spiked at approximately five
times the lowest concentration shown on the Quick™ Low Range test kit color chart (i.e., 3 ppb
X 5 = 15 ppb arsenic). Table 6-8 provides the standard deviation for the seven replicate samples
for the technical and non-technical operator for both the color chart and Quick ™ Arsenic Scan
results, and the calculated MDLs.
31
-------
Table 6-8. Detection Limit Results for Quick™ Low Range Test Kit
Technical
Technical
Non-
Operator
Operator
Technical
Non-Technical
Technical
Quick™
Quick™
Operator
Operator
Operator
Arsenic
Arsenic
Color
Quick™
Color Chart
Scan #1
Scan #2
Chart
Arsenic Scan #1
Sample ID
(PPb)
(ppb)
(ppb)
(ppb)
(ppb)
CAA-24 Rep 1
12
9.70
9.70
8
4.8
CAA-24 Rep 2
13
12.80
12.8
8
3
CAA-24 Rep 3
13
10.40
11.2
8
4.8
CAA-24 Rep 4
12
11.20
12.20
8
7.3
CAA-24 Rep 5
13
10.40
11.2
12
9
CAA-24 Rep 6
13
12.80
13.6
12
9
CAA-24 Rep 7
15
12.20
12.8
12
7.3
Standard Deviation
1.00
1.26
1.32
2.14
2.30
Method Detection Limit
3.1
4.0
4.1
6.7
7.2
(ppb)
6.2.5 Matrix Interference Effects
Matrix interference effects were assessed by comparing the calculated bias for the samples
containing low-level and high-level concentrations of interferences with the bias reported for the
other PT samples containing arsenic only (Table 6-5). An examination of these results indicates
that low levels of interfering compounds did not appear to affect the detection of arsenic, with
biases ranging from -48.7% to 16.2% for both the technical and non-technical operators, and
color chart and Quick™ Arsenic Scan methods. However, high levels of interferences appear to
have affected the arsenic levels measured by the Quick™ Low Range test kit for both the color
chart and Quick™ Arsenic Scan results, as evidenced by positive biases ranging from 65.6% to
107%.
6.2.6 Operator Bias
Operator bias was evaluated by comparing the color chart and Quick™ Arsenic Scan Unit #1
results above the detection limit produced by the technical and non-technical operators for all PT
and environmental samples (the non-technical operator did not use the Quick™ Arsenic Scan
Unit #2). Linear regression results for the two sets of data are shown in Figure 6-3. The slopes of
the regressions made from measurements by the non-technical operator suggest that the color
chart results tended to be higher, and the Quick™ Arsenic Scan results made by the technical
operator tended to be higher. Paired t-tests of the two sets of data indicated that the color chart
results were not significantly different at a 5% significance level; however, the Quick™ Arsenic
Scan results were significantly different.
32
-------
140
1:1 line
120
y= 1.20x-1.81
R = 0.94
100
o 80
O
re
o
A Color Chart
¦ Arsenic Quick Scan Unit #1
Linear (Color Chart)
~ Linear (Arsenic Quick Scan Unit #1)
c
sz
o
0)
c
o
z
y = 0.79x + 0.26
R = 0.95
40
20
0
20
40
60
80
100
120
Technical Operator
Figure 6-3. Comparison of Quick™ Low Range Test Results for Technical and Non-
Technical Operators
6.2.7 Inter-Unit Reproducibility
Inter-unit reproducibility was evaluated by comparing the data for the two Quick™ Arsenic Scan
units used by the technical operator. Only results above the detection limit were included in the
analysis. A linear regression of the two sets of data is shown in Figure 6-4. The regression line
almost exactly corresponded to the 1:1 line, indicating that the performance of the two units was
very similar. A paired t-test of the two sets of data indicated that the results were not
significantly different at a 5% significance level.
6.2.8 Rate of False Positives/False Negatives
Tables 6-9 and 6-10 show the data and results for the rates of false positives and false negatives,
respectively, obtained from the Quick™ Low Range test kit. All PT and environmental samples
were included in this evaluation.
33
-------
120
1:1 line
100
R = 0.!
80
a
'E
=>
~ Quick Arsenic Scan
— ~ Linear (Quick Arsenic Scan)
40
20
0
20
40
60
80
100
120
Unit #1
Figure 6-4. Comparison of Two Quick™ Arsenic Scan Units
As shown in Table 6-9, 32 samples had arsenic concentrations at or below 10 ppb as measured
by the reference analyses. For these samples, the Quick™ Low Range test kit color chart results
were >10 ppb and greater than 125% of the reference measurement for one sample for the
technical operator and four samples for the non-technical operator, yielding false positive rates of
3% and 12.5% respectively. The rates of false positives for the Quick™ Arsenic Scan units were
3% and 0% for the technical operator (Units #1 and #2, respectively) and 3% for the non-
technical operator (Unit #1 only).
Twenty-two samples had arsenic concentrations above 10 ppb as measured by the reference
analyses (Table 6-10). For these samples, the Quick™ Low Range test kit color chart results
were <10 ppb and less than 75% of the reference measurement for none of the samples for the
technical operator and three samples for the technical operator, yielding false negative rates of
0% and 14%, respectively. The rates of false negatives for the Quick™ Arsenic Scan units were
19% and 14% for the technical operator (Units #1 and #2, respectively), and 9.5% for the non-
technical operator (Unit #1 only).
34
-------
Table 6-9. Rate of False Positives for Quick™ Low Range Test Kits
False Positive (Y/N)
Non-
Technical
Technical
Non-
Technical
Description
Sample
ID
Replicate
Technical
Operator
Color
Chart
Operator
Quick™
Arsenic
Scan #1
Operator
Quick™
Arsenic
Scan #2
Technical
Operator
Color
Chart
Operator
Quick™
Arsenic
Scan #1
1 ppb As
CAA-25
1
N
N
N
N
N
CAA-25
2
N
N
N
N
N
CAA-25
3
N
N
N
N
N
CAA-25
4
N
N
N
N
N
3 ppb As
CAA-27
1
N
N
N
N
N
CAA-27
2
N
N
N
N
N
CAA-27
3
N
N
N
N
N
CAA-27
4
N
N
N
N
N
10 ppb As
CAA-1
1
N
N
N
Y
N
CAA-1
2
N
N
N
Y
N
CAA-1
3
N
Y
N
Y
Y
CAA-1
4
N
N
N
Y
N
Battelle
CAA-37
1
N
N
N
N
N
drinking water
CAA-37
2
N
N
N
N
N
CAA-37
3
N
N
N
N
N
CAA-37
4
N
N
N
N
N
Ayer treated
CAA-41
1
N
N
N
N
N
water
CAA-41
2
N
N
N
N
N
CAA-41
3
N
N
N
N
N
CAA-41
4
N
N
N
N
N
Falmouth Pond
CAA-43
1
N
N
N
N
N
water
CAA-43
2
N
N
N
N
N
CAA-43
3
N
N
N
N
N
CAA-43
4
N
N
N
N
N
Taunton River
CAA-47
1
N
N
N
N
N
water
CAA-47
2
N
N
N
N
N
CAA-47
3
N
N
N
N
N
CAA-47
4
N
N
N
N
N
10 ppb As +
CAA-33
1
Y
N
N
N
N
low level
CAA-33
2
N
N
N
N
N
interferents
CAA-33
3
N
N
N
N
N
CAA-33
4
N
N
N
N
N
Total number of samples
32
32
32
32
32
Total false positive
1
1
0
4
1
Percent false positive
3%
3%
0%
12.5%
3%
35
-------
Table 6-10. Rate of False Negatives for Quick™ Low Range Test Kits
False Negative (Y/N)
Non-
Technical
Technical
Non-
Technical
Technical
Operator
Operator
Technical
Operator
Sample
Operator
Quick™
Quick™
Operator
Quick™
Color
Arsenic
Arsenic
Color
Arsenic
Description
ID
Replicate
Chart
Scan #1
Scan #2
Chart
Scan #1
30 ppb As
CAA-29
1
N
N
N
N
N
CAA-29
2
N
N
N
N
N
CAA-29
3
N
N
N
N
N
CAA-29
4
N
N
N
N
N
100 ppb As
CAA-31
1
N
N
N
N
N
CAA-31
2
N
N
N
N
N
CAA-31
3
N
N
N
N
N
CAA-31
4
N
N
N
N
N
Battelle drinking
water LFM
CAA-38
1
N
Y
Y
Y
Y
Ayer untreated
CAA-39
1
N
N
N
N
N
water
CAA-39
2
N
N
N
N
N
CAA-39
3
N
N
N
N
N
CAA-39
4
N
Y
Y
N
N
Ayer untreated
water LFM
CAA-40
1
N
Y
N
N
N
Ayer treated water
LFM
CAA-42
1
N
Y
Y
Y
Y
Falmouth Pond
water LFM
CAA-46
1
N
N
N
Y
N
Taunton River
water LFM
CAA-48
1
N
N
N
N
N
10 ppb As +
CAA-35
1
N
N
N
N
N
high level
CAA-35
2
N
N
N
N
N
interferents
CAA-35
3
N
N
N
N
N
CAA-35
4
N
N
N
N
N
Total number of samples
21
21
21
21
21
Total number of false negatives
0
4
3
3
2
Percent of false negatives
0%
19%
14%
14%
9.5%
6.3 Other Factors
During testing activities, the technical and non-technical operators were instructed to keep a
record of their comments on ease of use, reliability, portability, and generation of waste
materials. This section summarizes these observations and other comments pertaining to any
problems encountered during testing. Cost information is also presented.
36
-------
6.3.1 Ease of Use
The technical and non-technical operator both reported that the Quick™ Low Range test kit was
very easy to use. The test kit instructions were clear and easy to follow. Although the manu-
facturer provided instructions for diluting samples above the 30 ppb arsenic level, the non-
technical operator sometimes had difficulty successfully performing dilutions and correctly
converting the results to a final concentration. The three scoops used to sequentially add reagents
were color coordinated, which facilitated the efficient operation of the test kit. The sample
bottles were of moderate size and were relatively easy to handle, although the narrow neck
sometimes caused spillage during the addition of reagents. The test kit materials were readily
transported to the Battelle storage shed where environmental samples were tested.
6.3.2 Analysis Time
The average total analysis time for a sample was about 15 minutes at a sample temperature of
24°C. The manufacturer provided a modified protocol that specified increased reaction times for
samples below 24°C. The test kit enabled two samples to be run concurrently without any
confusion.
6.3.3 Reliability
The Quick™ Low Range test kits operated reliably throughout the period of the test. Extra care
had to be taken to ensure that the caps to the reaction vessels were completely dry before
proceeding with further analyses, and in some cases the presence of water droplets on the test
pad prevented an accurate reading and required the reanalysis of the sample. After the analysis of
some of the environmental surface water samples, black particulate matter was noted on the test
pad, presumably because of the presence of organic material in the water sample.
6.3.4 Waste Material
The waste generated by the Quick™ Low Range test kit was manageable. The vendor's
instructions provide a warning that hydrogen and arsine are generated during the test and
recommend that testing be conducted in a well-ventilated area away from open flames and other
sources. MSDSs should be reviewed before handling any chemicals. Instructions for the disposal
of residual materials were clear and complete. The residual liquid in the reaction vessel was
allowed to settle before disposal in order to let particulates accumulate on the bottom. A dilute
hydrochloric acid solution was used to clean the reaction vessel prior to subsequent analyses.
Disposal of this waste in an appropriate manner must be taken into consideration.
6.3.5 Cost
The listed price for a Quick™ Low Range test kit for the analysis of 50 samples is $179.99. The
Quick™ Arsenic Scan is available as an option for an additional cost of $1,599.99.
37
-------
Chapter 7
Performance Summary
The Quick™ Low Range test kit was verified by evaluating the following parameters:
¦ Accuracy
¦ Precision
¦ Linearity
¦ MDL
¦ Matrix interference effects
¦ Operator bias
¦ Inter-unit reproducibility
¦ Rate of false positives/false negatives.
The quantitative assessment of accuracy indicated that the relative bias for the color chart ranged
from -38% to 239% for the technical operator and -81% to 579% for the non-technical operator.
The relative bias for the Quick™ Arsenic Scan ranged from -93% to 99% for the technical
operator and -86% to 66% for the non-technical operator. The overall agreement for the color
chart results based on an assessment of whether the result was assigned to the correct color block
indicated that the total percent agreement was 81% for the technical operator and 74% for the
non-technical operator.
Precision was assessed by analyzing four replicates of each sample. For the technical operator,
precision expressed as a RSD ranged from 0% to 10% for the color chart and 5% to 23% for the
Quick™ Arsenic Scan. For the non-technical operator, RSDs ranged from 0% to 23% for the
color chart and 0% to 42% for the Quick™ Arsenic Scan. These results exclude samples where
one or more of the replicate results was not detected by the Quick™ Low Range test kit.
The linearity of response was evaluated by plotting the test kit results against the reference
analysis results for the PT samples. The equations for the linear regressions that were performed
to evaluate linearity are summarized in Table 7-1. The slope, y-intercept, and correlation
coefficient corresponding to a linear response that exactly matched reference concentrations
would be 1.0, 0, and 1.0, respectively.
The MDL was assessed by analyzing seven replicates of a sample spiked at a level
approximately five times the manufacturer's estimated detection limit for the color chart (i.e.,
3 ppb X 5 = 15 ppb). The MDLs calculated using the precision data from these replicates ranged
from 3.1 ppb to 6.7 ppb for the color charts and 4.0 ppb to 7.2 ppb for the Quick™ Arsenic Scan.
38
-------
Table 7-1. Summary of Linear Regression Equations for Test Kit and Reference Results
Correlation
Coefficient
Description
Slope
0.83
0.90
0.85
0.87
0.68
Intercept (R)
2.61 0.9992
2.78 0.9805
0.83 0.9972
0.41 0.9939
0.99 0.9660
Color chart, technical operator
Color chart, non-technical operator
Quick™ Arsenic Scan #1, technical operator
Quick™ Arsenic Scan #2, technical operator
Quick™ Arsenic Scan #1, non-technical operator
Results for samples containing low and high levels of interfering compounds indicated that low
levels of interferents did not appear to affect the detection of arsenic, with biases ranging from -
48.7% to 16.2%. However, high levels of interferences appear to have affected the arsenic levels
measured by the Quick™ Low Range test kit for the color chart and Quick™ Arsenic Scan
results, as evidenced by positive biases ranging from 65.6% to 107%.
An evaluation of Quick™ Low Range test kit results for the technical and non-technical
operators suggested that the color chart measurements made by the non-technical operator tended
to be higher and the Quick™ Arsenic Scan measurements made by the technical operator tended
to be higher. Paired t-tests of the two sets of data indicated that the color chart results were not
significantly different at a 5% significance level; however, the Quick™ Arsenic Scan results
were significantly different for the two operators. The regression equations were as follows:
where x is the technical operator andj' is the non-technical operator.
Inter-unit reproducibility was evaluated by comparing the data for the two Quick™ Arsenic Scan
units used by the technical operator. A linear regression of the two sets of data indicated that the
results closely corresponded. A paired t-test of the two sets of data indicated that the results were
not significantly different at a 5% significance level. The regression equation was as follows:
where x is Unit #1 and^ is Unit #2.
A false positive was defined as a test kit result that was greater than 10 ppb and greater than
125%) of the reference concentration, when the reference concentration is less than or equal to
10 ppb. The rates of false positives for the technical and non-technical operators using the color
charts were 3% and 12.5%, respectively. The rates of false positives for the Quick™ Arsenic
Scan units were 3% and 0% for the technical operator (Units #1 and #2, respectively) and 3% for
the non-technical operator (Unit #1 only). A false negative was defined as a test kit result that
was less than or equal to 10 ppb and less than 75% of the reference concentration, when the
Color chart
Quick™ Arsenic Scan
y= 1.20x- 1.81, R = 0.94
y = 0.79x + 0.26, R= 0.95
Quick™ Arsenic Scan y = 1,00x + 1.17, R = 0.99
39
-------
reference concentration was greater than 10 ppb. The false negative rates for the technical and
non-technical operators using the color charts were 0% and 14%, respectively. The rates of false
negatives for the Quick™ Arsenic Scan units were 19% and 14% for the technical operator
(Units #1 and #2, respectively) and 9.5% for the non-technical operator (Unit #1 only).
The Quick™ Low Range test kits were easy to use and readily transportable to the field. The
time to analyze one sample was approximately 15 minutes at a temperature range of 24°C to
30°C (longer reaction times are required for samples below this temperature range). Two
samples were run concurrently without difficulty. The sample bottles were of moderate size and
were relatively easy to handle, although the narrow neck sometimes caused spillage during the
addition of reagents. The cost for a 50-sample test kit with the color chart is listed as $179.99.
Replacement reagents and supplies are not available; kits are provided as a complete set because
reagents, test strips, and color charts are made to perform optimally with each other, according to
the vendor. The Quick™ Arsenic Scan is available as an option for an additional cost of
$1,599.99.
40
-------
Chapter 8
References
1. Test/QA Plan for Verification of Portable Analyzers, Battelle, Columbus, Ohio, Version 2.0.
December 8, 2000.
2. U. S. EPA Method 200.8, Determination of Trace Elements in Waters and Wastes by
Inductively Coupled Plasma Mass Spectrometry, Revision 5.5, October 1999.
3. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Pilot, Version
4.0, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio,
December, 2002.
4. U.S. Code of Federal Regulations, Title 40, Part 136, Appendix B.
41
------- |