July 2002
Environmental Technology
Verification Report
Quick™
Arsenic Test Kit
Prepared by
£EPA
Barteiie
. . . Putting Technology To Work
Battelle
Under a cooperative agreement with
U.S. Environmental Protection Agency
etV ety etV

-------
July 2002
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Quick™
Arsenic Test Kit
by
Adam Abbgy
Thomas Kelly
Charles Lawrie
Karen Riggs
Battelle
Columbus, Ohio 43201

-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency and recommended for public release.
Mention of trade names or commercial products does not constitute endorsement or
recommendation by the EPA for use.
11

-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality and
to supply cost and performance data to select the most appropriate technology for that assess-
ment. In 1997, through a competitive cooperative agreement, Battelle was awarded EPA funding
and support to plan, coordinate, and conduct such verification tests for "Advanced Monitoring
Systems for Air, Water, and Soil" and report the results to the community at large. Information
concerning this specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/centers/centerl.html.
111

-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. In particular we would like to thank
A. J. Savage, Raj Mangaraj, Daniel Turner, and Bea Weaver of Battelle. We also acknowledge the
assistance of AMS Center stakeholders Vito Minei, Dennis Goldman, Geoff Dates, and Marty
Link, who reviewed the test/QA plan and verification reports.
IV

-------
Contents
Notice	ii
Foreword 	 iii
Acknowledgments 	 iv
List of Abbreviations 	 viii
1.	Background	1
2.	Technology Description 	2
3.	Test Design and Procedures	3
3.1	Introduction	3
3.2	Test Design	3
3.3	Test Samples 	4
3.3.1	QC Samples 	4
3.3.2	PT Samples	6
3.3.3	Environmental Samples	6
3.4	Reference Analysis 	7
3.5	Verification Schedule 	7
4.	Quality Assurance/Quality Control	9
4.1	QC for Reference Method	9
4.2	Audits	11
4.2.1	Performance Evaluation Audit	11
4.2.2	Technical Systems Audit	11
4.2.3	Audit of Data Quality	12
4.3	QA/QC Reporting	12
4.4	Data Review	12
5.	Statistical Methods	14
5.1	Accuracy 	14
5.2	Precision	15
5.3	Linearity	15
5.4	Method Detection Limit 	15
5.5	Matrix Interference Effects	16
v

-------
5.6	Operator Bias	16
5.7	Rate of False Positives/False Negatives	16
6.	Test Results	17
6.1	Accuracy 	17
6.2	Precision	24
6.3	Linearity	24
6.4	Method Detection Limit 	27
6.5	Matrix Interference Effects	27
6.6	Operator Bias	28
6.7	Rate of False Positives/False Negatives	28
6.8	Other Factors	31
6.8.1	Costs 	31
6.8.2	Data Completeness	31
7.	Performance Summary	32
8.	References	34
Figures
Figure 2-1. Industrial Test Systems, Inc., Quick™ Arsenic Test Kit	2
Figure 6-1. Comparison of Quick™ Test Kit to
Reference Method Results from PT Samples 	26
Tables
Table 3-1.	Test Samples for Verification of the Quick™ Test Kit	5
Table 3-2.	Schedule of Verification Test Days	8
Table 4-1.	Reference Method QCS Analysis Results	10
Table 4-2.	Reference Method LFMl Analysis Results	10
Table 4-3.	Reference Method Duplicate Analysis Results	11
Table 4-4.	Reference Method PE Audit Results	11
Table 4-5.	Summary of Data Recording Process 	13
Table 6-la.	Results from Laboratory Performance Test Sample Analyses 	18
vi

-------
Table 6-lb. Results from Drinking Water Analyses	19
Table 6-lc. Results from Freshwater Analyses 	20
Table 6-2a. Accuracy of the Quick™ Test Kit with Laboratory Performance
Test Samples	21
Table 6-2b. Accuracy of the Quick™ Test Kit with Drinking Water Samples	22
Table 6-2c. Accuracy of the Quick™ Test Kit with Freshwater Samples	23
Table 6-3. Summary of Qualitative Accuracy Results	23
Table 6-4a. Precision Results for Quick™ Test Kit from Laboratory Performance
Test Samples	25
Table 6-4b. Precision Results for Quick™ Test Kit from Drinking Water Samples	26
Table 6-5a. Results from Laboratory Performance Test Samples
with Low-Level Interferences	27
Table 6-5b. Results from Laboratory Performance Test Samples with
High-Level Interferences 	28
Table 6-6. Rate of False Positives from Quick™ Test Kit	29
Table 6-7. Rate of False Negatives from Quick™ Test Kit 	30
vii

-------
List of Abbreviations
AMS
Advanced Monitoring Systems
ASTM
American Society for Testing and Materials
DW
drinking fountain water
EPA
U.S. Environmental Protection Agency
ETV
Environmental Technology Verification
FW
freshwater
HDPE
high-density polyethylene
HI
high interference
ICPMS
inductively coupled plasma mass spectrometry
LBC
Little Beaver Creek
LC
Lytle Creek
LFM
laboratory-fortified matrix
LI
low interference
MDL
method detection limit
NIST
National Institute of Standards and Technology
ppb
parts per billion
ppm
parts per million
PE
performance evaluation
PT
performance test
QA
quality assurance
QA/QC
quality assurance/quality control
QC
quality control
QCS
quality control standard
QMP
Quality Management Plan
RB
reagent blank
RPD
relative percent difference
RSD
relative standard deviation
SR
Stillwater River
TSA
technical systems audit
TW
treated well water
WW
well water
Vlll

-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) has created the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the ETV
Program is to further environmental protection by substantially accelerating the acceptance and
use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing
high-quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative tech-
nologies by developing test plans that are responsive to the needs of stakeholders, conducting
field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance
(QA) protocols to ensure that data of known and adequate quality are generated and that the
results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
recently evaluated the performance of four portable analyzers for arsenic in water. This
verification report presents the procedures and results of the verification test for Industrial Test
Systems, Inc., Quick™ test kit arsenic analysis systems. The Quick™ test kit is an inexpensive,
portable, rapid device designed for on-site analysis of arsenic in water.
1

-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This verification report provides
results for the verification testing of the Quick™ test kit for arsenic in water. Following is a
description of the test kit, based on information provided by the vendor. The information
The Quick™ test kit can be used to test for total
arsenic in water. Up to 2.0 mg/L of hydrogen
sulfide is tolerated without test result interference,
and up to 5 parts per million (ppm) of antimony is
tolerated. The Quick™ test kit consists primarily of
two reaction bottles, two caps for holding the test
strip, three spoons, three bottles of reagent, and
one bottle of arsenic test strips in a waterproof,
plastic case. The three reagents are added
sequentially to the water sample and shaken. A test
strip is placed into the turret of the cap. The test
strip is exposed to arsine gas evolved from the
sample solution, resulting in a color change in the
test strip. When the reaction is complete, the test
strip is compared with a color chart provided with the kit. The intensity of the yellow/brown color
developed on the test strip relative to the color chart is proportional to the arsenic concentration
in the sample and, therefore, provides a semi-quantitative analysis of the arsenic concentration.
The color chart consists of the gradations: 0, 5, 10, 20, 40, 60, 100, 200, 300, and 500 parts per
billion (ppb). The kits are available in three sizes: for two tests, 50 tests, or 100 tests.
provided below was not verified in this test.
Figure 2-1. Industrial Test Systems, Inc.,
Quick™ Arsenic Test Kit
2

-------
Chapter 3
Test Design and Procedures
3.1 Introduction
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Portable Analyzers,(1) The verification was based on comparing the arsenic results
from the Quick™ test kit to those from a laboratory-based reference method. The reference
method for arsenic analysis was inductively coupled plasma mass spectrometry (ICPMS),
performed according to EPA Method 200.8(2) The Quick™ test kit does not require calibration,
but relies on comparisons to a color chart provided with the test kit to allow semi-quantitative
measurements of arsenic concentrations. The test kit was verified by analyzing laboratory-
prepared performance test samples, treated and untreated drinking water, and fresh surface water,
with both the test kit and the reference method.
3.2 Test Design
The Quick™ test kit was verified in terms of its performance on the following parameters:
#	Accuracy
#	Precision
#	Linearity
#	Method detection limit (MDL)
#	Matrix interference effects
#	Operator bias
#	Rate of false positives/false negatives.
All preparation and analyses were performed according to the manufacturer's recommended pro-
cedures. Results from the Quick™ test kit were recorded manually. The results from the Quick™
test kits were compared to those from the reference method to assess accuracy, linearity, and
detection limit. Multiple aliquots of performance test samples and drinking water samples were
analyzed to assess precision.
Identical sets of samples were analyzed independently by two separate operators (a technical and
a non-technical Battelle staff member). The technical operator was a research technician at
Battelle with three years of laboratory experience and a B.S degree. The non-technical operator
3

-------
was a part-time temporary helper at Battelle with a general education development certificate.
Because the reagents of the Quick™ test kits are consumed in use, it was not feasible for the two
operators to switch kits as a means of quantitatively assessing operator bias. However, each
operator used multiple kits in order to analyze all the samples, so it is assumed that kit-to-kit
variability was similar for both operators. Consequently, qualitative observations could be made
on operator bias.
Matrix interference effects were assessed by challenging the test kit with performance test
samples of known arsenic concentrations containing both low-level and high-level interferences.
False positives and negatives were evaluated relative to the recently established 10-ppb maximum
contaminant level for arsenic in drinking water. In addition to the analytical results, the time
required for sample analysis and operator observations concerning the use of the test kit (e.g.,
frequency of calibration, ease of use, maintenance) were recorded.
In a few instances, the test kit operator interpolated between the test kit gradations in reporting an
arsenic value. This is not unusual in use of such kits, and typically resulted in an arsenic reading
midway between two gradation values (e.g., 30 ppb, between gradations of 20 and 40 ppb).
3.3 Test Samples
Three types of samples were used in the verification test, as shown in Table 3-1: quality control
(QC) samples, performance test (PT) samples, and environmental water samples.
The QC and PT samples were prepared from National Institute of Standards and Technology
(NIST) traceable purchased standards. Under the Safe Drinking Water Act, the EPA lowered the
maximum contaminant level for arsenic from 50 ppb to 10 ppb, effective in January 2006.
Therefore, the QC sample concentrations for arsenic were targeted at that 10-ppb level. The PT
samples were targeted to range from 10% to 1,000% of that level, i.e., from 1 to 100 ppb. The
environmental water samples were collected from various drinking water and surface water
sources. All samples were analyzed using the Quick™ test kits and a reference method. Every
tenth sample was analyzed twice by the reference method to document the reference method's
precision.
3.3.1 QC Samples
As Table 3-1 indicates, prepared QC samples included laboratory reagent blanks (RB),
laboratory-fortified matrix (LFM) samples, and quality control samples. The RB samples
consisted of water collected from the same tap and were exposed to handling and analysis
procedures identical to the other prepared samples. These samples were used to help ensure that
no sources of contamination were introduced during sample handling and analysis. Two types of
LFMs were prepared. The LFMf samples consisted of aliquots of environmental samples that
were spiked in the field to increase the analyte concentration by 10 ppb of arsenic. These samples
were analyzed by the test kits in the field both before and after spiking. The spike solution used
4

-------
for the LFMf samples was prepared in the laboratory and brought to the field site. The LFMl
samples were aliquots of environmental samples that were spiked in the laboratory to increase the
analyte concentration by 25 ppb of arsenic. These samples were used to help identify whether
matrix effects influenced the reference method results. At least 10% of all the prepared samples
analyzed were RBs, and at least one sample taken from each sampling site was an LFMf.
Table 3-1. Test Samples3 for Verification of the Quick™ Test Kit
Type of Sample
Sample Characteristics
Concentration
No. of
Samples

Reagent Blank (RB)b
~0
10% of all

Laboratory Fortified Mixture (LFMp)b
10 ppb above native level
1 per site
Quality Control
LFMl13
25 ppb above native level
6

Quality Control Sample (QCS)b
10 ppb
10% of all

Prepared arsenic solution (PT6)
25 ppb
7

Prepared arsenic solution (PT1)
1 ppb
4

Prepared arsenic solution (PT2)
3 ppb
4

Prepared arsenic solution (PT3)
10 ppb
4
Performance Test
Prepared arsenic solution (PT4)
30 ppb
4

Prepared arsenic solution (PT5)
100 ppb
4

Prepared arsenic solution
10 ppb with low
Q

spiked with interference (LI)
interference
O

Prepared arsenic solution
10 ppb with high
Q

spiked with interference (HI)
interference
o

Columbus municipal drinking water
(DW)
Unknown
4

Well water (WW)
Unknown
4
Environmental
Treated well water (TW)
Unknown
4

Stillwater River (SR)
Unknown
4

Lytle Creek (LC)
Unknown
4

Little Beaver Creek (LBC)
Unknown
4
" Listing is for clarity; samples were analyzed in random order for the verification testing.
b See Section 3.3.1 for descriptions of these samples.
5

-------
Quality control standards (QCS) were used as calibration checks to verify that the Quick™ test
kit reference instrument was properly calibrated and reading within defined control limits. These
standards were purchased from a commercial supplier and were subject only to dilution as
appropriate. Calibration of the test kit and the reference instrument was verified using a QCS
before and after the testing period, as well as after every tenth sample. An additional independent
QCS was used in a performance evaluation (PE) audit of the reference method.
3.3.2	PT Samples
The two types of PT samples used in this verification test (Table 3-1) were prepared in the
laboratory using tap water as the water source. One type of PT solution contained arsenic at
various concentrations and was prepared specifically to determine Quick™ test kit accuracy,
linearity, and detection limit. To determine the detection limit of the Quick™, a solution with a
concentration five times the vendor's estimated detection limit was used. Seven non-consecutive
replicate analyses of this 25-ppb arsenic solution were made to obtain precision data with which
to estimate the MDL. Five other solutions were prepared to assess the linearity over a 1- to
100-ppb range of arsenic concentrations. Four aliquots of each of these solutions were pre-
pared and analyzed separately to assess the precision of the test kit, as well as the linearity.
The second type of PT sample was used to assess the effects of matrix interferences on the per-
formance of the Quick™ test kit. These samples were solutions with known concentrations of
arsenic spiked with potentially interfering species likely to be found in typical water samples. One
sample (designated LI) contained low levels of interferences that consisted of 1 ppm of iron,
3 ppm of sodium chloride, and 0.1 ppm of sulfide per liter at a pH of 6. The second sample
(designated HI) contained high levels of interferences that consisted of 10 ppm of iron, 30 ppm of
sodium chloride, and 1.0 ppm of sulfide per liter at a pH of 3. Eight replicate samples of each of
these solutions were analyzed.
3.3.3	Environmental Samples
Drinking water samples listed in Table 3-1 include Columbus municipal water collected from a
Battelle drinking fountain (DW), well water (WW), and treated well water (TW) from a school
near Columbus, Ohio. The WW was pumped from a 250-foot well and collected directly from an
existing spigot with no purging. The TW was treated by running the WW through a Greensand
filtration system in the basement of the school. These samples were collected directly from the
tap into 2-L high-density polyethylene (HDPE) containers. Four aliquots of each sample were
analyzed in the field at the time of collection by each set of the test kits being verified. One
aliquot of each sample was preserved with nitric acid and returned to Battelle for reference
analysis. The remaining collected sample was stored at 4°C for later use, if necessary.
Freshwater (FW) samples from the Stillwater River (SR), Lytle Creek (LC), and the Little Beaver
Creek (LBC) (in Ohio) were collected in 2-L HDPE containers. The samples were collected near
the shoreline by submerging the containers no more than one inch below the surface of the water.
Each body of water was sampled at four distinct locations. An aliquot of each sample was
6

-------
analyzed in the field at the time of collection by each set of each test kit being verified. One
aliquot of each sample was preserved with nitric acid and returned to Battelle for reference
analysis. The remaining collected sample was preserved and stored at 4°C for later use, if
necessary.
3.4	Reference Analysis
The reference arsenic analysis was performed using a Perkin Elmer Sciex Elan 6000 ICPMS
according to EPA Method 200.8, Revision 5.5.(2) The sample was introduced through a peristaltic
pump by pneumatic nebulization into a radiofrequency plasma where energy transfer processes
cause desolvation, atomization, and ionization. The ions were extracted from the plasma through
a pumped vacuum interface and separated on the basis of their mass-to-charge ratio by a
quadrupole mass spectrometer. The ions transmitted through the quadrupole were registered by a
continuous dynode electron multiplier, and the ion information was processed by a data handling
system.
The ICPMS was tuned, optimized, and calibrated daily. The calibration was performed using a
minimum of five calibration standards at concentrations ranging between 0.1 and 250 ppb, and a
required correlation coefficient of a minimum of 0.999. Internal standards were used to correct
for instrument drift and physical interferences. These standards were introduced in line through
the peristaltic pump and analyzed with all blanks, standards, and samples.
3.5	Verification Schedule
The verification test took place over a 19-day period from October 25 to November 12, 2001. The
environmental samples were collected and analyzed over the seven-day period from November 2
through November 8, 2001. Table 3-2 shows the daily testing activities that were conducted
during these periods. In all field locations, the samples were analyzed shortly after collection
using the Quick™ test kit by both the technical and the non-technical Battelle staff member. The
reference analyses on all samples were performed on December 21, 2001, approximately six
weeks after sample collection.
7

-------
Table 3-2. Schedule of Verification Test Days
Test Day	Testing Location	Activity
10/25-11/12/01
Battelle
Preparation and analysis of PT and associated QC
samples.
10/25/01
Battelle
Collection and analysis of DW and associated QC
samples within Battelle.
11/02/01
Ohio Field Location
Collection and analysis of WW samples, TW samples,
and associated QC samples at Licking Valley Middle
School.
11/06/01
Ohio Field Location
Collection and analysis of environmental and
associated QC samples at four locations on Little
Beaver Creek.
11/07/01
Ohio Field Location
Collection and analysis of environmental and
associated QC samples at four locations on Lytle
Creek.
11/08/01
Ohio Field Location
Collection and analysis of environmental and
associated QC samples at four locations on the
Stillwater River.
8

-------
Chapter 4
Quality Assurance/Quality Control
Quality assurance/quality control (QA/QC) procedures were performed in accordance with the
quality management plan (QMP) for the AMS Center(3) and the test/QA plan for this verification
test.(1)
4.1 QC for Reference Method
Field and laboratory RB samples were analyzed to ensure that no sources of contamination were
present. The test/QA plan stated that if the analysis of an RB sample indicated a concentration
above the MDL for the reference instrument, any contamination source was to be corrected and
proper blank readings achieved before proceeding with the verification test. A total of three field
RB and one laboratory RB were analyzed. All of the blanks analyzed were below the 0.1-ppb
reference MDL for arsenic.
The instrument used for the reference method was initially calibrated using 11 calibration
standards, with concentrations ranging between 0.1 and 250 ppb of arsenic. The accuracy of the
calibration also was verified after the analysis of every 10 samples by analyzing a 25-ppb QCS. If
the QCS analysis differed by more than ±10% from the true value of the standard, the instrument
was recalibrated before continuing the test. As shown in Table 4-1, the QCS analyses were
always within this required range. The maximum bias from the standard in any QCs analysis was
6.04%.
LFMl samples were analyzed to assess whether matrix effects influenced the results of the
reference method. The percent recovery (R) of these LFMl samples was calculated from the
following equation:
C - C
R = 	x 100	(1)
S
where Cs is the analyzed concentration of the spiked sample, C is the analyzed concentration of
the unspiked sample, and 5 is the concentration equivalent of the analyte spike. If the percent
recovery of an LFMl fell outside the range from 85 to 115%, a matrix effect was suspected. As
shown in Table 4-2, all of the LFMl sample results were well within this range, so no matrix
effect on the reference analyses is inferred.
9

-------
Table 4-1. Reference Method QCS Analysis Results
Measured Actual
Sample ID Date of Analysis Arsenic (ppb)	Arsenic (ppb)	Percent Bias
QCS 12/21/2001 24.1 25.0 3.56%
QCS 12/21/2001 23.5 25.0 6.04%
QCS 12/21/2001 23.8 25.0 4.64%
QCS 12/21/2001 23.9 25.0 4.32%
QCS	12/21/2001	244	25_0	2.52%
Table 4-2. Reference Method LFMl Analysis Results
Unspiked Sample Spiked Sample Spiked Amount
LFMl
Date of
Arsenic
Arsenic
Arsenic
Percent
Sample ID
Analysis
(ppb)
(ppb)
(ppb)
Recovery
Laboratory RB
12/21/01
<0.1
23.8
25.0
95.3%
Field QCS
12/21/01
10.9
35.7
25.0
99.0%
DW LFMf
12/21/01
10.6a
34.6
25.0
96.2%
LBC 3 Duplicate
12/21/01
2.26
26.6
25.0
97.5%
LC 4
12/21/01
1.37
26.3
25.0
99.7%
SR4
12/21/01
1.88
26.4
25.0
98.0%
1 Amount of arsenic in the sample after it was spiked in the field.
Duplicate samples were analyzed to assess the precision of the reference analysis. The relative
percent difference (RPD) of the duplicate sample analysis was calculated from the following
equation:
RPD =
(C-CD)
(C + CD)/ 2
: 100
(2)
Where C is the concentration of the sample analysis, and CD is the concentration of the sample
duplicate analysis. If the RPD was greater than 10%, the instrument was recalibrated before
continuing the test. As shown in Table 4-3, the RPDs for the duplicate analysis were all less than
10%. The maximum RPD in any duplicate analysis was 4%.
10

-------
Table 4-3. Reference Method Duplicate Analysis Results
Sample ID
Date of Analysis
Sample Arsenic
(ppb)
Duplicate
Sample Arsenic
(PPb)
RPD
PT QCS
12/21/2001
9.80
9.81
0%
PT1 (tap)
12/21/2001
1.76
1.76
0%
WW 1
12/21/2001
86.6
86.1
1%
LBC4
12/21/2001
2.54
2.44
4%
SR QCS
12/21/2001
9.33
9.37
0%
4.2 Audits
4.2.1 Performance Evaluation Audit
A PE audit was conducted to assess the quality of the reference measurements made in this
verification test. For the PE audit, an independent, NIST-traceable, reference material was
obtained from a different commercial supplier than the calibration standards and the field QCS.
The PE standard was prepared from Claritas PPT™ Grade Standard purchased through SPEX
CertiPrep. Accuracy of the reference method was determined by comparing the measured arsenic
concentration using the verification test standards to those obtained using the independently
certified PE standard. Percent difference was used to quantify the accuracy of the results.
Agreement of the standard within 10% was required for the measurements to be considered
acceptable. Failure to achieve this agreement would have triggered recalibration of the reference
instrument with the original QC standards and a repeat of the PE comparison. As shown in Table
4-4, the PE sample analysis was well within this required range.
Table 4-4. Reference Method PE Audit Results


Measured
Actual Concentration


Date of
Arsenic
Arsenic
Percent
Sample ID
Analysis
(ppb)
(ppb)
Agreement
PE-1
12/21/01
23.7
25.0
5.2%
4.2.2 Technical Systems Audit
The Battelle Quality Manager conducted a technical systems audit (TSA) between October 22
and December 21, 2001, to ensure that the verification test was being performed in accordance
with the test/QA plan(1) and the AMS Center QMP.(3) The standard solution preparation and PT
sample preparation were observed on October 22, the environmental testing (drinking water) on
11

-------
October 25, the testing with PT samples on October 26, and the reference method performance
on December 21. As part of the audit, the reference standards and method used were reviewed,
actual test procedures were compared to those specified in the test/QA plan, and data acquisition
and handling procedures were reviewed. Observations and findings from this audit were docu-
mented and submitted to the Verification Test Coordinator for response. No findings were
documented that required any corrective action. The records concerning the TSA are
permanently stored with the Battelle Quality Manager.
4.2.3 Audit of Data Quality
At least 10% of the data acquired during the verification test was audited. Battelle's Quality
Manager traced the data from the initial acquisition, through reduction and statistical analysis, to
final reporting, to ensure the integrity of the reported results. All calculations performed on the
data undergoing the audit were checked.
4.3 QA/QC Reporting
Each assessment and audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the
QMP for the ETV AMS Center.(3) Once the assessment report was prepared, the Verification Test
Coordinator ensured that a response was provided for each adverse finding or potential problem
and implemented any necessary follow-up corrective action. The Battelle Quality Manager
ensured that follow-up corrective action was taken. The results of the TSA and the audit of data
quality were sent to the EPA.
4.4 Data Review
Records generated in the verification test received a one-over-one review within two weeks of
generation before these records were used to calculate, evaluate, or report verification results.
Table 4-5 summarizes the types of data recorded. The review was performed by a Battelle
technical staff member involved in the verification test, but not the staff member that originally
generated the record. The person performing the review added his/her initials and the date to a
hard copy of the record being reviewed.
12

-------
Table 4-5. Summary of Data Recording Process
Data to be Responsible	Where	How Often	Disposition of
Recorded	Party	Recorded	Recorded	Data3
Dates, times of
test events
Battelle
Laboratory
record books
or ETV field
data sheets
Start/end of test
event
Used to
organize/check test
results; manually
incorporated in data
spreadsheets as
necessary
Test parameters
(temperature,
analyte/
interferant
identities, and
Quick™ test kit
results)
Battelle
Laboratory
record books
or ETV field
data sheets
When set or
changed, or as
needed to document
test
Used to
organize/check test
results, manually
incorporated in data
spreadsheets as
necessary
Reference method
sample analysis,
chain of custody,
and results
Battelle
Laboratory
record books,
data sheets, or
data
acquisition
system, as
appropriate
Throughout sample
handling and analysis
process
Transferred to
spreadsheets
1 All activities subsequent to data recording are carried out by Battelle.
13

-------
Chapter 5
Statistical Methods
The statistical methods presented in this chapter were planned for verifying the performance
factors listed in Section 3.2. In a few cases, qualitative comparisons are reported.
5.1 Accuracy
When possible, accuracy was assessed relative to the results obtained from the reference
analyses. Samples were analyzed by both the reference method and the test kit being verified. For
each sample, accuracy was expressed in terms of a relative bias (B) as calculated from the
following equation:
x 100	(3)
where d is the difference between the reading from the Quick™ test kit and that from the
reference method, and CR is the reference measurement.
Because of the semi-quantitative nature of the visual test kit results, it was not possible to make
this determination for many of the results. For this reason, all of the data also were judged by a
qualitative measure that was not specified in the test/QA plan. If the result from the test kit agreed
within 25% of the reference result, the measurement was considered accurate; if it did not, the
measurement was considered not to be accurate. The percentage of accurate measurements was
determined for each of the three types of water samples as calculated from the following
equation:
Y
A = — x 100	(4)
T
where A is the percent of accurate measurements, 7is the number of measurements within the
±25% criterion, and T is the total number of measurements. The criterion of 25% for agreement
was based on the measurement resolution of the several portable arsenic analyzers tested and on
scientific judgment of the required degree of accuracy for these analyzers. Readings below the
B
a
14

-------
detection limit (e.g., <10 ppb) were judged to be in agreement with the reference result if the
reference value was in the specified "less than" range.
5.2 Precision
When possible, the standard deviation (S) of the results for the replicate samples was calculated
and used as a measure of Quick™ test kit precision at each concentration.
S =
k=l
1/2
(5)
where n is the number of replicate samples, Ck is the concentration measured for the kth sample,
and C is the average concentration of the replicate samples. The instrumental precision at each
concentration was reported in terms of the relative standard deviation (RSD), e.g.,
C
RSD
x 100
(6)
5.3 Linearity
Linearity was assessed by linear regression of Quick™ test kit results against the reference
results, with linearity characterized by the slope, intercept, and correlation coefficient (r).
Linearity was tested using PT samples over the range 1 to 100 ppb of arsenic. The color chart for
the Quick™ test kit has a range of concentration from 5 to 500 ppb. If the concentration of
arsenic for any sample is greater than 500 ppb, a smaller sample size can be used to extend the
linearity beyond 500 ppb.
5.4 Method Detection Limit
The MDL for the Quick™ test kit was assessed from the seven replicate analyses of a fortified
sample with an analyte concentration of 25 ppb, i.e., five times the manufacturer's estimated
detection limit of 5 ppb. The MDL was calculated from the following equation:
MDL = t x S	(7)
where t (= 3.14) is the Student's t value for a 99% confidence level with n = 7, and S is the
standard deviation of the replicate samples.(4)
15

-------
5.5 Matrix Interference Effects
The effect of interfering matrix species on the response of the Quick™ test kit to arsenic is
typically calculated as the ratio of the difference in analytical response to the concentration of
interfering species. For example, if adding 500 ppb of an interfering species results in a difference
of 10 ppb in the analytical result, the relative sensitivity of the test kit to that interferant would be
calculated as 10 ppb/500 ppb = 2%. In this test, three interfering species were added to the
samples, all at either low or high concentrations (Section 3.3.2). Thus, it is not possible to
determine which of these compounds would be responsible for any observed interferences. Only
qualitative observations could be made assessing whether there was a positive or negative effect
due to matrix interferences.
5.6	Operator Bias
To assess operator bias for the Quick™ test kit, in all tests the results obtained from each operator
were compiled independently and subsequently compared. However, because of the semi-
quantitative nature of the test kit data and the inability of the operators to independently use the
same test kits, quantitative assessments of operator bias could not be made. Qualitative
observations were made concerning the results from the two operators.
5.7	Rate of False Positives/False Negatives
The rates of false positives and false negatives of the Quick™ test kit were assessed relative to the
10-ppb target arsenic level. A false positive result is defined as any result reported to be equal to
or greater than the guidance level (10 ppb) and greater than 125% of the reference value, when
the reference value is less than that guidance level. Similarly, a false negative result is defined as
any result reported below the guidance level and less than 75% of the reference value, when the
reference value is greater than that guidance level. The rates of false positives and false negatives
were expressed as a percentage of total samples analyzed for each type of sample.
16

-------
Chapter 6
Test Results
The results of the verification test of the Quick™ test kits are presented in this section.
6.1 Accuracy
Tables 6-la-c present the measured arsenic results from analysis of the PT, drinking water, and
FW samples, respectively. Both reference analyses and Quick™ test kit results are shown in the
tables, and Quick™ test kit results are shown for both the technical and non-technical operators.
Some Quick™ test kit results could not be distinguished from blank sample results and were
assigned a value of <5 ppb.
The field spike results indicate apparent inconsistencies in some of the spike concentrations. The
WW LFMf and LBC-4 LFMf samples apparently were not spiked in the field and the TW LFMf
sample may have been spiked twice. However, these spiking errors have no effect on the
usefulness of the data.
Tables 6-2a-c show the percent accuracy of the Quick™ test kit results. Shown in the second and
third columns in each of Tables 6-2a-c are the percent bias values determined according to
Equation 3, in Section 5.1. Bias was not calculated for values reported as <5 ppb. The percent
bias values that are shown in Tables 6-2a-c range from 8 to 83% for the non-technical operator
and 8 to 84% for the technical operator for the PT samples, 8 to 92% for the non-technical
operator and 8 to 54% for the technical operator for the drinking water samples, and 2 to 320%
for both the non-technical operator and for the technical operator for the FW samples. In general,
the larger bias values were associated with lower arsenic concentrations.
In addition to the quantitative bias results, the qualitative accuracy was compared using
Equation 4 in Section 5.1. The fourth and fifth columns in Tables 6-2a-c show the assignment of
each Quick™ test kit result, in terms of whether that result fell within 25% of the reference value,
or within a corresponding "less-than" range. The results of this qualitative evaluation of accuracy
are shown in Table 6-3, which lists the overall percent of results meeting the qualitative accuracy
criteria for each operator and sample type. Table 6-3 shows that the qualitative accuracy for the
Quick™ test kit for the PT samples was 71% for the non-technical operator and 55% for the
technical operator. The qualitative accuracy for the drinking water samples was 57% for the non-
technical operator and 52% for the technical operator. The qualitative accuracy for the FW
17

-------
Table 6-la. Results from Laboratory Performance Test Sample Analyses

Non-Technical
Technical
Reference Method3
Sample
Arsenic (ppb)
Arsenic (ppb)
Arsenic (ppb)
Laboratory RB
<5
<5
<0.1
Laboratory RB
NA
<5
<0.1
Laboratory RB
NA
<5
<0.1
Laboratory RB
NA
<5
<0.1
Laboratory RB
NA
<5
<0.1
Laboratory RB
NA
<5
<0.1
QCS
10
20
10.9
QCS
10
20
10.9
QCS
10
20
10.9
QCS
NA
20
10.9
QCS
NA
20
10.9
PT1-1
<5
<5
1.76
PT1-2
<5
<5
1.76
PT1-3
<5
<5
1.76
PT1-4
<5
<5
1.76
PT2-1
<5
5
3.97
PT2-2
<5
5
3.97
PT2-3
<5
5
3.97
PT2-4
<5
5
3.97
PT3-1
5
10
10.9
PT3-2
10
10
10.9
PT3-3
10
10
10.9
PT3-4
10
10
10.9
PT4-1
30
40
34.8
PT4-2
30
40
34.8
PT4-3
20
40
34.8
PT4-4
30
20
34.8
PT5-1
100
100
113
PT5-2
100
100
113
PT5-3
100
100
113
PT5-4
100
100
113
PT6-1
5
20
29.6
PT6-2
5
20
29.6
PT6-3
10
20
29.6
PT6-4
10
20
29.6
PT6-5
10
20
29.6
PT6-6
10
20
29.6
PT6-7
20
20
29.6
" Only one aliquot of each sample was analyzed by the reference method (except for the laboratory RB). Multiple
aliquots of each sample were analyzed by Quick™ test kit.
NA: Not analyzed.
18

-------
Table 6-lb. Results from Drinking Water Analyses

Non-Technical
Technical
Reference Method3
Sample
Arsenic (ppb)
Arsenic (ppb)
Arsenic (ppb)
Laboratory RB
<5
<5
<0.1
QCS
10
10
10.9
DW-1
<5
<5
0.87
DW-2
<5
<5
0.87
DW-3
<5
<5
0.87
DW-4
<5
<5
0.87
DWLFMp
5
5
10.6
Laboratory RB
<5
<5
<0.1
QCS
10
10
10.9
WW-1
100
60
86.6
WW-2
60
60
86.6
WW-3
60
40
86.6
WW-4
60
60
86.6
WWLFMp
70
60
82.1
Laboratory RB
<5
<5
<0.1
QCS
5
10
10.9
TW-1
10
40
26.0
TW-2
10
40
26.0
TW-3
10
40
26.0
TW-4
50
40
26.0
TWLFMp
40
60
50.8
" Only one aliquot of each sample was analyzed by the reference method. Multiple aliquots of each sample were analyzed
by Quick™ test kit.
19

-------
Table 6-lc. Results from Freshwater Analyses

Non-Technical
Technical
Reference Method a
Sample
Arsenic (ppb)
Arsenic (ppb)
Arsenic (ppb)
Laboratory RB
<5
<5
<0.1
QCS
10
10
9.33
SR-1
<5
<5
1.73
SR-2
<5
5
1.72
SR-2 Duplicate
<5
5
1.71
SR-3
<5
<5
2.03
SR-4
<5
5
1.88
SR-1 LFMp
10
20
11.6
Laboratory RB
<5
<5
<0.1
QCS
10
10
9.43
LC-1
<5
5
2.13
LC-2
<5
5
1.30
LC-3
<5
5
1.44
LC-4
<5
<5
1.37
LC-4 Duplicate
<5
5
1.36
LC-3 LFMp
10
10
12.0
Laboratory RB
<5
<5
<0.1
QCS
10
10
9.81
LBC-1
<5
5
2.48
LBC-2
<5
5
2.60
LBC-3
<5
<5
2.14
LBC-3 Duplicate
<5
<5
2.26
LBC-4
<5
<5
2.54
LBC-4 LFM,
10
10
2.38
20

-------
Table 6-2a. Accuracy of the Quick™ Test Kit with Laboratory Performance Test Samples
Sample
Bias"
Non-Technical
Bias3
Technical
Within Range (Y/N)b
Non-Technical
Within Range (Y/N)b
Technical
Laboratory RB
c
-
Y
Y
Laboratory RB
NA
-

Y
Laboratory RB
NA
-

Y
Laboratory RB
NA
-

Y
Laboratory RB
NA
-

Y
Laboratory RB
NA
-

Y
QCS
8%
84%
Y
N
QCS
8%
84%
Y
N
QCS
8%
84%
Y
N
QCS
NA
84%

N
QCS
NA
84%

N
PT1-1
-
-
Y
Y
PT1-2
-
-
Y
Y
PT1-3
-
-
Y
Y
PT1-4
-
-
Y
Y
PT2-1
-
26%
Y
N
PT2-2
-
26%
Y
N
PT2-3
-
26%
Y
N
PT2-4
-
26%
Y
N
PT3-1
54%
8%
N
Y
PT3-2
8%
8%
Y
Y
PT3-3
8%
8%
Y
Y
PT3-4
8%
8%
Y
Y
PT4-1
14%
15%
Y
Y
PT4-2
14%
15%
Y
Y
PT4-3
43%
15%
N
Y
PT4-4
14%
43%
Y
N
PT5-1
12%
12%
Y
Y
PT5-2
12%
12%
Y
Y
PT5-3
12%
12%
Y
Y
PT5-4
12%
12%
Y
Y
PT6-1
83%
32%
N
N
PT6-2
83%
32%
N
N
PT6-3
66%
32%
N
N
PT6-4
66%
32%
N
N
PT6-5
66%
32%
N
N
PT6-6
66%
32%
N
N
PT6-7
32%
32%
N
N
s Percent bias calculated according to Equation 3, Section 5.1.
b Y = result within ±25% of reference, or reference value within < range; N = result not within ±25% of reference, or reference
value not within < range.
c Non-detect, no calculation of bias can be made.
NA: not analyzed.
21

-------
Table 6-2b. Accuracy of the Quick™ Test Kit with Drinking Water Samples
Sample
Bias3
Non-Technical
Bias3
Technical
Within Range
(Y/N)b
Non-Technical
Within Range
(Y/N)b
Technical
Laboratory RB
_C
-
Y
Y
QCS
8%
8%
Y
Y
DW-1
-
-
Y
Y
DW-2
-
-
Y
Y
DW-3
-
-
Y
Y
DW-4
-
-
Y
Y
DW LFM,
53%
53%
N
N
Laboratory RB
-
-
Y
Y
QCS
8%
8%
Y
Y
WW-1
15%
31%
Y
N
WW-2
31%
31%
N
N
WW-3
31%
54%
N
N
WW-4
31%
31%
N
N
WWLFMp
15%
27%
Y
N
Laboratory RB
-
-
Y
Y
QCS
54%
8%
N
Y
TW-1
62%
54%
N
N
TW-2
62%
54%
N
N
TW-3
62%
54%
N
N
TW-4
92%
54%
N
N
TWLFMp
21%
18%
Y
Y
s Percent bias calculated according to Equation 3, Section 5.1.
b Y = result within ±25% of reference, or reference value within < range; N = result not within ±25% of reference, or reference
value not within < range.
c Non-detect, no calculation of bias can be made.
22

-------
Table 6-2c. Accuracy of the Quick™ Test Kit with Freshwater Samples
Sample
Bias"
Non-Technical
Bias3
Technical
Within Range
(Y/N)b
Non-Technical
Within Range
(Y/N)b
Technical
Laboratory RB
_C
-
Y
Y
QCS
7%
7%
Y
Y
SR-1
-
-
Y
Y
SR-2
-
191%
Y
N
SR-2 Duplicate
-
192%
Y
N
SR-3
-
-
Y
Y
SR-4
-
166%
Y
N
SR-1 LFMp
14%
72%
Y
N
Laboratory RB
-
-
Y
Y
QCS
6%
6%
Y
Y
LC-1
-
135%
Y
N
LC-2
-
285%
Y
N
LC-3
-
247%
Y
N
LC-4
-
-
Y
Y
LC-4 Duplicate
-
268%
Y
N
LC3 LFMp
17%
17%
Y
Y
Laboratory RB
-
-
Y
Y
QCS
2%
2%
Y
Y
LBC-1
-
102%
Y
N
LBC-2
-
92%
Y
N
LBC-3
-
-
Y
Y
LBC-3 Duplicate
-
-
Y
Y
LBC-4
-
-
Y
Y
LBC-4 LFM,
320%
320%
N
N
1 Percent bias calculated according to Equation 3, Section 5.1.
b Y = result within ±25% of reference, or reference value within < range; N = result not within ±25% of reference, or reference
value not within < range.
c No calculation of bias can be made.
Table 6-3. Summary of Qualitative Accuracy Results

Percent Accurate
Percent Accurate

Within 25%
Within 25%

(Non-Technical Operator)
(Technical Operator)
Laboratory performance test samples
71%
55%
Drinking water samples
57%
52%
Freshwater samples
96%
54%
23

-------
samples was 96% for the non-technical operator and 54% for the technical operator. Many of the
Quick™ results judged qualitatively accurate were the result of sample concentrations below the
manufacturer's estimated detection limit of 5 ppb. For the 25 samples in Tables 6-la and b with
reference arsenic values between 26 and 113 ppb, the qualitative accuracy was 40% or less with
both operators.
6.2 Precision
Tables 6-4a and b, respectively, show the data used to evaluate the RSD of the Quick™ test kit
results for the replicate laboratory PT and drinking water samples, along with the percent RSD
value for each set of replicate analysis. The percent RSD was determined according to Equation 6
in Section 5.2. Percent RSD was not calculated if all of the results for a set of replicates were <5
ppb. These data sets illustrate the consistency in the Quick™ test kit replicate analyses. Seven of
the 14 replicate sets for the PT and QCS samples (Table 6-4a) showed an RSD of 0%. The results
for three of the replicate sets were <5 ppb. The remaining replicate sets for the non-technical
operator had an RSD ranging from 29 to 50%, and the remaining replicate set for the technical
operator had an RSD of 29%. For the drinking water samples (Table 6-4b), all results for two of
the replicate sets were <5 ppb. The remaining sets had an RSD of 29 to 100% for the non-
technical operator and 0 to 18% for the technical operator.
6.3 Linearity
The linearity of the Quick™ test kit readings was assessed by means of a linear regression of the
Quick™ test kit results against the reference method results, using the 27 data points from the PT
samples (Table 6-1 a). In this regression, results reported as below detection limit by the Quick™
test kit were assigned a value of half the detection limit (2.5 ppb). Figure 6-1 shows a scatter plot
of the Quick™ test kit data from both non-technical and the technical operators versus the
reference method results. The one-to-one line is also shown in Figure 6-1.
A linear regression of the data in Figure 6-1 gives the following regression equations:
with the Quick™ test kit for the non-technical operator,
ppb = 0.90 (±0.086) x (reference, ppb) - 5.2 (±4.1) ppb,
with r = 0.974, and
with the Quick™ test kit for the technical operator,
ppb = 0.88 (±0.056) x (reference, ppb) - 0.45 (±2.7) ppb,
with r = 0.988
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
Both regressions show slopes that are significantly different from 1.0.
24

-------
Table 6-4a. Precision Results for Quick™ Test Kit from Laboratory Performance Test
Samples

Reference
Non-Technical3
Technical
Concentration (ppb)
Arsenic (ppb)
Arsenic (ppb)
QCS
10.9
10
20
QCS

10
20
QCS

10
20
QCS


20
QCS


20
%RSD

0
0
PT1-1
1.76
<5
<5
PT1-2

<5
<5
PT1-3

<5
<5
PT1-4

<5
<5
%RSD

_b
_b
PT2-1
3.97
<5
5
PT2-2

<5
5
PT2-3

<5
5
PT2-4

<5
5
%RSD

_b
0
PT3-1
10.9
5
10
PT3-2

10
10
PT3-3

10
10
PT3-4

10
10
%RSD

29
0
PT4-1
34.8
30
40
PT4-2

30
40
PT4-3

20
40
PT4-4

30
20
%RSD

29
29
PT5-1
113
100
100
PT5-2

100
100
PT5-3

100
100
PT5-4

100
100
%RSD

0
0
PT6-1
29.6
5
20
PT6-2

5
20
PT6-3

10
20
PT6-4

10
20
PT6-5

10
20
PT6-6

10
20
PT6-7

20
20
%RSD

50
0
s For the purpose of calculating %RSD, all "less than" values are considered zero.
b No %RSD could be calculated.
25

-------
Table 6-4b. Precision Results for Quick™ Test Kit from Drinking Water Samples

Reference
Non-Technical3
Technical3
Concentration (ppb)
Arsenic (ppb)
Arsenic (ppb)
DW-1
0.87
<5
<5
DW-2

<5
<5
DW-3

<5
<5
DW-4

<5
<5
% RSD

_b
_b
WW-1
86.6
100
60
WW-2

60
60
WW-3

60
40
WW-4

60
60
% RSD

29
18
TW-1
26.0
10
40
TW-2

10
40
TW-3

10
40
TW-4

50
40
%RSD

100
0
" For the purpose of calculating %RSD, all "less than" values are considered zero.
b No %RSD could be calculated.
« Non-Technical Arsenic, ppb
¦ Technical Arsenic, ppb
	Linear (Non-Technical Arsenic, ppb)
	Linear (Technical Arsenic, ppb)
	Linear (1:1 Line)
140
120
100
¦Q
Q.
Q.
O
*3
a
o
20
40
60
80
100
120
140
Reference Arsenic, ppb
Figure 6-1. Comparison of Quick™ Test Kit to Reference Method Results from
PT Samples
26

-------
6.4	Method Detection Limit
The manufacturer's estimated detection limit for the Quick™ test kit is 5 ppb. An attempt was
made to determine the MDL by analyzing seven replicate samples at approximately 25 ppb (PT6
samples, Table 6-la). The Quick™ results for both operators were all less than the reference
value, but in particular the technical operator's results were all identical (20 ppb), providing no
variation with which to quantitatively assess the MDL.(4) The non-technical operator reported
arsenic between 5 and 20 ppb. Since the Quick ™ test kit is only semi-quantitative, no MDL was
calculated from these data. Qualitative indication of the Quick™ test kit MDL can be obtained
from the results of the PT2 and PT3 samples (Table 6-la) of concentrations 3.97 and 10.9 ppb,
respectively. With the 3.97-ppb samples, the non-technical operator reported results of <5 ppb,
whereas the technical operator reported results of 5 ppb. With the 10.9-ppb samples, all Quick™
results were 10 ppb except for one result of 5 ppb with the non-technical operator.
6.5	Matrix Interference Effects
Tables 6-5a and b show the analytical results from laboratory performance test samples
containing about 10.5 ppb arsenic, with low and high levels of interference, respectively. The
Quick™ test kit produced positive readings on all the matrix interference samples with both
operators, with a small increase in readings with the higher interference levels. For example, the
non-technical operator reported 10 ppb in five of eight analyses of the LI samples, with three
readings of 5 ppb, and reported 10 ppb in seven of eight analyses, with only one reading of 5 ppb.
for the HI samples. Similarly, the technical operator reported six of eight values at 10 ppb and
two at 5 ppb with the LI samples, but five of eight at 10 ppb and three at 20 ppb with the HI
samples. These results indicate a minor tendency toward higher readings (3 ppb on average) from
the Quick™ test kit at the higher interference levels. Because of the study design, it was not
possible to determine which ion was responsible for the observed result.
Table 6-5a. Results from Laboratory Performance Test Samples with Low-Level
Interferences

Non-Technical
Arsenic (ppb)
Technical
Arsenic (ppb)
LI-1
10
5
LI-2
5
10
LI-3
5
10
LI-4
10
10
LI-5
10
10
LI-6
10
5
LI-7
5
10
LI-8
10
10
" Only one aliquot of LI solution was analyzed by the reference method. Eight aliquots of LI solution were analyzed by
Quick™ test kits.
27

-------
Table 6-5b. Results from Laboratory Performance Test Samples with High-Level
Interferences

Non-Technical
Arsenic (ppb)
Technical
Arsenic (ppb)
HI-1
10
10
HI-2
5
10
HI-3
10
10
HI-4
10
20
HI-5
10
10
HI-6
10
20
HI-7
10
10
HI-8
10
20
" Only one aliquot of HI solution was analyzed by the reference method. Eight aliquots of HI solution were analyzed
by Quick™ test kits.
6.6	Operator Bias
The effect of operator skill level does not appear to be a major factor with the Quick™ test kit.
The non-technical operator had a higher percentage of accurate results, although the greater
frequency of non-detects with the non-technical operator played a part in that outcome. On the
other hand, the technical operator had fewer false positive and negative results (see Section 6.7).
6.7	Rate of False Positives/False Negatives
Tables 6-6 and 6-7, respectively, show the data and results for the rates of false positives and false
negatives obtained from the Quick™ test kit. All PT and environmental samples (Table 3-1) were
considered for this evaluation.
Table 6-6 shows that 24 samples had reference arsenic concentrations less than the target decision
level of 10 ppb. Of the samples tested by the non-technical operator, in only one sample did the
Quick™ test kit results indicate a concentration of 10 ppb or higher. The result was a false
positive rate of 4% relative to the 10 ppb value. The samples tested by the technical operator had
a false positive rate of 0%, with no Quick™ test kit results at or above the 10-ppb decision level.
Table 6-7 shows that 43 samples had reference arsenic concentrations greater than the target
decision level of 10 ppb. In seven of the 43 samples, the analyte was detected at a level less than
10 ppb by the non-technical operator (i.e., a false negative rate of 16%). The technical operator
reported only two such results, for a false negative rate of 5%.
28

-------
Table 6-6 Rate of False Positives from Quick™ Test Kit
Non-Technical
Technical
Reference
Non-Technical
Technical

Arsenic
Arsenic
Method
False Positive
False Positive

(PPb)
(PPb)
Arsenic (ppb)
(Y/N)
(Y/N)
PT1-1
<5
<5
1.76
N
N
PT1-2
<5
<5
1.76
N
N
PT1-3
<5
<5
1.76
N
N
PT1-4
<5
<5
1.76
N
N
PT2-1
<5
5
3.97
N
N
PT2-2
<5
5
3.97
N
N
PT2-3
<5
5
3.97
N
N
PT2-4
<5
5
3.97
N
N
DW-1
<5
<5
0.87
N
N
DW-2
<5
<5
0.87
N
N
DW-3
<5
<5
0.87
N
N
DW-4
<5
<5
0.87
N
N
SR-1
<5
<5
1.73
N
N
SR-2
<5
5
1.72
N
N
SR-3
<5
<5
2.03
N
N
SR-4
<5
5
1.88
N
N
LC-1
<5
5
2.13
N
N
LC-2
<5
5
1.3
N
N
LC-3
<5
5
1.44
N
N
LC-4
<5
<5
1.37
N
N
LBC-1
<5
<5
2.48
N
N
LBC-2
<5
<5
2.6
N
N
LBC-3
<5
<5
2.14
N
N
LBC-4
10
<5
2.54
Y
N
Total number of applicable samples


24
24
Total false positive



1
0
Percent false positive



4%
0%
Y = yes
N = no
29

-------
Table 6-7 Rate of False Negatives from Quick™ Test Kit

Non-Technical
Arsenic
(PPb)
Technical
Arsenic
(PPb)
Reference Method
Arsenic
(PPb)
Non-Technical
False Negative
(Y/N)
Technical
False Negative
(Y/N)
PT3-1
5
10
10.9
Y
N
PT3-2
10
10
10.9
N
N
PT3-3
10
10
10.9
N
N
PT3-4
10
10
10.9
N
N
PT4-1
30
40
34.8
N
N
PT4-2
30
40
34.8
N
N
PT4-3
20
40
34.8
N
N
PT4-4
30
20
34.8
N
N
PT5-1
100
100
113
N
N
PT5-2
100
100
113
N
N
PT5-3
100
100
113
N
N
PT5-4
100
100
113
N
N
PT6-1
5
20
29.6
Y
N
PT6-2
5
20
29.6
Y
N
PT6-3
10
20
29.6
N
N
PT6-4
10
20
29.6
N
N
PT6-5
10
20
29.6
N
N
PT6-6
10
20
29.6
N
N
PT6-7
20
20
29.6
N
N
LI-1
10
5
10.6
N
Y
LI-2
5
10
10.6
Y
N
LI-3
5
10
10.6
Y
N
LI-4
10
10
10.6
N
N
LI-5
10
10
10.6
N
N
LI-6
10
5
10.6
N
Y
LI-7
5
10
10.6
Y
N
LI-8
10
10
10.6
N
N
HI-1
10
10
10.4
N
N
HI-2
5
10
10.4
Y
N
HI-3
10
10
10.4
N
N
HI-4
10
20
10.4
N
N
HI-5
10
10
10.4
N
N
HI-6
10
20
10.4
N
N
HI-7
10
10
10.4
N
N
HI-8
10
20
10.4
N
N
WW-1
100
60
86.6
N
N
WW-2
60
60
86.6
N
N
WW-3
60
40
86.6
N
N
WW-4
60
60
86.6
N
N
TW-1
10
40
26.0
N
N
TW-2
10
40
26.0
N
N
TW-3
10
40
26.0
N
N
TW-4
50
40
26.0
N
N
Total number of applicable samples
Total false negative
Percent false negative

43
7
16%
43
2
5%
Y = yes
N = no
30

-------
6.8 Other Factors
The operators felt that the Quick™ test kit was easy to use and free of maintenance. The kit is
lightweight, easy to transport by car, and can be carried through fields and wooded areas. The
reaction bottles, however, are tall, narrow, and lightweight, making them susceptible to falling
over with a moderate breeze.
The Quick™ test kit allows two samples to be analyzed simultaneously. The total reaction time is
less than 15 minutes. The reagents are ready to use and do not require preparation. Three sizes of
scoops are included in the Quick™ test kit, making it easy to add the three reagents to the
sample. However, the narrow top of the reaction bottles makes it difficult to add the reagents. The
reagent bottles can be cleaned and reused. However, the operators experienced some difficulty
with the reagents sticking to the reaction vessel. This can be remedied by washing in a dilute acid
solution.
This kit requires no liquids or concentrated acids, making it safe and easy to carry in the field. The
solid reagents contain no toxic materials.
6.8.1	Costs
The Quick™ test kit is available in three sizes. The smallest kit costs $12.99 and is capable of
analyzing two samples. The 50-sample test kit costs $79.99. The large kit, capable of analyzing
100 samples, sells for $139.99.
6.8.2	Data Completeness
All portions of the verification test were completed, and all data that were to be recorded were
successfully acquired. The non-technical operator analyzed only one of the three required
laboratory reagent blanks, otherwise data completeness was 100%.
31

-------
Chapter 7
Performance Summary
An assessment of quantitative accuracy showed that percent bias values ranged from 8 to 83%
for the non-technical operator and 8 to 84% for the technical operator for the PT samples. The
percent bias ranged from 8 to 92% for the non-technical operator and 8 to 54% for the technical
operator for the drinking water samples. For the FW samples, the percent bias ranged from 2 to
320% for both the non-technical and technical operators. An additional qualitative criterion for
accuracy was the percentage of samples for which the Quick™ test kit result was within 25% of
the reference result or within a corresponding "less than" range. By this criterion, the Quick™
test kit yielded a qualitative accuracy for the PT samples of 71% for the non-technical operator
and 55%) for the technical operator. The qualitative accuracy for the drinking water samples was
57%) for the non-technical operator and 52% for the technical operator. The qualitative accuracy
for the freshwater samples was 96% for the non-technical operator and 54% for the technical
operator.
Percent RSD data illustrate consistency in the Quick™ test kit replicate analyses. Seven of the
14 replicate sets for the PT samples showed an RSD of 0% (i.e., all replicate results were
identical). The remaining replicate sets for the non-technical operator had an RSD ranging from
29 to 50%), and the remaining replicate set for the technical operator had an RSD of 29%. For the
drinking water samples, the RSDs for the non-technical operator ranged from 29 to 100%), and
the RSDs for the technical operator ranged from 0 to 18%).
The linearity of response of the Quick™ test kit was assessed using the PT samples containing
2 to 112 ppb arsenic. The linear regression for the Quick™ test kit results for the non-technical
operator was ppb = 0.90 (±0.086) x (reference, ppb) - 5.2 (±4.1) ppb, with a correlation coefficient
(r) of 0.974. The corresponding equation for the results for the technical operator was ppb = 0.88
(±0.056) x (reference, ppb) - 0.45 (±2.7) ppb, with a correlation coefficient (r) of 0.988.
The manufacturer's estimated detection limit for the Quick™ test kit is 5 ppb. Seven replicate
samples of 25-ppb arsenic produced Quick™ readings of 5 to 20 ppb with the non-technical
operator and seven identical readings of 20 ppb with the technical operator. No MDL was
calculated quantitatively from these data.
The Quick™ test kit showed a minor tendency toward higher readings (3 ppb on average) with
higher levels of sodium chloride, iron, sulfide, and acidity. Because of the study design, it was not
possible to determine which ion was responsible for the observed result.
32

-------
The operator skill level does not appear to be a major factor determining Quick™ test kit results.
The rates of false positives and false negatives for the Quick™ test kit were assessed relative to the
reference method using 10 ppb of arsenic as the decision level. The rate of false positives for the
Quick™ test kit was 4% for the non-technical operator and 0% for the technical operator. The rate
of false negatives was 16% for the non-technical operator and 5% for the technical operator.
The Quick™ test kit is available in three sizes. The smallest is capable of analyzing two samples
and costs $12.99. The 50-sample test kit costs $79.99. The large kit, capable of analyzing
100 samples, sells for $139.99. The test kit allows two samples to be analyzed simultaneously. The
total reaction time is less than 15 minutes. The reagents are ready to use and do not require
preparation. Three scoop sizes are included in the Quick™ test kit, making addition of the reagents
simple, but the size and shape of the reaction vessels limit the ease of use of the test kit.
33

-------
Chapter 8
References
1.	Test/QA Plan for Verification of Portable Analyzers, Battelle, Columbus, Ohio, Version 2.0.
2.	U.S. EPA Method 200.8, Determination of Trace Elements in Waters and Wastes by
Inductively Coupled Plasma Mass Spectrometry, Revision 5.5, April 1991.
3.	Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Pilot,
Version 2.0, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus,
Ohio, October 2000.
4.	U.S. Code of Federal Regulations, Title 40, Part 136, Appendix B.
34

-------