-------
Table 6-3a. Relative Standard Deviation of Performance Test Sample Measurements
Non-Technical Operator
Technical Operator
Reference
Concentration Method
(mg/L) (RSD)
Unit #1
(RSD)
Unit #2
(RSD)
Unit #1
(RSD)
Unit #2
(RSD)
0.030 8%
0%
11%
22%
8%
0.100 7%
0%
3%
7%
6%
0.200 2%
3%
3%
3%
2%
0.400 2%
3%
3%
4%
4%
0.800 1%
NA(a)
NA
NA
NA
(a) NA = calculation of precision not appropriate when result was outside the detectable range of the Thermo Orion
AQ4000.
Table 6-3b. Relative Standard Deviation of Surface Water Measurements
Non-Technical Operator
Technical Operator
Reference
Method
Sample Description (RSD)
Unit #1
(RSD)
Unit #2
(RSD)
Unit #1
(RSD)
Unit #2
(RSD)
Alum Creek LFM 8%
19%
20%
6%
7%
Olentangy River LFM 2 %
6%
4%
0%
1%
Table 6-3c. Relative Standard Deviation of U.S. Drinking Water Measurements
Non-Technical Operator
Reference
Technical Operator
Method
Unit #1
Unit #2
Unit #1
Unit #2
Sample Description
(RSD)
(RSD)
(RSD)
(RSD)
(RSD)
Des Moines, IA, LFM
3%
7%
7%
3%
5%
Flagstaff, AZ, LFM
12%
3%
5%
4%
7%
Montpelier, VT, LFM
2%
4%
3%
14%
13%
Seattle, WA, LFM
2%
10%
11%
3%
4%
Tallahassee, FL, LFM
2%
16%
18%
1%
5%
30
-------
Table 6-3d. Relative Standard Deviation of Columbus, OH, Drinking Water Measurements
Non-Technical Operator Technical Operator
Reference
Method Unit #1 Unit #2 Unit #1 Unit #2
Sample Description (RSD) (RSD) (RSD) (RSD) (RSD)
City Water LFM - Outdoor
Field Site
4%
NA'a)
NA
NA
NA
City Water LFM - Indoor
Field Site
4%
5%
9%
5%
6%
City Water LFM - Lab
4%
3%
5%
4%
4%
Well Water LFM -
Outdoor Field Site
13%
NA
NA
43%
35%
Well Water LFM - Indoor
Field Site
13%
18%
20%
5%
5%
Well Water LFM - Lab
NA
NA
NA
NA
NA
(a) NA = calculation of precision not appropriate when result was outside the detectable range of the Thermo Orion
AQ4000.
6.3 Linearity
The linearity of the Thermo Orion AQ4000 was assessed by using a linear regression of the PT
results against the reference method results (Table 6-la). Figures 6-1 and 6-2 show scatter plots
of the results from the non-technical and technical operator, respectively versus the reference
results. A dotted regression line with a slope of unity and intercept of zero also is shown in
Figures 6-1 and 6-2.
o>
E
y = 0.8708X+ 0.0026
r2 = 0.9964
0 o
V O
1 *—
C O
is
a>
a.
O
0.1
0.2
0.3
0.4
0.5
Reference Method Results (mg/L)
Figure 6-1. Non-Technical Operator Linearity Results
31
-------
0.45
nr
0.4
*u>
£
0.35
-—1
0.3
d
c
o
0.25
o
0.2
o
0.15
V
0.1
Q.
o
0.05
0
y = 0.8288X + 0.0123
r2 = 0.9852
0.1 0.2 0.3 0.4
Reference Method Results (mg/L)
0.5
Figure 6-2. Technical Operator Linearity Results
A lineal- regression of the data in Figure 6-1 for the non-technical operator gives the following
regression equation:
y (non-technical operator results in mg/L)=0.871 (± 0.020) x (reference result in mg/L)
+ 0.003 (± 0.004) mg/L with r=0.996 and N=33.
A lineal- regression of the data in Figure 6-2 for the technical operator gives the following
regression equation:
y (technical operator results in mg/L)=0.829 (± 0.038) x (reference result in mg/L)
+ 0.012 (± 0.008) mg/L with r=0.985 and N=33.
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
Only the technical operator's intercept is significantly different from zero, and the r values are
both above 0.980. Both slopes are significantly different from unity at the 95% confidence
interval, but the slopes from each operator are statistically the same. This deviation from unity
indicates a low bias in the results generated by the Thermo Orion AQ4000 compared with the
results produced by the reference method.
6.4 Method Detection Limit
The manufacturer's estimated detection limit for the Thermo Orion AQ4000 is 0.020 mg/L. The
MDL(4) was determined by analyzing seven replicate samples at a concentration of 0.1 mg/L.
Table 6-4 shows the results of the MDL assessment. The MDL determined as described in
Equation (6) of Section 5.4 was approximately 0.01 mg/L for Thermo Orion AQ4000 when used
by the non-technical operator and approximately 0.02 mg/L when used by the technical operator.
32
-------
Table 6-4. Results of Method Detection Limit Assessment
Non-Technical Operator Technical Operator
MDL Cone.
Unit #1
Unit #2
Unit #1
Unit #2
(mg/L)
(mg/L)
(mg/L)
(mg/L)
(mg/L)
0.100
0.08
0.079
0.09
0.092
0.100
0.09
0.089
0.09
0.095
0.100
0.08
0.078
0.09
0.092
0.100
0.08
0.080
0.09
0.089
0.100
0.08
0.080
0.10
0.103
0.100
0.08
0.076
0.08
0.079
0.100
0.08
0.080
0.08
0.090
Std Dev
0.004
0.004
0.007
0.007
t (n=7)
3.140
3.140
3.140
3.140
MDL (mg/L)
0.012
0.013
0.022
0.023
6.5 Inter-Unit Reproducibility
The inter-unit reproducibility of the Thermo Orion AQ4000 was assessed by using a linear
regression of the results produced by one Thermo Orion AQ4000 plotted against the results
produced by the other Thermo Orion AQ4000. The results from all of the samples that had
detectable amounts of cyanide (including the PT, surface, and drinking water samples) produced
by both operators were included in this regression. Figure 6-3 shows a scatter plot of the results
from both Thermo Orion AQ4000s.
A linear regression of the data in Figure 6-3 for the inter-unit reproducibility assessment gives the
following regression equation:
y (Unit #1 result in mg/L)=0.999 (± 0.015) x (Unit #2 result in mg/L) + 0.004 (± 0.002)
mg/L with r2=0.994 and N=112.
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
The slope is not significantly different from unity, while the intercept is significantly different
from zero. These data indicate that the two Thermo Orion AQ4000s functioned very similarly to
one another.
33
-------
0.4
0.35
J 0.3
|* 0.25
~ 0.2
s a15
1 0.1
0.05 -
0 -I 1 1 1 1 1 1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
Unit #2 (mg/L)
Figure 6-3. Inter-Unit Reproducibility Results
6.6 Lethal or Near-Lethal Dose Response
Samples at 50.0-, 100-, and 250-mg/L concentrations (close to what may be lethal if a volume the
size of a typical glass of water was ingested) were prepared and analyzed by the Thermo Orion
AQ4000. Upon breaking the ampoule in the sample, the color of the sample changed within five
seconds to brilliant purple and, after approximately 35 more seconds, to blood red. The change
was much more rapid than for any of the PT samples. The PT samples took about 30 seconds to
even produce a small change in the color of the sample and took the full 15-minute reaction time
to reach its analysis color of a clear, light purple. When these samples with lethal/near-lethal
concentrations were inserted into the Thermo Orion AQ4000 after the full reaction time, the
digital readout read "over range."
6.7 Operator Bias
The possible difference in results produced by the non-technical and technical operator was
assessed by using a linear regression of the results produced by the non-technical operator plotted
against the results produced by the technical operator. The results from all of the samples that had
detectable amounts of cyanide (including the PT, surface, and drinking water samples) from both
technologies were included in this regression. Figure 6-4 shows a scatter plot of the results from
both analyzers.
A lineal- regression of the data in Figure 6-4 for the operator bias assessment gives the following
regression equation:
y (non-tech result in mg/L)=1.000 (± 0.061) x (tech result in mg/L) - 0.013
(± 0.009) mg/L with r=0.905 and N=112.
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
The slope is not significantly different from unity, while the intercept is significantly different
from zero. These data indicate that there was veiy little difference in results generated by the non-
technical operator compared with those of the technical operator.
34
-------
0.4
_0-35
ra ? 0.3 -
o
"E E 0.25
¦g r 0.2 -
^ ^ 0.15 -
o a> 0.1 -
z O 0.05 -
y= 0.9999x-0.0127
%• V,
r2 = 0.9048
_
0.05 0.1 0.15 0.2 0.25
Technical Operator (mg/L)
0.3
0.35
0.4
Figure 6-4. Non-Technical vs. Technical Operator Bias
Results
6.8 Field Portability
The Thermo Orion AQ4000 was operated in laboratory and field settings during this verification
test. Tables 6-Id, 6-2d, and 6-3d show the results of these measurements. From an operational
standpoint, the Thermo Orion AQ4000 was easily transported to the field setting, and the samples
were analyzed in the same fashion as they were in the laboratory. No functional aspects of the
Thermo Orion AQ4000 were compromised by performing the analyses in the field setting.
However, performing analyses under extremely cold conditions (sample water temperatures
between 4 and 6°C) negatively affected the performance of the Thermo Orion AQ4000 reagents.
The low temperatures apparently slowed the chemical reaction rates, which caused the decreased
color change in the LFM samples.
Table 6-2d shows the bias of the samples analyzed in the field setting (indoors with sample
temperatures of approximately 16°C and outdoors with sample temperatures of 4 to 6°C) and of
the identical samples analyzed at the laboratory at approximately 20°C. The well and Columbus,
OH, city water samples were both dechlorinated as described in Section 3.5.1. In addition,
because the well water sample had a pungent odor, lead carbonate was added after NaOH
preservation to check for the presence of sulfides. The lead carbonate did not turn black. Such a
color change would have indicated the presence of sulfides. Nonetheless, there was a 41 to 48%
bias in the indoor Columbus, OH, city water measurements and a 27 to 43% bias in the indoor
well water measurements. Because there was an apparent matrix interference in the reference
measurement (see Table 4-2), the well water biases were calculated using the fortified
concentration (0.200 mg/L) as the reference concentration.
The apparent matrix interference in the well water LFM seemed to progressively mask the
cyanide in the LFM sample after it was spiked and analyzed at the indoor field setting (producing
a 27 to 43% bias from initial fortification) because, by the time the well water LFM samples were
analyzed by the Thermo Orion AQ4000 at the laboratory two days after initial fortification, there
was no detectable cyanide (100% bias from initial fortification). These same samples were
analyzed using the reference method eight days after initial fortification, and the result was below
the MDL of the reference method (Table 4-2). Because there was an apparent time-dependent
35
-------
matrix interference, the data measured in the well water using the Thermo Orion AQ4000 in the
field setting cannot be meaningfully compared with the result produced from the identical
samples analyzed with the Thermo Orion AQ4000 in the laboratory.
The bias in the Columbus, OH, water indoor LFM sample (41 to 48%) was similar to the bias in
the Columbus, OH, water LFM sample analyzed at the laboratory location (36 to 42%). The
apparent matrix interference causing the large biases did not further mask the cyanide in the LFM
sample as evidenced by the similar biases at the field location and at the laboratory two days
later. These data support the qualitative assessment that the Thermo Orion AQ4000 functions
properly when operated in field locations.
6.9 Ease of Use
The Thermo Orion AQ4000 and AQ4006 cyanide reagents and Auto-Test™ cuvettes were easy
to operate. The instructions were clear, and the sample and reagents were easily measured using a
graduated cylinder, syringe, and a dropper bottle. It was convenient that adding reagents did not
require strict mixing and reaction times. The operators only had to hold strictly to the 15-minute
color development reaction time. Not having to keep track of several short mixing/ reaction times
after adding each reagent streamlined the analysis and increased sample throughput. The Thermo
Orion AQ4000 recognized the Auto-Test™ cuvettes when they were inserted and a 15-minute
timer appeared on the digital readout. When analyzing large sample sets, this timer had to be
overridden before every sample analysis. While the sample handling and analysis were very easy,
the pH of each sample had to be adjusted to between 10.5 and 11 using NaOH and hydrochloric
acid. This step required the availability of acid and base, pH paper or meter, and some knowledge
of pH adjustment. Instructions for pH adjustment were not provided. Because the color change
took place within the Auto-Test™ cuvettes and they were disposable, cleanup was simple and
free of mess. Only the graduated cylinder used for measuring the sample and adding reagents
needed to be rinsed between samples.
6.10 Sample Throughput
Sample preparation, including accurate volume measurement and the addition of reagents, took
only one to two minutes per sample. After performing the sample preparation, a 15-minute period
of color development is required before sample analysis. Therefore, if only one sample is
analyzed, it would take approximately 17 minutes. However, both operators were able to stagger
the start of the color development period every two minutes for subsequent samples, so a typical
sample set of 12 analyses took 30 to 40 minutes.
36
-------
Chapter 7
Performance Summary
Biases for the Thermo Orion AQ4000 ranged from 4 to 23% for the PT samples; 10 to 26% for
the surface water samples; 6 to 51% for the drinking water samples from around the country; and
27 to 100% for the Columbus, OH, drinking water samples. In the analyses of surface water
samples from the Olentangy River that the reference method reported as less than 0.005 mg/L,
the Thermo Orion AQ4000 displayed "over range" for five of the 16 samples, suggesting a
cyanide concentration that was outside the calibration range of the Thermo Orion AQ4000. The
manufacturer has stated that the "over range" result also is displayed if a sample is outside (i.e.,
either above or below) the calibration range of the Thermo Orion AQ4000.
The RSDs ranged from 0 to 22% for the PT samples; 0 to 20% for the surface water samples; 1 to
18% for the drinking water samples from around the country; and 3 to 20% for the Columbus,
OH, drinking water samples analyzed at the indoor field site.
A linear regression of the linearity data for the non-technical operator gives the following
regression equation:
y (non-technical operator results in mg/L)=0.871 (± 0.020) x (reference result in mg/L)
+ 0.003 (± 0.004) mg/L with r2=0.996 and N=33.
A linear regression of the data for the technical operator gives the following regression equation:
y (technical operator results in mg/L)=0.829 (± 0.038) x (reference result in mg/L)
+ 0.012 (± 0.008) mg/L with r2=0.985 and N=33.
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
Only the technical operator's intercept is significantly different from zero, and the r2 values are
both above 0.980. The linearity of the Thermo Orion AQ4000 was not dependent on which
operator was performing the analyses. The slope of the linear regression was significantly less
than unity in both instances. This deviation from unity indicates a low bias in the results
generated by the Thermo Orion AQ4000 compared with the results produced by the reference
method.
The MDL was determined to be approximately 0.01 mg/L for the Thermo Orion AQ4000 when
used by the non-technical operator and approximately 0.02 mg/L for the Thermo Orion AQ4000
when used by the technical operator.
37
-------
A linear regression of the data for the inter-unit reproducibility assessment gives the following
regression equation:
y (Unit #1 result in mg/L)=0.999 (± 0.015) x (Unit #2 result in mg/L) + 0.004 (± 0.002)
mg/L with 1^=0.994 and N=112.
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
The slope is not significantly different from unity, while the intercept is significantly different
from zero. These data indicate that the technologies functioned very similarly to one another.
When performing the analysis on samples containing lethal/near-lethal concentrations of cyanide,
the difference in the color development was remarkable. Upon breaking the ampoule in the
sample, the color of the sample changed within five seconds to brilliant purple and, after
approximately 35 more seconds, to blood red. The change was much more rapid than for any of
the PT samples. When the samples were inserted into the Thermo Orion AQ4000 after the full
reaction time, the digital readout read "over range." Even without using the AQ4000 colorimeter,
the reagents and Auto-Test™ cuvettes would be useful for a first responder seeking to find out
whether a toxic level of cyanide is present in a drinking water sample. The presence of such
concentrations could be confirmed within minutes by visual observation of the color development
process.
A linear regression of the data for the operator bias assessment gives the following regression
equation:
y (non-tech result in mg/L)= 1.000 (± 0.061) x (tech result in mg/L) - 0.013
(± 0.009) mg/L with r2=0.905 and N=112.
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
The slope is not significantly different from unity, while the intercept is significantly different
from zero. These data indicate that there was very little difference in results generated by the non-
technical operator compared with those of the technical operator.
From an operational standpoint, the Thermo Orion AQ4000 was easily transported to the field
setting, and the samples were analyzed in the same fashion as they were in the laboratory. No
functional aspects of the Thermo Orion AQ4000 were compromised by performing the analyses
in the field setting. However, performing analyses under extremely cold conditions negatively
affected the performance of the Thermo Orion AQ4000 reagents. The low temperatures
apparently slowed the chemical reaction rates, which caused the decreased color change in the
LFM samples.
The Thermo Orion AQ4000 and AQ4006 cyanide reagents and Auto-Test™ cuvettes were easy
to operate. The instructions were clear, and the sample and reagents were easily measured using a
graduated cylinder, syringe, and a dropper bottle. The Thermo Orion AQ4000 recognized the
Auto-Test™ cuvettes when they were inserted, and a 15-minute timer appeared on the digital
readout. When analyzing large sample sets, this timer had to be overridden before every sample
analysis. While the sample handling and analysis were easy, the pH of each sample had to be
adjusted to between 10.5 and 11.0 using NaOH and HC1. This step required the availability of
38
-------
acid and base, pH paper or meter, and some knowledge of pH adjustment. Instructions for pH
adjustment were not included in the manufacturer's instructions. The Auto-Test™ cuvettes made
cleanup and waste disposal simple and mess free. Only the graduated cylinder used for measuring
the sample and adding reagents needed to be rinsed between samples.
Since the Thermo Orion AQ4000 did not require strict mixing/reaction time periods after adding
each reagent, and the Auto-Test™ cuvettes automatically measured the volume of sample added
to the final reaction vessel, the analysis process was conducive to analyzing large numbers of
samples consecutively. Each sample was entirely prepared within one or two minutes, and then
the 15-minute color development period started. If only one sample is analyzed, sample through-
put would take approximately 17 minutes. However, both operators were able to stagger the start
of the color development period every two minutes for subsequent samples, so a typical sample
set of 12 analyses took 30 to 40 minutes.
39
-------
Chapter 8
References
1. Test/QA Plan for Verification of Portable Analyzers for Detection of Cyanide in Water,
Battelle, Columbus, OH, January 2003.
2. U.S. EPA Method 335.1, Cyanides Amenable to Chlorination, 1974, in "Methods for
Chemical Analysis of Water and Wastes," EPA/600/4-79/020, March 1983.
3. United States Environmental Protection Agency, National Primary Drinking Water
Standards, EPA/816-F-02-013, July 2002.
4. Code of Federal Regulations, Title 40, Part 136, Appendix B, Definition and Procedure
for the Determination of the Method Detection Limit-Revision 1.11.
5. Quality Management Plan (QMP)for the ETV Advanced Monitoring Systems Center,
Version 4.0, U.S. EPA Environmental Technology Verification Program, Battelle,
Columbus, OH, December 2002.
40
-------