August 2001
Environmental Technology
Verification Report
NECl F-NTK
Nitrate Test Kit
Prepared by
Balteiie
. . . Putting Technology To Work
Battel le
Under a cooperative agreement with
&EPA U.S. Environmental Protection Agency
ETVElV ElV
-------
August 2001
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
NECi F-NTK
Nitrate Test Kit
by
Adam Abbgy
Thomas Kelly
Charles Lawrie
Karen Riggs
Battelle
Columbus, Ohio 43201
1
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency and recommended for public release.
Mention of trade names or commercial products does not constitute endorsement or
recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification Organizations oversee and report verification activities based on testing and Quality
Assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six technology centers. Information about
each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality and
to supply cost and performance data to select the most appropriate technology for that assess-
ment. In 1997, through a competitive cooperative agreement, Battelle was awarded EPA funding
and support to plan, coordinate, and conduct such verification tests for "Advanced Monitoring
Systems for Air, Water, and Soil" and report the results to the community at large. Information
concerning this specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/07/07_main.htm.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. In particular we would like to thank
Patricia Holowecky, Andrew Montgomery, Wayne Trulli, and Lynda Short of Battelle. We also
acknowledge the participation of Ellen Campbell of NECi in this verification test.
IV
-------
Contents
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations viii
1. Background 1
2. Technology Description 2
3. Test Design and Procedures 3
3.1 Introduction 3
3.2 Test Design 3
3.3 Test Samples 4
3.3.1 QC Samples 4
3.3.2 PT Samples 6
3.3.3 Environmental Samples 6
3.4 Reference Analysis 7
3.5 Verification Schedule 7
4. Quality Assurance/Quality Control 9
4.1 QC for Reference Method 9
4.2 Audits 11
4.2.1 Performance Evaluation Audit 11
4.2.2 Technical Systems Audit 12
4.2.3 Audit of Data Quality 12
4.3 QA/QC Reporting 12
4.4 Data Review 13
5. Statistical Methods 14
5.1 Accuracy 14
5.2 Precision 15
5.3 Linearity 15
5.4 Method Detection Limit 15
5.5 Matrix Interferences 16
v
-------
5.6 Operator Bias 16
5.7 Rate of False Positives/False Negatives 16
6. Test Results 17
6.1 Round 1 Results 17
6.1.1 Accuracy 17
6.1.2 Precision 22
6.1.3 Linearity 22
6.1.4 Method Detection Limit 24
6.1.5 Matrix Interferences 24
6.1.6 Operator Bias 26
6.1.7 Rate of False Positives/False Negatives 26
6.1.8 Other Factors 26
6.2 Round 2 Results 29
6.2.1 Accuracy 29
6.2.2 Linearity 35
6.2.3 Detection Limit 36
6.2.4 Operator Bias 36
7. Performance Summary 37
8. References 39
Appendix A: Data Recording Sheet A-l
Appendix B: Vendor Comments B-l
Figures
Figure 2-1. NECi F-NTK Nitrate Test Kit 2
Figure 6-1. Comparison of F-NTK Results to Reference Results from Round 2, Phase 2 ... 36
Tables
Table 3-1. Test Samples for Nitrate-N Used in Verification of the F-NTK Test Kit 5
Table 3-2. Schedule of Round 1 Verification Test Days 8
Table 4-1. Reference Method QCS Analysis Results 10
vi
-------
Table 4-2. Reference Method LFM Results 11
Table 4-3. Reference Method PE Audit Results 12
Table 4-4. Summary of Data Recording Process 13
Table 6-la. Results from Round 1 Laboratory Performance Test Sample Analyses 18
Table 6-lb. Results from Round 1 Drinking Water Analyses 18
Table 6-lc. Results from Round 1 Freshwater Analyses 19
Table 6-ld. Results from Round 1 Salt Water Analyses 19
Table 6-2a. Accuracy of the F-NTK Test Kit with Laboratory Performance Test
Samples for Round I 20
Table 6-2b. Accuracy of the F-NTK Test Kit with Drinking Water Samples for Round I .... 20
Table 6-2c. Accuracy of the F-NTK Test Kit with Freshwater Samples for Round I 21
Table 6-2d. Accuracy of the F-NTK Test Kit with Salt Water Samples for Round I 21
Table 6-3. Summary of Qualitative Accuracy Results from Round 1 of Testing 22
Table 6-4a. Precision Results for F-NTK Kits from Laboratory Performance Test
Samples for Round I 23
Table 6-4b. Precision Results for F-NTK from Drinking Water Samples for Round I 24
Table 6-5. Method Detection Limit Results for the F-NTK Kits for Round I 25
Table 6-6a. Data from Laboratory Performance Test with Low-Level Interferences for
Round I 25
Table 6-6b. Data from Laboratory Performance Test with High-Level Interferences for
Round I 25
Table 6-7. Rate of False Positives from F-NTK Test Kit for Round I 27
Table 6-8. Rate of False Negatives from F-NTK Test Kit for Round I 28
Table 6-9. Field Results from Freshwater Samples, Round 2, First Phase 30
Table 6-10. Laboratory Results from Reanalysis of Freshwater Samples, Round 2,
Second Phase 31
vii
-------
Table 6-11. Performance of the F-NTK Test Kit Relative to 1-ppm Precision on the
Field Freshwater Samples, Round 2, First Phase 33
Table 6-12. Performance of the F-NTK Test Kit Relative to 1-ppm Precision on the
Laboratory Reanalysis of Freshwater Samples, Round 2, Second Phase 34
Table 6-13. Summary of F-NTK Results Relative to 1-ppm Precision for Round 2 35
Vlll
-------
List of Abbreviations
AC
Alum Creek reservoir water
AMS
Advanced Monitoring Systems
ASTM
American Society for Testing and Materials
BDL
below detection limit
CW
community water
DW
drinking water
EPA
U.S. Environmental Protection Agency
ETV
Environmental Technology Verification
HDPE
high-density polyethylene
IC
ion chromatograph
LFM
laboratory-fortified matrix
MB-B
Massachusetts Bay bottom water
MB-S
Massachusetts Bay surface water
MCL
maximum contaminant limit
MDL
method detection limit
NECi
Nitrate Elimination Co., Inc.
OR
Olentangy River water
ppb
parts per billion
ppm
parts per million
PE
performance evaluation
PT
performance test
QA
quality assurance
QC
quality control
QCS
quality control standard
QMP
Quality Management Plan
RB
reagent blank
RSD
relative standard deviation
SR
Scioto River water
TSA
technical systems audit
WW
well water
IX
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) has created the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by substantially accelerating the acceptance
and use of improved and cost-effective technologies. ETV seeks to achieve this goal by provid-
ing high quality, peer-reviewed data on technology performance to those involved in the design,
distribution, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of regulators, buyers and vendor organizations; and with the full participation of
individual technology developers. The program evaluates the performance of innovative tech-
nologies by developing test plans that are responsive to the needs of stakeholders, conducting
field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance
protocols to ensure that data of known and adequate quality are generated and that the results are
defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. This verification
report presents the procedures and results of the verification test for the Nitrate Elimination Co.,
Inc. (NECi) F-NTK™ nitrate test kit.
1
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of environ-
mental monitoring technologies for air, water, and soil. This verification report provides results
for the verification testing of the F-NTK nitrate test kit. The following description of the F-NTK
is based on information provided by the vendor, Nitrate Elimination Co., Inc. (NECi).
The NECi F-NTK provides the reagents and equipment necessary for analyzing for nitrate,
nitrite, and total nitrate-nitrogen (nitrate-N) in environmental water samples and water-extracts of
soil, plant tissue, and some foods. The F-NTK uses an enzyme-based (nitrate-N reductase)
nitrate-N testing method and contains no toxic or hazardous chemicals. With the F-NTK, nitrate-
N can be analyzed in two ranges of 0.5 to 10 parts per million (ppm) nitrate-N (1 ppm = 1 mg per
liter) with a precision of ±1 ppm nitrate-N or 0.05 to 1.0 ppm nitrate-N with a precision of
±0.1 ppm nitrate-N.
The sample results from the F-NTK nitrate
test kit are evaluated using three nitrate-N
standards, which are mixed and developed by
the user, and a precision color chart for
estimating nitrate-N content. All the
necessary tools for conducting the nitrate-N
tests are supplied in the F-NTK, including a
test tube rack. The test kit provides semi-
quantitative estimates of nitrate content. For
more accurate quantitative data, the assay can
be read using a colorimeter at 540 nm.
Figure 2-1. NECi F-NTK Nitrate Test Kit
2
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Portable Analyzers {l) The verification was based on comparing results from the
F-NTK test kit to those from a reference method. The reference method for nitrate-N analysis
uses ion chromatography (IC) according to EPA Method 300.1(2) The F-NTK test kit was cali-
brated using standards supplied with the kit. The kit was tested by analyzing laboratory-prepared
performance test samples and drinking water, residential tap water, reservoir, river, and ocean
water with both the test kit and the reference method. The F-NTK test kit was evaluated for
accuracy, precision, linearity, method detection limit, matrix interference effects, and operator
bias, as well as ease of use, cost, and sample throughput.
3.2 Test Design
Two sets of the F-NTK test kit were tested independently by challenging them with test samples
representative of those likely to be analyzed using the F-NTK. Each set of F-NTK test kits was
used to analyze the full set of samples for nitrate-N. The preparation, calibration, and analyses
were performed according to the manufacturer's recommended procedures. Results from the F-
NTK test kit were recorded manually. In addition to the analytical results, the time required for
sample analysis and operator observations concerning the use of the test kit (e.g., frequency of
calibration, ease of use, maintenance) were recorded. The results from the F-NTK tests were
compared to those from the reference method to quantitatively assess accuracy, linearity, and
detection limit. Multiple aliquots of performance test samples and drinking water samples were
analyzed to assess precision.
For each set of the test kit, identical sets of samples were analyzed independently by two separate
operators (a technical and a non-technical Battelle staff member) to test for the existence of
operator bias on the test kit performance. The technical operator was a research technician at
Battelle with three years of laboratory experience and a B.S degree. The non-technical operator
was a part-time laboratory helper at Battelle and a student at Ohio State University. Interferences
and matrix effects were assessed by challenging the test kit with performance test samples of
known nitrate-N concentrations containing both low-level and high-level interferences. Sample
throughput was estimated based on the time required to analyze a sample set. Performance
3
-------
parameters, such as ease of use and reliability, were evaluated based on documented observations
of the operators.
3.3 Test Samples
Three types of samples were used in the verification test as shown in Table 3-1: quality control
(QC) samples, performance test (PT) samples, and environmental water samples.
The QC and PT samples were prepared from purchased standards. The EPA has set the maxi-
mum contaminant limit (MCL) for nitrate at 10 ppm nitrate-N, under the Safe Drinking Water
Act. However, the vendor suggested that nitrate concentrations in the natural environment are
rarely above 2.0 ppm nitrate-N. For this reason, the QC sample concentrations for nitrate-N were
targeted at a 2 ppm midpoint. The PT samples ranged from 10% to 1,000% of that level, i.e.,
from 0.2 to 20 ppm. The environmental water samples were collected from various drinking
water and surface water sources. All samples were analyzed using the two F-NTK test kits and by
a laboratory reference method. Every tenth sample was analyzed twice by the reference method,
to document the reference method's precision.
3.3.1 QC Samples
As Table 3-1 indicates, prepared QC samples included both laboratory reagent blanks (RB) and
laboratory-fortified matrix (LFM) samples. The RB samples consisted of American Society for
Testing and Materials (ASTM) Type II deionized water, and were exposed to identical handling
and analysis procedures as other prepared samples. These samples were used to help ensure that
no sources of contamination were introduced in the sample handling and analysis procedures.
Two types of LFMs were prepared. The LFMf samples consisted of aliquots of environmental
samples that were spiked in the field to increase the analyte concentration by 2 ppm nitrate-N.
The spike solution used to prepare the LFMf was prepared in the laboratory and brought to the
field site. The LFMl samples were aliquots of environmental samples that were spiked in the
laboratory to increase the analyte concentration by 2.0 ppm nitrate-N. These samples were used
to help identify whether matrix effects had an influence on the analytical results. At least 10% of
all the prepared samples analyzed were RBs, and at least one sample taken from each sampling
site was an LFM.
Quality control standards (QCS) were used as calibration checks to verify that the F-NTK and the
reference instrument were properly calibrated and reading within defined control limits. These
nitrate-N standards were purchased from a commercial supplier and were subject only to dilution
as appropriate. Calibration of the test kit and the reference instrument was verified using a QCS
before and after the testing period, as well as after every tenth sample. An additional independent
QCS was used in a performance evaluation audit of the reference method.
4
-------
Table 3-1. Test Samples3 for Nitrate-N Used in Verification of the F-NTK Test Kit
Type of Sample
Sample Characteristics
Nitrate-N
Concentration
No. of
Samples/Analysis
Quality Control
RBh
~0
10% of all
LFMb
2 ppm above native level
1 per site
QCSb
2 ppm
10% of all
Performance Test
For the determination of
detection limit for nitrate
(PT6)
2.5 ppm (Five times the
manufacturer's estimated
detection limit)
1/7
Nitrate (PT1)
0.2 ppm
1/4
Nitrate (PT4)
0.6 ppm
1/4
Nitrate (PT2)
2.0 ppm
1/4
Nitrate (PT5)
6.0 ppm
1/4
Nitrate (PT3)
20 ppm
1/4
Analyte spiked with
interference (LI)
3.0 ppm with low
interference
1/8
Analyte spiked with
interference (HI)
3.0 ppm with high
interference
1/8
Environmental
Drinking water (DW)
Unknown
1/4
Community water (CW)
Unknown
1/4
Well water (WW)
Unknown
1/4
Alum Creek Reservoir (AC)
Unknown
4/1
Olentangy River (OR)
Unknown
4/1
Scioto River (SR)
Unknown
4/1
Massachusetts Bay surface
water (MB-S)
Unknown
4/1
Massachusetts Bay water at
sediment/water column
interface (MB-B)
Unknown
4/1
a Listing is for clarity; samples were analyzed in randomized order for the verification testing.
bSee Section 3.3.1 for descriptions of these samples.
5
-------
3.3.2 PT Samples
The two types of PT samples used in this verification test (Table 3-1) were prepared in the
laboratory using ASTM Type II water as the water source. One type of PT solution contained
nitrate-N at various concentrations and was prepared specifically to determine F-NTK accuracy,
linearity, and detection limit. To determine the detection limit of the test kit, a solution with a
concentration five times the vendor's reported detection limit was used (i.e., 2.5 ppm nitrate-N).
Seven nonconsecutive replicate analyses of this solution were made to obtain precision data with
which to determine the method detection limit. Additionally, solutions were prepared to assess
the linearity over a broad range of nitrate-N concentration. Four aliquots of each of these solu-
tions were prepared and analyzed separately to assess the precision of the test kit.
The second type of PT sample helped establish the effects of potential matrix interferences on
the performance of the F-NTK. These samples were prepared from solutions with known concen-
trations of nitrate-N and were spiked with potentially interfering species likely to be found in
typical water samples. One sample contained low levels of interferences that consisted of 1 mg of
iron, 3 mg of sodium chloride, and 0.1 mg of sulfate per liter at a pH of 6. The second sample
contained high levels of interferences that consisted of 10 mg of iron, 30 mg of sodium chloride,
and 1.0 mg of sulfate per liter at a pH of 3. Eight replicate samples of each of these solutions
were analyzed.
3.3.3 Environmental Samples
Drinking water samples listed in Table 3-1 include drinking water from a Battelle drinking
fountain (DW), and well water (WW) and community drinking water (CW) from two residential
sites in Columbus, Ohio. These samples were collected directly from the tap into 2-L high-
density polyethylene (HDPE) containers. Four (100-mL) aliquots of each sample were analyzed
in the field at the time of collection by each set of the test kit being verified. One (100-mL)
aliquot of each sample was returned to Battelle for reference analysis. The remaining collected
sample was stored for later use, if necessary. These aliquots were stored at approximately 4°C
and analyzed within appropriate holding times.
Freshwater samples from the Alum Creek Reservoir (AC), the Olentangy River (OR), and the
Scioto River (SR) (in Columbus, Ohio) were collected in 500-mL HDPE containers. The samples
were collected at the surface of the water near the shoreline by submerging the containers no
more than one inch below the surface of the water. Each body of water was sampled at four
distinct locations. The samples were split into three 100-mL aliquots. One aliquot of each sample
was analyzed in the field at the time of collection by each set of the test kit being verified. The
third aliquot of each sample was returned to Battelle for reference analysis. The remaining
collected sample was stored for later use, if necessary. These aliquots were stored at approxi-
mately 4°C and analyzed within appropriate holding times.
Three 100-mL aliquots of salt water were collected at the surface of the Massachusetts Bay
(MB-S) and from the sediment/water column interface (MB-B) at four distinct locations. One
aliquot of each sample was analyzed at the time of collection by each set of the test kit being
6
-------
verified. One aliquot of each sample was returned to Battelle for analysis by the reference
method.
3.4 Reference Analysis
The reference nitrate-N analysis was performed using a Dionex DX-500 ion chromatograph (IC)
according to EPA Method 300.1. The IC was equipped with a guard column, a separator column,
an anion suppressor, and a conductivity detector. A cleanup step and a pretreatment cartridge
were used for saltwater analysis. The instrument was calibrated with a minimum of three nitrate-
N standards bracketing the range of samples to be analyzed. The samples were introduced into
the instrument via an autosampler and swept through the columns using a 21 mM sodium
hydroxide solution. The columns separated the nitrate-N from other anions and potential inter-
ferents. The nitrate-N was detected by the conductivity detector, providing a detection limit of
0.02 ppm for nitrate-N. The analytical results were converted from nitrate to nitrate-N.
3.5 Verification Schedule
Round 1 of the F-NTK verification test took place over a 10-day period in January 2001, at
Battelle's laboratories in Columbus, Ohio, and in a subsequent three-day period in February 2001
at Battelle's Ocean Sciences Laboratory in Duxbury, Massachusetts. These two locations allowed
collection of a variety of drinking water and surface water samples for use in the verification.
Table 3-2 shows the daily testing activities that were conducted during these periods. During
January 2001, separate days were devoted to laboratory analysis of samples at Battelle, and to
collection and analysis of samples in the field. In all field locations, the collected samples were
analyzed using the F-NTK test kits shortly after collection, by both the technical and the non-
technical Battelle staff member. In February, the F-NTK kits were used on board a ship to
analyze surface water collected from Massachusetts Bay.
As described in Section 6 of this report, the comparison between the F-NTK results and the
reference results was poor for the freshwater field samples during Round 1. As a result, the
vendor requested that the freshwater field sample comparison be repeated. That retesting with
freshwater field samples was conducted at the vendor's expense, and is designated Round 2
testing in this verification. For this round of testing, the F-NTK kits incorporated new packaging
developed by the vendor to better preserve the reagents and make the kits easier to use. Further
comments on the changes in the kits are provided by the vendor in Appendix B of this report.
The Round 2 testing consisted of two phases. In the first phase, the freshwater field sampling and
analysis portion of the verification (shown as days 5, 6, and 7 in Table 3-2), was repeated by
Battelle technical and non-technical staff. Water samples were collected from Alum Creek
Reservoir, the Olentangy River, and the Scioto River on June 19 and 20, 2001. In addition to the
aliquots analyzed by the F-NTK and reference methods, additional aliquots of all field samples
from this sampling were stored at Battelle. Those aliquots were then used in the second phase of
7
-------
Table 3-2. Schedule of Round 1 Verification Test Days
Test Day
Testing Location
Activity
Day One
1/8-1/9/01
Battelle Columbus
Laboratory
Analysis of PT samples and associated QC samples.
Day Two
1/10/01
Battelle Columbus
Laboratory and a
Columbus Field Location
Collection and analysis of drinking water samples and
LFM samples within Battelle and at a residential site
using well water.
Day Three
1/11/01
Columbus Field Location
Collection and analysis of environmental samples and
LFM samples at a residential site using community
water.
Day Four
1/12/01
Battelle Columbus
Laboratory
Analysis of PT samples and associated QC samples.
Day Five
1/16/01
Columbus Field Location
Collection and analysis of environmental samples and
LFM samples at four locations on the Olentangy
River.
Day Six
1/17/01
Columbus Field Location
Collection and analysis of environmental samples and
LFM samples at four locations on the Scioto River.
Day Seven
1/18/01
Columbus Field Location
Collection and analysis of environmental samples and
LFM samples at four locations on the Alum Creek
reservoir.
Day Eight
2/5/01
Transport to Battelle
Duxbury, MA
Shipping and handling of analyzers undergoing
verification to field test site.
Day Nine
2/7/01
Duxbury, MA, Field
Location
Collection and analysis of environmental samples and
LFM samples at salt water locations; shipping of
environmental samples to Columbus for subsequent
reference analysis.
Round 2, on July 11, 2001. For that phase, a representative of NECi came to Battelle, and used
the F-NTK kits alongside a Battelle operator, to analyze the water samples collected in the field
in the first phase of Round 1. All data from both Round 1 and Round 2 of testing are presented in
this report, however, the data from Round 2 are more representative of the current F-NTK kits.
8
-------
Chapter 4
Quality Assurance/Quality Control
Quality assurance/quality control (QA/QC) procedures were performed in accordance with the
quality management plan (QMP) for the AMS Center(4) and the test/QA plan for this verification
test.(1)
4.1 QC for Reference Method
Field RB and laboratory blank samples were analyzed to ensure that no sources of contamination
were present. The test/QA plan stated that if the analysis of an RB sample indicated a concen-
tration above the method detection limit (MDL) for the reference instrument, any contamination
source was to be corrected and proper blank readings achieved before proceeding with the verifi-
cation test. A total of 36 laboratory blanks and four field reagent blanks were analyzed, and all of
the blanks analyzed were below the 0.02 ppm detection limit for nitrate-N.
The accuracy of the reference method was verified at the beginning and end of each day that
reference analyses were performed. The instrument used for the reference method was initially
calibrated according to the procedures specified in the reference method. Instrument calibration
was then verified using an appropriate QCS. If the QCS analysis differed by more than ± 10%
from the true value of the standard, the instrument was to be recalibrated before continuing the
test. As shown in Table 4-1, all of the QCS analyzed in both Round 1 and Round 2 were within
this required range, the maximum bias from the standard in any QCS analysis being 5.6%.
LFM samples were analyzed to assess whether matrix effects influenced the results of the
reference method. The percent recovery (R) of the spiked samples was calculated from the
following equation:
C -C
R = — x 100 (1)
s
where Cs is the analyzed concentration of the spiked sample, C is the analyzed concentration of
the unspiked sample, and 5 is the concentration equivalent of the analyte spike. If the percent
recovery of an LFM fell outside the range from 85 to 115%, a matrix effect was suspected. As
shown in Table 4-2, all of the LFMl samples from both rounds of testing were within this range,
and all but three of the LFMf samples from both rounds of testing were within this range. One
9
-------
Table 4-1. Reference Method QCS Analysis Results
Sample ID
Date of Analysis
Measured
Nitrate-N (ppm)
Actual
Nitrate-N (ppm)
Percent Bias
QCS-Initial
1/12/2001
4.96
5.00
0.8
QCS
1/12/2001
5.05
5.00
1.0
QCS
1/12/2001
5.04
5.00
0.8
QCS
1/12/2001
4.94
5.00
1.2
QCS-Final
1/12/2001
4.94
5.00
1.2
QCS-Initial
1/13/2001
4.98
5.00
0.4
QCS
1/13/2001
4.99
5.00
0.2
QCS-Final
1/13/2001
5.04
5.00
0.8
QCS-Initial
1/15/2001
5.11
5.00
2.2
QCS
1/15/2001
4.9
5.00
2.0
QCS-Final
1/15/2001
4.97
5.00
0.6
QCS-Initial
1/19/2001
4.89
5.00
2.2
QCS
1/19/2001
4.76
5.00
4.8
QCS
1/19/2001
4.79
5.00
4.2
QCS
1/19/2001
4.72
5.00
5.6
QCS-Final
1/19/2001
4.74
5.00
5.2
QCS-Initial
2/12/2001
4.8
5.00
4.0
QCS
2/12/2001
4.86
5.00
2.8
QCS-Final
2/12/2001
4.88
5.00
2.4
QCS-Initial
6/21/2001
4.96
5.00
0.8
QCS
6/21/2001
5.03
5.00
0.6
QCS
6/21/2001
5.01
5.00
0.2
QCS-Final
6/21/2001
5.10
5.00
2.0
river water sample from Round 1 showed a slightly high recovery, and one river sample from
Round 2 had a low recovery. The salt water sample also showed a low recovery. For the LFMf
samples, however, a matrix effect cannot be confirmed because other spiked samples do not
show a consistent pattern of recovery values.
10
-------
Table 4-2. Reference Method LFM Analysis Results
Unspiked Sample Spiked Sample Spiked Amount
Spiked
Sample ID
Date of
Analysis
Nitrate-N
(ppm)
Nitrate-N
(ppm)
Nitrate-N
(ppm)
Percent
Recovery
SPK-DW3 LFM,
01/12/2001
4.86
6.90
2.00
102.0
SPK-WW3 LFM,
01/12/2001
<0.02
1.72
2.00
86.0
SPK-CW3 LFM,
01/12/2001
1.64
3.38
2.00
87.0
SPK-OR LFMf
01/12/2001
2.15
4.50
2.00
117.5
SPK-SR LFMf
01/12/2001
6.78
8.91
2.00
106.5
WW1 LFMl
01/12/2001
<0.02
0.61
0.68
90.7
CW1 LFM
01/12/2001
1.67
2.42
0.68
110.7
SPK-AC LFMF
01/13/2001
2.01
3.73
2.00
86.0
SPK-MB LFMF
01/13/2001
0.09
1.43
2.00
67.0
DW1 LFMl
01/13/2001
4.64
5.36
0.68
105.0
OR1 LFMl
01/19/2001
0.75
1.46
0.68
104.7
AC1 LFMl
01/19/2001
1.77
2.38
0.68
89.3
MB1-BLFMl
2/09/2001
0.07
0.66
0.68
86.7
SPK-OR LFMF
06/21/2001
4.70
6.16
2.00
73.0
SPK-SR LFMF
06/21/2001
6.40
8.19
2.00
89.5
SPK-AC LFMF
06/21/2001
3.03
5.41
2.00
119.0
OR1 LFMl
06/21/2001
4.17
4.93
0.68
111.8
SRI LFM L
06/21/2001
6.54
6.82
0.68
41.2
AC1 LFMl
06/21/2001
3.14
3.74
0.68
88.2
a Average of four duplicates.
b Results after samples were diluted 1:3 prior to spike.
c BDL = below detection limit.
4.2 Audits
4.2.1 Performance Evaluation Audit
A performance evaluation (PE) audit was conducted to assess the quality of the reference
measurements made in this verification test. For the PE audit, an independent nitrate standard
was obtained from a different vendor than the one that supplied the calibration standards. The
performance evaluation standard was prepared and analyzed on January 19, 2001. Accuracy of
the reference instrument was determined by comparing the measured nitrate-N concentration
using the verification test standards to those obtained using the certified PE standard. Percent
difference was used to quantify the accuracy of the results. The PE sample for the nitrate-N was a
National Institute of Standards and Technology-traceable standard certified by THERMO Orion.
Agreement of the standard within 10% was required for the measurements to be considered
acceptable. Failure to achieve this agreement would have triggered recalibration of the
11
-------
instalments with the original QC standards and a repeat of the performance evaluation compari-
son. As shown in Table 4-3, the PE sample analyzed was well within this required range.
Table 4-3. Reference Method PE Audit Results
Measured
Actual Concentration
Date of
Nitrate-N
Nitrate-N
Percent
Sample ID
Analysis
(ppm)
(ppm)
Agreement
PE-1
01/19/2001
2.91
3.00
-3.0
4.2.2 Technical Systems Audit
The Battelle Quality Manager conducted a technical systems audit from January 8 to 16, 2001, to
ensure that the verification test was being performed in accordance with the test/QA plan and the
AMS Center QMP. The standard/solution preparation was observed on January 8; on January 9
the performance test; on January 11 the environmental testing (drinking water); on January 12 the
interference testing and reference method operations; and on January 16 environmental testing
(Olentangy River). As part of the audit, the reference standards and method used were reviewed,
actual test procedures were compared to those specified in the test/QA plan, and data acquisition
and handling procedures were reviewed. Observations and findings from this audit were docu-
mented and submitted to the Verification Test Coordinator for response. No findings were
documented that required any corrective action. The records concerning the technical systems
audit (TSA) are permanently stored with the Battelle Quality Manager.
4.2.3 Audit of Data Quality
At least 10% of the data acquired during the verification test was audited. Battelle's Quality
Manager traced the data from the initial acquisition, through reduction and statistical analysis, to
final reporting, to ensure the integrity of the reported results. All calculations performed on the
data undergoing the audit were checked.
4.3 QA/QC Reporting
Each assessment and audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the
QMP for the ETV AMS Center.(4) Once the assessment report was prepared, the Verification Test
Coordinator ensured that a response was provided for each adverse finding or potential problem
and implemented any necessary follow-up corrective action. The Battelle Quality Manager
ensured that follow-up corrective action was taken. The results of the TSA and the audit of data
quality were sent to the EPA.
12
-------
4.4 Data Review
Records generated in the verification test received a one-over-one review within two weeks of
generation, before these records were used to calculate, evaluate, or report verification results.
Table 4-4 summarizes the types of data recorded. The review was performed by a Battelle
technical staff member involved in the verification test, but not the staff member that originally
generated the record. The person performing the review added his/her initials and the date to a
hard copy of the record being reviewed.
Table 4-4. Summary of Data Recording Process
Data to be
Responsible
Where
How Often
Disposition of
Recorded
Party
Recorded
Recorded
Data3
Dates, times of
Battelle
Laboratory
Start/end of test
Used to
test events
record books
organize/check test
or ETV field
results; manually
data sheets
incorporated in data
spreadsheets as
necessary
Test parameters
Battelle
Laboratory
When set or changed,
Used to
(temperature,
record books
or as needed to
organize/check test
analyte/
or ETV field
document test
results, manually
interferant
data sheets
incorporated in data
identities, and
spreadsheets as
F-NTK results)
necessary
Reference method
Battelle
Laboratory
Throughout sample
Transferred to
sample analysis,
record books,
handling and analysis
spreadsheets
chain of custody,
data sheets, or
process
and results
data
acquisition
system, as
appropriate
11 All activities subsequent to data recording are carried out by Battelle.
13
-------
Chapter 5
Statistical Methods
The statistical methods presented in this chapter were planned for verifying the performance
factors listed in Section 3.1. However, in some cases the F-NTK test kits being verified yielded
only semi-quantitative results that did not lend themselves well to statistical evaluation. In such
cases qualitative comparisons are reported.
5.1 Accuracy
When possible, accuracy was assessed relative to the results obtained from the reference
analyses. Samples were analyzed by both the reference method and the test kit being verified.
The accuracy was expressed in terms of a relative average bias (B) as calculated from the
following equation:
where d is the average difference between the readings from the F-NTK kit and those from the
reference method, and CR is the average of the reference measurements.
Because of the semi-quantitative nature of the test kit results, it was not possible to make this
determination for much of the data. For this reason, all of the data were judged by a qualitative
measure. If the result from the test kit agreed with the reference result within the test kit's stated
precision capability of ± 1 ppm, the measurement was considered accurate; if it did not, the
measurement was considered not to be accurate. The percentage of accurate measurements was
determined for each of the four types of water samples as calculated from the following equation:
where A is the percent of accurate measurements, Y is the number of measurements within the ±1
ppm criteria, and T is the total number of measurements.
B = J-xlOO
Cr
(2)
Y
A = —xlOO
T
(3)
14
-------
The results were analyzed independently for the readings obtained from the two operators to
determine if significant operator bias exists.
5.2 Precision
When possible, the standard deviation (S) of the results for the replicate samples was calculated
and used as a measure of F-NTK precision at each concentration.
S =
t
k=i
1/2
(4)
where n is the number of replicate samples, Ck is the concentration measured for the kth sample,
and C is the average concentration of the replicate samples. The instrumental precision at each
concentration was reported in terms of the relative standard deviation (RSD), e.g.,
S
RSD =
C
xlOO
(5)
In some cases, F-NTK analysis results could only be quantified as within a range (e.g., 1 to 5
ppm). In those cases, both extremes of the range (i.e., 1 ppm and 5 ppm) were used as results to
calculate a range of RSD values.
5.3 Linearity
Linearity was assessed by linear regression of F-NTK and reference results. However, the F-NTK
test kit is semi-quantitative and was calibrated using 1-, 5-, and 10-ppm standards provided with
the test kit. Because the color developed by the successive standards often was barely
distinguishable, quantitative results were not often obtained with which to perform this
assessment.
5.4 Method Detection Limit
The MDL for the test kit was assessed from the seven replicate analyses of a fortified sample
with an analyte concentration of five times the vendor's estimated detection limit as described in
40 CFR part 136 Appendix B(3). The MDL was calculated from the following equation:
MDL = t xS (6)
where t is the Student's value for a 99% confidence level with n = 7, and S is the standard
deviation of the replicate samples. At times the operators could only report the results as a range
15
-------
of values. Because of this, the MDL was calculated as a range, using the lowest value and the
highest value of the range reported.
5.5 Matrix Interferences
The effect of interfering matrix species on the response of the test kit to a given analyte is
typically calculated as the ratio of the difference in analytical response to the concentration of
interfering species. For example, if the addition of 500 parts per billion (ppb) of an interfering
species results in a difference of 10 ppb in the analytical result, the relative sensitivity of the test
kit to that interferant would be calculated as 10 ppb/500 ppb = 2%. In this test, there were few
quantitative results for use in assessing the matrix interferences; therefore, only qualitative
observations could be made concerning matrix interferences.
5.6 Operator Bias
To assess operator bias for the test kit, the results obtained from each operator were compiled
independently and subsequently compared. However, because of the qualitative nature of the test
kit data, the existence of statistically significant operator bias could not be determined through a
Student's Mest of the data as planned. Qualitative observations are made concerning the results
from the two operators.
5.7 Rate of False Positives/False Negatives
The rate of false positives/false negatives of the test kit was assessed relative to the 2 ppm target
nitrate-N level. Analytes reported as being above that level by the test kit, but below that level by
the reference method, were considered false positives. Analytes reported as being below 2 ppm
level by the test kit, but reported as above that level by the reference method, were considered
false negatives. The rate of false positives/false negatives was expressed as a percentage of total
samples analyzed for each matrix.
16
-------
Chapter 6
Test Results
The results of the verification test of the F-NTK nitrate test kits are presented in this section.
Results from Round 1 and Round 2 of testing are presented separately, because of the different
test schedules, procedures, and test kit involved. Note that Appendix B of this report contains
vendor comments on the kits and test results, along with Battelle responses on some items.
6.1 Round 1 Results
6.1.1 Accuracy
Tables 6-la-d list the measured nitrate-N results from analysis of the four different types of water
samples in Round 1. Both reference and F-NTK results are shown in the tables, and F-NTK
results are shown for both the technical and non-technical operators. Sometimes F-NTK results
could only be quantified in terms of a range of values (e.g., 1 to 5 ppm). This occurred when the
difference in color between the 1-, 5-, and 10-ppm standards generated from the test kit was so
slight that the operator could not make a precise determination. Some samples could not be
distinguished from blank samples, and so were assigned a value of 0 ppm. No attempt was made
to dilute and reanalyze samples with concentrations greater than 10 ppm.
Tables 6-2a-d show the results of evaluating the accuracy of the F-NTK Round 1 results listed in
Tables 6-la-d. Shown in the first two columns in each of Tables 6-2a-d are the percent bias
values determined according to Equation 2, in Section 5.1. Clearly, relatively few of the F-NTK
results from Round 1 were quantitative enough to allow calculation of percent bias. In fact, for
the salt water field samples (Table 6-2d, respectively), no quantitative calculation of bias could
be made. The percent bias values that are shown in Tables 6-2 a-c range from 10 to over 560%,
indicating inconsistent behavior from the F-NTK kits in Round 1.
In the absence of quantitative bias results, the qualitative accuracy comparison using Equation 3
in Section 5.1 was used. The third and fourth columns in Tables 6-2 a-d show the assignment of
each F-NTK result, in terms of whether that result fell within ±1 ppm of the reference value.
Note that some of the F-NTK results were reported as a range of concentrations; those results
may be assigned as both meeting and not meeting the ±1 ppm criterion in Tables 6-2a-d. The
results of this qualitative evaluation of accuracy are shown in Table 6-3, which lists the overall
percent of results meeting the ±1 ppm accuracy criterion, for each operator and sample type in
17
-------
Table 6-la. Results from Round 1 Laboratory Performance Test Sample Analyses
Non-Technical Staff
Technical Staff
Reference Test
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
PT1-1
< 1
< 1
0.19
PT1-2
< 1
< 1
PT1-3
< 1
< 1
PT1-4
< 1
< 1
PT2-1
< 1
10
1.87
PT2-2
< 1
10
PT2-3
< 1
10
PT2-4
< 1
10
PT3-1
> 10
> 10
18.9
PT3-2
> 10
> 10
PT3-3
> 10
> 10
PT3-4
> 10
> 10
PT4-1
< 1
< 1
0.50
PT4-2
< 1
< 1
PT4-3
< 1
< 1
PT4-4
< 1
< 1
PT5-1
5
1-5
5.54
PT5-2
5
1-5
PT5-3
5
< 1
PT5-4
5
< 1
Table 6-lb. Results from Round 1 Drinking Water Analyses
Non-Technical Staff
Technical Staff
Reference Test
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
BF-1
5
5
4.71
BF-2
5
5
5.06
BF-3
5
5
4.62
BF-4
5
5
5.05
TH-1
< 1
<5
<0.02
TH-2
< 1
<5
<0.02
TH-3
< 1
<5
<0.02
TH-4
< 1
<5
<0.02
PH-1
1
< 1
1.67
PH-2
1
< 1
1.57
PH-3
1
< 1
1.66
PH-4
1
< 1
1.66
18
-------
Table 6-lc. Results from Round 1 Freshwater Analyses
Non-Technical Staff
Technical Staff
Reference Test
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
OR-1
0a
< 1
0.75
OR-2
0
< 1
2.68
OR-3
0
< 1
2.15
OR-4
0
< 1
2.56
SRI
< 1
2
6.79
SR2
< 1
1
6.43
SR3
> 10
1
6.87
SR4
10
1
6.78
AC-1
0
< 1
1.77
AC-2
0
< 1
2.01
AC-3
0
< 1
1.99
AC-4
0
< 1
1.98
a Entry of zero indicates sample
was indistinguishable from blank.
Table 6-ld. Results from Round 1 Salt Water Analyses
Non-Technical Staff
Technical Staff
Reference Test
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
MB1-S
< 1
< 1
0.07
MB1-B
< 1
< 1
0.07
MB2-S
< 1
< 1
0.05
MB2-B
< 1
< 1
0.07
MB3-S
< 1
< 1
<0.02
MB3-B
< 1
< 1
0.06
MB4-S
< 1
< 1
<0.02
MB4-B
< 1
< 1
0.09
19
-------
Table 6-2a. Accuracy of the F-NTKTest Kit with Laboratory Performance Test Samples for
Round 1
Percent Bias3
Non-Technical
Percent Bias3
Technical
Within Range (Y/N)b
Non-Technical Staff
Within Range (Y/N)b
Technical Staff
PT1-1
c
--
Y
Y
PT1-2
—
--
Y
Y
PT1-3
—
--
Y
Y
PT1-4
—
--
Y
Y
PT2-1
—
535
Y/N
N
PT2-2
—
535
Y/N
N
PT2-3
—
535
Y/N
N
PT2-4
—
535
Y/N
N
PT3-1
—
--
Y
Y
PT3-2
—
--
Y
Y
PT3-3
—
--
Y
Y
PT3-4
—
~
Y
Y
PT4-1
—
~
Y
Y
PT4-2
—
~
Y
Y
PT4-3
—
~
Y
Y
PT4-4
—
--
Y
Y
PT5-1
10
10-82
Y
Y/N
PT5-2
10
10-82
Y
Y/N
PT5-3
10
~
Y
N
PT5-4
10
~
Y
N
a Percent bias calculated according to Equation 2, Section 5-1.
ppm of reference. c No calculation of bias can be made.
b Y = result within ±1 ppm of reference; N=result not within ±1
Table 6-2b. Accuracy of the F-NTK Test Kit with Drinking Water Samples for Round 1
Percent Bias3
Non-Technical
Percent Bias3
Technical
Within Range (Y/N)b
Non-Technical Staff
Within Range (Y/N)b
Technical Staff
BF-1
6
6
Y
Y
BF-2
1
1
Y
Y
BF-3
8
8
Y
Y
BF-4
1
1
Y
Y
TH-1
c
--
Y
Y
TH-2
"
--
Y
Y
TH-3
"
~
Y
Y
TH-4
"
--
Y
Y
PH-1
40
~
Y
Y/N
PH-2
36
~
Y
Y/N
PH-3
40
~
Y
Y/N
PH-4
40
~
Y
Y/N
a Percent bias calculated according to Equation 2, Section 5-1. b Y=result within ±1 ppm of reference; N=result not within
±1 ppm of reference. c No calculation of bias can be made.
20
-------
Table 6-2c. Accuracy of the F-NTK Test Kit with Freshwater Samples for Round 1
Percent Bias3
Non-Technical
Percent Bias3
Technical
Within Range (Y/N)b
Non-Technical Staff
Within Range (Y/N)b
Technical Staff
OR-1
100
565
Y
N
OR-2
100
86
N
N
OR-3
100
132
N
N
OR-4
100
95
N
N
SR-1
c
--
N
N
SR-2
"
--
N
N
SR-3
"
--
N
N
SR-4
48
~
N
N
AC-1
100
--
N
Y/N
AC-2
100
~
N
N
AC-3
100
~
N
Y/N
AC-4
100
~
N
Y/N
a Percent bias calculated according to Equation 2, Section 5-1.
b Y = result within ±1 ppm of reference; N=result not within ±1 ppm of reference.
c No calculation of bias can be made.
Table 6-2d. Accuracy of the F-NTK Test Kit with Salt Water Samples for Round 1
Percent Bias3
Non-Technical
Percent Bias3
Technical
Within Range (Y/N)b
Non-Technical Staff
Within Range (Y/N)b
Technical Staff
MB1-S
c
--
Y
Y
MB1-B
"
--
Y
Y
MB2-S
"
~
Y
Y
MB2-B
"
~
Y
Y
MB3-S
"
~
Y
Y
MB3-B
"
--
Y
Y
MB4-S
"
~
Y
Y
MB4-B
"
~
Y
Y
a Percent bias calculated according to Equation 2, Section 5-1.
b Y = result within ±1 ppm of reference; N=result not within ±1 ppm of reference.
c No calculation of bias can be made.
21
-------
Table 6-3. Summary of Qualitative Accuracy Results from Round 1 of Testing
Percent Accuracy
Non-Technical staff
Percent Accuracy
Technical staff
Laboratory performance test samples
80-100
60-70
Drinking water samples
100
67-100
Freshwater samples
8
0-25
Salt water samples
100
100
Round 1 of testing. Table 6-3 shows that the qualitative accuracy of the F-NTK kits was
relatively high for laboratory and drinking water samples, with both technical and non-technical
operators. However, for freshwater field samples, the qualitative accuracy results were only 8%
for the technical operator, and 0 to 25% for the non-technical operator. For the salt water
samples, the qualitative accuracy was 100% with both operators. This result occurred because all
reference results for the salt water samples were less than 0.1 ppm, and all F-NTK results for
those samples were less than 1 ppm.
6.1.2 Precision
Tables 6-4a and b, respectively, show the data used to evaluate the relative standard deviation of
the F-NTK results for the replicate laboratory performance test samples and drinking water
samples in Round 1. Also shown in the tables is the percent RSD value for each set of replicate
analyses. Calculation of precision was complicated by the qualitative nature of the F-NTK
results. In most cases, percent RSD could not be calculated quantitatively, because all F-NTK
results were "greater than" or "less than" values. However, these data sets do illustrate
consistency in the F-NTK replicate analyses. In other cases, all F-NTK results for replicate
samples were equal. These cases exhibit the reproducibility achievable with the kits, and result in
percent RSD values of zero. Five of the 16 cases had this outcome. For the 5-ppm laboratory test
samples, the percent RSD for the samples analyzed by the technical staff member was 115%.
These results are based on assuming a value of zero for the "less than" F-NTK results.
6.1.3 Linearity
As is evident from the preceding sections, the F-NTK test kits yielded semi-quantitative results in
Round 1 testing. The kits were calibrated using standards included in the test kit, at nominal
concentrations of 1, 5, and 10 ppm. The color of the standards was compared with the color on
the chart included in the sample kits. In all cases, the prepared standards were used to determine
the results of the sample analyses. A new set of standards was analyzed each day by the non-
technical and technical operators. The operators observed wide variation in the coloration of the
standards, ranging from standards matching the color on the chart, to no color appearing in the
1-ppm standard, while the 5- and 10-ppm standard showed slight coloration, to absolutely no
color being formed in any of the standards. At no time did a more concentrated standard appear
lighter in color relative to a less concentrated standard. Overall, this variability in F-NTK-01
standards prevented any quantitative evaluation of linearity of response in Round 1.
22
-------
Table 6-4a. Precision Results for F-NTK Kits from Laboratory Performance Test Samples
for Round 1
Non-Technical Staff
Technical Staff
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
PT1-1
< 1
< 1
PT1-2
< 1
< 1
PT1-3
< 1
< 1
PT1-4
< 1
< 1
% RSD
a
--
PT2-1
< l
10
PT2-2
< l
10
PT2-3
< l
10
PT2-4
< l
10
% RSD
--
0%
PT3-1
> 10
> 10
PT3-2
> 10
> 10
PT3-3
> 10
> 10
PT3-4
> 10
> 10
% RSD
"
"
PT4-1
< 1
< 1
PT4-2
< 1
< 1
PT4-3
< 1
< 1
PT4-4
< 1
< 1
%RSD
"
"
PT5-1
5
1-5
PT5-2
5
1-5
PT5-3
5
< 1
PT5-4
5
< 1
%RSD
0%
115%b
a No % RSD calculated.
b For the purpose of calculating standard deviation, "less than" values were considered to be zero.
23
-------
Table 6-4b. Precision Results for F-NTK Kits from Drinking Water Samples for Round 1
Sample
Non-Technical StafP
Nitrate-N (ppm)
Technical StafP
Nitrate-N (ppm)
DW BF-1
5
5
DW BF-2
5
5
DW BF-3
5
5
DW BF-4
5
5
% RSD
0%
0%
WWTH-1
< 1
<5
WWTH-2
< 1
<5
WWTH-3
< 1
<5
WWTH-4
< 1
<5
% RSD
a
—
CW PH-1
l
< 1
CW PH-2
l
< 1
CW PH-3
l
< 1
CW PH-4
l
< 1
%RSD
0%
a No % RSD calculated.
6.1.4 Method Detection Limit
The method detection limit of the F-NTK kits used in Round lwas determined based on seven
replicate samples at a concentration of 2.5 ppm (five times the vendor's reported detection limit
of 0.5 ppm). The data, and parameters needed for the calculation of MDL by Equation 6 in
Section 5.4, are shown in Table 6-5. Shown are the values of S and t needed for the calculation,
and the resulting values for the MDL. The calculated MDL for the non-technical operator was 3.0
to 5.4 ppm and for the technical operator 10.8 to 11.2 ppm (Table 6-5). Thus, the analyte level in
the test sample was less than the determined MDL. These results are clearly influenced by the
large variability in F-NTK results. For example, results with the technical operator ranged from
<1 to 10 ppm in analysis of the 2.5 ppm sample. This variability is not considered characteristic
of the F-NTK kits; further comments on detection capabilities in the absence of this variability
are presented in Section 6.2.
6.1.5 Matrix Interferences
Tables 6-6a and b show the analytical results from Round 1 laboratory performance test samples
with low and high levels of interferences, respectively. The non-technical operator did not detect
any analyte in any of the low interference samples (Table 6-6a), and the technical operator did
not detect any analyte in three of those samples. In five of the eight samples spiked with a low
amount of interferences, the technical operator detected analyte at 167 to 333% of the true value
of the analyte. For the samples spiked with a high level of interferences (Table 6-6b), both
24
-------
Table 6-5. Method Detection Limit Results for the F-NTK Kits for Round 1 for Round 1
Non-Technical Staff
Technical Staff
Sample
Nitrate-N (ppm)a
Nitrate-N (ppm)a
PT6-1
< 1
1-5
PT6-2
< 1
1-5
PT6-3
2
10
PT6-4
1-5
1-5
PT6-5
2-3
1-5
PT6-6
2
< 1
PT6-7
2
< 1
Std. Dev'n. (S)
0.951 - 1.732
3.45-3.56
t at n=7b
3.14
3.14
MDLC
3.0-5.4
10.8-11.2
a For the purpose of calculating standard deviation, all "less than" values are considered to be zero.
ht is the Student's value for a 99% confidence level.
c MDL = t x S (see Section 5.4).
Table 6-6a. Data from Laboratory Performance Test Samples with Low-Level Interferences
for Round 1
Non-Technical Staff
Technical Staff
Reference Method
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
LI-1
< 1
<5
3.023
LI-2
< 1
<5
3.023
LI-3
< 1
<5
3.023
LI-4
< 1
5
3.023
LI-5
< 1
5
3.023
LI-6
< 1
10
3.023
LI-7
< 1
10
3.023
LI-8
< 1
10
3.02a
aLI solution only analyzed
once by reference method. Eight aliquots of single solution were
analyzed by F-NTK test kits.
Table 6-6b. Data from Laboratory Performance Test Samples with High-Level
Interferences for Round 1
Non-Technical Staff
Technical staff
Reference Method
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
HI-1
> 10
> 10
3.T
HI-2
> 10
> 10
3.T
HI-3
> 10
> 10
3.T
HI-4
> 10
> 10
3.T
HI-5
> 10
> 10
3.T
HI-6
> 10
> 10
3.T
HI-7
> 10
> 10
3.T
HI-8
> 10
> 10
3.T
aHI solution only analyzed once by reference method. Eight aliquots of single solution were analyzed by F-NTK test kits.
25
-------
operators reported values greater than 333% of the true value in all samples. These results
suggest that the higher level of interferences caused overestimation of the nitrate-N level to a
much greater extent than did the lower interference levels. However, the variability in test kit
behavior is likely to have affected these results, and no quantitation of interference effects can be
made.
6.1.6 Operator Bias
No evaluation of the effect of operator skill level could be made in Round 1 of testing because of
the variability and qualitative nature of the test results. The results of the Round 1 tests indicated
that the uncertainty of the individual test kit used, especially due to the color development of its
standards, was much greater than the variability due to the operator skill level.
6.1.7 Rate of False Positives/False Negatives
Tables 6-7 and 6-8, respectively, show the data and results for the rates of false positives and
false negatives obtained from the test kit in Round 1 of testing. All performance test and
environmental samples (Table 3-1) were considered for this evaluation. However, F-NTK values
reported as ranges spanning the 2-ppm midpoint value, or as a "less than" value that was greater
than the 2-ppm midpoint value, were deemed not applicable to this evaluation.
Table 6-7 shows that there were 32 samples with reference analyte concentrations less than the
target midpoint of 2 ppm. The samples tested by the technical operator showed four samples that
were not applicable to this analysis, and four others in which F-NTK results exceeded 2 ppm.
The result was a false positive rate of 14% relative to the 2 ppm value. The samples tested by the
non-technical operator had a false positive rate of 0%, with no F-NTK results above the 2-ppm
midpoint value.
Table 6-8 shows that there were 43 samples with reference analyte concentrations greater than
the target midpoint of 2 ppm. The samples tested by the technical operator had a false negative
rate of 32%. In 11 of the samples, the analyte was detected at a level less than 2 ppm, and nine of
the samples were not applicable. The samples tested by the non-technical operator had a false
negative rate of 38%. In 16 of the samples, the analyte was detected at a level less than 2 ppm,
and one of the samples was not applicable.
6.1.8 Other Factors
The operators felt the F-NTK test kit was easy to use and free of maintenance. The kit was easy
to transport by car, boat, and airplane. The sample test tube rack included in the F-NTK-01 made
it easy for the operators to perform the analysis in the field with rough terrain or bumpy seas. The
test kit allowed analysis of three standards and five samples. The preparation of the reagents in
each kit takes approximately 45 minutes, and the analysis of the standards and samples take
approximately 45 minutes each.
26
-------
Table 6-7. Rate of False Positives from F-NTK Test Kit for Round 1
Technical Result
Non-Technical
Reference Method
Technical
Non-Technical
Nitrate-N
Result Nitrate-N
Result Nitrate-N
False Positive
False Positive
Sample
(ppm)
(ppm)
(ppm)
(Y/N)
(Y/N)
PT1-1
< 1
< 1
0.19
N
N
PT1-2
< 1
< 1
0.19
N
N
PT1-3
< 1
< 1
0.19
N
N
PT1-4
< 1
< 1
0.19
N
N
PT2-1
10
< 1
1.87
Y
N
PT2-2
10
< 1
1.87
Y
N
PT2-3
10
< 1
1.87
Y
N
PT2-4
10
< 1
1.87
Y
N
PT4-1
< 1
< 1
0.50
N
N
PT4-2
< 1
< 1
0.50
N
N
PT4-3
< 1
< 1
0.50
N
N
PT4-4
< 1
< 1
0.50
N
N
WW-1
<5
< 1
<0.02
NA
N
WW-2
<5
< 1
<0.02
NA
N
WW-3
<5
< 1
<0.02
NA
N
WW-4
<5
< 1
<0.02
NA
N
CW-1
< 1
1
1.67
N
N
CW-2
< 1
1
1.57
N
N
CW-3
< 1
1
1.66
N
N
CW-4
< 1
1
1.66
N
N
OR-1
< 1
0
0.75
N
N
AC-1
< 1
0
1.77
N
N
AC-3
< 1
0
1.99
N
N
AC-4
< 1
0
1.98
N
N
MB1-S
< 1
< 1
0.07
N
N
MB1-B
< 1
< 1
0.07
N
N
MB2-S
< 1
< 1
0.05
N
N
MB2-B
< 1
< 1
0.07
N
N
MB3-S
< 1
< 1
<0.02
N
N
MB3-B
< 1
< 1
0.06
N
N
MB4-S
< 1
< 1
<0.02
N
N
MB4-B
< 1
< 1
0.09
N
N
Total number of applicable samples
28
32
Total false positives
4
0
Percent false positives
14%
0%
NA = Not applicable
Y = Yes
N = No
27
-------
Table 6-8. Rate of False Negatives from F-NTK Test Kit for Round 1
Sample
Technical
Result Nitrate-N
(nnm)
Non-Technical
Result Nitrate-N
(nnm)
Reference Method
Result Nitrate-N (nnm)
Technical
False Negative
(Y/N)
Non-Technical
False Negative
(Y/N)
PT3-1
> 10
> 10
18.87
N
N
PT3-2
> 10
> 10
18.87a
N
N
PT3-3
> 10
> 10
18.87a
N
N
PT3-4
> 10
> 10
18.87a
N
N
PT5-1
1-5
5
5.54
NA
N
PT5-2
1-5
5
5.54a
NA
N
PT5-3
< 1
5
5.54a
Y
N
PT5-4
< 1
5
5.54a
Y
N
PT6-1
1-5
< 1
2.38
NA
Y
PT6-2
1-5
< 1
2.38a
NA
Y
PT6-3
10
2
2.38a
N
N
PT6-4
1-5
1-5
2.38a
NA
NA
PT6-5
1-5
2-3
2.38a
NA
N
PT6-6
< 1
2
2.38a
Y
N
PT6-7
< 1
2
2.38a
Y
N
DW-1
5
5
4.71
N
N
DW-2
5
5
5.06
N
N
DW-3
5
5
4.62
N
N
DW-4
5
5
5.05
N
N
LI-1
<5
< 1
3.02
NA
Y
LI-2
<5
< 1
3.02a
NA
Y
LI-3
<5
< 1
3.02a
NA
Y
LI-4
5
< 1
3.02a
N
Y
LI-5
5
< 1
3.02a
N
Y
LI-6
10
< 1
3.02a
N
Y
LI-7
10
< 1
3.02a
N
Y
LI-8
10
< 1
3.02a
N
Y
HI-1
> 10
> 10
3.69
N
N
HI-2
> 10
> 10
3.69a
N
N
HI-3
> 10
> 10
3.69a
N
N
HI-4
> 10
> 10
3.69a
N
N
HI-5
> 10
> 10
3.69a
N
N
HI-6
> 10
> 10
3.69a
N
N
HI-7
> 10
> 10
3.69a
N
N
HI-8
> 10
> 10
3.69a
N
N
OR-2
< 1
0
2.68a
Y
Y
OR-3
< 1
0
2.15
Y
Y
OR-4
< 1
0
2.56
Y
Y
SR-1
2
< 1
6.79
N
Y
SR-2
1
< 1
6.43
Y
Y
SR-3
1
>10
6.87
Y
N
SR-4
1
10
6.78
Y
N
AC-2
< 1
0
2.01
Y
Y
Total number of applicable samples
Total false negatives
Percent false negatives
34
11
32%
42
16
38%
NA = Not applicable
Y = Yes
N = No
a Only one sample analyzed by reference method. Multiple aliquots of sample were analyzed by F-NTK test kit.
28
-------
Each F-NTK test kit includes five reagent packets. Each packet contains enough reagent to
analyze three standards and five samples. When more than five samples were analyzed in a day,
additional reagent packets were used without preparing additional standards.
The biggest difficulty in Round 1 was the large difference from one reagent pack to another in
the development of the color in the nitrate-N standards. At times it was difficult to discern the
faint gradations in the pink color in the standards and samples. There were times when the 1 ppm
standard appeared as colorless as the blank. One set of standards had no color development at all.
The operators' impression was that the analysis of the samples was accurate when the color of
the standards developed completely and was comparable to the color chart provided in the kit.
However, they felt that when an additional reagent packet was required, the samples could not be
compared accurately to the standards prepared from the previous packet.
6.1.8.1 Costs
The cost of one F-NTK test kit is $30. The kit contains five reagent packets, each of which is
sufficient to conduct five sample analyses.
6.1.8.2 Data Completeness
All portions of the verification test were completed, and all data that were to be recorded were
successfully acquired. Thus, data completeness was 100%.
6.2 Round 2 Results
Round 2 of this verification involved analysis of freshwater field samples, blanks, and spiked
samples only. The first phase of Round 2 involved the same Battelle operators as in Round 1; the
second phase of Round 2 involved the Battelle technical operator and a vendor representative.
The results of the second round of the verification test of the F-NTK are presented in this section.
6.2.1 Accuracy
Table 6-9 lists the F-NTK results and reference data obtained in the field in the first phase of
Round 2 by the technical and non-technical Battelle operators. Table 6-10 lists the F-NTK results
and reference data obtained in the laboratory in the second phase of Round 2 by the vendor
representative and the Battelle technical operator.
29
-------
Table 6-9. Field Results from Freshwater Samples, Round 2, First Phase
Non-Technical Staff
Technical Staff
Reference Test
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
OR-Blank
<1
< 1
<0.02
QCS
1.5
<1
1.89
OR-1
3
<1
4.17
OR-2
3
3
3.99
OR-3
4
< 1
3.91
OR-4
1.5
<1
4.70
LFMf
1.5
1
6.16
SR-Blank
<1
<1
<0.02
QCS
3
2
1.84
SRI
1.5
<1
6.54
SR2
<1
<1
6.48
SR3
4
4
6.27
SR4
1.5
2
6.40
LFMf
3
2
8.19
AC-Blank
<1
<1
<0.02
QCS
<1
<1
1.84
AC-1
2
< 1
3.14
AC-2
<1
< 1
3.09
AC-3
<1
<1
2.96
AC-4
<1
< 1
3.03
LFM,
<1
< 1
5.41
Inspection of Table 6-9 shows that in the field testing that comprised the first phase of Round 2,
neither the technical nor non-technical staff obtained high accuracy with the F-NTK test kits.
Although the field and spiked samples contained approximately 3 to 8 ppm of nitrate-N, many of
the F-NTK results were reported as < 1 ppm. Such non-detect values indicate reasonable
accuracy when analyzing the three blank samples, given the 1 ppm resolution of the F-NTK kits.
However, both operators reported < 1 ppm results even with samples containing as much as
6.5 ppm nitrate-N (sample SR2, Table 6-9). Of the 18 non-blank samples in Table 6-9, quantita-
tive results (i.e., not less-than values) were reported for 12 samples by the non-technical operator,
and for just six samples by the technical operator. When those subsets of the results are treated by
Equation 2 (Section 5.1) to determine the relative average bias of the F-NTK kits, a relative
average bias of 52.8% is found for the results with the non-technical operator, and 58.4% for the
results with the technical operator.
The F-NTK vendor indicated that the erratic performance of the F-NTK kits in the first phase of
Round 2 was unusual. As a result a second phase of Round 2 was conducted, consisting of a
reanalysis of aliquots of the same freshwater field samples. This time the analysis was performed
in the laboratory by the technical operator and a vendor representative. In this second phase of
30
-------
Round 2, the performance of the F-NTK kits was greatly improved, in that all the 1, 5, and 10
ppm standards included in the kits gave definite and easily distinguishable color formation. The
F-NTK and reference results from that second phase are shown in Table 6-10, which lists results
from the F-NTK kits with both the vendor representative and the Battelle technical operator.
Table 6-10. Laboratory Results from Reanalysis of Freshwater Samples, Round 2,
Second Phase
Vendor
Battelle
Representative
Technical Staff
Reference Test
Sample
Nitrate-N (ppm)
Nitrate-N (ppm)
Nitrate-N (ppm)
OR-Blank
<1
<1
<0.02
QCS
2
2
1.89
OR-1
5
5
4.17
OR-2
3
4
3.99
OR-3
5
5
3.91
OR-4
2
4
4.70
LFMf
3
5
6.16
SR-Blank
<1
<1
<0.02
QCS
2
2
1.84
SRI
3
5
6.54
SR2
5
3
6.48
SR3
4
3
6.27
SR4
5
3
6.40
LFMf
5
5
8.19
AC-Blank
<1
<1
<0.02
QCS
1
1
1.84
AC-1
4
4
3.14
AC-2
2
2
3.09
AC-3
3
2
2.96
AC-4
4
3
3.03
LFMf
5
5
5.41
Those data indicate much better accuracy relative to the reference results, compared to the first
phase of Round 2. With both the vendor and Battelle operator, the F-NTK kits correctly gave
< 1 ppm values only for the three blank samples, and gave quantitative results for all of the other
18 samples. When those 18 sample results are treated by Equation 2 (Section 5.1) to determine
the relative average bias of the F-NTK kits, a relative average bias of 31.4% is found for the
results with the vendor operator, and 28.9% for the results with the Battelle technical operator.
31
-------
For consistency with the data evaluation in Section 6.1.1, the results of Round 2 have also been
evaluated in terms of the frequency with which F-NTK results were within the expected 1 ppm
precision relative to the reference results. Tables 6-11 and 6-12 show the results of this compari-
son for each sample in the first and second phases of Round 2 (Tables 6-9 and 6-10, respec-
tively). Table 6-13 summarizes the results, listing for each operator in each phase of Round 2 the
percentage of results within 1 ppm of the reference. The second phase results are clearly better
than the first phase results. For the second phase, 57% of the results with the vendor operator and
62% of the results with the Battelle operator were within the stated 1-ppm precision of the kits.
Furthermore, if the precision criterion is relaxed slightly, to 1.1 ppm, the resulting percentages
increase substantially, i.e., to 67% with the vendor operator and 72% with the Battelle operator.
These results show that the F-NTK kits are capable of approximately 1-ppm resolution at nitrate-
N concentrations of less than 10 ppm.
The improved performance in the second phase of Round 2 appears related to the improved
packaging of the F-NTK kits. The Battelle operators noted that the kits used in that phase of
testing produced clear and consistent color changes for all standards and solutions, unlike the
inconsistent to non-existent changes observed in some of the previous test activities. As a final
evaluation of the F-NTK kits, the kit that did not produce color change in the first phase of
Round 2 was used in conjunction with a kit from the second phase of Round 2. In this evaluation,
2 ppm and 5 ppm nitrate-N standards were analyzed using the new version of the kit, but with
one reagent at a time from the old kit substituted for the corresponding reagent in the new kit.
This evaluation showed that, whereas the buffer, color reagent, and NADH reagent in the old kit
worked normally, the enzyme reagent in the old kit was inactive. This finding is consistent with
the highly variable results seen in parts of this verification test, and suggests that the previous
means of packaging the F-NTK test kits may have resulted in loss of enzyme activity. In any
case, the results from the second phase of Round 2 clearly indicate that the current packaging
approach of the F-NTK kits preserves the enzyme activity.
32
-------
Table 6-11. Performance of the F-NTK Test Kit Relative to 1-ppm Precision on the Field
Freshwater Samples, Round 2, First Phase
Within Range of
Within Range of
1 ppm (Y/N)
1 ppm (Y/N)
Sample
Non-Technical Staff
Technical Staff
OR-Blank
Ya
Y
QCS
Y
N
OR-1
N
N
OR-2
Y
Y
OR-3
Y
N
OR-4
N
N
LFM,
N
N
SR-Blank
Y
Y
QCS
N
Y
SRI
N
N
SR2
N
N
SR3
N
Y
SR4
N
N
LFM,
N
N
AC-Blank
Y
Y
QCS
N
N
AC-1
N
N
AC-2
N
N
AC-3
N
N
AC-4
N
N
LFM,
N
N
a Y = result within 1 ppm of reference; N = result not within
1 ppm of reference.
33
-------
Table 6-12. Performance of the F-NTK Test Kit Relative to 1 ppm Precision on the
Laboratory Reanalysis of Freshwater Samples, Round 2, Second Phase
Sample
Within Range of
1 ppm (Y/N)
Vendor Representative
Within Range of
1 ppm (Y/N)
Technical Staff
OR-Blank
Ya
Y
QCS
Y
Y
OR-1
Y
Y
OR-2
Y
Y
OR-3
N
N
OR-4
N
Y
LFMf
N
N
SR-Blank
Y
Y
QCS
Y
Y
SRI
N
N
SR2
N
N
SR3
N
N
SR4
N
N
LFMf
N
N
AC-Blank
Y
Y
QCS
Y
Y
AC-1
Y
Y
AC-2
N
N
AC-3
Y
Y
AC-4
Y
Y
LFMf
Y
Y
a Y = result within 1 ppm of reference; N = result not within 1 ppm of reference.
34
-------
Table 6-13. Summary of F-NTK Results Relative to 1 ppm Precision for Round 2
Percent of Results
Test Phase
F-NTK Operator
Within 1 ppm of Reference
First
Non-technical Battelle
29
Technical Battelle
19
Second
Vendor Representative
57
Technical Battelle
62
6.2.2 Linearity
The linearity of the F-NTK readings was assessed by means of a linear regression of the F-NTK
results against the reference results, using the 21 data points from the second phase of Round 2
(Table 6-10). In this regression, results reported as below detection limit by either the F-NTK or
the reference method were assigned a value of half the detection limit. For the data in Table 6-10,
only the blank values showed non-detect results. Therefore the blank results were each assigned
values of 0.01 ppm for the reference value and 0.5 ppm for the F-NTK value. Figure 6-1 shows a
scatter plot of the F-NTK data from both the vendor and the Battelle technical operators, versus
the reference nitrate-N results. The one-to-one line is also shown in Figure 6-1.
A linear regression of the data in Figure 6-1 gives the following regression equations:
for the F-NTK with the vendor operator,
F-NTK, ppm = 0.538 (±0.208) x (Reference, ppm) + 1.02 (±0.926) ppm,
with r = 0.779;
for the F-NTK with the Battelle technical operator,
F-NTK, ppm = 0.545 (±0.204) x (Reference, ppm) + 1.00 (±0.908) ppm,
with r = 0.789,
where the values in parentheses represent the 95% confidence interval of the slope and intercept.
The slopes are significantly different from 1.0, and the intercepts are significantly different from
zero.
35
-------
10
~ Vendor Representative
X Technical Operator
1:1 Line
X«K
4 6
Reference Nitrate-N, ppm
10
Figure 6-1. Comparison of F-NTK Results to Reference Results from Round 2, Phase 2
6.2.3 Detection Limit
In Round 2 of verification of the F-NTK kits, the method detection limit was not determined
according to the procedure used in Section 6.1.4. However, there was clear color development in
all the F-NTK 1 ppm standards, and in all the 2 ppm QCS samples shown in Table 6-10. These
results indicate that the F-NTK kits with new packaging are capable of detecting nitrate-N at
levels at least as low as 1 ppm.
6.2.4 Operator Bias
The regression results presented in Section 6.2.2 are closely similar for the F-NTK kits with two
different operators. In fact, consideration of the 95% confidence intervals of the regression
results shows that neither the two slope values nor the two intercepts are significantly different
from one another. Thus, at least in this case, there is no evidence of any operator bias in the
readings of the F-NTK test kits.
36
-------
Chapter 7
Performance Summary
The F-NTK test kits demonstrated inconsistent performance in much of this verification test, in
the form of widely different levels of color formation from nitrate-N standards when different
test reagent kits were used. The current packaging for the F-NTK kits appears to have resolved
that problem. A second round of testing was requested and funded by the vendor because of this
inconsistent performance. Unfortunately, performance of some of the tested kits was affected by
this variability, and only qualitative assessments could be made of some of the kit's performance
characteristics, including precision, detection limit, effect of operator skill level, and effects of
interferences.
Quantitative accuracy could not be assessed in Round 1 testing. In Round 2, the average percent
bias of the current F-NTK test kit relative to the reference method was 28.9% with a technically
trained Battelle operator, and 31.4% with a vendor operator, at nitrate-N levels of about 2 to
8 ppm in surface freshwater samples. An additional criterion for accuracy was the percentage of
samples for which the F-NTK result was within 1 ppm of the reference result. By this criterion
the F-NTK kits yielded accurate results for 80 to 100% of laboratory performance samples with a
non-technical operator, and 60 to 70% with a technical operator. The corresponding percentages
for drinking water samples were 100% with the non-technical operator and 67 to 100% with the
technical operator. For surface freshwater samples, the corresponding averages were 57% with a
vendor operator and 62% with a Battelle technical operator. For saltwater samples, the F-NTK
kits with both technical and non-technical operators correctly indicated that nitrate-N levels were
below 1 ppm in 100% of the samples.
In Round 1, no quantitative evaluation of linearity of response could be made due to variation in
color of standards prepared from the F-NTK. In Round 2, the linearity of response of the current
F-NTK kits was assessed using surface freshwater samples containing about 2 to 8 ppm nitrate-
N. The linear regression equation for F-NTK results with the vendor representative as operator
was: F-NTK ppm nitrate-N = 0.538 x (Reference ppm) + 1.02 ppm, with a correlation coefficient
(r) of 0.779. The corresponding equation for results with the Battelle technical operator was:
F-NTK ppm nitrate-N = 0.545 x (Reference ppm) + 1.00 ppm, with a correlation coefficient (r)
of 0.789.
The precision of the F-NTK kits could not be evaluated quantitatively in Round 1, because of
frequent non-detect results. However, when detectable results were obtained the F-NTK kits
typically gave the same result for each replicate analysis of a single sample. The determination of
37
-------
the F-NTK detection limit was similarly hampered by variability in the response of the kits in
Round 1, but 1 ppm levels were readily measured with the current version of the kits that was
used in the second phase of Round 2 testing.
In Round 1 testing, the presence of high levels of iron, NaCl, sulfate, and acid resulted in greater
overestimation of nitrate-N levels than did low levels of these interferents. However, no
quantitation of interference effects could be made.
The rates of false positives and false negatives of the F-NTK test kits were assessed relative to
the reference method in Round 1, using 2 ppm nitrate-N as the decision level. The rate of false
positives of the F-NTK test kit was 14% when used by the technical operator, and zero when
used by the non-technical operator. The rate of false negatives was 32% with the technical
operator and 38% with the non-technical operator.
Each F-NTK kit costs $30, and includes five packets of reagents, each of which is capable of
analyzing three standards and five samples. The test kit allowed analysis of three standards and
five samples. The preparation of the reagents in each kit takes approximately 45 minutes, and
the analysis of the standards and samples take approximately 45 minutes each. The F-NTK test
kit was easy to use, easy to transport, and required no maintenance. The test results show no
strong effect of the operator skill level on F-NTK results. Data completeness in the test was
100%.
38
-------
Chapter 8
References
1. Test/QA Plan for Verification of Portable Analyzers, Battelle, Columbus, Ohio,
December 8, 2000.
2. Methods for the Determination of Inorganic Substances in Environmental Samples, EPA
16001R-93-100 - August 1993.
3. U.S. Code of Federal Regulations, Title 40, Part 136 Appendix B.
4. Quality Management Plan (QMP) for the E TV Advanced Monitoring Systems Pilot,
Version 2.0, U.S. EPA Environmental Technology Verification Program, Battelle,
Columbus, Ohio, October 2000.
39
-------
Appendix A
Data Recording Sheet
A-l
-------
ETV Field Data Sheet
T echni ci an:
Time:
Date:
Air Temperature: (F) (C)
Humidity:
Barometric Pressure: (in Hg)
Weather Conditions:
Sample Location:
Total Amount Collected: (L)
Water Temperature: (F) (C)
Sample ID
Results
Notes
Comments:
Technician's Signature: Date:
Technical Reviewer's Signature: Date:
A-2
-------
Appendix B
Vendor Comments
B-l
-------
Appendix B
Vendor Comments
(Battelle notes in parentheses and italics)
1. How the NECi Nitrate Assay Works
Enzymes are highly effective protein catalysts that accelerate specific chemical reactions in living
systems. The enzyme used in the NECi test kits is nitrate reductase (NaR), which we purify from
corn seedlings or yeast. The biological electron donor NADH (nicotinamide adenine
dinucleotide, a B vitamin) provides the electrons required for the reduction. The catalytic rate of
NaR is about 200 nitrate to nitrite conversions per second per molecule of NaR. The reaction is
irreversible and goes to completion:
NADH + NITRATE --> NITRITE + NAD+ + OH"
The resulting nitrite is then detected using standard Griess reaction chemistry. Nitrite in an acid
solution will react with sulfanilamide and N-Naphthylethylenediamine (often called NED) to
form a pink product. The intensity of the color is directly proportional to the quantity of nitrite in
the solution. (The more color, the more nitrite is present.) Because all of the nitrate in the sample
has been converted - reduced — to nitrite by the nitrate reductase enzyme, the pink color is also
directly proportional to the amount of nitrate that was present in the sample. In NECi's lab format
nitrate test kits, the acid used is 3N HC1, and test results are read using a photometer or
Microplate reader. A less corrosive, solid organic acid is used in our Field and Consumer kits to
make them safer and more environmentally friendly. Nitrate content in these kits is determined
based on visual comparison to nitrate standards and color charts when a photometer is not
available.
NECi's nitrate assay is therefore comparable to EPA Standard Method 353.2 & 353.3, APHA
4500-N03E & F, USGS 1-2545-90, etc. The difference is that the nitrate reduction step is
catalyzed by a rapid, selective, and highly specific enzyme rather than by cadmium, a toxic heavy
metal. It is also comparable to the reagent system used in many conventional nitrate test kits and
methods: again, the difference is that the toxic and less specific cadmium or zinc catalyst has
been replaced by a protein.
Substances that can interfere with NaR activity have been fully studied and are limited to NaCl
(NECi has saltwater kits that overcome this problem), and millimolar levels of chlorate and
heavy metals.
NECi designed all test kits to give a strong color response to nitrate so that users can easily see
the difference between 1 and 10 ppm nitrate-N in the Standard range kits, and between 0.05 -
1.0 ppm in our Low Range products. Color intensity of the 10 ppm standard yields an absorbance
at 540nm (A540) between 0.6 and 0.8 absorbance units (AU), i.e., a very dark pink solution. The
kits are designed so that virtually 100% of the nitrate in a 50 /A sample of a 10 ppm nitrate-N
B-2
-------
solution will be reduced to nitrite within 15 minutes. This is achieved by carefully controlling the
amount of enzyme used per assay.
The Field kit reagent system was developed with funding from the Small Business Innovation
Research program of the US Dept of Agriculture. A list of publications and abstracts describing
the assay is included at the end of this document.
2. Standard Curves and Product Stability
The Field kits are designed so that they can be stored at 4°C or colder for at least six months; the
stability of the NaR (enzyme) reagent is not guaranteed after that. The freeze-dried desiccant
packets of NaR will retain sufficient activity to perform within NECi specifications after being
exposed to up to six days of 85 °F (typical summer weather); this permits NECi to ship at
ambient temperature and allows users to bring kits to a field site. The NADH reagent also needs
to be handled as specified. NADH is stable for long periods in the dry form, but loses viability
(becomes oxidized to NAD+) once it is in solution; NECi suggests a four hour window unless the
reagent is stored below 5°C.
Once the NaR has been dissolved, it is stable for:
• Up to four hours once diluted in the Assay Buffer
• 24 -36 hours at 85 °F if reconstituted with NECi Enzyme Diluent
• 3-5 days at 4°C (standard refrigerator) in Enzyme Diluent
• Up to 12 months when stored in a freezer in Enzyme Diluent.
All NECi kits come with nitrate standards. Users are instructed to run at least one nitrate standard
every time they run an assay (use the kit). We provide three concentrations of nitrate standard
with each kit, and recommend using all three. This insures the user that 1) they have followed the
instructions, and 2) all reagents are working. The color development of the standards also
compensates for any degradation in reagent quality that may have occurred over time. Any data
obtained without running at least one nitrate standard concurrently with samples is invalid. We
also stress that the 10 ppm standard should yield a decidedly pink color (or dark gray when the
user is colorblind).
NECi's QC procedures specify that a standard curve generated by the Standard Range Field
Kits will yield a Blank of less than 0.025 AU at 540nm, and a line with a slope of at least
0.06 AU/ppm (meaning that the 10.0 ppm nitrate standard gives an absorbance at 540nm of at
least 0.6 AU) with linearity of 0.95 or better.
Additional comments:
NECi has developed a Field Kit designed for analysis of low levels of nitrate in sea water, which
has been used by Woods Hole and other demanding researchers for determination of nitrate
levels between 0.05 - 1.0 ppm nitrate. Battelle adopted a literal interpretation of what constitutes
a different product, which prevented us from providing our Low Range Seawater Kit format for
testing.
B-3
-------
(Battelle Note: ETVrequirements are that a verification must address a single technology. The
identification of the F-NTK kit as the subject of this verification was clearly specified in the ETV
Vendor Agreement thatNECi signed in order to participate in this test. That Vendor Agreement
states that "ETV verification applies only to the single model of the subject technology submitted
by the vendor for verification. ")
NECi made changes in packaging format between Rounds 1 and 2 of the verification tests,
because we are still refining the kits to be as user-friendly as possible without loss of accuracy or
major increase in cost per test. Our customers had not liked the new format because they felt
there were too many mixing steps. Since every step any user (novice or otherwise) makes
increases the chance of introducing error in following any protocol, we decided to simplify the
format. Any changes in kit formats have been limited to things such as number of assays per
batch of reagent, or in how many premixed or premeasured reagents we can afford to provide.
The data from the second round of testing would have been unacceptable at NECi, falling far
outside our QC/QA specifications. We are especially concerned with the Interference and
Standard Addition studies data. The spiking studies, in which a known amount of nitrate was
added to a solution containing no or little nitrate, failed to show any response to the added nitrate.
NECi has performed hundreds of spiking studies, in deionized water, tap water, lake water, water
from fish tanks, plant extracts, tissue culture media, etc. All yielded quantitative response to
added nitrate. Some of this data has been published (see for example the Current Protocols
chapter, reference number 4 in the NECi Publications list below). We do not understand why the
data from the verification test show no response whatsoever. None of the labs to whom we have
sent prototypes and products has ever reported such a result to us.
The Interference data is also puzzling. The data from the High Level study show increased
nitrate values in the presence of moderate levels of iron, sulfate, and sodium chloride. These
results cannot be supported in the literature. Sulfate has no affect, inhibitory or stimulating, on
nitrate reductase activity. Metals can be inhibitors, but only at higher concentrations and when no
EDTA is present in the buffer (we include this chelator in the Assay Buffer in all kits). Sodium
chloride is a known inhibitor of the enzyme at 0.3%, its concentration in seawater, meaning that,
if anything, the response to nitrate would have been decreased. See the review by WH Campbell
(,Annual Reviews in Plant Physiology and Plant Molecular Biology, 1999, reference number 1
in the Recent Academic Publications list below) for confirmation of these statements. And we
cannot find any information in the literature that these concentrations of these chemicals can
interfere in any way with the Griess reagents. We must conclude that there was a problem with
this part of the work that had nothing to do with our reagent system.
(Battelle Note: All instructions for use of the F-NTK kits were followed in performing the
verification test, and all test procedures were conducted and documented as specified in the
test/QA plan for this verification. Technical Systems Audits confirmed that the procedures were
performed properly. The literature on potential interferences notwithstanding, the observations
from the matrix interference test were as presented in this report. Those results may well have
been affected by the variability observed in test kit response in Round 1 of testing. In any case,
the report makes no quantitative conclusions about the extent of interferences.)
B-4
-------
We have never understood why the verification team rejected all suggestions to read the results
of the testing in a photometer after the color observations had been recorded. We make very
specific claims about test kit performance in our literature, and there is no way to prove or
disprove our claims without use of a photometer. In addition, it would have been helpful for
Battelle personnel to gauge the accuracy of their color judgment by comparing their observations
with instrument data. We would be embarrassed to sell a product that could not differentiate
between 5 and 10 ppm. I have made presentations at conferences, and we have written articles
that have been published, asserting that our test kits provide semi-quantitative data for nonskilled
users. We have user data that further confirms our results. However, it remains impossible to
evaluate or troubleshoot because we have only subjective information regarding color
development.
(Battelle Note: The F-NTK test kits were tested according to their intended use, i.e., as Field
Nitrate Test Kits. For that reason, the test/QA plan for this verification calledfor visual reading
and comparison of the color development of samples and standards. NECi approved that peer-
reviewed test/QA plan before testing began. Furthermore, the primary observation of the Round
1 tests was that color development in F-NTK standards was weak to non-existent. Comparison to
ion chromatographic reference results would not be improved by use of a colorimeter, when
color development is absent to begin with. Finally, the comments above, that F-NTK perfor-
mance claims can only be evaluated using a colorimeter, seem to suggest that the F-NTK kits
cannot meet those claims in the field without the use of a colorimeter. This is contradictory to the
stated use of the kits as field visual testing kits.)
Comments prepared by: Ellen R Campbell
Vice President
Nitrate Elimination Company, Inc. (NECi)
334 Hecla Street
Lake Linden, Michigan 49945
(906) 296-1000
NECi PUBLICATIONS on Nitrate Reductase:
1. Hyde G.E., J. A. Wilberding, A.L. Meyer, E.R. Campbell and W.H. Campbell (1989)
Monoclonal antibody-based immunoaffinity chromatography for purifying corn and squash
NADH:nitrate reductases. Evidence for an interchain disulfide bond in nitrate reductase. Plant
Molecular Biology 13: 233-246.
2. Campbell, E.R., J.S. Corrigan and W. H. Campbell (1997) Field determination of nitrate using
nitrate reductase. In: Field Analytical Methods for Hazardous Wastes and Toxic Chemicals,
Conference Proceedings, Air & Waste Management Assoc., pp. 851-860.
3. Glazier, S.A., E.R. Campbell and W.H. Campbell (1998) Construction and characterization of
nitrate reductase-based amperometric electrode and nitrate assay of fertilizers and drinking water.
Analytical Chemistry 70/8:1511-1515.
B-5
-------
4. Campbell, E.R. and W.H. Campbell (1998) Determination of nitrate in aqueous matrices
using nitrate reductase. In: Current Protocols in Field Analytical Chemistry, Supplement I,
J. Wiley & Sons, 5A1.1 -5A1.15.
5. Campbell, E.R., T.P.K. Skidmore, L.A. Winowiecki and W.H. Campbell (2001) A new trend
in nitrate analysis: enzyme-based field test for nitrate. American Environmental Laboratory.
6. Campbell, E.R. (2000) Nitrate and Health. Focus 10,000, Minnesota's Lakeside Mag. Fall
2000: 8-9.
7. Campbell ER, LA Winowiecki, M Shea, WH Campbell (2000) New nitrate measurement
tools to assist in nutrient management. Conference Proceedings, Water Environment Federation
Animal Residuals Management Conference 2000, published as a CD-Rom, available from
WEF.
8. Patton CJ, AE Fischer, WH Campbell and ER Campbell (submitted June 2001)
NADH:Nitrate Reductase - A nontoxic alternative to cadmium for colorimetric nitrate
determination in natural water by air-segmented continuous-flow analysis. Submitted to
Environmental Science & Technology.
ABSTRACTS AND PRESENTATIONS:
1. World '96 World Environmental Congress, Cincinnati, OH Oct 26-29, 1996. "Enzymatic
nitrate elimination technology". Abstract, poster presentation, W.H. Campbell and E.R.
Campbell
2. AOAC International 111th Annual Meeting, San Diego, CA Sep 7-11, 1997. "Enzyme-based
measurement of nitrate in water and food samples: improved sensitivity and selectivity with
reduced environmental impact." Abstract, poster presentation, E.R. Campbell and W.H.
Campbell.
3. US/Egypt Partnership for Investment in Biotechnology Workshop, Cairo, Egypt, Feb 10-11,
1998. Sponsored by the Fogarty Center, NIH.
4. PittCon '98, New Orleans, LA, Mar 1-5, 1998. "An enzyme-based field test kit for nitrate".
Abstract and poster 1884P, E.R. Campbell, V.L. Salo and W.H. Campbell.
5. Campbell, E.R. and W.H. Campbell (1998) Enzyme-based nitrate detection: from test kits to
biosensors. In: EnviroAnalysis '98 Conference Proceedings, Ottawa, Canada, pp 49-53.
6. Campbell, E.R. and W.H. Campbell (1999) Nitrate measurement with biosensor technology.
In: Appalachian Rivers II, Conference Proceedings, DOE/Federal Energy Technology Center,
Morgantown, WV (published on CD-ROM).
B-6
-------
7. Campbell, E.R., L.A. Winowiecki, and W.H. Campbell (2000) "An enzyme-based field test
for nitrate". Abstract and platform presentation, On-Site 2000 conference, Las Vegas, NV,
Jan23- 26.
8. Campbell ER, CJ Patton, AE Fischer, & WH Campbell (2000) Environmentally benign nitrate
analysis. Abstract, poster, and panel presentation at USEPA National Environmental Monitoring
Technology Conference, Boston, MA, 19-20 Sept 2000.
Recent Academic Publications on Nitrate Reductase by WH Campbell, NECi President and
Chief Scientist:
1. Campbell, Wilbur H. (1999) Nitrate Reductase Structure, Function and Regulation:
Bridging the Gap between Biochemistry and Physiology, Annual Review of Plant
Physiology and Plant Molecular Biology 50:227-303.
2. George, G. N., Mertens, J. A., and Wilbur H. Campbell (1999) Structural Changes
Induced by Catalytic Turnover at the Molybdenum Site of Arabidopsis Nitrate Reductase.
Journal of American Chemical Society, 121(41):9730-9731.
3. Mertens, J. A., Campbell, Wilbur Ft., Skipper, L., and David J. Lowe (1999) Electron
Transfer from FAD to Heme-Fe in Plant NADH:Nitrate Reductase. In: Flavins and
Flavoproteins 1999, S. Ghisla et al., eds., Agency for Scientific Publication, Berlin,
pp. 131 - 134. ISBN 3-00-005128-7.
4. Mertens JA, N Shiraishi, WH Campbell (2000) Recombinant expression of molybdenum
reductase fragments of plant nitrate reductase at high levels in Pichia Pastoris. Plant
Physiol. 123:743-756.
5. Skipper L, WH Campbell, JA Mertens & DJ Lowe (2001) Pre-steady-state kinetic
analysis of recombinant arabisopsis NADH:nitrate reductase: Rate-limiting processes in
catalysis. Journal of Biological Chemistry, 276 (29):26995-27002.
B-7
------- |