October 2005
Environmental Technology
Verification Report
ROSEMOUNT ANALYTICAL
MODEL WQS CONTINUOUS
MULTI-PARAMETER WATER
QUALITY MONITOR
Prepared by
Battelle
Baireiie
'I he Business
-------
THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
&EPA
U.S. Environmental Protection Agency
Battelle
'fne Business oj Innovation
ETV Joint Verification Statement
TECHNOLOGY TYPE: MULTI-PARAMETER WATER MONITORS FOR
DISTRIBUTION SYSTEMS
APPLICATION:
MONITORING DRINKING WATER QUALITY
TECHNOLOGY NAME: Model WQS
COMPANY: Rosemount Analytical
ADDRESS:
WEB SITE:
E-MAIL:
2400 Barranca Parkway
Irvine, CA 92606
www.emersonprocess.com
Richard.Baril @ EmersonProcess.com
PHONE: 949-757-8500
FAX: 949-863-9159
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology Verification (ETV)
Program to facilitate the deployment of innovative or improved environmental technologies through performance
verification and dissemination of information. The goal of the ETV Program is to further environmental protection
by accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal
by providing high-quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies. Information and ETV
documents are available at www.epa.gov/etv.
ETV works in partnership with recognized standards and testing organizations, with stakeholder groups
(consisting of buyers, vendor organizations, and permitters), and with individual technology developers. The
program evaluates the performance of innovative technologies by developing test plans that are responsive to the
needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and pre-
paring peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA)
protocols to ensure that data of known and adequate quality are generated and that the results are defensible.
The Advanced Monitoring Systems (AMS) Center, one of six technology areas under ETV, is operated by Battelle
in cooperation with EPA's National Exposure Research Laboratory. The AMS Center evaluated the performance
of the Rosemount Analytical Multi-Parameter/Optical Water Quality System (Model WQS) in continuously
measuring free chlorine, temperature, conductivity, pH, and oxidation-reduction potential (ORP) in drinking water.
This verification statement provides a summary of the test results.
-------
VERIFICATION TEST DESCRIPTION
The performance of the WQS unit was assessed in terms of its accuracy, response to injected contaminants, inter-
unit reproducibility, ease of use, and data acquisition. The verification test was conducted between August 9 and
October 28, 2004, and consisted of three stages, each designed to evaluate a particular performance characteristic
of the WQS unit. All three stages of the test were conducted using a recirculating pipe loop at the U.S. EPA's Test
and Evaluation Facility in Cincinnati, Ohio.
In the first stage of this verification test, the accuracy of the measurements made by the WQS units was evaluated
during nine, 4-hour periods of stable water quality conditions by comparing each WQS unit measurement to a grab
sample result generated each hour using a standard laboratory reference method and then calculating the percent
difference (%D). The second stage of the verification test involved evaluating the response of the WQS units to
changes in water quality parameters by injecting contaminants (nicotine, arsenic trioxide, and aldicarb) into the
pipe loop. Two injections of three contaminants were made into the recirculating pipe loop containing finished
Cincinnati drinking water. The response of each water quality parameter, whether it was an increase, decrease, or
no change, was documented and is reported here. In the first phase of Stage 3 of the verification test, the
performance of the WQS units was evaluated during 52 days of continuous operation, throughout which references
samples were collected once daily. The final phase of Stage 3 (which immediately followed the first phase of Stage
3 and lasted approximately one week) consisted of a two-step evaluation of the WQS performance to determine
whether this length of operation would negatively impact the results from the WQS. First, as during Stage 1, a
reference grab sample was collected every hour during a 4-hour analysis period and analyzed using the standard
reference methods. Again, this was done to define a formal time period of stable water quality conditions over
which the accuracy of the WQS could be evaluated. Second, to evaluate the response of the WQS unit to
contaminant injection after the extended deployment, the duplicate injection of aldicarb, which was also included
in the Stage 2 testing, was repeated. In addition, a pure E. coli culture, including the E. coli and the growth
medium, was included as a second injected contaminant during Stage 3. Inter-unit reproducibility was assessed by
comparing the results of two identical units operating simultaneously. Ease of use was documented by technicians
who operated and maintained the units, as well as the Battelle Verification Test Coordinator.
QA oversight of verification testing was provided by Battelle and EPA. Battelle QA staff conducted a technical
systems audit, a performance evaluation audit, and a data quality audit of 10% of the test data.
This verification statement, the full report on which it is based, and the test/QA plan for this verification test are all
available at www.epa.gov/etv/centers/centerl.html.
TECHNOLOGY DESCRIPTION
The following description of the WQS unit was provided by the vendor and does not represent verified
information.
The WQS unit measures pH, ORP, conductivity, temperature, and free chlorine in drinking water. The system
combines user-specified instruments and sensors to create a customized system for monitoring water quality. The
WQS unit does not need added reagents and uses minimum process flows of less than 183 milliliters per minute.
The WQS unit uses three basic electrochemical principles of operation: millivolt measurements for pH and ORP,
conductance/resistance measurements for conductivity, and amperometric/polarographic measurements for
chlorine residuals. The WQS unit continuously monitors each parameter to provide constant surveillance of water
quality events to ensure that acceptable water quality conditions are maintained. The WQS unit includes a sensor,
cables, and instruments to measure water quality parameters. The verified WQS unit was 26 inches high and
32 inches wide. The width varies by system from 26 inches to 50 inches wide. The data output from the system is
available as 4/20 mA analog, highway addressable remote transducer (HART®) or Foundation fieldbus® (HI), RS-
485, Ethernet, or Modbus RTU digital outputs. It uses 115/230-volt alternating current or 24-volt direct current.
During this verification test, a Fluke (Everette, Washington) data logger was configured to the WQS unit to record
the data every 30 seconds. The data logger was connected to a laptop computer that stored the data onto its hard
-------
drive as a delimited text file that was easily imported into a spreadsheet. The costs of the units as configured for
the verification test ranged from $12,000 to $15,000. In addition, calibration reagents cost approximately $200
annually.
VERIFICATION OF PERFORMANCE
Evaluation Parameter
Stage 1—
Accuracy
Stage 2 — Response
to Injected
Contaminants
Stage 3 — Accuracy
During Extended
Deployment
Stage 3 — Accuracy
After Extended
Deployment
Stage 3 — Response
to Injected
Contaminants
Injection Summary
Inter-unit
Reproducibility
(Unit 2 vs. Unit 1)
Ease of Use and Data
Acquisition
Units 1 and 2,
range of %D (median)
Nicotine
Arsenic
trioxide
Aldicarb
Reference
WQS
Reference
WQS
Reference
WQS
Units 1 and 2,
range of %D (median)
Unit 1, %D
Unit 2, %D
E. coli
Aldicarb
Reference
WQS
Reference
WQS
Free
Chlorine
-11.1 to 96.7
(14.5)
-
-
-
(b)
-
-
-36.2 to 68.3
(1.6)
-1.1
-2.2
-
-
-
(b)
Tem-
perature
-5.9 to 1.5
(-1.7)
NC
NC
NC
NC
NC
NC
-4.1 to 2.4
(-0.2)
0.6
0.2
NC
NC
NC
NC
Conductivity
2.9 to 5.3
(4.2)
NC
NC
+
+
NC
NC
3.4 to 6.7
(5.2)
5.1
5.3
+
NC
NC
NC
pH
-7.4 to -1.1
(-3.0)
NC
NC
+
+
NC
NC
-2.8 to 1.8
(-1.2)
-0.6
-0.9
-
-
-
NC
ORP
(a)
-
-
-
-
-
-
(a)
(a)
(a)
-
-
-
-
For a reason that is not clear, aldicarb altered the pH, as measured by the reference method,
during the Stage 3 injections, but not during the Stage 2 injections.
Slope (intercept)
r2
p-value
0.48 (0.45)
0.271
0.367
1.01 (-0.19)
0.999
0.882
1.00(0.26)
1.00
0.787
0.97 (0.25)
0.958
0.832
0.97 (-4.38)
0.950
0.01 l(c)
With the exception of ORP, the t-test indicated that the sensors on each unit were performing
similarly. For ORP, the linear correlation between the two units was very high, but the extremely
small variability in the signal caused the difference between the two units to be statistically
significant. Although the free chlorine sensors were not highly correlated with one another, the
large variability in their measurements prevented the t-test from determining a significant
difference between the units.
Based on the performance of the free chlorine sensors, calibration and membrane replacement
may have to occur periodically to maintain accurate measurements, especially those involving
response to injected contaminants. Also, the regular variability in free chlorine and pH
measurements may prevent observing small changes in those water quality parameters.
-------
Original signed by Gregory A. Mack 10/17/05 Original signed by Andrew P. Avel 1/17/06
Gregory A. Mack Date Andrew P. Avel Date
Assistant Division Manager Acting Director
Energy, Transportation, and Environment Division National Homeland Security Research Center
Battelle U.S. Environmental Protection Agency
NOTICE: ETV verifications are based on an evaluation of technology performance under specific, predetermined
criteria and the appropriate quality assurance procedures. EPA and Battelle make no expressed or implied
warranties as to the performance of the technology and do not certify that a technology will always operate as
verified. The end user is solely responsible for complying with any and all applicable federal, state, and local
requirements. Mention of commercial product names does not imply endorsement.
-------
October 2005
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
ROSEMOUNT ANALYTICAL
MODEL WQS CONTINUOUS MULTI-
PARAMETER WATER QUALITY MONITOR
by
Ryan James
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six verification centers. Information about
each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/centers/centerl.html.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We would like to thank Roy Haught
and John Hall of the U.S. Environmental Protection Agency's (EPA's) Test and Evaluation
(T&E) Facility (operated by Shaw Environmental, Inc. [Shaw]) in Cincinnati, Ohio, for hosting
the verification test. The U.S. EPA primary contract to Shaw provided significant support in
interfacing the continuous monitors with the pipe loop, as well as facilitating the experimental
plan. The T&E Facility's contribution included providing the reference analyses and operating
the pipe loop, as well as reviewing the test/quality assurance (QA) plan and the reports. In
addition, we would like to thank Steve Allgeier of EPA's Office of Water, Gary Norris and Alan
Vette of the EPA National Exposure Research Laboratory, Lisa Olsen of the U.S. Geological
Survey, Matthew Steele of the City of Columbus Water Quality Assurance District, and Ron
Hunsinger of East Bay Municipal Utility District, who also reviewed the test/QA plan and/or the
reports.
IV
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations viii
1 Background 1
2 Technology Description 2
3 Test Design 3
3.1 Introduction 3
3.2 Test Stages 3
3.2.1 Stage 1, Accuracy 4
3.2.2 Stage 2, Response to Injected Contaminants 4
3.2.3 Stage 3, Extended Deployment 5
3.3 Laboratory Reference and Quality Control Samples 5
3.3.1 Reference Methods 5
3.3.2 Reference Method Quality Control Samples 6
4 Quality Assurance/Quality Control 8
4.1 Audits 8
4.1.1 Performance Evaluation Audit 8
4.1.2 Technical Systems Audit 8
4.1.3 Audit of Data Quality 9
4.2 Quality Assurance/Quality Control Reporting 9
4.3 Data Review 9
5 Statistical Methods 11
5.1 Accuracy 11
5.2 Response to Injected Contaminants 11
5.3 Inter-unit Reproducibility 12
6 Test Results 13
6.1 Accuracy 14
6.2 Response to Injected Contaminants 18
6.3 Extended Deployment 22
6.4 Accuracy and Response to Injected Contaminants After Extended Deployment ... 27
-------
6.5 Inter-unit Reproducibility 31
6.6 Ease of Use and Data Acquisition 32
7 Performance Summary 33
8 References 34
Figures
Figure 2-1. Rosemount Analytical WQS Unit 2
Figure 6-1. Stage 2 Contaminant Injection Results for Free Chlorine 19
Figure 6-2. Stage 2 Contaminant Injection Results for ORP 20
Figure 6-3. Stage 2 Contaminant Injection Results for pH 20
Figure 6-4. Stage 2 Contaminant Injection Results for Conductivity 21
Figure 6-5. Extended Deployment Results for Free Chlorine 23
Figure 6-6. Extended Deployment Results for pH 23
Figure 6-7. Extended Deployment Results for ORP 24
Figure 6-8. Extended Deployment Results for Conductivity 24
Figure 6-9. Extended Deployment Results for Temperature 25
Figure 6-10. Stage 3 Contaminant Injection Results for Free Chlorine 28
Figure 6-11. Stage 3 Contaminant Injection Results for ORP 29
Figure 6-12. Stage 3 Contaminant Injection Results for pH 29
Figure 6-13. Stage 3 Contaminant Injection Results for Conductivity 30
VI
-------
Tables
Table 3-1. Reference Methods 6
Table 3-2. Reference Analyses and Quality Control Samples 7
Table 4-1. Performance Evaluation Audit
and Reference Method Duplicate Analysis Results 9
Table 4-2. Summary of Data Recording Process 10
Table 6-1. Summary of Test Stages and Type of Data Presentation 13
Table 6-2a. Accuracy Evaluation Under Various Conditions—Free Chlorine 14
Table 6-2b. Accuracy Evaluation Under Various Conditions—Temperature 15
Table 6-2c. Accuracy Evaluation Under Various Conditions—Conductivity 16
Table 6-2d. Accuracy Evaluation Under Various Conditions—pH 17
Table 6-3. Effect of Contaminant Injections Prior to Extended Deployment 19
Table 6-4. Accuracy During Extended Deployment 26
Table 6-5. Post-Extended Deployment Results 27
Table 6-6. Effect of Contaminant Injections After Extended Deployment 28
Table 6-7. Inter-unit Reproducibility Evaluation 31
vn
-------
List of Abbreviations
AMS Advanced Monitoring Systems
°C degree centigrade
C12 free chlorine
DI deionized
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
|iS/cm microSiemens per centimeter
mg/L milligram per liter
NIST National Institute of Standards and Technology
ORP oxidation reduction potential
%D percent difference
PE performance evaluation
PVC polyvinyl chloride
QA quality assurance
QC quality control
QMP quality management plan
SD standard deviation
T&E Test and Evaluation
TSA technical systems audit
WQS Water Quality System
Vlll
-------
Chapter 1
Background
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
evaluated the performance of the Rosemount Analytical Multi-Parameter/Optical Water Quality
System (WQS) in continuously measuring free chlorine, temperature, conductivity, pH, and
oxidation-reduction potential (ORP) in drinking water. Continuous multi-parameter water
monitors for distribution systems were identified as a priority technology verification category
through the AMS Center stakeholder process.
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This verification report provides
results for the verification testing of the WQS water quality monitor. Following is a description
of the WQS unit, based on information provided by the vendor. The information provided below
was not verified in this test.
The WQS unit (Figure 2-1) measures pH, oxidation-reduction potential (ORP), conductivity,
temperature, and free chlorine (C12) in drinking water. The system combines user-specified
instruments and sensors to create a customized system for monitoring water quality. The WQS
unit does not need added reagents and uses minimum process flows of less than 183 milliliters
per minute. The WQS unit uses three basic electrochemical principles of operation: millivolt
measurements for pH and ORP, conductance/resistance measurements for conductivity, and
amperometric/polarographic measurements for chlorine residuals. The WQS unit continuously
monitors each parameter to provide constant
surveillance of water quality events to ensure
that acceptable water quality conditions are
maintained.
The WQS unit includes a sensor, cables, and
instruments to measure water quality
parameters. The verified WQS unit was
26 inches high and 32 inches wide. The width
varies by system from 26 inches to 50 inches
wide. The data output from the system is
available as 4/20 mA analog, highway
addressable remote transducer (HART®) or
Foundation fieldbus® (HI), RS-485, Ethernet,
or Modbus RTU digital outputs. It uses
115/230-volt alternating current or 24-volt
direct current.
Figure 2-1. Rosemount Analytical WQS Unit
During this verification test, a Fluke (Everette, Washington) data logger was configured to the
WQS unit to record the data every 30 seconds. The data logger was connected to a laptop
computer that stored the data onto its hard drive as a delimited text file that was easily imported
into a spreadsheet. The costs of the units as configured for the verification test ranged from
$12,000 to $15,000. In addition, calibration reagents cost approximately $200 annually.
-------
Chapter 3
Test Design
3.1 Introduction
The multi-parameter water monitors tested consisted of instrument packages that connect to or
are inserted in distribution system pipes for continuous monitoring. Also included in this
technology category were technologies that can be programmed to automatically sample and
analyze distribution system water at regular intervals. The minimum requirement for
participation in this verification test was that the water monitors were able to measure residual
chlorine, as well as at least one other water quality parameter. Residual chlorine is a particularly
important water quality parameter because changes in its concentration can indicate the presence
of contamination within a distribution system, and chlorination is a very common form of water
treatment used by water utilities in the United States.
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Multi-Parameter Water Monitors for Distribution Systems^ and assessed the
performance of the WQS units in continuously monitoring pH, conductivity, free chlorine, ORP,
and temperature in terms of the following:
• Accuracy
• Response to injected contaminants
• Inter-unit reproducibility
• Ease of use and data acquisition.
Accuracy was quantitatively evaluated by comparing the results generated by two WQS units to
grab sample results generated by a standard laboratory reference method. Response to injected
contaminants was evaluated qualitatively by observing whether the measured water quality
parameters were affected by the injection of several contaminants. Inter-unit reproducibility was
assessed by comparing the results of two identical WQS units operating simultaneously. Ease of
use was documented by technicians who operated and maintained the WQSs, as well as the
Battelle Verification Test Coordinator.
3.2 Test Stages
This verification test was conducted between August 9 and October 28, 2004, and consisted of
three stages, each designed to evaluate a particular performance characteristic of the WQS unit.
All three stages of the test were conducted using a recirculating pipe loop at the U.S. EPA's Test
and Evaluation (T&E) Facility in Cincinnati, Ohio. The recirculating pipe loop consisted of
ductile iron pipe, 6 inches in diameter and 100 feet long, which contained approximately
240 gallons of Cincinnati drinking water with a flow rate of approximately 1 foot/second. The
-------
water within the pipe loop had a residence time of approximately 24 hours. Water from the pipe
loop was plumbed to two WQS units by a section of 2-inch polyvinyl chloride (PVC) pipe in
series with a shut-off valve with a ribbed nozzle that was connected to the WQS units with a
1/2-inch PVC hose and a hose clamp. Reference samples of approximately 1 liter (enough
volume to perform all the required analyses) to be analyzed by each standard laboratory
reference method were collected from the reference sample collection valve located
approximately 15 feet from the WQS units on the PVC pipe.
3.2.1 Stage 1, Accuracy
During the first stage of this verification test, the accuracy of the measurements made by both
the WQS units was evaluated by comparing the results from each unit to the result generated by
a standard laboratory reference method. Stage 1 testing simulated the characteristics of a variety
of water quality conditions by changing two variables: pH and temperature. Using nine sets of
pH and temperature conditions, this evaluation consisted of separate four-hour testing periods of
continuous analysis, with reference method sampling and analysis every hour. Four sets of
conditions involved varying only the pH by injecting the pipe loop with a steady stream of
sodium bisulfate. These sets consisted of pHs of approximately 7, 8, and 9 pH units (ambient pH
at the T&E Facility was between 8 and 9) and a temperature between 21 and 23 degrees
centigrade (°C) (T&E Facility ambient during time of testing). Two other sets of conditions
included water temperature between 12 and 14°C and pHs of approximately 7 and 8; and two
sets at approximately these pHs, but at a temperature of approximately 27°C. One set (Set 2) was
repeated as Set 3. The pipe loop ambient conditions were analyzed at the start and end of this
stage. Prior to each testing period with unique conditions, the water in the pipe loop was allowed
to equilibrate until the pH and temperature were at the desired level, as determined by the
standard reference methods. This equilibration step took approximately 12 hours from the time
the sodium bisulfate was added (to decrease pH) or the temperature was adjusted (using a chiller
connected to the pipe loop) until testing occurred.
3.2.2 Stage 2, Response to Injected Contaminants
The second stage of the verification test involved testing the response of the WQS units to
changes in water quality parameters by injecting contaminants into the pipe loop. Two injections
of three contaminants were made into the recirculating pipe loop containing finished Cincinnati
drinking water. Each injection was made over a period of approximately 15 seconds by
connecting the injection tank to the pipe loop's recirculating pump. The three contaminants were
nicotine, arsenic trioxide (adjusted to pH 12 to get it into solution), and aldicarb. With the
exception of the first nicotine injection, each of these contaminants was dissolved in approxi-
mately 5 gallons of pipe loop water that had been dechlorinated using granular carbon filtration
to prevent degradation of the contaminant prior to injection. Upon injection, concentrations of
these contaminants within the pipe loop water were approximately 10 milligrams per liter
(mg/L). For the first nicotine injection, however, not enough nicotine to attain this concentration
was available so the available nicotine was dissolved into 2 gallons of the dechlorinated pipe
loop water and injected. The resulting nicotine concentration in the pipe loop was approximately
6 mg/L. Because the qualitative change in water quality parameters was similar for both nicotine
injections despite the concentration difference, it was not necessary to repeat the 10 mg/L
injection of nicotine. For all three sets of injections, a reference sample was collected prior to the
injection and again at 3, 15, and 60 minutes after the injection. The difference between reference
-------
method results occurring before and then again after each injection indicated the directional
change in water quality caused by the injected contaminant. For each injected contaminant, the
results from the WQS units were evaluated based on how well their directional change matched
that of the reference method result. After each injection, the pipe loop was allowed to
re-equilibrate for approximately 12 hours so that each WQS unit returned to a steady baseline.
Injected contaminants were obtained from Sigma-Aldrich (St. Louis, Missouri) or ChemService
(West Chester, Pennsylvania) and were accompanied by a certificate of analysis provided by the
supplier. Battelle QA staff audited the gravimetric preparation of these solutions.
3.2.3 Stage 3, Extended Deployment
In the first phase of Stage 3 of the verification test, the performance of the WQS units was
evaluated during 52 days of continuous operation. The WQS unit required no regularly
scheduled maintenance during this deployment. To track the performance of the WQS unit with
respect to the reference results, reference samples were collected and analyzed for the selected
parameters at least once per day (excluding weekends and holidays) for the duration of Stage 3.
All continuously measured data were graphed, along with the results from the reference
measurements, to provide a qualitative evaluation of the data. Throughout the duration of the
deployment, the average percent difference (%D), as defined in Section 5.1, between the results
from the WQS units and those from the reference methods was evaluated.
The final phase of Stage 3 (which immediately followed the first phase of Stage 3 and lasted
approximately one week) consisted of a two-step evaluation of the WQS unit performance after
the 52-day extended deployment to determine whether this length of operation would negatively
affect the results from the WQS. First, while the WQS units were continuously operating, a
reference sample was collected every hour during a 4-hour analysis period and analyzed using
the standard reference methods. This was done to define a formal time period of stable water
quality conditions for the accuracy of the WQS to be evaluated. Second, to evaluate the response
of the WQS unit to contaminant injection after the extended deployment, the duplicate injection
of aldicarb, which was also included in the Stage 2 testing, was repeated. In addition, a pure
E. coli culture, including the E. coli and the growth medium, was included as a second injected
contaminant during Stage 3. E. coli was intended as an injected contaminant during Stage 2, but
was not available until later in the test. During this contaminant injection component of Stage 3,
reference samples were collected as they were during Stage 2.
3.3 Laboratory Reference and Quality Control Samples
The WQS units were evaluated by comparing their results with laboratory reference measure-
ments. The following sections provide an overview of the applicable procedures, analyses, and
methods.
3.3.1 Reference Methods
To eliminate the possibility of using stagnant water residing in the reference sample collection
valve (dead volume) as the reference samples, the first step in the reference sample collection
procedure included collecting and discarding (from the reference sample collection valve)
approximately 1 L of water, which was estimated to be approximately 10 times the dead volume
5
-------
of the reference sample collection value. Then, from the same valve, approximately 1 L of water
was collected in a glass beaker and carried directly to a technician, who immediately began the
reference analyses. All the analyses were performed within minutes of sample collection. The
standard laboratory methods used for the reference analyses are shown in Table 3-1. Also
included in the table are method detection limits and quality control (QC) measurement
tolerances. Battelle technical staff collected the reference samples, and technical staff at the T&E
Facility performed the analyses. The T&E Facility provided calibrated instrumentation,
performed all method QA/QC, and provided calibration records for all instrumentation. The
T&E Facility provided reference sample results upon the analysis of the reference samples
(within one day). Because previous work at the T&E facility® showed that the laboratory
reference method for ORP using a grab sample is not directly comparable to a continuous
measurement in a flowing pipe, accuracy results were not included for ORP. ORP reference and
continuous measurement results were, however, included for the purpose of a qualitative data
evaluation in the figures showing continuous data and reference method results. Although the
ORP reference value may not be equivalent to the continuous measurement, changes in the
continuous measurements were evaluated with the reference results to determine whether the
sensor was identifying increases and decreases correctly.
Table 3-1. Reference Methods
Parameter
Method
Reference Instruments
Method Detection
Limit
Acceptable
Differences for
QC Measurements
pH EPA150.1(3)
Conductivity SM 2510(4)
Corning 320 pH meter
YSI556 multi-parameter
water monitor
NA
2 microSiemens/
centimeter
(H/Scm)
tO.3 pH units
±25 %D
Free chlorine
ORP(a)
Temperature
SM 4500-G(5)
SM 2580-B(6)
EPA170.1(7)
Hach 2400 portable
spectrophotometer
YSI 556 multi-parameter
water monitor
Hach 21 OOP turbidimeter
0.01 mg/L as C12
NA
NA
±25 %D
±25 %D
±1°C
(a) The reference method for measuring ORP is not directly comparable because of the difference in potential in a
flowing pipe compared to that measured in a grab sample.
NA = not applicable.
3.3.2 Reference Method Quality Control Samples
As shown in Table 3-2, duplicate reference samples were collected and analyzed once daily
during Stages 1 and 2 and weekly during Stage 3. Also, laboratory blanks consisting of
American Society for Testing and Materials Type n deionized (DI) water were analyzed with the
same frequency. Reference analyses of these blank samples were most important for free
chlorine because it was the only parameter that needed confirmation of the lack of contamina-
tion. For the other parameters, the performance evaluation (PE) audit confirmed the accuracy of
the method and the absence of contamination. Duplicate measurements had to be within the
acceptable differences provided in Table 3-1.
-------
Table 3-2. Reference Analyses and Quality Control Samples
1:
2:
3:
3:
Stage
Accuracy
Response to
injected
contaminants
Extended
deployment
Post-extended
deployment
accuracy
Reference Reference
Sampling Sample Samples per QC Samples per Total QC
Periods (length) Frequency Period Period Samples
One at start, one
9 (4 hours) every hour
thereafter
One pre-
injection;
6 (one injection) one at 3, 15, and
60 minutes post-
injection
^ fc<^ A \ Once eacri
1 (52 days) , ,
3 weekday
1 (4 hours) Same as Stage 1
One duplicate and
5 one DI water blank
daily
One duplicate and
4 one DI water blank
daily
One duplicate and
37 one DI water blank
each week
5 Same as Stage 1
18
12
16
2
3: Response to
injected 4 (one injection) Same as Stage 2
contaminants
Same as Stage 2
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed in accordance with the quality management plan (QMP) for
the AMS Center(8) and the test/QA plan (1) for this verification test.
4.1 Audits
4.1.1 Performance Evaluation Audit
A PE audit was conducted to assess the quality of the reference measurements made in this
verification test. With the exception of temperature, each type of reference measurement was
compared with a National Institute of Standards and Technology (NIST)-traceable standard
reference water sample. The standard reference water samples had certified values of each water
quality parameter that were unknown to the analyst. These samples were analyzed in the same
manner as the rest of the reference analyses to independently confirm the accuracy of the
reference measurements. The temperature PE audit was performed by comparing two
independent thermometer results. As Table 4-1 shows, all PE audit results were within the
acceptable differences provided in Table 3-1. The percent difference (%D) was calculated using
the following equation.
C - C
%D= — - - - xlOO%
where CR is the reference method result, and CN is the NIST value for each water quality
parameter (or, for temperature, data from the second thermometer). Other QC data collected
during this verification test were reference method duplicate analysis results, which are also
shown in Table 4-1. Because pH units are measured on a logarithmic, rather than linear, scale,
and the measurement of temperature is extremely precise; the quality control metrics for those
two parameters were the absolute units rather than percent difference.
4.1.2 Technical Systems Audit
The Battelle Quality Manager performed a technical systems audit (TSA) to ensure that the
verification test was performed in accordance with the AMS Center QMP,(8) the test/QA plan,(1)
published reference methods, and any standard operating procedures used by the T&E Facility.
The TSA noted no adverse findings. A TSA report was prepared, and a copy was distributed to
the EPA AMS Center Quality Manager.
-------
Table 4-1. Performance Evaluation Audit and Reference Method Duplicate Analysis
Results
Parameter
pH
Conductivity (|iS/cm)
Free chlorine (mg/L)
Temperature (°C)
NIST
Standard
Value
9.26
1,920
4.19
23.80(a)
PE Audit
Reference
Method
Result Difference
9.18 -0.08 pH unit
1,706 -11.1%
3.62 -13.6%
23.80 0.00°C
Duplicate Analysis
Average of
Absolute Values Range of
of Difference Difference
0.04 pH unit 0.0 to 0. 13 pH unit
0.25% -1.9 to 0.7%
2.62% -7.3 to 2.1%
0.02°C -0.18to0.29°C
ORP was not included in the accuracy evaluation because of the lack of an appropriate reference method.
-------
Table 4-2. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test
events
Calibration
information (WQS
unit and reference
methods)
WQS unit results
Where Recorded
ETV data sheets
and testing
notebook
ETV data sheets
and testing
notebook
Recorded
electronically by
How Often
Recorded
Start/end of test and
at each change of a
test parameter
Upon each
calibration
Recorded
continuously
By
Whom
Battelle
and T&E
Facility
Battelle
and T&E
Facility
Battelle
Disposition of
Data
Used to
organize/check test
results; manually
incorporated in
data spreadsheets
as necessary
Manually
incorporated in
data spreadsheets
as necessary
Delimited text files
each WQS unit and
then downloaded to
computer daily
Reference method
procedures
ETV laboratory
record books or
data recording
forms
Throughout sample
analysis process
T&E
Facility
Transferred to
spreadsheets or
laboratory record
book
10
-------
Chapter 5
Statistical Methods
The statistical methods presented in this chapter were used to verify the WQS unit's accuracy,
response to injected contaminants, and inter-unit reproducibility.
5.1 Accuracy
Throughout this verification test, results from the WQS unit were compared to the results
obtained from analysis of a grab sample by the reference methods. During Stage 1, the percent
difference (%D) between these two results was calculated using the following equation:
%D= Cm ~CR xlOO%
CR
where CR is the result determined by the reference method and Cm is the result from a WQS unit;
the WQS unit results were recorded every 30 seconds, whereas collecting the reference samples
took only a few seconds. Therefore, Cm was the measurement recorded closest to the time the
reference sample was collected. Water quality stability, as well as the stability of each sensor,
was evaluated during the four-hour time period when reference samples were analyzed every
hour for each of the parameters. Ideally, if the result from a WQS unit and a reference method
measurements were the same, there would be a percent difference of zero. During Stages 2 and
3, the continuous data, graphed with the reference method results, were visually examined to
evaluate the response of the WQS unit to the injection of contaminants and their stability over an
extended deployment. During the accuracy and contaminant injection components of Stage 3, the
data were evaluated as they were for Stages 1 and 2, respectively.
5.2 Response to Injected Contaminants
To evaluate the response (i.e., the increase or decrease of water quality parameter measured by
the WQS units) to contaminant injections, the pre- and post-injection reference samples were
graphed as individual data points, along with the continuous measurements. The reference
results showed the effect of each injection on the chemistry of the water in the pipe loop, and the
continuous results from the WQS unit highlighted its response to such changes.
11
-------
5.3 Inter-unit Reproducibility
The results obtained from two identical WQS units were compared to assess inter-unit
reproducibility. Each time a reference sample was collected and analyzed (approximately
127 times throughout this verification test), the results from each WQS unit were compared to
evaluate whether the two WQS units were generating similar results. This was done in two ways.
First, the results from one were graphed against the results of the other unit. In this evaluation, a
slope of unity and coefficient of determination (r2) of 1.0 would indicate ideal inter-unit
reproducibility. Slopes above 1.0 may indicate a high bias from Unit 2 (graphed on the y-axis) or
a low bias for Unit 1 with respect to each other. Similarly, slopes below 1.0 may indicate a low
bias for Unit 2 or a high bias for Unit 1, again with respect to each other. Second, the data from
each unit were included in a paired t-test, with the assumption that the data from each unit had
equal variances. The t-test calculated the probability of obtaining the subject results from the two
units if there was no significant difference between their results. Therefore, probability values
(p-values) of less than 0.05 (i.e., less than a 5% probability that this data set would be generated
if there actually was no difference between the two units) indicated a significant difference
between the two units. In addition, the results from both units were graphed together for the
Stages 2 and 3 results, allowing a visual comparison.
12
-------
Chapter 6
Test Results
As mentioned previously, this verification test was conducted in three stages that focused on
three different aspects of multi-parameter water monitors for distribution systems. The three
stages are summarized in Table 6-1. The first stage consisted of an evaluation (with varied pHs
and temperatures) of the accuracy of each WQS unit sensor: free chlorine, temperature,
conductivity, and pH. ORP also was measured; but, because a laboratory reference measurement
equivalent to the on-line continuous measurement was not available, ORP was not included in
the accuracy evaluation. Note also that one of the WQS units was equipped with a mono-
chloramine sensor; however, because Cincinnati is a chlorinated system, the monochloramine
levels are very low. Therefore, monochloramine results were not included in this report. The
second stage of the verification test consisted of an evaluation of the response of the WQS units
to the injection of several contaminants into the pipe loop. The third stage consisted of deploying
the WQS unit for 52 consecutive days with minimal intervention for maintenance. In addition,
contaminant injections were performed at the close of Stage 3 to confirm that the WQS units
were still responsive to contaminant injection after the extended deployment. Two WQS units
were tested to evaluate inter-unit reproducibility. In addition, required maintenance and
operational characteristics were documented throughout the verification test. This chapter
provides the results of the three testing stages, the inter-unit reproducibility data, and ease of use
information.
Table 6-1. Summary of Test Stages and Type of Data Presentation
Stage Summary Data Presentation
1 Accuracy when pH and temperature Table of percent differences between WQS
were varied units and reference measurements
2 Response to contaminant injection Graphs of WQS unit measurements and
reference measurements, table showing the
effect of injections on reference and WQS
measurements
3 Extended deployment with minimal Graphs of WQS unit measurements with
maintenance along with post-extended reference measurements, table showing
deployment accuracy and response to average percent differences throughout
contaminant injections extended deployment, table showing the
effect of injections on reference and WQS
measurements
13
-------
6.1 Accuracy
Tables 6-2a-d list the data from the accuracy evaluation performed during the first stage of the
verification test. During four-hour periods, the water quality conditions were held stable, and
reference samples were collected and analyzed five times, once at the start of the designated test
period and four times at one-hour increments thereafter. Because reference sample collection
took just a few seconds, and the results from the WQS units were recorded every 30 seconds, the
water quality parameter measurement at the time closest to reference sample collection was
compared to the reference sample. For each unit, this approach resulted in five paired WQS units
and reference results for each of the nine sets of water conditions used to simulate pH and
temperature variations at a water utility. The average and standard deviations of these five results
are shown in the tables below, as well as the percent difference between the average results of
both WQS units and the average of the reference results.
Table 6-2a. Accuracy Evaluation Under Various Conditions—Free Chlorine
Set
1
2
3
4
5
6
7
8
9
Conditions
ambient pH,
ambient temperature
decreased pH,
ambient temperature
decreased pH,
ambient temperature
decreased pH,
ambient temperature
ambient pH,
decreased temperature
decreased pH,
decreased temperature
ambient pH,
increased temperature
decreased pH,
increased temperature
ambient pH,
ambient temperature
Reference
Average (SD)
[mg/L]
0.91 (0.08)
0.78 (0.02)
0.65 (0.01)
0.29 (0.02)
0.41 (0.08)
1.47(0.06)
0.60 (0.04)
0.54 (0.05)
0.91 (0.03)
Unitl
Average (SD)
[mg/L]
1.05 (0.03)
0.79 (0.04)
0.63 (0.01)
0.26 (0.01)
0.57 (0.02)
1.52 (0.06)
0.60 (0.03)
0.48 (0.05)
1.67(0.11)
%D
15.4
1.3
-3.1
-10.3
39.0
3.4
0.0
-11.1
83.5
Unit 2
Average (SD)
[mg/L]
1.32(0.11)
0.91 (0.06)
0.73 (0.02)
0.29 (0.01)
0.70 (0.03)
1.67(0.06)
1.18(0.02)
0.85 (0.03)
1.32(0.04)
%D
45.1
16.7
12.3
0.0
70.7
13.6
96.7
57.4
45.1
14
-------
Table 6-2b. Accuracy Evaluation Under Various Conditions—Temperature
Reference
Unit 1 Unit 2
Average (SD) Average (SD) Average (SD)
Set Conditions [°C] [°C] %D [°C] %D
1 ambient pH, 22.66(0.33) 22.17(0.25) -2.2 22.09(0.25) -2.5
ambient temperature
2 decreased pH, 22.73(0.23) 22.26(0.22) -2.1 22.20(0.21) -2.3
ambient temperature
3 decreasedpH, 21.61(0.16) 21.50(0.11) -0.5 21.42(0.11) -0.9
ambient temperature
4 decreasedpH, 21.93(0.15) 21.65(0.04) -1.3 21.61(0.06) -1.5
ambient temperature
5 ambient pH, 13.82(0.44) 13.21(0.16) -4.4 13.12(0.18) -5.1
decreased temperature
6 decreased pH, decreased 12.63(0.26) 12.06(0.24) -4.5 11.88(0.26) -5.9
temperature
7 ambient pH, 26.60(0.27) 27.01(0.16) 1.5 26.97(0.15) 1.4
increased temperature
8 decreased pH, increased 26.69(0.23) 26.96(0.15) 1.0 26.90(0.14) 0.8
temperature
9 ambient pH, 22.79(0.21) 22.49(0.33) -1.3 22.37(0.39) -1.8
ambient temperature
15
-------
Table 6-2c. Accuracy Evaluation Under Various Conditions—Conductivity
Reference
Average (SD)
Unitl
Unit 2
Set
Conditions
Average (SD)
[[iS/cm]
ambient pH, 451(1)
ambient temperature
decreased pH, 486(10)
ambient temperature
decreased pH, 503 (6)
ambient temperature
decreased pH, 694(12)
ambient temperature
ambient pH, 412(1)
decreased temperature
decreased pH, decreased 501 (10)
temperature
ambient pH, 447(1)
increased temperature
decreased pH, increased 529 (2)
temperature
ambient pH, 442(1)
ambient temperature
465 (2)
500(11)
526 (7)
730 (12)
426 (2)
516(11)
463 (3)
548 (3)
460(1)
%D
3.1
2.9
4.6
5.2
3.4
3.0
3.6
3.6
4.1
Average (SD)
[[iS/cm] % D
468(1) 3.8
503(11) 3.5
528 (7) 5.0
731 (12)
431 (2)
523(11)
466 (2)
552 (3)
462 (0)
5.3
4.6
4.4
4.3
4.3
4.5
16
-------
Table 6-2d. Accuracy Evaluation Under Various Conditions—pH
Reference
Unit 1 Unit 2
Average (SD) Average (SD) Average (SD)
Set
1
2
3
4
5
6
7
8
9
Conditions
ambient pH,
ambient temperature
decreased pH,
ambient temperature
decreased pH,
ambient temperature
decreased pH,
ambient temperature
ambient pH,
decreased temperature
decreased pH,
decreased temperature
ambient pH,
increased temperature
decreased pH,
increased temperature
ambient pH,
ambient temperature
[pH Unit]
8.76 (0.02)
7.89 (0.09)
7.52 (0.04)
6.73(0.12)
8.48 (0.02)
7.31 (0.08)
8.37 (0.05)
7.60 (0.06)
8.74 (0.01)
[pH Unit]
8.60 (0.07)
7.53 (0.20)
7.09 (0.04)
6.24 (0.06)
8.36(0.12)
7.14(0.13)
8.19(0.11)
7.12(0.13)
8.54(0.13)
%D
-1.8
-4.6
-5.7
-7.3
-1.4
-2.3
-2.2
-6.3
-2.3
[pH Unit]
8.70(0.14)
7.59 (0.20)
7.27 (0.10)
6.23 (0.07)
8.39 (0.06)
6.95 (0.09)
8.17 (0.06)
7.18 (0.06)
8.51 (0.06)
%D
-0.7
-3.8
-3.3
-7.4
-1.1
-4.9
-2.4
-5.5
-2.6
Of the parameters that were evaluated for accuracy, the free chlorine sensors generated the
largest range of percent differences compared to the reference method. For free chlorine, the
range of percent differences (with the median shown in parentheses) was from -11.1 to 96.7
(14.5); for temperature, -5.9 to 1.5 (-1.7); for conductivity 2.9 to 5.3 (4.2); and for pH, -7.4 to
-1.1 (-3.0).1 The chlorine sensor was calibrated by the vendor prior to the verification test, but
was not recalibrated during Stage 1. There was no obvious trend in the performance of the
chlorine sensors. For Set 1, Unit 1 had a %D of 15.4% and Unit 2 had a %D of 45.1%. For Sets 2
through 4, both sensors' results were less than 20% different from the reference results. For Set
5, both sensors were more than 40% different than the reference measurement. Thereafter, Unit
1's agreement with the reference measurement improved considerably for Sets 6 through 8, but
the percent difference of Unit 1 was more than 80% for Set 9. The agreement of Unit 2 with the
reference measurement only improved for Set 6, whereas for the remaining sets, the continuous
and reference method were at least 45% different from each other. The standard deviations for
the reference method demonstrate that the variability in both the reference and continuous
measurements was generally less than 10%.
1 Throughout this report, median values are provided when a range of values is presented. The median of a set of
positive and negative numbers provides a good indicator of the overall direction of the percent differences in the
data set (i.e., whether most values were positive or negative). The disadvantage is that, unless the signs of all the
data are the same, information about the magnitude of change is not available from the median. In summary, the
medians in this report provide the direction, not magnitude, of difference information.
17
-------
The temperature sensors (Table 6-2b) generated very small percent differences with respect to
the reference method at ambient temperatures (between -2.5% and -0.5%), slightly larger
negative percent differences (-5.9% to -4.4%) resulted when the temperature of the water in the
pipe loop was decreased, and small positive percent differences (0.8% to 1.5%) resulted when
the temperature of the pipe loop water was increased. This trend in percent differences is likely
due to the reference sample collection and analysis procedure. Reference samples were carried to
a laboratory bench approximately 25 feet from the reference sample collection valve. Therefore,
upon sample collection, the reference sample immediately began equilibrating with the ambient
air, thus causing a slight increase in water temperature in the brief time period between reference
sampling and analysis.
The conductivity and pH results (Tables 6-2c and 6-2d) produced very small percent differences
from the reference method. Across all sets of conditions, the percent differences for the
conductivity measurements were between 0 and 7%, indicating that the WQS units were always
slightly higher than the reference method.
6.2 Response to Injected Contaminants
Six injections of contaminants were performed during the second stage of this verification test;
i.e., duplicate injections of nicotine, arsenic trioxide, and aldicarb. Table 6-3 shows the
directional change of each reference and WQS measurement in response to the contaminant
injections. In general, free chlorine, and ORP were the only parameters clearly affected (for both
the reference and continuous measurements) by all six injections. Both the reference and
continuous measurement for both of these water quality parameters decreased upon injection of
contaminants. There was one exception during the second arsenic trioxide injection; Unit 1's
chlorine sensors didn't respond to the contaminant injection, while the Unit 2 chlorine sensor did
respond. Figures 6-1 through 6-4 show the responses of free chlorine, ORP, pH, and
conductivity. The blue and yellow lines on the graphs represent the measurements made by each
WQS unit, and the magenta data points represent the results from the laboratory reference
method. Because accuracy was the focus of the first stage of verification testing, percent
differences between the WQS units and the reference method results are not presented here;
however, the reference method results are included in these figures to confirm that the
fluctuations in the continuous results are due to changes in water chemistry as the result of the
injected contaminants. The figures are divided with vertical lines that define the approximate
time period for each injection. Each injection time period defined on the figures is approximately
24 hours, but the times vary somewhat depending on when chlorine was added to restore the
system to pre-injection conditions. The contaminant that was injected and whether it was the first
or second replicate are shown at the top of each section of the figures. For each injection, at least
four reference sample results were collected, and are included in these figures. The first occurred
within approximately one hour prior to contaminant injection during a period of stable water
quality conditions. The next three reference data points were from samples collected 3, 15, and
60 minutes after contaminant injection. For some of the injections, another reference sample was
collected the following day to show that the pipe loop system had recovered or was in the
process of recovering after the injection. This final reference data point also served as the first
reference sample collected for some of the injections, representing the stable baseline just prior
to injection.
18
-------
Table 6-3. Effect of Contaminant Injections Prior to Extended Deployment
Parameter
Free chlorine
Temperature
Conductivity
pH
ORP
Nicotine
Reference WQS
-
NC NC
NC NC
NC NC
-
Arsenic Trioxide
Reference WQS
_ (a)
NC NC
+ +
+ +
-
Aldicarb
Reference
-
NC
NC
NC
-
WQS
-
NC
NC
NC
-
-------
auu
800-
700-
600
500-
400-
300-
200-
100-
n
Nicotine 1
f* H
,
I/
Nicotine 2
U'
#
ft.
Arsenic 1
1
t-
*
A
A
1
I
1
1
1
U
Arsenic 2
A
•
;
r
U
1
Aldicarb i
1
1
/
V
Idicarb
2
0
*
i
\
' — Unitl
Reference
Unit 2
Each section 'separated by vertical lines) represents approximately 24 houis
Figure 6-2. Stage 2 Contaminant Injection Results for ORP
9.4
X
Q_
9.2 -
9 -
8.8
8.6
Nicotnel Nicotine 2 Arsenic! Arsenic 2 Aldiccrt 1 Aldicarb 2
8.2
1/1
8
Each section (separated by vertical lines i represents approximately 24 hours.
Figure 6-3. Stage 2 Contaminant Injection Results for pH
20
-------
ouu -
490
480
470 -
o
1 460 -
o
§ 450 -
§ 440 -
E 430
420-
410 -
400-
Eac
Nicotine 1
j^*\l
sX
» *
Nkotine 2
I
I
V
\
*
Arsenic 1
1
1
1
W**"si i
u)
*
+ *
Arsenic 2
\
\
*
*
Aldicart) 1
•r M— *^*^hjH
.'
Aldicarb 2
***-T-^
*-
#
— Unitl
» Reference
Unit 2
h section (separated by vertical lines! represents approximately 24 hours
Figure 6-4. Stage 2 Contaminant Injection Results for Conductivity
Figure 6-1 shows how the measurement of free chlorine was affected by the contaminant
injections. Prior to the injections, the free chlorine level was maintained at approximately
1 mg/L, as is evidenced by the reference method data point at the far left of the figure. However,
the measurements of the WQS units at this time were between 1.5 and 2 mg/L for Unit 1 and
approximately 1.25 mg/L for Unit 2, in both cases considerably higher than the reference method
measurement. When nicotine was injected for the first time, the free chlorine sensors detected a
drop in free chlorine of approximately 0.5 mg/L, while the reference measurement indicated that
nicotine had reacted almost completely with the chlorine in the pipe loop water, taking the
concentration to near zero. Following the drop in chlorine concentration corresponding to the
first injection, the sensors recovered to readings similar to those before the first injection. The
chlorine sensors responded to the second injection of nicotine as they did following the first
injection. However, after that injection, the sensors did not return to their pre-injection readings,
but remained steady at their respective post-injection concentration levels. In addition, Unit 2
drifted from approximately 1 mg/L to approximately 0.5 mg/L before the first arsenic trioxide
injection. Unit 1 remained at a concentration of approximately 1 mg/L prior to the arsenic
injection. The sharp drop in chlorine shown by both sensors between the second nicotine
injection and the first arsenic injection was not due to a contaminant injection, but to a brief
change in the pipe loop water chlorine level unrelated to the verification test. However, the Unit
2 chlorine sensor did not recover fully from that drop in chlorine. Figure 6-1 shows that both
chlorine sensors have a consistent variability throughout this stage. In 3- to 4-minute intervals,
the measurements oscillated by 5 to 20%. This variability is shown visually by the rather wide
trace as opposed to the very thin trace shown for ORP. The response of each sensor is still clear,
but small changes in chlorine concentration are obviously more difficult to detect.
21
-------
The first injection of arsenic trioxide caused a decrease in free chlorine as measured by the WQS
units as well as the reference method. After the free chlorine concentration reached its minimum
point after injection, the pipe loop was restored to approximately pre-injection conditions by
adding sodium hypochlorite. The WQS units recovered to approximately their pre-injection
levels. Upon the second injection of arsenic trioxide, the reference method measurement again
dropped almost completely to zero, as did the Unit 2 measurements. However, Unit 1 did not
respond at all to the drop in free chlorine recognized by Unit 2 and the reference method. Both
units responded to the addition of sodium hypochlorite to restore the pipe loop to pre-injection
conditions. Unit 1 measured an increased concentration of approximately 1.75 mg/L, while Unit
2 recovered to a concentration measurement slightly lower than what it had been prior to the
injection of arsenic (approximately 0.4 mg/L).
The free chlorine sensors on both units responded to the injection of aldicarb and then returned
to approximately their pre-injection concentrations. After the first injection of aldicarb,
Rosemount staff directed the verification staff to recalibrate the chlorine sensors. This is shown
by the abrupt drop in Unit 1's measurement and the abrupt increase in Unit 2's measurement to
match the first reference result of the final aldicarb injection. Both units responded similarly to
this injection, but they did not drop to a chlorine concentration as low as was measured by the
reference measurement.
The ORP in water is dependent on the occurrence of oxidation-reduction chemical reactions
within the water. Therefore, when free chlorine is reacting with injected contaminants, it can be
expected that the ORP would be affected. Figure 6-2 shows that ORP tracked the concentration
of free chlorine upon injection of the contaminants. The free chlorine reacted with the
contaminants, and the concentration dropped, as did the ORP. It is difficult to determine if the
change in ORP is in response to the drop in free chlorine or to the presence of the contaminant
itself. Note the steep decline in reference free chlorine concentration upon each injection.
Similarly, there is a steep decline in the ORP measurement.
Figures 6-3 and 6-4 show the injection results for pH and conductivity, the water quality
parameters that were affected only by the injection of arsenic trioxide. This effect may have been
due to the pH adjustment required to get this contaminant into solution.
6.3 Extended Deployment
Figures 6-5 through 6-9 show the continuous measurements from both WQS units during the
52-day extended deployment stage of the verification test. Those measurements are represented
by the blue and yellow lines, while the results of the reference samples, collected once daily
throughout this deployment, are represented by the magenta symbols. The x-axis on each figure
represents the period of time between September 1, 2004, and October 22, 2004, while the y-axis
gives the results of each water quality measurement. Data points were recorded every 30 seconds
during the verification test; but, for the extended deployment figures, only data points collected
approximately every 2 minutes were depicted. This was done so that a standard spreadsheet
could be used to generate these figures. This approach was inconsequential to interpreting the
figures.
22
-------
Duiation of Stage 3 52 days
Figure 6-5. Extended Deployment Results for Free Chlorine
7
Figure 6-6. Extended Deployment Results for pH
23
-------
Duration of Stage 3: 52 days
Figure 6-7. Extended Deployment Results for ORP
.JUU
450-
E 400-
1
n
§ 350-
(/)
o
0
£ 300-
250-
onn
s~*\
f **** 1
M/ \
»** \
*\
*^
'^S^
\
i
A
E.ent,,
/
xr
\ x^ ^ /^ **
** » *** •*
— Unitl
* Reference
Unit 2
Duration of Stage 3: 52 clays
Figure 6-8. Extended Deployment Results for Conductivity
24
-------
Duiation of Stage 3: 52 days
Figure 6-9. Extended Deployment Results for Temperature
The objective of this stage of the verification test was to evaluate the performance of the WQS
unit over an extended period of time with minimal intervention to simulate a situation in which
the units may be deployed at a remote location. The continuous trace was evaluated visually to
see whether any aspects of the data were notable. A second, more quantitative, evaluation was
then performed to get an indication of the accuracy of the extended deployment measurements.
This evaluation, much like the accuracy evaluation conducted during the first stage of testing,
included calculating the percent differences between the average continuous measurements and
average reference sample results throughout the extended deployment, as well as the standard
deviation of each of those measurements. The standard deviation of the results provided a means
to evaluate the stability of the water conditions during Stage 3, as well as how the standard
deviations of the continuous measurements differed from the standard deviations of the reference
measurements. Similar relative standard deviations between the continuous and reference
measurements indicate that the variability was mostly dependent on the water conditions and not
due to systematic variability in the WQS unit results. (Note that the reference results were only
generated during business hours, so any fluctuations occurring during off hours are not reflected
in the standard deviation of the reference results. Because of this, free chlorine, a parameter that
varied at times during weekends when the supply of chlorine ran low, might have been expected
to have a larger variability than other more stable parameters.) Table 6-4 lists the percent
differences, along with the average and standard deviations of the reference and continuous
results during the extended deployment. The range and median (see the footnote in Section 6.1
for direction on interpreting the median) percent difference for each water quality parameter, as
measured for each reference sample analyzed during the extended deployment, are also given.
25
-------
Table 6-4. Accuracy During Extended Deployment
Parameter
Free chlorine
Temperature
Conductivity
pH
Reference
Average
(SD)(a)
0.95 (0.10)
22.83 (0.35)
333 (57)
8.72 (0.07)
Unitl
Average (SD)(a)
1.00(0.19)
22.81 (0.25)
349 (57)
8.65 (0.13)
%D
5.3
-0.1
4.8
-0.8
Unit 2
Average (SD)(a)
0.97(0.13)
22.72 (0.28)
351 (57)
8.63 (0.10)
%D
2.1
-0.5
5.4
-1.0
Both WQS Units
%D Range
(median)
-36.2 to 68.3 (1.6)
-4.1 to 2.4 (-0.2)
3.4 to 6.7 (5.2)
-2.8 to 1.8 (-1.2)
a) Free chlorine, mg/L; temperature, °C; conductivity, |^S/cm; pH, pH units.
For free chlorine, visual inspection of the data in Figure 6-5 revealed that at the start of Stage 3,
the WQS units' measurements were similar to the reference results, but drifted lower over the
following several days (1 day = 1 magenta symbol). Thereafter, until approximately one-third of
the way through the extended deployment, the free chlorine measurements were biased low with
respect to the reference measurements. At that point (free chlorine Event #1 in Figure 6-5), the
Rosemount representative directed the verification staff to recalibrate the free chlorine sensors
based on the reference method result. For several days, both WQS units tracked the free chlorine
reference measurements rather well until the measured chlorine concentrations drifted slightly
high for approximately two weeks (free chlorine Event #2). After the Rosemount representative
changed the membranes and calibrated both chlorine sensors (free chlorine Event #3), the WQS
units consistently tracked the free chlorine reference measurements for the remainder of the
extended deployment. During the entire extended deployment, the percent differences for both
WQS units ranged from -36.2 to 68.3, with a median of 1.6. The average free chlorine
concentration, as measured by the reference method, was 0.95 ± 0.10 mg/L.
The measurements from the pH, ORP, conductivity, and temperature sensors are shown in
Figures 6-6 through 6-9. The pH sensor was recalibrated at the same time as the chlorine sensor
(pH Event #1 in Figure 6-6) ; and, with the exception of the two reference measurements prior to
recalibration, the accuracy after calibration was similar to that during the rest of the extended
deployment, with percent differences ranging from -2.8 to 1.8 and a median of -1.2. The ORP
and conductivity sensors were verified by Rosemount staff using standard solutions at the same
time as the pH and chlorine sensors were calibrated (Event #1 in Figures 6-7 and 6-8). This
intervention did not change the results from either of those sensors, but was done only to confirm
the accurate measurement of the standard. The temperature sensor was allowed to operate
without intervention throughout the extended deployment. In Figure 6-7, the ORP results are
shown along with a laboratory reference method result. The ORP reference method does not
provide a reliable result for water in a flowing pipe,(2) but it can be used to evaluate a trend in the
decrease and increase in the ORP, as it was in Stage 2 for the contaminant injections. The Unit 1
and 2 conductivity results tracked the reference method results throughout the extended
deployment. The temperature results from both Units 1 and 2 varied regularly because the test
was conducted in a facility where the water temperature was heavily affected by the outdoor
temperature; therefore, the water temperature changed as a function of the high and low for the
day. However, Unit 2 temperature results appeared to be biased low with respect to Unit 1 and
the reference method.
26
-------
The regular variability in the free chlorine results that was discussed in Section 6.2 continued to
be observed during this stage of the verification test. In fact, the degree of variability seemed to
increase slightly from the start of this stage to the end. A similar variability was observed in the
Stage 3 pH results. Again, the overall effect of this variability seemed small, but it may prevent
small changes in free chlorine or pH from being noticed. With the exception of free chlorine, the
standard deviations of the WQS measurements were similar in magnitude to those of the
reference measurements, indicating that most of the variability in the measurements is due to the
actual variability in the water quality parameters rather than substandard performance.
6.4 Accuracy and Response to Injected Contaminants After Extended Deployment
After the 52-day deployment of the WQS units with minimal intervention, their performance
was evaluated during a 4-hour period of ambient pH and temperature during which reference
samples were collected hourly. The results of this evaluation are given in Table 6-5. With the
exception of free chlorine, these results were comparable to those collected at the start of the
verification test. The free chlorine results measured after extended deployment generated percent
differences of approximately 2%. In contrast, the percent differences at the close of Stage 1 were
greater than 45%. Between the end of Stage 1 and the start of the post-extended deployment
accuracy evaluation, the chlorine sensors had been calibrated twice and the membranes had been
replaced.
Table 6-5. Post-Extended Deployment Results
Parameter
Free chlorine
Temperature
Conductivity
PH
Reference
Average (SD)(a)
0.92 (0.02)
22.66(0.16)
356(1)
8.59 (0.01)
Unitl
Average
(SD)(a)
0.91(0.03)
22.79 (0.08)
374(1)
8.54 (0.05)
%D
-1.1
0.6
5.1
-0.6
Unit 2
Average
(SD)(a)
0.90 (0.03)
22.71 (0.09)
375 (1)
8.51 (0.05)
%D
-2.2
0.2
5.3
-0.9
1 Free chlorine, mg/L; temperature, °C; conductivity, |^S/cm; pH, pH units.
A second evaluation of the response to injected contaminants after the extended deployment
used four contaminants. Two were a repeat of the aldicarb injections performed during Stage 2
and two were injections of E. coli, which was not available for injection during the earlier stage
of the test. Table 6-6 and Figures 6-10 through 6-13 show the directional change of each
reference and WQS measurement in response to the contaminant injections. In general, free
chlorine, ORP, and pH were the parameters clearly affected (for the reference results and all but
one of the continuous measurements) for all four injections.
27
-------
Table 6-6. Effect of Contaminant Injections After Extended Deployment
Parameter
Free chlorine
Temperature
Conductivity
pH
ORP
E. coli
Reference WQS
-
NC NC
+ NC
-
-
Aldicarb
Reference WQS
_(*)
NC NC
NC NC
NC
-
-------
Each section (separated by vertical lines) represents approximately 24 hours
Figure 6-11. Stage 3 Contaminant Injection Results for ORP
Figure 6-12. Stage 3 Contaminant Injection Results for pH
29
-------
420
410
400
E
o
«S
£ 390 H
E
g>
<§ 380
L.
o
E
370
360
350
E. colil E.oa*2 AldicartS
Aldicart 4
Each section (separated by vertical lines) represents approximately 24 hours
Figure 6-13. Stage 3 Contaminant Injection Results for Conductivity
For free chlorine, the reference concentration decreased from approximately 1 mg/L to near zero
upon each of the four injections. The free chlorine sensors started out this portion of the test
measuring concentrations similar to the reference results; however, upon injection, the WQS
units' measurements did not drop lower than 0.5 mg/L. For the first three injections of this stage,
the Unit 1 sensor responded to the contaminant injection in a similar way to the Unit 2 sensor.
When the final injection of aldicarb was made, the Unit 2 sensor did not respond at all even
though the reference method clearly indicated an immediate drop in the free chlorine
concentration to nearly zero. The Unit 1 free chlorine concentration did decrease, but not to the
level of the reference measurement. It also was notable that both free chlorine sensors recovered
adequately to the pre-injection water conditions after the E. coli injections, but after the first
aldicarb injection, recalibration was required to bring the sensor back to the pre-injection
conditions. Because of Unit 2's lack of response to the final aldicarb injection, it seems that the
membrane of the Unit 2 sensor may have become clogged or fouled during the contaminant
injections. The ORP response was, as during Stage 2, consistent across all four injections. For
the pH measured by the two WQS units and the reference method, a brief decrease was observed
upon injecting the culture of E. coli, and the pipe loop quickly returned to the baseline pH. In
addition, the pH measured by the reference method decreased very slightly during the aldicarb
injections. This was an effect that had not been observed during the Stage 2 aldicarb injections,
so it was unexpected. There was also an increase in the conductivity measurement upon injection
of the E. coli. These slight changes in pH and conductivity due to the aldicarb and E. coli
injections, respectively, were measured by the reference method, but were not indicated through
visual observation by the WQS measurements.
30
-------
6.5 Inter-unit Reproducibility
Two WQS units were compared throughout the verification test to determine whether they
generated results that were similar to one another. This was done using the WQS data collected
whenever a reference sample was collected throughout the verification test. Two evaluations
were performed to make this comparison. First, the results from Unit 2 were graphed on the
y-axis, those from Unit 1 were graphed on the x-axis, and a line was fitted to the data. Second, a
t-test assuming equal variances was performed on those same data. For the linear regression
analysis, if both WQS units reported the identical result, the slope of such a regression would be
unity (1), the intercept zero (0), and the coefficient of determination (r2) 1.0. The slope can
indicate whether the results are biased in one direction or the other, while the coefficient of
determination provides a measure of the variability of the results. The t-test shows whether the
sensors generated statistically similar data. Small p-values (<0.05 at a 5% confidence level)
would suggest that the results from the two units are significantly different from one another.
Table 6-7 gives the slope, intercept, and coefficient of determination for the inter-unit
reproducibility evaluation and the p-value for the t-test performed for each sensor.
Table 6-7. Inter-unit Reproducibility Evaluation
Parameter
Free chlorine
Temperature
Conductivity
PH
ORP
Slope
0.48
1.01
1.00
0.97
0.97
Intercept
0.45
-0.19
0.26
0.25
-4.38
r2
0.271
0.999
1.00
0.958
0.950
t-test p-value
0.367
0.882
0.787
0.832
0.011
Shading indicates that the difference between the results of the two sensors was statistically significant.
As seen in Table 6-7, all of the sensors, except free chlorine, had coefficients of determination
greater than 0.95 and slopes greater than 0.97, indicating that their results were very similar and
repeatable. When a t-test was performed on this data, the p-values were much larger than 0.05
for pH, conductivity, free chlorine, and temperature, suggesting that the two sensors of each type
were not significantly different from one another. However, for ORP, even though the regression
data suggested that the results from each sensor were highly correlated with one another, the
extremely small amount of variability in the ORP measurements caused the t-test result to
suggest that there was, in fact, a significant difference between the results of the sensors. Even
though this difference in performance was statistically significant, the magnitude in difference
between the two sensors was small. In addition to the inter-unit statistical evaluation, the results
for all four sensors were confirmed through a visual evaluation of the figures throughout Chapter
6. For temperature, conductivity, and pH, the results from the two WQS units are graphed nearly
on top of one another; while for ORP, a small, but consistent, difference was evident.
The free chlorine sensor had a lower coefficient of determination and a slope that deviated from
unity by greater than 50%. This lower correlation was observed in the figures when Unit 2
drifted to chlorine concentrations different from Unit 1, or when the two sensors responded
differently to contaminant injections. However, even though the sensors were not as highly
correlated with one another as the other sensors, the overall larger variability in the sensor
measurements kept the t-test from determining the results as significantly different from one
31
-------
another. This result was also observed in the figures through frequent overlap of each sensor's
line due to the variability in the signal.
6.6 Ease of Use and Data Acquisition
Throughout the verification test, the verification staff was not required to perform any routine
maintenance. However, on three occasions, the chlorine sensors were recalibrated by Rosemount
or by verification staff (at the direction of Rosemount) to match the reference sample measure-
ment. The chlorine sensor membranes were replaced once during the verification test and debris
deposited into the flow cells was cleaned out at that time. Based on the performance of the WQS
free chlorine sensors, these maintenance activities may have to be performed periodically to
maintain accurate measurements, especially those involving response to injected contaminants.
This would require a means of measuring the chlorine concentration of the water, as well as a
site visit to perform this maintenance. No other maintenance was necessary during the test.
A Fluke data logger was configured with a laptop PC to download the data to the PC's hard drive
in real time. The files were saved as delimited text files for subsequent import into a spreadsheet.
The data logger and laptop are not a standard feature of the Rosemount WQS.
32
-------
Chapter 7
Performance Summary
Evaluation Parameter
Stage 1—
Accuracy
Stage 2 — Response
to Injected
Contaminants
Stage 3 — Accuracy
During Extended
Deployment
Stage 3 — Accuracy
After Extended
Deployment
Stage 3 — Response
to Injected
Contaminants
Injection Summary
Units 1 and 2,
range of %D (median)
Nicotine
Arsenic
trioxide
Aldicarb
Reference
WQS
Reference
WQS
Reference
WQS
Units 1 and 2,
range of %D (median)
Unit 1, %D
Unit 2, %D
E. coli
Aldicarb
Reference
WQS
Reference
WQS
Free Chlorine
-11.1 to 96.7
(14.5)
-
-
-
(b)
-
-
-36.2 to 68.3
(1.6)
-1.1
-2.2
-
-
-
(b)
Tem-
perature
-5.9 to 1.5
(-1.7)
NC
NC
NC
NC
NC
NC
-4.1 to 2. 4
(-0.2)
0.6
0.2
NC
NC
NC
NC
Conductivity
2.9 to 5.3
(4.2)
NC
NC
+
+
NC
NC
3.4 to 6.7
(5.2)
5.1
5.3
+
NC
NC
NC
pH
-7.4 to -1.1
(-3.0)
NC
NC
+
+
NC
NC
-2.8 to 1.8
(-1.2)
-0.6
-0.9
-
-
-
NC
ORP
(a)
-
-
-
-
-
-
(a)
(a)
(a)
-
-
-
-
For a reason that is not clear, aldicarb altered the pH, as measured by the reference method,
during the Stage 3 injections, but not during the Stage 2 injections.
Slope (intercept)
r2
p-value
0.48 (0.45)
0.271
0.367
1.01 (-0.19)
0.999
0.882
1.00(0.26)
1.00
0.787
0.97 (0.25)
0.958
0.832
0.97 (-4.38)
0.950
0.01 l(c)
Reproducibility
(Unit 2 vs. Unit 1)
With the exception of ORP, the t-test indicated that the sensors on each unit were performing
similarly. For ORP, the linear correlation between the two units was very high, but the
extremely small variability in the signal caused the difference between the two units to be
statistically significant. Although the free chlorine sensors were not highly correlated with one
another, the large variability in their measurements prevented the t-test from determining a
significant difference between the units.
Ease of Use and Data
Acquisition
Based on the performance of the free chlorine sensors, calibration and membrane replacement
may have to occur periodically to maintain accurate measurements, especially those involving
response to injected contaminants. Also, the regular variability in free chlorine and pH
measurements may prevent observing small changes in those water quality parameters.
-------
Chapter 8
References
1. Test/QA Plan for Verification of Multi-Parameter Water Monitors for Distribution
Systems, Battelle, Columbus, Ohio, August 2004.
2. Personal communication with John Hall, U.S. EPA, July 23, 2004.
3. U.S. EPA, EPA Method 150.1, pH, in Methods for Chemical Analysis of Water and
Wastes, EPA/600/4-79/020, March 1983.
4. American Public Health Association, et al., SM 2510, Conductivity, in Standard
Methods for the Examination of Water and Wastewater, 19th Edition, Washington,
D.C., 1997.
5. American Public Health Association, et al., SM 4500-G, Residual Chlorine, in
Standard Methods for the Examination of Water and Wastewater, 19th Edition,
Washington, D.C., 1997.
6. American Public Health Association, et al., SM 2580-B, Electrochemical Potential, in
Standard Methods for the Examination of Water and Wastewater. 19th Edition,
Washington, D.C., 1997.
7. U.S. EPA, EPA Method 170.1, Temperature, in Methods for Chemical Analysis of
Water and Wastes, EPA/600/4-79/020, March 1983.
8. Quality Management Plan (QMP)for the ETV Advanced Monitoring Systems Center,
Version 5.0, U.S. EPA Environmental Technology Verification Program, Battelle,
Columbus, Ohio, March 2004.
34
------- |