November 2005
Environmental Technology
Verification Report
HACH COMPANY WATER DISTRIBUTION
MONITORING PANEL AND THE EVENT
MONITOR™ TRIGGER SYSTEM
Prepared by
Battelle
Baireiie
Ira Business erf Innovation
Under a cooperative agreement with
SV tlJT\ U.S. Environmental Protection Agency
ET1/ET1/
-------
November 2005
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
HACH COMPANY WATER DISTRIBUTION
MONITORING PANEL AND THE EVENT
MONITOR™ TRIGGER SYSTEM
by
Ryan James
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
v>EPA
U.S. Environmental Protection Agency
ET/
Battelle
Business i>_/ Innovation
ETV Joint Verification Statement
TECHNOLOGY TYPE: MULTI-PARAMETER WATER MONITORS
FOR DISTRIBUTION SYSTEMS
APPLICATION:
MONITORING DRINKING WATER QUALITY
TECHNOLOGY NAME: Water Distribution Monitoring Panel and Event
Monitor™ Trigger System
COMPANY:
ADDRESS:
WEB SITE:
E-MAIL:
Hach Company
P.O. Box 389
Loveland, Colorado 80538
www.hach.com
dkroll @hach.com
PHONE: 800-227-4224
FAX: 970-669-2932
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology Verification (ETV)
Program to facilitate the deployment of innovative or improved environmental technologies through performance
verification and dissemination of information. The goal of the ETV Program is to further environmental protection
by accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal
by providing high-quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies. Information and ETV
documents are available at www.epa.gov/etv.
ETV works in partnership with recognized standards and testing organizations, with stakeholder groups
(consisting of buyers, vendor organizations, and permitters), and with individual technology developers. The
program evaluates the performance of innovative technologies by developing test plans that are responsive to the
needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and pre-
paring peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA)
protocols to ensure that data of known and adequate quality are generated and that the results are defensible.
The Advanced Monitoring Systems (AMS) Center, one of six technology areas under ETV, is operated by Battelle
in cooperation with EPA's National Exposure Research Laboratory. The AMS Center evaluated the performance
of the Hach Company Water Distribution Monitoring Panel (WDMP), as well as the Hach Event Monitor™
Trigger System (EMTS), in continuously measuring total chlorine, turbidity, temperature, conductivity, pH, and
total organic carbon (TOC) in drinking water. This verification statement provides a summary of the test results.
-------
VERIFICATION TEST DESCRIPTION
The performance of the WDMP and EMTS units was assessed in terms of their accuracy, response to injected
contaminants, inter-unit reproducibility, ease of use, and data acquisition. The verification test was conducted
between August 9 and November 12, 2004, and consisted of four stages, each designed to evaluate a particular
performance characteristic of the WDMP and EMTS units. The first three stages of the test were conducted using a
recirculating pipe loop at the U.S. EPA's Test and Evaluation Facility in Cincinnati, Ohio. Stage 4 used a single-
pass pipe at the same facility.
In the first stage of this verification test, the accuracy of the measurements made by the WDMP units was
evaluated during eight, 4-hour periods of stable water quality conditions by comparing each WDMP unit
measurement to a grab sample result generated each hour using a standard laboratory reference method and then
calculating the percent difference (%D). The second stage of the verification test involved evaluating the response
of the WDMP units to changes in water quality parameters caused by injecting contaminants (nicotine, arsenic
trioxide, and aldicarb) into the pipe loop. Two injections of three contaminants were made into the recirculating
pipe loop containing finished Cincinnati drinking water. Grab samples were collected prior to the contaminant
injections and at 3, 15, and 60 minutes after injection to confirm the response of each water quality parameter,
whether it was an increase, decrease, or no change. In the first phase of Stage 3 of the verification test, the
performance of the WDMP units was evaluated during 52 days of continuous operation, throughout which
reference samples were collected once daily. The final phase of Stage 3 (which immediately followed the first
phase of Stage 3 and lasted approximately one week) consisted of a two-step evaluation of the WDMP to
determine whether this length of operation would negatively affect results. First, as during Stage 1, a reference
grab sample was collected every hour during a 4-hour analysis period and analyzed using the standard reference
methods. Again, this was done to define a formal time period of stable water quality conditions over which the
accuracy of the WDMP could be evaluated. Second, to evaluate the response of the WDMP to contaminant
injection after the extended deployment, the duplicate injection of aldicarb, which was also included in the Stage 2
testing, was repeated. In addition, a pure E. coli culture, including the E. coli and the growth medium, was
included as a second injected contaminant during Stage 3. The fourth and final stage of the verification test
involved testing whether the EMTS detected the injection of 13 contaminants (aldicarb, arsenic trioxide,
colchicine, dichlorvos, dicamba, E. coli bacteria, glyphosate, lead nitrate, mercuric chloride, methanol, nicotine,
potassium ferricyanide, and sodium fluoroacetate), as well as whether it correctly identified the contaminants.
Because the Stage 4 results were qualitative, grab samples were not collected. Throughout the test, inter-unit
reproducibility was assessed by comparing the results of two identical units operating simultaneously. Ease of use
was documented by technicians who operated and maintained the units, as well as the Battelle Verification Test
Coordinator.
QA oversight of verification testing was provided by Battelle and EPA. Battelle QA staff conducted a technical
systems audit, a performance evaluation audit, and a data quality audit of 10% of the test data.
This verification statement, the full report on which it is based, and the test/QA plan for this verification test are all
available at www.epa.gov/etv/centers/centerl.html.
-------
TECHNOLOGY DESCRIPTION
This verification report provides results for the verification testing of the Hach WDMP, as well as the EMTS,
which functions in concert with the WDMP. For the purposes of this report, the astroTOC online ultraviolet (UV)
TOC analyzer was considered a part of the WDMP, even though the TOC analyzer is actually a stand-alone
continuous monitor. Following is a description of the combined system, based on information provided by the
vendor. The information provided below was not verified in this test.
The WDMP contains online monitors for free or total chlorine, pH, turbidity, electrolytic conductivity,
temperature, sample pressure, and TOC. Chlorine residual is measured by a Hach CL17 chlorine analyzer. The
CL17 collects a water sample every 2.5 minutes and uses the EPA-approved colorimetric diethyl-p-phenylene
diamine method. The CL17 uses minimal reagents and a mixing system that operates with no moving parts,
including a self-cleaning stir bar in the sample chamber. A differential pH electrode, which uses a pH buffer as a
reference point, measures pH. Turbidity is measured using a Hach 1720D process turbidimeter. The sample flows
continuously through a patented bubble removal system that vents entrained air from the sample, eliminating
interference in low-level turbidity measurement. Incandescent light is directed from the sensor head assembly
down into the turbidimeter body and is scattered by suspended particles in the sample. A sensor detects light
scattered at 90 degrees from the incident beam, which is a measure of the turbidity in the water. Electrolytic
conductivity is continuously measured by a two-electrode cell. Temperature is measured by the
temperature-sensing element in the conductivity cell.
The astroTOC UV analyzer combines a chemical and UV oxidation technique in a low-temperature reactor to
measure the TOC. A 4-20 mA analog signal carries the TOC information to the EMTS. The WDMP is fed by a
single, 1/2-inch sample line. Free-flowing waste drains through a single outlet. A sample line runs from the
WDMP to the astroTOC, which has a drain line from a single outlet.
The EMTS integrates the multiple sensor outputs from the WDMP and astroTOC. Once each minute, software
applies an algorithm (patent pending) to the sensor measurements, calculating the site's water quality baseline.
The EMTS alarms when the trigger signal exceeds a trigger threshold, indicating an "event." The EMTS may be
equipped with an agent library containing profiles of various contaminants. The EMTS also contains a plant event
library that has no entries when the system is first installed. If an event occurs and its signature cannot be matched
to any signature in the agent library, the plant event library is searched for a match. If a match is found, the event is
reported. If no match is found, the signature for the event is stored in case the event recurs. In addition to a trigger
signal alarm, the EMTS can also alarm on high/low parameter excursions. It logs all input data, trigger signal
values, and diagnostic data in a database that can be extracted to a memory stick. Operators can view and recall
logged data for each parameter and the trigger signal using a touch-screen. The EMTS can also act as "slave" on
an RS485 Modbus network to provide data whenever polled by a Modbus "master."
The combined system of the WDMP and the EMTS, designed for wall or rack mounting, is approximately 3.3 feet
tall by 6.6 feet wide. The WDMP costs $12,800, the EMTS costs $8,450, and the online TOC analyzer costs
$14,076 in the recommended configuration for a total cost of approximately $35,000 for the units tested. The
monthly cost for consumables is approximately $260.
-------
VERIFICATION OF PERFORMANCE
Evaluation Parameter
Stage 1—
Accuracy
Stage 2 —
Response to
Injected
Contaminants
Stage 3—
Accuracy During
Extended
Deployment
Stage 3—
Accuracy After
Extended
Deployment
Stage 3—
Response to
Injected
Contaminants
Injection
Summary
Inter-unit
Reproducibility
(Unit 2 vs. Unit 1)
Stage 4—
Contaminant
Identification
Ease of Use and
Data Acquisition
Units 1 and 2, range
of %D (median)
Nicotine
Arsenic
trioxide
Aldicarb
Reference
WDMP
Reference
WDMP
Reference
WDMP
Units 1 and 2, range
of %D (median)
Unit 1, %D
Unit 2, %D
E. coli
Aldicarb
Reference
WDMP
Reference
WDMP
Total
Chlorine
-47.4 to 4.5
(-3.9)
-
-
-
-
-
-
-15.9 to 6.9
(-3.2)
-4.9
-4.9
-
-
-
-
Turbidity
-53.9 to -1.3
(-34.1)
(a)
+
(a)
+
(a)
+
-81.1 to
245.5 (-21.3)
-5.9
-11.8
+(b)
+
+(b)
+
Tem-
perature
-3.0 to 44.3
(-0.2)
NC
NC
NC
NC
NC
NC
-7.4 to 8.5
(-0.1)
-0.2
4.6
NC
NC
NC
NC
Conductivity
-15.5 to 8.1
(2.2)
NC
NC
+
+
NC
NC
-1.8 to 9.6
(4.8)
6.7
0.3
+
+
NC
NC
pH
-6.6 to 3.1
(0.9)
NC
NC
+
+
NC
NC
-2.7 to 0.5
(-0.9)
-2.2
0.2
-
-
-
-
TOC
-64.7 to 147.5
(-14.8)
+
+
NC
NC
+
+
-47.3 to 103.0
(-6.9)
-20.5
3.4
+
+
+
+
Total chlorine and TOC were dramatically affected by injections of nicotine, E. coli, and aldicarb; and
turbidity, pH, and conductivity were affected by some or all of the injections, but not as consistently as total
chlorine and TOC. Aldicarb altered the pH during Stage 3, but not Stage 2.
Slope (intercept)
r2
p-value
0.98
(0.03)
0.994
0.779
0.97
(0.005)(c)
0.88 l(c)
0.884(c)
0.72
(7.68)
0.758
5.5 x 10'6
0.92
(4.19)
0.961
0.006
1.06
(-0.40)
0.919
0.517
0.97
(0.31)
0.991
0.374
With the exception of temperature and conductivity, both units generated similar results.
Each time a contaminant was injected, the EMTS detected a deviation in baseline conditions, causing a
"trigger event." Eleven of 13 contaminants were correctly identified at some point during the injection time.
Ferricyanide and lead nitrate were identified correctly 100% of the time. The rest of the injected
contaminants were identified as a contaminant other than themselves at some point throughout the duration
of the injection. Only nicotine and arsenic trioxide were never correctly identified.
Neither the WDMPs nor the EMTSs required daily operator attention. Hach Company staff adjusted the
flows on the turbidity and total chlorine meters as needed to keep them at the required levels and rebooted
the EMTS when it was not displaying data properly. The chlorine sensors and turbidimeters needed periodic
cleaning, and the TOC analyzer was calibrated three times.
(a) Relatively large uncertainty in the reference measurements made it difficult to detect a significant change.
(b) Magnitude of change different between duplicate injections.
(c) Outlier excluded.
+/- = Parameter measurement increased/decreased upon injection.
NC = No change in response to the contaminant injection.
-------
Original signed by Gregory A. Mack 11/23/05 Original signed by Andrew P. Avel 1/17/06
Gregory A. Mack Date Andrew P. Avel Date
Assistant Division Manager Acting Director
Energy, Transportation, and Environment Division National Homeland Security Research Center
Battelle U.S. Environmental Protection Agency
NOTICE: ETV verifications are based on an evaluation of technology performance under specific, predetermined
criteria and the appropriate quality assurance procedures. EPA and Battelle make no expressed or implied
warranties as to the performance of the technology and do not certify that a technology will always operate as
verified. The end user is solely responsible for complying with any and all applicable federal, state, and local
requirements. Mention of commercial product names does not imply endorsement.
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
accelerating the entrance of new environmental technologies into the marketplace. Verification
organizations oversee and report verification activities based on testing and quality assurance
protocols developed with input from major stakeholders and customer groups associated with the
technology area. ETV consists of six verification centers. Information about each of these
centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/centers/centerl.html.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We would like to thank Roy Haught
and John Hall of the U.S. Environmental Protection Agency's (EPA's) Testing and Evaluation
(T&E) Facility (operated by Shaw Environmental, Inc.) in Cincinnati, Ohio, for hosting the
verification test. The T&E Facility's contribution included providing the reference analyses and
operating the pipe loop, as well as reviewing the test/quality assurance (QA) plan and the
reports. In addition, we would like to thank Steve Allgeier of EPA's Office of Water, Gary
Norris and Alan Vette of the EPA National Exposure Research Laboratory, Lisa Olsen of the
U.S. Geological Survey, Ron Hunsinger of East Bay Municipal Utility District, and
Matthew Steele of the City of Columbus Water Quality Assurance Laboratory, who also
reviewed the test/QA plan and/or the reports.
IV
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations ix
1 Background 1
2 Technology Description 2
3 Test Design 5
3.1 Introduction 5
3.2 Test Stages 6
3.2.1 Stage 1, Accuracy 6
3.2.2 Stage 2, Response to Injected Contaminants 7
3.2.3 Stage 3, Extended Deployment 7
3.2.4 Stage 4, Contaminant Identification 8
3.3 Laboratory Reference and Quality Control Samples 9
3.3.1 Reference Methods 9
3.3.2 Reference Method Quality Control Samples 9
4 Quality Assurance/Quality Control 11
4.1 Audits 11
4.1.1 Performance Evaluation Audit 11
4.1.2 Technical Systems Audit 12
4.1.3 Audit of Data Quality 12
4.2 Quality Assurance/Quality Control Reporting 12
4.3 Data Review 12
5 Statistical Methods 14
5.1 Accuracy 14
5.2 Response to Injected Contaminants 14
5.3 Inter-unit Reproducibility 15
5.4 Contaminant Identification 15
6 Test Results 17
6.1 Accuracy 18
6.2 Response to Injected Contaminants 24
-------
6.3 Extended Deployment 29
6.4 Accuracy and Response to Injected Contaminants After Extended Deployment ... 35
6.5 Inter-unit Reproducibility 39
6.6 Contaminant Identification 41
6.7 Ease of Use and Data Acquisition 45
7 Performance Summary 47
8 References 48
Appendix A. Hach Company Review A-l
Figures
Figure 2-1. Hach Company WDMP 2
Figure 2-2. Hach Company EMTS 3
Figure 6-1. Stage 2 Contaminant Injection Results for Total Chlorine 25
Figure 6-2. Stage 2 Contaminant Injection Results for TOC 26
Figure 6-3. Stage 2 Contaminant Injection Results for Turbidity 26
Figure 6-4. Stage 2 Contaminant Injection Results for pH 27
Figure 6-5. Stage 2 Contaminant Injection Results for Conductivity 27
Figure 6-6. Extended Deployment Results for Total Chlorine 30
Figure 6-7. Extended Deployment Results for TOC 30
Figure 6-8. Extended Deployment Results for Turbidity 31
Figure 6-9. Extended Deployment Results for pH 31
Figure 6-10. Extended Deployment Results for Temperature 32
Figure 6-11. Extended Deployment Results for Conductivity 32
Figure 6-12. Stage 3 Contaminant Injection Results for Total Chlorine 37
Figure 6-13. Stage 3 Contaminant Injection Results for TOC 37
vi
-------
Figure 6-14. Stage 3 Contaminant Injection Results for Turbidity 38
Figure 6-15. Stage 3 Contaminant Injection Results for pH 38
Figure 6-16. Stage 3 Contaminant Injection Results for Conductivity 39
Tables
Table 3-1. Stage 4 Injected Contaminants 8
Table 3-2. Reference Methods 10
Table 3-3. Reference Analyses and Quality Control Samples 10
Table 4-1. Performance Evaluation Audit Results
and Reference Method Duplicate Analysis 12
Table 4-2. Summary of Data Recording Process 13
Table 6-1. Summary of Test Stages and Type of Data Presentation 17
Table 6-2a. Accuracy Evaluation Under Various Conditions—Total Chlorine 18
Table 6-2b. Accuracy Evaluation Under Various Conditions—Turbidity 19
Table 6-2c. Accuracy Evaluation Under Various Conditions—Temperature 20
Table 6-2d. Accuracy Evaluation Under Various Conditions—Conductivity 21
Table 6-2e. Accuracy Evaluation Under Various Conditions—pH 22
Table 6-2f. Accuracy Evaluation Under Various Conditions—Total Organic Carbon 23
Table 6-3. Effect of Contaminant Injections Prior to Extended Deployment 25
Table 6-4. Accuracy During Extended Deployment 33
Table 6-5. Post-Extended Deployment Results 36
Table 6-6. Effect of Contaminant Injections After Extended Deployment 36
Table 6-7. Inter-unit Reproducibility Evaluation 40
vii
-------
Table 6-8. Contaminant Identification—Number and Quality of Matches 42
Table 6-9. Classification Rate Levels 45
Vlll
-------
List of Abbreviations
AMS Advanced Monitoring Systems
°C degree centigrade
DI deionized
EMTS Event Monitor™ Trigger System
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
L liter
|iSiemens/cm microSiemens per centimeter
mg/L milligram per liter
mV millivolt
NIST National Institute of Standards and Technology
ntu nephelometric turbidity unit
%D percent difference
PE performance evaluation
PVC polyvinyl chloride
QA quality assurance
QC quality control
QMP quality management plan
SD standard deviation
T&E Test and Evaluation
TOC total organic carbon
TSA technical systems audit
UV ultraviolet
WDMP Water Distribution Monitoring Panel
IX
-------
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
evaluated the performance of the Hach Company Water Distribution Monitoring Panel (WDMP)
in continuously measuring total chlorine, turbidity, temperature, conductivity, pH, and total
organic carbon (TOC) in drinking water, as well as the Event Monitor™ Trigger System's
(EMTS) ability to identify contaminants as they are injected into a pipe of flowing drinking
water. Continuous multi-parameter water monitors for distribution systems were identified as a
priority technology verification category through the AMS Center stakeholder process.
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This verification report provides
results for the verification testing of the WDMP as well as the EMTS, which functions in concert
with the WDMP. For the purposes of this report, the astroTOC online ultraviolet (UV) TOC
analyzer was considered a part of the WDMP, even though the TOC analyzer is actually a stand-
alone continuous monitor. Following is a description of the combined system, based on
information provided by the vendor. The information provided below was not verified in this
test.
:
Figure 2-1. Hach Company WDMP
The WDMP (Figure 2-1) contains online
monitors for free or total chlorine, pH,
turbidity, conductivity, temperature, sample
pressure, and TOC. Chlorine residual is
measured by a Hach CL17 chlorine
analyzer. The CL17 collects a water sample
every 2.5 minutes and uses the
EPA-approved colorimetric
diethyl-p-phenylene diamine method. The
CL17 uses minimal reagents and a mixing
system that operates with no moving parts,
including a self-cleaning stir bar in the
sample chamber. A differential pH
electrode, which uses a pH buffer as a
reference point, measures pH. Turbidity is
measured using a Hach 1720D process
turbidimeter. The sample flows
continuously through a patented bubble
removal system that vents entrained air
from the sample, eliminating interference in
low-level turbidity measurement.
Incandescent light is directed from the
sensor head assembly down into the
turbidimeter body and is scattered by
suspended particles in the sample. A sensor
detects light scattered at 90 degrees from
the incident beam, which is a measure of
-------
the turbidity in the water. Conductivity is continuously measured by a two-electrode cell.
Temperature is measured by the temperature-sensing element in the conductivity cell.
The astroTOC UV analyzer combines a chemical and UV oxidation technique in a
low-temperature reactor to measure the TOC. A 4-20 mA analog signal carries the TOC
information to the EMTS. The WDMP is fed by a single, 1/2-inch sample line. Free-flowing
waste drains through a single outlet. A sample line runs from the WDMP to the astroTOC, which
has a drain line from a single outlet.
The EMTS (Figure 2-2) integrates the multiple sensor outputs from the WDMP and astroTOC.
Once each minute, software applies an algorithm (patent pending) to the sensor measurements,
calculating the site's water quality
baseline. The EMTS alarms when the
trigger signal exceeds a trigger threshold,
indicating an "event." The EMTS may be
equipped with an agent library containing
profiles of various contaminants. The
EMTS also contains a plant event library
that has no entries when the system is first
installed. If an event occurs and its
signature cannot be matched to any
signature in the agent library, the plant
event library is searched for a match. If a
match is found, the event is reported. If no
match is found, the signature for the event
is stored for future reference in case the
event recurs. Thus, the plant event library
records the signatures for events that occur
at that location, over time. Operators can
label plant event profiles by name and
severity level, allowing recognition of
Figure 2-2. Hach Company EMTS
events that have previously been found, thus decreasing the frequency of unknown alarms over
time. The ability of the EMTS to learn over time in no way compromises its ability to trigger on
events as soon as deployment occurs. Also, once a baseline measurement has been established
(taking a few minutes), any deviation from baseline (excursion) matching the water quality
parameter pattern of an agent found in the supplied agent library will be classified as such.
Unlike traditional monitoring/classification systems, the EMTS is able to trigger on excursions
that are not as yet found in either the agent library or plant library and alert the operator to an
unknown excursion that represents a significant change in water quality. Operator input is
required to investigate and name the unknown event.
In addition to a trigger signal alarm, the EMTS can also alarm on predetermined high/low
parameter settings. It logs all input data, trigger signal values, and diagnostic data in a database
that can be extracted to a memory stick. Operators can view and recall logged data for each
parameter and the trigger signal using a touch-screen. The EMTS can also act as "slave"on an
RS485 Modbus network to provide data whenever polled by a Modbus "master."
-------
The combined system of the WDMP and the EMTS, designed for wall or rack mounting, is
approximately 3.3 feet tall by 6.6 feet wide. The WDMP costs $12,800, the EMTS costs $8,450,
and the online TOC analyzer costs $14,076 in the recommended configuration for a total cost of
approximately $35,000 for the units tested. The monthly cost for consumables is approximately
$260.
-------
Chapter 3
Test Design
3.1 Introduction
The multi-parameter water monitors tested consisted of instrument packages that connect to
distribution system pipes for continuous monitoring. Also included in this technology category
were technologies that can be programmed to automatically sample and analyze distribution
system water at regular intervals. The minimum requirement for participation in this verification
test was that the water monitors were able to measure residual chlorine, as well as at least one
other water quality parameter. Residual chlorine is a particularly important water quality
parameter because changes in its concentration can indicate the presence of contamination
within a distribution system, and chlorination is a very common form of water treatment used by
water utilities in the United States.
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Multi-Parameter Water Monitors for Distribution Systems^ and assessed the
performance of the WDMP units in continuously monitoring pH, conductivity, total chlorine,
TOC, temperature, and turbidity in terms of the following:
• Accuracy
• Response to injected contaminants
• Inter-unit reproducibility
• Ease of use and data acquisition.
In addition, the ability of the EMTS units to identify when a contaminant injection had occurred
and what contaminant had been injected was verified.
Accuracy was quantitatively evaluated by comparing the results generated by two WDMP units
to grab sample results generated by a standard laboratory reference method. Response to injected
contaminants was evaluated qualitatively by observing whether the measured water quality
parameters were affected by the injection of several contaminants. Inter-unit reproducibility was
assessed by comparing the results of two identical units operating simultaneously. Ease of use
was documented by technicians who operated and maintained the units, as well as the Battelle
Verification Test Coordinator. Contaminant identification was verified by reporting whether the
EMTS recognized an injection and correctly identified the presence of the contaminant during
the injection.
-------
3.2 Test Stages
This verification test was conducted between August 9 and November 12, 2004, and consisted of
four stages, each designed to evaluate a particular performance characteristic of the WDMP and
EMTS units. The first three stages of the test were conducted using a recirculating pipe loop at
the U.S. EPA's Test and Evaluation (T&E) Facility in Cincinnati, Ohio. The recirculating pipe
loop consisted of ductile iron pipe, 6 inches in diameter and 100 feet long, which contained
approximately 240 gallons of Cincinnati drinking water with a flow rate of approximately
1 foot/second. The water within the pipe loop had a residence time of approximately 24 hours.
Water from the pipe loop was plumbed to the two WDMP units by a section of 2-inch polyvinyl
chloride (PVC) pipe in series with a shut-off valve with a ribbed nozzle that was connected to
the WDMP and EMTS units with an 18-foot, 1/2-inch outside diameter hose. Reference samples
of approximately 1 liter (enough volume to perform all the required analyses) to be analyzed by
each standard laboratory reference method were collected from the reference sample collection
valve located on the PVC pipe.
The fourth stage of the verification test was conducted using a single-pass pipe at the same
facility. The single-pass pipe consisted of fiberglass-lined ductile iron pipe that was 3 inches in
diameter. The flow rate of the single-pass pipe was approximately 20 L/minute. The distance
between the injection portal and the WDMP and EMTS was approximately 82 feet. The WDMP
and EMTS were plumbed to a sampling valve in a manner similar to that described previously.
This stage of testing was conducted to accommodate the Hach EMTS. No other technologies
participated.
3.2.1 Stage 1, Accuracy
During the first stage of this verification test, the accuracy of the measurements made by the
WDMP units was evaluated by comparing the results from each unit to the result generated by a
standard laboratory reference method. Stage 1 testing simulated the characteristics of a variety of
water quality conditions by changing two variables: pH and temperature. Using nine sets of pH
and temperature conditions, this evaluation consisted of separate four-hour testing periods of
continuous analysis, with reference method sampling and analysis every hour. Four sets of
conditions involved varying only the pH by injecting the pipe loop with a steady stream of
sodium bisulfate. Those sets consisted of pHs of approximately 7, 8, and 9 pH units (ambient pH
at the T&E Facility ambient was between 8 and 9) and a temperature between 21 and 23 degrees
centigrade (°C) (T&E Facility ambient during the time of testing). Two other sets included
changing the water temperature to between 12 and 14°C and testing at pHs of approximately 7
and 8; and, finally, two sets at approximately these pHs, but at a temperature of approximately
27°C. One set (Set 2) was repeated as Set 3. The pipe loop ambient conditions were analyzed at
the start and end of this stage. Prior to each testing period with unique conditions, the water in
the pipe loop was equilibrated until the pH and temperature were at the desired level, as
determined by the standard reference methods. This equilibration step took approximately
12 hours from the time the sodium bisulfate was added (to decrease pH) or the temperature was
adjusted.
-------
3.2.2 Stage 2, Response to Injected Contaminants
The second stage of the verification test involved testing the response of the WDMP units to
changes in water quality parameters caused by injecting contaminants into the pipe loop. Two
injections of three contaminants were made into the recirculating pipe loop containing finished
Cincinnati drinking water. Each injection was made over a period of approximately 15 seconds
by connecting the injection tank to the pipe loop's recirculating pump. The three contaminants
were nicotine, arsenic trioxide (adjusted to pH 12 to get it into solution), and aldicarb. With the
exception of the first nicotine injection, each of these contaminants was dissolved in
approximately 5 gallons of pipe loop water that had been dechlorinated using granular carbon
filtration to prevent degradation of the contaminant prior to injection. Upon injection, concen-
trations of these contaminants within the pipe loop were approximately 10 milligrams per liter
(mg/L). For the first nicotine injection, however, not enough nicotine to attain this concentration
was available so the available nicotine was dissolved into 2 gallons of the dechlorinated pipe
loop water and injected. The resulting nicotine concentration in the pipe loop was calculated to
be approximately 6 mg/L. Because the qualitative change (increase or decrease) in water quality
parameters was similar for both nicotine injections despite the concentration difference, it was
not necessary to repeat the 10 mg/L injection of nicotine. The concentration of injected
contaminants was not confirmed after injection; therefore, the concentration in the pipe loop is
based on the gravimetric measurements during solution prep and subsequent dilution in the pipe
loop. For all three sets of injections, a grab sample was collected prior to the injection and again
at 3, 15, and 60 minutes after the injection. The difference between reference method results
before and after injection indicated the approximate change in water quality caused by the
injected contaminant. For each injected contaminant, the results from the WDMP units were
evaluated based on how well their directional change matched that of the reference method
result. After each injection, the pipe loop was allowed to re-equilibrate for at least 12 hours so
that each WDMP unit returned to a steady baseline. Injected contaminants were obtained from
Sigma-Aldrich (St. Louis, Missouri) or ChemService (West Chester, Pennsylvania) and were
accompanied by a certificate of analysis provided by the supplier. Battelle QA staff audited the
gravimetric preparation of these solutions.
3.2.3 Stage 3, Extended Deployment
In the first phase of Stage 3 of the verification test, the performance of the WDMP units was
evaluated during 52 days of continuous operation. To track the performance of the WDMP units
with respect to the reference results, reference samples were collected and analyzed for the
selected parameters at least once per day (excluding weekends and holidays) for the duration of
Stage 3. All continuously measured data were graphed, along with the results from the reference
measurements, to provide a qualitative evaluation of the data. Throughout the duration of the
deployment, the average percent difference (%D), as defined in Section 5.1, between the results
from the WDMP units and those from the reference methods throughout the duration of the
deployment was evaluated to determine how well the WDMP unit results compared to the
reference method was evaluated.
The second phase of Stage 3 (which immediately followed the first phase of Stage 3 and lasted
approximately one week) consisted of a two-step evaluation of the WDMP unit performance
after the 52-day extended deployment to determine whether this length of operation would
negatively affect the results from the WDMP. First, while the WDMP units were continuously
7
-------
operating, a reference sample was collected every hour during a 4-hour analysis period and
analyzed using the standard reference methods. This was done to define a formal time period of
stable water quality conditions for the accuracy of the WDMP units to be evaluated. Second, to
evaluate the response of the WDMP units to contaminant injection after the extended
deployment, two injections of aldicarb, which were also included in the Stage 2 testing, were
repeated. In addition, a pure E. coli culture, including the E. coli and the growth medium, was
included as a second injected contaminant during Stage 3. E. coli was intended as an injected
contaminant during Stage 2, but was not available until later in the test. During this contaminant
injection component of Stage 3, reference samples were collected as they were during Stage 2.
3.2.4 Stage 4, Contaminant Identification
The purpose of the fourth and final stage of the verification test was to determine whether the
EMTS detected the injection of selected contaminants, as well as whether it correctly identified
each contaminant. Two separate injections of 13 contaminants (see Table 3-1) were made into
the single-pass pipe described previously. The contaminants were selected from 22 compounds
whose agent signatures were included in the Hach library at the time of testing. To protect the
integrity of this portion of the test, the Hach Company was not informed of the identity of the
contaminant injected until the contaminant identification data from each injection was in
Battelle's possession. Also, duplicate injections were never performed subsequent to one
another. All the contaminants were injected randomly once over approximately two days and,
thereafter, each contaminant was injected a second time, again in random order. To test the
capability of the EMTS appropriately, Hach suggested that the concentration of each
contaminant be at least 10 mg/L once injected into the pipe and that the injection duration be
10 to 20 minutes. To attain an in-pipe concentration well above that, a 10-L injection solution of
each contaminant was prepared at approximately 600 mg/L. This solution was injected for
20 minutes at a flow rate of 0.5 L/minute, while the pipe flow rate was approximately
20 L/minute. Because of the higher flow rate in the pipe, a dilution factor had to be applied to
calculate the resulting concentration within the pipe, which was approximately 15 mg/L. Of
course, concentrations at the leading and trailing edges of the sample slug were expected to be
less than 15 mg/L because of dilution with unspiked water in front of and behind the injected
slug of contaminant. Note that the concentrations of each contaminant were not confirmed using
analytical methodology. Table 3-1 lists the contaminants that were injected during this stage of
the verification test.
Table 3-1. Stage 4 Injected Contaminants
Injected Contaminants Approximate Concentration
Aldicarb, arsenic trioxide, colchicine, dichlorvos, dicamba, E. coli
bacteria, glyphosate, lead nitrate, mercuric chloride, methanol, 15 mg/L
nicotine, potassium ferricyanide, and sodium fluoroacetate.
-------
3.3 Laboratory Reference and Quality Control Samples
The WDMP units were evaluated by comparing their results with laboratory reference measure-
ments. The following sections provide an overview of the applicable procedures, analyses, and
methods.
3.3.1 Reference Methods
To eliminate the possibility of using stagnant water residing in the reference sample collection
valve (dead volume) as the reference samples, the first step in the reference sample collection
procedure included collecting and discarding approximately 1 L of water, which was estimated
to be approximately 10 times the dead volume of the reference sample collection value. Then,
from the same valve, approximately 1 L of water was collected in a glass beaker and carried
directly to a technician, who immediately began the reference analyses. With the exception of
TOC, all the analyses were performed within minutes of sample collection. TOC analyses were
performed within the method's required 14-day holding time period. The standard laboratory
methods used for the reference analyses are shown in Table 3-2. Also included in the table are
method detection limits and QC measurement differences. Battelle technical staff collected the
reference samples, and technical staff at the T&E Facility performed the analyses. The T&E
Facility provided calibrated instrumentation, performed all method QA/QC, and provided
calibration records for all instrumentation. The T&E Facility provided reference sample results
upon the analysis of the reference samples (within one day).
3.3.2 Reference Method Quality Control Samples
As shown in Table 3-3, duplicate reference samples were collected and analyzed once daily
during Stages 1 and 2 and weekly during Stage 3. Also, laboratory blanks consisting of
American Society for Testing and Materials Type n deionized (DI) water were analyzed with the
same frequency. Reference analysis of these blank samples were most important for total
chlorine, turbidity, and TOC because they were the only parameters that needed confirmation of
the lack of contamination. For the other parameters, the performance evaluation (PE) audit
confirmed the accuracy of the method and the absence of contamination. Duplicate measure-
ments had to be within the acceptable percent differences provided in Table 3-2. Because the
objective of Stage 4 was to verify the EMTS's ability to trigger when an injection was made and
to identify injected contaminants, no reference measurements were performed during this stage.
-------
Table 3-2. Reference Methods
Parameter
Method
Reference Instruments
Method Detection
Limit
Acceptable
Differences for
QC Measurements
pH
Conductivity
Total
chlorine
TOC
Temperature
Turbidity
EPA 150.1(2)
SM2510(3)
SM 4500-C1-G(4)
EPA415.1(5)
EPA 170.1(6)
EPA 180.1(7)
Corning 320 pH meter
YSI 556 multi-parameter
water monitor
Hach 2400 portable
spectrophotometer
Phoenix 8000 TOC
analyzer
YSI 556 multi-parameter
water monitor
Hach 21 OOP turbidimeter
NA
2 microSiemens per
centimeter (|iS/cm)
0.01 milligram per
liter (mg/L) as C12
0.01 mg/L
NA
0.067 nephelometric
turbidity unit (ntu)
±0.3 pH units
±25 %D
±25 %D
±25 %D
±1°C
±25 %D
NA = Not applicable.
Table 3-3. Reference Analyses and Quality Control Samples
1:
2:
3:
3:
Stage
Accuracy
Response to
injected
contaminants
Extended
deployment
Post-extended
deployment
accuracy
Reference
Sampling Sample
Periods (length) Frequency
One at start, one
9 (4 hours) every hour
thereafter
One pre-
injection;
6 (one injection) one at 3, 15, and
60 minutes post-
injection
1 /c-> A \ Once each
1 (52 days) , ,
3 weekday
1 (4 hours) Same as Stage 1
Reference
Samples per QC Samples per
Period Period
One duplicate and
5 one DI water blank
daily
One duplicate and
4 one DI water blank
daily
One duplicate and
37 one DI water blank
each week
5 Same as Stage 1
Total QC
Samples
18
12
16
2
3: Response to
injected
contaminants
4 (one injection) Same as Stage 2
Same as Stage 2
4: Contaminant
identification
No reference measurements collected
10
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed in accordance with the quality management plan (QMP) for
the AMS Center(8) and the test/QA plan (1) for this verification test.
4.1 Audits
4.1.1 Performance Evaluation Audit
A PE audit was conducted to assess the quality of the reference measurements made in this
verification test. With the exception of temperature, each type of reference measurement was
compared with a National Institute of Standards and Technology (NIST)-traceable standard
reference water sample. The standard reference water samples had certified values of each water
quality parameter that were unknown to the analyst. These samples were analyzed in the same
manner as the rest of the reference analyses to independently confirm the accuracy of the
reference measurements. The temperature PE audit was performed by comparing two
independent thermometer results. As Table 4-1 shows, all PE audit results were within the
acceptable differences provided in Table 3-2. The percent difference (%D) was calculated using
the following equation:
C - C
%D=— - - xlOO%
where CR was the reference method result and CN the NIST value for each respective water
quality parameter (or, for temperature, data from the second thermometer). Other QC data
collected during this verification test were reference method duplicate analysis results, which are
also shown in Table 4-1. With the exception of one duplicate measure of turbidity, all six
parameters were always within the differences defined in Table 3-2. Because pH units are
measured on a logarithmic, rather than linear, scale, and the measurement of temperature is
extremely precise; the quality control metrics for those two parameters were the absolute units
rather than percent difference. No corrective action was taken for the one turbidity measurement
(55.2%) that was outside the acceptable tolerance criteria. If this outlier is removed, the upper
range of percent difference was 18.2%, and the average absolute value of differences was 5.4%.
11
-------
Table 4-1. Performance Evaluation Audit and Reference Method Duplicate Analysis
Results
Parameter
pH (pH unit)
Conductivity (|iS/cm)
Total chlorine (mg/L)
Temperature (°C)
TOC (mg/L)
Turbidity (ntu)
NIST
Standard
Value
9.26
1,920
4.19
23.80(a)
11.8
20.0
PE
Reference
Method
Result
9.18
1,706
3.62
23.80
11.7
22.3
Audit
Difference
-0.08 pH unit
-11.1%
-13.6%
0.00°C
-0.8%
11.5%
Duplicate Analysis
Average of
Absolute Values of
Difference
0.04 pH unit
0.25%
2.62%
0.02°C
1.5%
7.49%
Range of
Difference
0.0 to 0.1 3 pH unit
-1.9 to 0.7%
-7.3 to 2.1%
-0.18to0.29°C
-5. 6 to 11.6%
-8.7 to 55.2%
-------
Table 4-2. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test
events
Calibration
information
(WDMP, EMTS,
and reference
methods)
WDMP and EMTS
results
Where Recorded
ETV data sheets
and testing
notebook
ETV data sheets
and testing
notebook
Recorded
electronically by
How Often
Recorded
Start/end of test and
at each change of a
test parameter
Upon each
calibration
Recorded
continuously
By
Whom
Battelle
and T&E
Facility
Battelle
and T&E
Facility
Battelle
Disposition of
Data
Used to organize/
check test results;
manually
incorporated in
data spreadsheets
as necessary
Manually
incorporated in
data spreadsheets
as necessary
Comma delimited
text files
each unit and then
downloaded to
computer daily
Reference method
procedures
ETV laboratory
record books or
data recording
forms
Throughout sample
analysis process
T&E
Facility
Transferred to
spreadsheets or
laboratory record
book
13
-------
Chapter 5
Statistical Methods
The statistical methods presented in this chapter were used to verify the WDMP units' accuracy,
response to injected contaminants, inter-unit reproducibility, and identification of injected
contaminants for EMTS units.
5.1 Accuracy
Throughout this verification test, results from the WDMP units were compared to the results
obtained from analysis of a grab sample by the reference methods. During Stage 1, the percent
difference (%D) between these two results was calculated using the following equation:
C - C
%D= — xlOO%
^R
where CR is the result determined by the reference method and Cm is the result from the WDMP
unit. The WDMP unit results were recorded every 30 seconds, while collecting the reference
samples took only a few seconds; therefore, Cm was the measurement recorded closest to the
time the reference sample was collected. Water quality stability, as well as the stability of each
sensor, was evaluated during the four-hour time period when reference samples were analyzed
every hour for each of the parameters. Ideally, if the result from a WDMP unit and a reference
method measurement were the same, there would be a percent difference of zero. It should be
noted that the formula for percent difference is sensitive to reference results that are small in
magnitude. For example, if the reference turbidity is 0.1 ntu, and the online instrument reads 0.2,
the percent difference is 100%. Alternatively, if the reference turbidity is 1.0 ntu, and the online
instrument reads 1.1, the percent difference is only 10%. During Stages 2 and 3, the continuous
data, graphed with the reference method results, were visually examined to evaluate the response
of the WDMP unit to the injection of contaminants and their stability over an extended
deployment. During the accuracy and contaminant injection components of Stage 3, the data
were evaluated as they were for Stages 1 and 2, respectively.
5.2 Response to Injected Contaminants
To evaluate the response (i.e., the increase or decrease of water quality parameter measured by
the WDMP units) to contaminant injections, the pre- and post-injection reference samples were
graphed as individual data points, along with the continuous measurements. The reference
14
-------
results showed the effect of each injection on the chemistry of the water in the pipe loop, and the
continuous results from the WDMP units highlighted its response to such changes.
5.3 Inter-unit Reproducibility
The results obtained from two identical WDMP units were compared to assess inter-unit
reproducibility. Each time a reference sample was collected and analyzed (approximately
138 times throughout this verification test), the results from each WDMP unit were compared to
evaluate whether the two units were generating similar results. This was done in two ways. First,
the results from one unit were graphed against the results of the other unit. In this evaluation, a
slope of unity and a coefficient of determination (r2) of 1.0 would indicate ideal inter-unit
reproducibility. Slopes above 1.0 may indicate a high bias from Unit 2 (graphed on the y-axis) or
a low bias for Unit 1 with respect to each other. Similarly, slopes below 1.0 may indicate a low
bias for Unit 2 or a high bias for Unit 1, again with respect to each other. Second, the data from
each unit were included in a paired t-test, with the assumption that the data from each unit had
equal variances. The t-test calculated the probability of obtaining the subject results from the two
units if there was no significant difference between their results. Therefore, probability values
(p-values) of less than 0.05 indicated a significant difference between the two units. In addition,
the results from both units were graphed together for the Stages 2 and 3 results, allowing a visual
comparison.
5.4 Contaminant Identification
During Stage 4 of the test, the ability of the EMTS to detect a contaminant injection was verified
by confirming that a trigger event (caused by the trigger signal's exceedence of a user-specified
threshold) occurred during the period of time that each WDMP unit was exposed to the
contaminant (approximately 20 minutes), as well as evaluating whether the injected contaminant
was identified correctly. The EMTS searches the agent signature library once per minute during
trigger events. During that search, if a match is found between the experimental data and the
agent library data, an agent alarm occurs. Because the experimental injection data may match the
EMTS data from several contaminants, more than one agent alarm can result from a single
search of the agent library, although some may have strong and some may have weak match
angles. That is, in an exposure time of approximately 20 minutes, more than 20 agent alarms can
result. Similarly, if there are no matches, the event is recorded as an "unknown" event. If an
agent alarm is the result of a library search, the EMTS reports the identified agent, as well as a
match angle, which is a unitless measure of the quality of the match between the experimental
data and the library data. Lower match angles correspond to better match quality and higher
match angles correspond to poorer match quality. While the detailed approach to obtaining the
match angle is proprietary, and the practice of determining the quality of a match is somewhat
subjective; in general, Hach suggests that match angles greater than 7 should be considered
questionable or weak matches; angles between 4 and 7 moderately good matches; and angles less
than 4 good matches. Overall, Hach suggests that the best way to interpret EMTS identification
data in a controlled experimental situation such as this ETV test is to evaluate
• Whether the injection or contamination event triggered an agent alarm
• How many times the correct contaminant was identified by the EMTS
15
-------
• What the quality of the match was, as determined from the match angle criteria described
above
• How many and how frequently other contaminants were found during the injection
• What the quality of the match was for those contaminants, again using the match angle
criteria.
In Chapter 6, the contaminant identification data are presented in a format that allows the
interpretation of the ETV data in this way. Each agent alarm is accounted for (whether correct or
not) in a column specifying the quality of the match.
Another approach to evaluating the identification data that may be easier to interpret, but that is
less rigorous because the match angles are not accounted for, is also presented in Chapter 6.
Using this approach, the fraction of agent alarms attributable to the injected contaminant (correct
identifications) was called the classification rate (CR) and was calculated using the following
equation:
Nr
CR=—xlOQ%
NT
where Nc is the number of correct identifications of the injected contaminant during each
injection period, and NT is the total number of contaminant identifications (no matter which
contaminant was identified) that occurred during an injection event. For example, if, upon
injection of aldicarb, the EMTS identified 20 contaminants, and 19 of them were correct, the
classification rate for aldicarb would be 95%. Alternatively, a correct identification for 1 out of
20 identifications would correspond to a classification rate of 5%.
To present the data in a more concise way, classification rates of greater than 70% for the
injected contaminant were assigned a Level 5; classification rates between 31% and 70%, a
Level 4; classification rates between 1% and 30%, a 3; classification rates of 0% with other
contaminants identified, a 2; and no detection of injection a 1.
16
-------
Chapter 6
Test Results
As mentioned previously, this verification test was conducted in four stages that focused on four
different aspects of multi-parameter water monitors for distribution systems. The four stages are
summarized in Table 6-1. The first stage consisted of an evaluation (with varied pHs and
temperatures) of the accuracy of each WDMP sensor: total chlorine, turbidity, temperature,
conductivity, pH, and TOC. The second stage of the verification test consisted of an evaluation
of the response of the WDMP units to the injection of several contaminants into the pipe loop.
The third stage consisted of deploying the WDMP units for 52 consecutive days with minimal
intervention for maintenance. In addition, contaminant injections were performed at the close of
Stage 3 to confirm that the WDMP units were still responsive to contaminant injection after the
extended deployment. The fourth stage evaluated the ability of the EMTS to detect the injection
of a contaminant, as well as to identify the contaminant. Throughout all stages of the test, two
WDMP units were tested to evaluate inter-unit reproducibility. In addition, required maintenance
and operational characteristics were documented throughout the verification test. This chapter
provides the results of the testing stages, the inter-unit reproducibility data, and ease of use
information.
Table 6-1. Summary of Test Stages and Type of Data Presentation
Stage Summary
Data Presentation
1 Accuracy when pH and temperature
were varied
Table of percent differences between WDMP
units and reference measurements
Response to contaminant injection
Graphs of WDMP unit measurements and
reference measurements, table showing the effect
of injections on both reference and WDMP
measurements
Extended deployment with minimal
maintenance along with post-extended
deployment accuracy and response to
contaminant injections
Graphs of WDMP unit measurements with
reference measurements, table showing average
percent differences throughout extended deploy-
ment, table showing the effect of injections on
both reference and WDMP measurements
Injection detection and contaminant
identification
Table of EMTS contaminant identifications that,
for each injection, gives the number of agent
alarms and associated match quality attributable
to each possible contaminant; table of
classification rate levels
17
-------
6.1 Accuracy
Tables 6-2a-f list the data from the accuracy evaluation performed during the first stage of the
verification test. During four-hour periods, the water quality conditions were held stable, and
reference samples were collected and analyzed five times, once at the start of the designated test
period and four times at one-hour increments thereafter. In evaluating accuracy in each four-hour
period, measurements from each reference sample were compared with the WDMP unit
measurement taken closest to the time of the reference sample collection and analysis. For each
unit, this approach resulted in five paired WDMP and reference results for each of the nine sets of
water conditions used to simulate pH and temperature variations at a water utility. The average
and standard deviations of these five results are shown in the tables below, as well as the percent
difference between the average results from both WDMP units and the average of the reference
results.
Table 6-2a. Accuracy Evaluation Under Various Conditions—Total Chlorine
Unitl
Set
Conditions
Reference
Average (SD) Average (SD)
[mg/L] [mg/L] % D
Unit 2
Average (SD)
[mg/L] % D
1 ambient pH,
ambient temperature
2 decreased pH, ambient
temperature
3 decreased pH, ambient
temperature
4 decreased pH, ambient
temperature
5 ambient pH, decreased
temperature
6 decreased pH, decreased
temperature
7 ambient pH, increased
temperature
8 decreased pH, increased
temperature
9 ambient pH, ambient
temperature
0.97(0.07) 0.96(0.01) -1.0
0.86 (0.02) 0.82 (0.01) -4.7
0.73 (0.01) 0.49 (0.09) -32.9
0.38 (0.03) 0.20 (0.06) -47.4
0.51(0.08) 0.50(0.01) -2.0
1.57(0.06) 1.63(0.07) 3.8
0.69 (0.01) 0.64 (0.01) -7.2
0.65 (0.07) 0.60 (0.05) -7.7
0.98 (0.02) 0.95 (0.03) -3.1
0.99 (0.00) 2.1
0.84 (0.02) -2.3
0.49 (0.09)
0.32 (0.01)
0.50 (0.01)
1.64(0.07)
0.65 (0.01)
0.60 (0.05)
0.96 (0.03)
-32.9
-15.8
-2.0
4.5
-7.7
-2.0
18
-------
Table 6-2b. Accuracy Evaluation Under Various Conditions—Turbidity
Set
1
2
3
4
5
6
7
8
9
Reference
Unitl
Average (SD) Average (SD)
Conditions [ntu] [ntu]
ambient pH,
ambient temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
ambient pH, decreased
temperature
decreased pH,
decreased temperature
ambient pH, increased
temperature
decreased pH, increased
temperature
ambient pH, ambient
temperature
1.27 (0.95)
1.14(0.40)
0.97 (0.33)
1.54(0.20)
0.89 (0.41)
0.99 (0.21)
0.92 (0.16)
1.00(0.35)
0.46(0.11)
0.59 (0.04)
0.98 (0.48)
0.69(0.11)
1.37(0.11)
0.45 (0.02)
0.48 (0.09)
0.44 (0.03)
0.69 (0.00)
0.27 (0.02)
Unit 2
Average (SD)
% D [ntu]
-53.5
-14.0
-28.9
-11.0
-49.4
-51.5
-52.2
-31.0
-41.3
0.63 (0.04)
0.79 (0.07)
0.68 (0.09)
1.52(0.52)
0.41 (0.03)
0.68 (0.01)
0.58 (0.01)
0.74 (0.00)
0.29 (0.02)
%D
-50.4
-30.7
-29.9
-1.3
-53.9
-31.3
-37.0
-26.0
-37.0
19
-------
Table 6-2c. Accuracy Evaluation Under Various Conditions—Temperature
Reference
Unit 1 Unit 2
Average (SD) Average (SD) Average (SD)
Set Conditions [°C] [°C] %D [°C] %D
1 ambient pH, 22.66(0.33) 22.74(0.38) 0.4 24.89(1.11) 9.8
ambient temperature
2 decreased pH, ambient 22.73(0.23) 22.55(0.16) -0.8 24.42(0.80) 7.4
temperature
3 decreased pH, ambient 21.66(0.08) 21.46(0.18) -0.9 21.44(0.17) -1.0
temperature
4 decreased pH, ambient 21.93(0.15) 21.28(0.13) -3.0 21.39(0.30) -2.5
temperature
ambient pH, decreased 13.82(0.44) 14.33(0.20) 3.7 19.53(0.36) 41.3
temperature
decreased pH, 12.63(0.26) 13.91(0.68) 10.1 18.22(0.99) 44.3
decreased temperature
ambient pH, increased 26.60(0.27) 26.35(0.38) -0.9 26.58(0.78) -0.1
temperature
8 decreased pH, increased 26.69(0.23) 26.58(0.38) -0.4 26.58(0.087) -0.4
temperature
9 ambient pH, ambient 22.79(0.21) 22.72(0.38) -0.3 23.97 (0.58) 5.2
temperature
20
-------
Table 6-2d. Accuracy Evaluation Under Various Conditions—Conductivity
Set
1
2
3
4
5
6
7
8
9
Conditions
ambient pH,
ambient temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
ambient pH, decreased
temperature
decreased pH,
decreased temperature
ambient pH, increased
temperature
decreased pH, increased
temperature
ambient pH, ambient
temperature
Reference
Average (SD)
(|j,S/cm)
451(1)
484 (10)
503 (6)
694 (12)
412(1)
501 (10)
447 (1)
529 (2)
442 (1)
Unitl
Average (SD)
[[iS/cm]
474 (3)
511 (12)
540 (8)
742 (13)
421 (2)
512(10)
483 (3)
571 (6)
469 (1)
%D
5.1
5.6
7.4
6.9
2.2
2.2
8.1
7.9
6.1
Unit 2
Average (SD)
([iS/cm)
439 (5)
409 (26)
540 (8)
693(11)
383 (3)
461 (9)
454 (5)
538 (8)
438 (3)
%D
-2.7
-15.5
7.4
-0.1
-7.0
-8.0
1.6
1.7
-0.9
21
-------
Table 6-2e. Accuracy Evaluation Under Various Conditions—pH
Reference
Unit 1 Unit 2
Average (SD) Average (SD) Average (SD)
Set Conditions [pH unit] [pH unit] % D [pH unit] % D
1 ambient pH, 8.76(0.02) 8.85(0.01) 1.0 9.03(0.03) 3.1
ambient temperature
2 decreased pH, ambient 7.89(0.09) 7.87(0.16) -0.3 7.37(0.70) -6.6
temperature
3 decreased pH, ambient 7.52(0.04) 7.33(0.04) -2.5 7.34(0.05) -2.4
temperature
4 decreased pH, ambient 6.73(0.12) 6.37(0.06) -5.3 6.42(0.07) -4.6
temperature
5 ambient pH, decreased 8.48(0.02) 8.55(0.02) 0.8 8.57(0.01) 1.1
temperature
6 decreased pH, decreased 7.31(0.08) 7.15(0.08) -2.2 7.18(0.08) -1.8
temperature
7 ambient pH, increased 8.37(0.05) 8.25(0.02) -1.4 8.32(0.02) -0.6
temperature
8 decreased pH, increased 7.60(0.06) 7.25(0.03) -4.6 7.29(0.02) -4.1
temperature
9 ambient pH, ambient 8.74(0.01) 8.60(0.01) -1.6 8.63(0.01) -1.3
temperature
22
-------
Table 6-2f. Accuracy Evaluation Under Various Conditions—Total Organic Carbon
Set
1
2
3
4
5
6
7
8
9
Reference
Unitl
Unit 2
Average (SD) Average (SD) Average (SD)
Conditions [mg/L] [mg/L] %D [mg/L]
ambient pH,
ambient temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
ambient pH, decreased
temperature
decreased pH, decreased
temperature
ambient pH, increased
temperature
decreased pH, increased
temperature
ambient pH, ambient
temperature
0.59 (0.03)
0.56 (0.05)
0.43 (0.25)
0.51 (0.02)
1.20(0.40)
0.48 (0.02)
0.57 (0.02)
0.54 (0.01)
0.55 (0.12)
0.87 (0.29) 47.5
0.36 (0.04) -35.7
0.22 (0.03) -48.8
0.18(0.11) -64.7
1.24(0.09) 3.3
0.18(0.08) -62.5
0.23 (0.08) -59.6
0.25 (0.09) -53.7
(a) (a)
1.46(0.05)
0.34 (0.06)
0.51 (0.04)
0.44 (0.04)
1.20(0.44)
0.70 (0.41)
0.48 (0.06)
0.51 (0.06)
0.53 (0.01)
%D
147.5
-39.3
18.6
-13.7
0.0
45.8
-15.8
-5.6
-3.6
(a)
Ongoing instrument maintenance resulted in a zero reading.
As can be seen in Tables 6-2a-f, for total chlorine, the percent differences (with the median
shown in parentheses) ranged from -47.4 to 4.5 (-3.9); for turbidity, -53.9 to -1.3 (-34.1); for
temperature, -3.0 to 44.3 (-0.2); for conductivity, -15.5 to 8.1 (2.2); for pH, -6.6 to 3.1 (0.9); and
for TOC, -64.7 to 147.5 (-14.8).l Across all of the water quality parameters that were measured,
the pH measurements had the smallest range of percent differences. Total chlorine, temperature,
and conductivity also had relatively small ranges, with the exception of one or two sets of
conditions. For total chlorine, with the exception of Sets 3 and 4, the range of percent differences
was -7.7 to 5.1. For temperature, if Unit 2's Sets 5 and 6 (the decreased temperature sets) were
not considered, the range of percent differences was -3.0 to 10.1; and, for conductivity, the only
percent difference greater than 10 was for Unit 2 during Set 2 (at -15.5%). For temperature, an
experimental design factor seemed to affect the accuracy of the measurements. Because of the
space requirement, the WDMP units were plumbed to the pipe loop from one floor below, with a
PVC tube (1/2-inch outside diameter) that was approximately 18 feet long. During the sets with
decreased temperature, by the time the water reached the WDMP units, the temperature had
begun equilibrating with the ambient air temperature, increasing or contributing to the positive
Throughout this report, median values are provided when a range of values is presented. The median of a set of
positive and negative numbers provides a good indicator of the overall direction of the percent differences in the
data set (i.e., whether most values were positive or negative). The disadvantage is that, unless the signs of all the
data are the same, information about the magnitude of change is not available from the median. In summary, the
medians in this report provide the direction, not magnitude, of difference information.
23
-------
percent differences. The temperature percent differences from the other sets of test conditions
were not as greatly affected because the ambient and increased water temperatures did not differ
as much as the ambient and decreased water temperatures did.
The turbidity measurements were consistently low with respect to the reference measurement.
As shown in Table 4-1, the comparison of the reference method to the NIST standard indicated
that the reference method was biased high. When combined with the Stage 1 results shown in
Table 6-2b, this may suggest that the turbidity reference result was typically higher than the
actual turbidity in the water. Note that, because of the relatively low turbidity in the Cincinnati
water, small absolute differences between the reference samples and the WDMP unit's
measurements increased the value of the percent differences between the two numbers. For
TOC, the results were somewhat inconsistent between the two units throughout this stage of the
verification test. The Unit 1 TOC analyzer was below the reference result by at least 35% in
eight out of the nine sets. However, the Unit 2 TOC results matched the reference results more
consistently, with six out of the nine results falling within 20% of the reference result.
The standard deviations of the reference and continuous measurements collected during each test
period were, with few exceptions, very small with respect to the average result. In only a few
instances was the standard deviation greater than 10% of the average result. The only exceptions
to this were for turbidity and TOC, which were not controlled as part of the verification test, but
were dependent on events occurring in the Cincinnati water utility. Also, small changes in
reference and continuous measurements corresponded to rather large relative changes because of
the low turbidity and low concentration of TOC in the Cincinnati water. Overall, the low
standard deviations show that the water conditions during the test periods were very stable and
that there was little variability in the measurements.
6.2 Response to Injected Contaminants
Six injections of contaminants were performed during the second stage of this verification test;
i.e., duplicate injections of nicotine, arsenic trioxide, and aldicarb. Table 6-3 shows the
directional change of each reference and WDMP measurement in response to the contaminant
injections. In general, total chlorine was the only parameter clearly affected (for both the
reference and continuous measurements) by all six injections. Figures 6-1 through 6-5 show the
water quality parameters for which there was a response. The blue and yellow lines on the
graphs represent the measurements made by each WDMP unit, and the magenta data points
represent the results from the laboratory reference method. Because accuracy was the focus of
the first stage of verification testing, percent differences between the WDMP units and the
reference method results are not presented here; however, the reference method results are
included in these figures to confirm that the fluctuations in the continuous results are due to
changes in water chemistry as the result of the injected contaminants. The figures are divided
with vertical lines that define the approximate time interval that the sensor measurement was
affected by the injection. Normally, the sensors were allowed to return to baseline overnight
after the injections. Therefore, each injection time period defined on the figures is approximately
24 hours. The times vary somewhat, however, depending on when chlorine was added to restore
the system to pre-injection conditions. The contaminant that was injected and whether it
24
-------
Table 6-3. Effect of Contaminant Injections Prior to Extended Deployment
Nicotine
Parameter
Total chlorine
Turbidity
Temperature
Conductivity
pH
TOC
Reference
_
(a)
NC
NC
NC
+
WDMP
_
+
NC
NC
NC
+
Arsenic Trioxide
Reference WDMP
_
(a)
NC NC
+ +
+ +
NC NC
Aldicarb
Reference
_
(a)
NC
NC
NC
+
WDMP
_
+
NC
NC
NC
+
"" Relatively large uncertainty in the reference measurements made it difficult to detect a significant change.
+/- = Parameter measurement increased/decreased upon injection.
NC = No change in response to the contaminant injection.
O)
E
1.4
1.2
1
0.8
0.6
0.4
0.2
0 J
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-1. Stage 2 Contaminant Injection Results for Total Chlorine
— UniM
• Reference
Unit 2
25
-------
Nicotine 2
Arsenic )
Arsenic 2
Aldicarb
Aldicarb 2
— Unitl
• Reference
Unit 2
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-2. Stage 2 Contaminant Injection Results for TOC
1
Nicotine 1 4 Nicotine 1
0.9 -
0.8
0.7
10.6 ^
\rsenic 1
Arsenic 2
ent#1
Ale
carbl
Aldicarb 2
Event #2
— Unitl
• Reference
Unit 2
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-3. Stage 2 Contaminant Injection Results for Turbidity
26
-------
Z>.*J
9 -
8.5 -
8 -
I -7 1-
^7.5 -
7 -
6.5 -
6 -
C C -
Nicotine 1
V
1
— —
Nicotine 2
n
•
Arsenic 1
J
*" V»
\
Arseni c 2
i
L
K<-
t#i
Aldicarb 1
*-_— . t
^d^^^*'*^^^
Aldicarb
• *
*
2
— Unitl
» Reference
Unit 2
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-4. Stage 2 Contaminant Injection Results for pH
Nicotine 1
Nicotine 2
Arsenic 1
Arsenic 2
Aldicarb 1
Aldicarb
700
650 -
600 -\
I
v 550 H
E
o 500 -
450 H
400
350 J
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-5. Stage 2 Contaminant Injection Results for Conductivity
Event #1
— Unitl
* Reference
Unit 2
27
-------
was the first or second replicate are shown at the top of each section of the figures. For each
injection, at least four reference sample results were collected, and are included in these figures.
The first occurred within approximately one hour prior to contaminant injection during a period
of stable water quality conditions. The next three reference data points were from samples
collected 3, 15, and 60 minutes after contaminant injection. For some of the injections, another
reference sample was collected the following day to show that the pipe loop system had
recovered or was in the process of recovering after the injection. This final reference data point
also served as the first reference sample collected for some of the injections, representing the
stable baseline just prior to injection. Following the first aldicarb injection, it appears as if
Unit 2's data may be missing. At this time, Unit 2 reverted unexpectedly to recording data every
hour rather than every 30 seconds, as was done for the rest of the injections. There was no
explanation for why this happened, but it was the only occurrence throughout the verification
test. The hourly measurements are visible on some figures, but are overwritten by the Unit 1
measurements on some others.
Figure 6-1 shows how the measurement of total chlorine was affected by the contaminant
injections. Prior to the injections, the total chlorine level was maintained between approximately
1 and 1.2 mg/L, as is evidenced by the magenta data points near the start of each section of the
figure. In each case, within one hour of contaminant injection, the total chlorine level, as
measured by the laboratory reference method, dropped to approximately 0.5 mg/L for nicotine
(total chlorine Event #1 in Figure 6-1) and near zero for the other two contaminants (for an
example, see total chlorine Event #2). Upon injection, the vertical drop in the line representing
the total chlorine concentration made it clear that the chlorine sensor on the WDMP units were
responsive to the drop in total chlorine levels. For each injection, the drop in total chlorine levels
was followed by the restoration of the pipe loop system to approximately pre-injection
conditions through the addition of sodium hypochlorite. This is shown in Figure 6-1 by the
rapidly increasing total chlorine concentration after the sensor reaches a low point in total
chlorine concentration (for an example, see total chlorine Event #3).
As shown in Figure 6-2, the TOC measurement responded significantly to nicotine (TOC
Event #1 in Figure 6-2) and aldicarb (TOC Event #2), the two organic contaminants. Arsenic
trioxide is an inorganic compound and, as expected, did not increase the TOC concentration in
the pipe loop water (TOC Event #3). The WDMP TOC measurements tracked the reference
results very well for each of the nicotine and aldicarb injections.
Figure 6-3 shows the turbidity measurements during the contaminant injections. As for the other
water parameters, the reference samples were collected prior to the contaminant injection and at
3, 15, and 60 minutes following the contaminant injection. Therefore, each cluster of magenta
symbols on the figure indicate when a contaminant injection had occurred. However, for all the
injections except the second arsenic injection, the level of turbidity measured by the reference
method decreased from the time the pre-injection reference sample was collected until the
subsequent reference samples were collected and analyzed. This suggests that 1) the contaminant
injections did not increase the turbidity in the flowing water or, 2) that the uncertainty in the
reference measurements was too large to determine whether turbidity was significantly affected.
Because the continuous turbidity measurement of the WDMP seemed to increase at least slightly
(through a visual inspection of the data) with each injection, it seems that the latter scenario is
more likely to be the case. For the nicotine injections, the change in turbidity according to the
WDMP measurements was very small, while the changes during the arsenic and aldicarb
28
-------
injections were more pronounced. Because each of these contaminants was dissolved in water
prior to injection, these observed increases could have been due to changes in the optical
properties of the water resulting from the dissolution of the contaminant or the co-injection of
small amounts of air, introducing a few bubbles into the pipe, thus causing an increase in
turbidity. Note that just prior to the arsenic injection (turbidity Event #1 in Figure 6-3) and just
prior to the final aldicarb injection (turbidity Event #2), there were turbidity spikes that were not
due to the injection of any contaminants. Apparently, some outside perturbations in the water
system caused these brief events.
For both pH and conductivity, there was a small increase measured by both the reference and
continuous measurements during the injection of arsenic trioxide only (see pH and conductivity
Events #1 in Figures 6-4 and 6-5, respectively). For both water quality parameters, the increase
may have been due to the pH adjustment required to get this contaminant into solution.
6.3 Extended Deployment
Figures 6-6 through 6-11 show the continuous measurements from both WDMP units during the
52-day extended deployment stage of the verification test. Those measurements are represented
by the blue and yellow lines, while the results of the reference samples, collected once daily
throughout this deployment, are represented by the magenta symbols. The x-axis on each figure
represents the period of time between September 1, 2004, and October 22, 2004, and the y-axis
gives the results of each water quality measurement. Data points were recorded every 30 seconds
during the verification test; but, for the extended deployment figures, only data points collected
approximately every 2 minutes were depicted. This was done so that a standard spreadsheet
could be used to generate these figures. This approach was inconsequential to interpreting the
figures.
The objective of this stage of the verification test was to evaluate the performance of the WDMP
units over an extended period of time with minimal intervention to simulate a situation in which
the units may be deployed at a remote location. The continuous trace was evaluated visually to
see whether any aspects of the data were noteworthy. A second, more quantitative, evaluation
was then performed to get an indication of the accuracy of the extended deployment measure-
ments. This evaluation, much like the accuracy evaluation conducted during the first stage of
testing, included calculating the percent differences between the average continuous measure-
ments and average reference sample results throughout the extended deployment, as well as the
standard deviation of each of those measurements. The standard deviation of the results
indicated the stability of the water conditions during Stage 3, as well as how the standard
deviations of the continuous measurements differed from the standard deviations of the reference
measurements. Similar standard deviations between the continuous and reference measurements
indicate that the variability was mostly dependent on the water conditions and not due to
systematic variability in the WDMP unit results. (Note that reference results were only generated
during business hours, so any fluctuations occurring during off hours are not reflected in the
standard deviations of the reference results. Because of this, total chlorine, a parameter that
varied at times during weekends when the supply of chlorine ran low, might have been expected
to have a larger variability than other more stable parameters.) Table 6-4 lists the percent
differences, along with the average and standard deviations of the reference and continuous
29
-------
D>
E
— Unitl
* Reference
Unit 2
Duration of Stage 3: 52 days
Figure 6-6. Extended Deployment Results for Total Chlorine
— Unitl
• Reference
Unit 2
Duration of Stage 3: 52 days
Figure 6-7. Extended Deployment Results for TOC
30
-------
10 n
9 -
8 -
7 -
6 -
5
4
3 H
2
1
0
— UniM
• Reference
Unit 2
Event#2
L L/, .,•• \rnjw*.
JV—Hi*,._j.^ ^ ** .. iy»v->#jw—J.
Duration of Stage 3: 52 days
Figure 6-8. Extended Deployment Results for Turbidity
8.9
8.8 -
8.7 -
8.6 -
8.5 -
8.4
8.3
8.2
— UniM
Reference
Unit 2
Duration of Stage 3: 52 days
Figure 6-9. Extended Deployment Results for pH
31
-------
—Unit1
' Reference
Unit 2
Duration of Stage 3: 52 clays
Figure 6-10. Extended Deployment Results for Temperature
600
500 -
£ 400
a:
300 -
CO
o
I 200
100 -
0
.***»
— Unitl
Reference
Unit 2
Duration of Stage 3: 52 days
Figure 6-11. Extended Deployment Results for Conductivity
32
-------
Table 6-4. Accuracy During Extended Deployment
Parameter
Total chlorine
Turbidity00
Temperature
Conductivity
pH
TOC
Reference
Average
(SD)(a)
1.05 (0.10)
0.38 (0.26)
22.83 (0.34)
334 (55)
8.72 (0.07)
0.79 (0.17)
Unitl
Average
(SD)(a)
1.02(0.09)
0.35 (0.31)
22.57 (0.38)
357 (57)
8.56 (0.07)
0.62 (0.09)
%D
-2.9
-7.9
-1.1
6.9
-1.8
-21.5
Unit 2
Average
(SD)(a)
1.03(0.10)
0.24(0.17)
23.50 (0.82)
335 (54)
8.70 (0.07)
0.89(0.19)
%D
-1.9
-36.8
2.9
0.3
-0.2
12.7
Both Units %D
Range (median)
-15.9 to 6.9 (-3.2)
-8 1.1 to 245. 5 (-2 1.3)
-7 .4 to 8.5 (-0.1)
-1.8 to 9.6 (4.8)
-2.7 to 0.5 (-0.9)
-47.3 to 103.0 (-6.9)
-------
The TOC measurements during the extended deployment are shown in Figure 6-7. Notable
aspects of the data include a sharp increase in the Unit 1 concentration measurement near the
start of this stage (TOC Event #1 in Figure 6-7), two spikes in the Unit 2 measurements nearly
halfway through this stage (TOC Event #2), and a convergence of both TOC analyzer
measurements near the end of the extended deployment (TOC Event #3). The increase in the
Unit 1 measurement occurred when the liquid nitrogen supply ran out, thus causing the result to
be measured as total carbon (which would be expected to be higher) rather than TOC. The first
Unit 2 spike during Event #2, which actually overlays an identical spike in the Unit 1 results that
is not visible in Figure 6-7, corresponds to stopping the flow to perform maintenance activities
on both units. A day later, the second Unit 2 spike corresponds to the calibration of both units.
However, there was not a signal spike in the Unit 1 results. Finally, the convergence of the
results from both WDMP units near the end of this stage indicates another calibration of both
units. Overall, the Unit 1 measurements tracked the reference measurements more accurately
prior to the first calibration, and the Unit 2 results did so after the calibration. This is also
observed in the statistical evaluation of the data. Prior to the first calibration, the average percent
difference for Unit 1 was 6.6 ± 18.7 and for Unit 2, 55.9 ± 27.6. After the first calibration, the
percent differences were -30.6 ± 8.0 and -6.2 ±9.1 for Units 1 and 2, respectively.
The turbidity measurements are shown in Figure 6-8. Throughout the extended deployment, the
baseline turbidity was less than 0.5 ntu most of the time except for several high turbidity events
that were not anticipated. Some of the smaller turbidity events seemed to correspond with
collection of the reference samples. The perturbation of water in the pipe may have caused brief
increases in turbidity. The one rather large turbidity event shown by the large peak near the
middle of the extended deployment (turbidity Event #1 in Figure 6-8) was not caused by
reference sampling or any other known event. However, it was clearly measured by both the
WDMPs and the reference method. Just after the large peak there is a spike in the signal of both
units, which occurred when flow was stopped to perform maintenance on the turbidity meter.
Because of the large difference between the data from the high turbidity event and the rest of the
test, these data were removed from the calculation of average and standard deviation. During the
second half of the extended deployment, Unit 1 generated results that were biased high with
respect to the Unit 2 and reference results and that, at times, displayed a high degree of
variability (turbidity Event #2). This persisted for approximately two weeks. The end of that
period of variable results from Unit 1 did not seem to correlate with any maintenance or
calibration performed on the WDMPs, but the signal stabilized for the remainder of the stage.
Overall, during the first half of the extended deployment, the WDMP turbidity measurements
were biased low with respect to the reference measurements. However, the reference measure-
ments during that time were higher than during the second half of the extended deployment
when the reference measurements were more in alignment with the WDMP measurements.
Because neither the water source nor the baseline results from the WDMP changed considerably
between the first and second halves of the extended deployment, it seems possible that the
reference method measurements were biased high during the first half. During the first half, the
average differences for Units 1 and 2 were 55% ± 23% and 59% ±21%, respectively. During the
second half, the difference for Unit 1 (excluding the high variability results) was -6% ±12% and
for Unit 2,-6% ±9%.
The pH, temperature, and conductivity measurements are shown in Figures 6-9 through 6-11.
Each of these sensors operated without intervention throughout this stage of the verification test.
34
-------
Each sensor tracked the sensor from the other WDMP unit rather well. However, in each case,
one unit's measurements more closely matched those generated by the reference method. For pH
and conductivity, Unit 2 was more accurate with respect to the reference result, while, for
temperature, Unit 1 was more accurate. The temperature measurements from both Units 1 and 2
showed regular variability because the test was conducted in a facility where the water
temperature was heavily affected by the outdoor temperature; therefore, the water temperature
changed as a function of the high and low temperature for the day. However, it is not apparent
why the Unit 2 measurements seem to have a wider range of daily temperature variation than
Unit 1. With the exception of the Unit 2 temperature measurements, the standard deviations for
each sensor across the entire extended deployment were either very similar or less than the
standard deviations for the reference method. This indicates that the variability observed in most
of the measurements was actually due to variability in the measured water quality parameters
rather than any systematic error in the sensors.
6.4 Accuracy and Response to Injected Contaminants After Extended Deployment
After the 52-day deployment of the WDMP units with minimal intervention, their performance
was evaluated during a 4-hour period of ambient pH and temperature during which reference
samples were collected hourly. The results of this evaluation are given in Table 6-5. The percent
differences between the WDMP units and the reference measurements during this post-extended
deployment accuracy evaluation for total chlorine, temperature, conductivity, and pH were, for
the most part, similar in magnitude to measurements during the Stage 1 accuracy evaluation. For
turbidity and TOC, however, the percent differences were considerably smaller than in Stage 1.
The WDMP turbidity measurements during Stage 1 were generally between 30% and 80%
below the reference method measurements while, during this stage, the percent differences were
approximately -6% and -12% for Units 1 and 2, respectively. Similarly, the TOC measurements
during Stage 1 varied widely between sets of conditions. The Unit 1 percent differences ranged
from -65% to 47% and Unit 2 from -39% to 148%. These results, and the extended deployment
results, both with average percent differences of less than 21% and relatively small standard
deviations, indicate that the TOC measurements were more accurate at the close of the
verification test than at the start. Apparently, calibration of the TOC analyzers during the
extended deployment stage was the reason for these improved results. It was not apparent what
caused the improved agreement between the reference and continuous measurements.
A second evaluation of the response to injected contaminants after the extended deployment
used four contaminants. Two were a repeat of the aldicarb injections performed during Stage 2
and two were injections of E. coli, which was not available for injection during the earlier stage
of the test. Table 6-6 gives the directional change of each reference and WDMP measurement in
response to the contaminant injections. In general, total chlorine and TOC were the two
parameters that were most obviously affected (through visible inspection of the data) by all four
injections. This is shown in Figures 6-12 and 6-13 (see total chlorine and TOC Events #1
through 4). In addition, the duplicate injections generated very similar changes in each of the
parameters. Turbidity, pH, and conductivity, shown in Figures 6-14 through 6-16, also were
affected by some or all of these injections, but the magnitude and consistency of change was not
as obvious as for total chlorine and TOC.
35
-------
Table 6-5. Post-Extended Deployment Results
Parameter
Total chlorine
Turbidity
Temperature
Conductivity
pH
TOC
Reference
Average (SD)
1.03 (0.03)
0.17 (0.02)
22.66 (0.16)
356(1)
8.59 (0.01)
0.88 (0.01)
Unitl
Average
(SD)
0.98 (0.02)
0.16(0.03)
22.61 (0.03)
380(1)
8.40 (0.01)
0.70 (0.01)
%D
-4.9
-5.9
-0.2
6.7
-2.2
-20.5
Unit 2
Average
(SD)
0.98 (0.02)
0.15(0.04)
23.70 (0.06)
357 (1)
8.61 (0.00)
0.91 (0.01)
%D
-4.9
-11.8
4.6
0.3
0.2
3.4
-------
£.
1.8 -
1.6 -
1.4
"oT
= 12
o
0 1 "
"ra
o 0.8-
0.6 -
0.4 -
E. co// 1
^
*>*
•
ly'J
+/
/'
0.2 -]
n I
E. co// 2
/]
J fc
V*
I &
w
I/
I
IH^B
•1
Aldicarb 3
*
i
€'
/'-""^
'
^
Aldicarb 4
— Unitl
i^~^~ — " * Reference
\!
1
I
I
-j—
Unit 2
w
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-12. Stage 3 Contaminant Injection Results for Total Chlorine
— Unitl
* Reference
Unit2
Each section (separated by vertical lines; represents approximately 24 hours.
Figure 6-13. Stage 3 Contaminant Injection Results for TOC
37
-------
— Unitl
* Reference
Unit 2
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-14. Stage 3 Contaminant Injection Results for Turbidity
— Unitl
* Reference
Unit 2
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-15. Stage 3 Contaminant Injection Results for pH
38
-------
.
01
500
480
460-
440-
420 -
400
55
S 380 i
E 360-
340-
320-
300
E CDltl 1
E. coli 2
Aldicarb 3
Aldicart 4
Nor>€TVEco/7ir1ecUon
— Unitl
* Reference
Unit 2
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-16. Stage 3 Contaminant Injection Results for Conductivity
The reference method indicated a decrease in pH corresponding to the injection of E. coli and
aldicarb. No change in pH was observed during the Stage 2 injections of aldicarb so the Stage 3
response was unexpected. The WDMP units both detected the change in pH due to the E. coli
injections (pH Event #1 in Figure 6-15), while the change during the aldicarb injections was
detected only by a small inflection of the pH signal (pH Event #2) rather than an obvious peak in
the negative direction, as was observed for the E. coli injections. Finally, conductivity increased
slightly in response to the injection of E. coli. Both the reference method and the WDMP units
detected this change during the first E. coli injection (conductivity Event #1 in Figure 6-16); but,
during the second injection, WDMP Unit 2 generated a zero reading because some air bubbles
had become trapped inside the conductivity sensor (conductivity Event #2). Unit 1 did detect the
change (conductivity Event #3). Note that the second peak (just after the E. coli injection) in the
TOC, turbidity, pH, and conductivity data (labeled non-ETV E. coli injection) reflects a second
injection of E. coli performed that day by the T&E facility staff that was not part of the ETV test.
6.5 Inter-unit Reproducibility
Two WDMP units were compared throughout the verification test to determine whether they
generated results that were similar to one another. This was done using the data collected
whenever a reference sample was collected throughout the verification test. Two evaluations
were performed to make this comparison. First, the results from each sensor from Unit 2 were
graphed on the y-axis, those from Unit 1 were graphed on the x-axis, and a linear regression line
was fitted to the data. For the linear regression analysis, if both units reported the identical result,
the slope of such a regression would be unity, the intercept zero (0), and the coefficient of
determination (r2) 1.0. The slope can indicate whether the results are biased in one direction or
39
-------
the other, while the coefficient of determination provides a measure of the variability of the
results. Second, a t-test assuming equal variances was performed on those same data. The t-test
shows whether the sensors generated statistically similar data. Small p-values (<0.05 at a 5%
confidence level) would suggest that the results from the two units are significantly different
from one another. Table 6-7 gives the slope, intercept, and coefficient of determination for the
inter-unit reproducibility evaluation and the p-value for the t-test performed for each sensor.
Table 6-7. Inter-unit Reproducibility Evaluation
Parameter
Total chlorine
Turbidity
Turbidity (outlier removed)
Temperature
Conductivity
pH
TOC
Slope
0.98
0.77
0.97
0.72
0.92
1.06
0.97
Intercept
0.03
0.09
0.005
7.68
4.19
-0.40
0.31
r2
0.994
0.696
0.881
0.758
0.961
0.919
0.991
t-test p-value
0.779
0.584
0.884
5.5 x 10'6
0.006
0.517
0.374
Shading indicates a significant difference between the two units.
As shown in Table 6-7, the total chlorine, pH, and TOC sensors had coefficients of
determination greater than 0.91 and slopes within 6% of unity, indicating that their results were
very similar and repeatable. Confirming that evaluation, the t-test p-values for those same
parameters were significantly greater than 0.05, indicating that each sensor was generating
statistically similar results. The turbidity measurement, however, generated a slope of 0.77 and a
coefficient of determination of 0.696, suggesting that the results from both units were not well
correlated with one another. However, those results were affected by a single data point in which
Unit 1 generated a result of 3.75 ntu and Unit 2 generated a result of 0.45 ntu. While the reason
for this outlier was not apparent, if this data point were removed, the slope would change to 0.97,
the intercept to 0.005, and the coefficient of determination to 0.881, indicating very similar
results between the two units. In addition, with or without the outlying data point, the t-test
results indicated that the two units were generating statistically similar turbidity data. Figures 6-8
and 6-14 confirm the statistical evaluation of inter-unit reproducibility for turbidity. With the
exception of Unit 1 about halfway through the extended deployment when it drifted high and
displayed a relatively high degree of variability, the results from both units tracked one another
well.
The conductivity meters had a coefficient of determination of 0.961 and a slope of 0.92,
indicating that the data were highly correlated with one another. The t-test generated p-values
significantly less than 0.05, which indicated that the results from the two conductivity sensors
were significantly different. This difference was driven by the small amount of variability in the
conductivity measurements; therefore, the small difference between the means of the two units
was statistically significant. The temperature measurements had a slope of 0.72 and a coefficient
of determination of 0.758, suggesting that the two units were generating statistically different
results. This result did not appear to be driven by outlying temperatures; and the t-test, with a
p-value of less than 0.05, also indicated that the results from each unit were, in fact, different.
Note that the offsets in the conductivity and temperature results (or from any of the parameters)
40
-------
do not affect the performance of the identification algorithm because the baseline is removed and
the identification is performed based only on deviations from baseline.
As discussed for turbidity, the inter-unit reproducibility results for each water quality parameter
were confirmed through a visual evaluation of the figures throughout Chapter 6. As the statistical
results indicated, all the parameters except temperature and conductivity (the two parameters that
had been determined to be significantly different from one another) were nearly overlapping
when plotted on the same axis, indicating that they were, indeed, extremely similar to one
another.
6.6 Contaminant Identification
Thirteen contaminants were injected (in duplicate) during Stage 4 of this verification test.
Section 3.2 describes the straight, single-pass pipe loop that was used. A total volume of 10 L of
each contaminant solution was pumped into the flowing pipe for approximately 20 minutes,
bringing the water to approximately 15 mg/L for each of the contaminants that were injected.
After the leading edge of the injected slug of contaminant reached the WDMP, if the trigger
signal (a proprietary combination of the monitored water quality parameters) exceeded a
specified threshold (trigger event), the EMTS searched the agent library for possible matches.
The EMTS produced an "agent alarm" whenever a trigger event occurred and the deviation in
baseline water quality parameters matched an agent signature in the agent library. When these
signatures were compared with the signatures from the agent library, the quality of the match
was evaluated with a metric called the match angle, which was described in Section 5.4.
Table 6-8 shows the contaminant identification data for each injection that was performed,
including the data from both units tested. A contaminant was never injected without the EMTS
exceeding the trigger threshold and producing a corresponding agent alarm. For both units, the
agent alarms occurred as few as eight times and as many as 79 times during the 20-minute
injection periods. No agent alarms occurred outside of the 20-minute injection periods. As
mentioned previously, each minute-by-minute search of the agent library can result in more than
one agent being identified, which is why more than 20 agent alarms can occur during a
20-minute injection. If the EMTS recognized a deviation from baseline, but the signature did not
match an agent in the library, the trigger event was identified and recorded as an "unknown"
event. Because the leading and trailing edges of the injected contaminant are dynamic, it is
possible that the injection event will generate alarms other than the injected contaminant.
In Table 6-8, the contaminants injected are presented in alphabetical order on left side, and
across the top are the contaminants that were identified by the EMTS agent alarms. There are
more contaminants across the top of the table because three contaminants were identified that
had not been injected. At the time of this test, the EMTS library was populated with 22 contam-
inants. All except one of the 13 contaminants that were injected were among the contaminants in
the EMTS library. The one exception was that pure glyphosate was used for the ETV test, while
the EMTS library was developed using Roundup™, the commercial preparation of glyphosate.
41
-------
Table 6-8. Contaminant Identification—Number and Quality of Matches
Injected
Contaminant
Metha-
midophos
to
W = weak, M = moderate, G = good.
-------
In Table 6-8, for each injected contaminant, the total number of agent alarms is given for each
replicate injection and for each unit. The three columns for each identified agent account for the
quality of the match angle. For example, for the first injection of aldicarb, Unit 1 reported that
nine of the agent alarms were considered weak matches to aldicarb, four were considered
moderate, and there were no good matches. Moving to the right across the table, it can be seen
that aldicarb was identified as colchicine, E. coli, fluoroacetate, and carbaryl by Unit 1 at some
point during the injection. The agent alarms that correctly matched an injected contaminant are
outlined with dark black; and the weak, moderate, and good matches are highlighted with tan,
blue, and yellow, respectively.
The agent alarms resulting from injected contaminants provide an effective way of evaluating
this data. The results for the injection of ferricyanide and lead nitrate clearly were distinct from
the rest of the contaminant injections because, for both units during both injections, agent alarms
were only attributed to those contaminants. The match angles for the ferricyanide alarm fell
entirely in the moderate or weak match quality categories. For the first lead nitrate injection, 39
out of 41 agent alarms were also in the weak or moderate categories, with two in the good match
category; however, for the second lead nitrate injection, 20 out of 43 agent alarms were in the
good match category.
Arsenic trioxide and nicotine were two other contaminants whose data were distinct from the
other contaminants. For these two contaminants, the agent alarms never corresponded to a
correct identification for either unit. Arsenic trioxide was identified most of the time as
glyphosate and less frequently as malathion. Nicotine was most often identified as aldicarb, but
also was identified as colchicine, dichlorvos, and carbaryl. Following ETV test, the Hach
Company updated its agent library with additional data for arsenic trioxide and nicotine to
determine why the results were not as they had anticipated. A summary of their independent
work is given in Appendix A.
The other nine contaminants were sometimes identified correctly and sometimes as another
contaminant. Because of the subjective nature of evaluating the quality of the matches, no
quantitative data analysis that accounts for the match angle will be performed here; but rather a
general discussion of the results presented in Table 6-8. This approach to comparing the agent
alarm results will focus on the data in the context of all the agent alarms that occurred across
both injections and both units to provide an overview of how the EMTS performed.
During the injection of aldicarb, 67 agent alarms were attributed to aldicarb, 45 to carbaryl, and
21 to E. coli. Colchicine, fluoroacetate, malathion, and methamidophos were also identified. Of
the alarms attributed to aldicarb, 21 had good match angles. The rest of the contaminants had
weak or moderate match angles.
Agent alarms during the injection of colchicine were attributed to colchicine and methanol
98 times each and to dichlorvos 78 times. Of the colchicine alarms, 71 were good matches, while
for the dichlorvos alarms, 64 were good matches. For the methanol alarms, 27 fell into that
category.
For the dicamba injection, 51 agent alarms were correctly attributed to dicamba compared with
10 or fewer agent alarms attributed to colchicine, dichlorvos, mercuric chloride, and methanol.
Fifteen of the 51 dicamba agent alarms were good matches.
43
-------
During the dichlorvos injection, 21 agent alarms correctly identified the contaminant, while
78 identified the contaminant as colchicine and 90 as methanol. All but one of the dichlorvos
alarms indicated weak matches, while several alarms for colchicine and methanol indicated good
matches.
The E. coli injection was correctly identified 20 times with weak or moderate match angles,
while it was identified as malathion 53 times, including seven good match angles. Also
identified during the E. coli injection were arsenic trioxide, colchicine, glyphosate, methanol,
carbonyl, and methamidophos between two and 11 times, mostly with weak and moderate match
angles.
The fluoroacetate injection generated 49 correct agent alarms, with all but two in the weak or
moderate match categories. Fluoroacetate was much less frequently (six or fewer times each)
identified as colchicine, methanol, malathion, and methamidophos.
The injection of glyphosate generated seven correct agent alarms even though Roundup™, the
commercial preparation of glyphosate that also includes other organic chemicals, rather than
pure glyphosate, was included in the EMTS library. Because of this, Roundup™ has a different
water quality parameter signature than pure glyphosate. This injection was also identified as
aldicarb (one time), dicamba (17 times), mercuric chloride (30 times), malathion (20 times), and
methamidophos (11 times). The only alarms that were good matches were for methamidophos.
After the ETV test, Hach updated its agent library with data for pure glyphosate. A summary of
Hach's independent work is given in Appendix A.
The mercuric chloride injection produced an almost equal number of agent alarms identifying it
correctly and as dicamba. Eighty-seven agent alarms were for mercuric chloride, with 16 good
matches and 56 moderate matches; and 86 alarms were attributable to dicamba, with 16 good
matches and 57 moderate matches.
Methanol was correctly identified 26 times, with 14 good matches. However, it was identified as
colchicine 32 times with five good matches and as dichlorvos 25 times with six good matches.
To summarize, the EMTS accuracy for detecting an injected contaminant was 100%; that is, in
all cases, the injection of a contaminant caused a deviation from the baseline measurement of the
water quality parameters significant enough to cause a trigger event resulting in an agent library
search. For 11 out of 13 contaminants, at some time during the injection, the correct contaminant
was identified during the search of the EMTS library. Two method blank injections of pipe loop
water did not cause trigger events; and, therefore, no injection was detected.
The data in Table 6-8 are difficult to interpret, but give a complete report of EMTS performance,
including total number of agent alarms, agent alarms that were attributable to the injected
contaminant (correct identification), and those that were not. Table 6-8 also shows the quality of
match that the agent alarms represented. To provide a more concise way of presenting the data,
the fraction of agent alarms attributable to the injected contaminant (correct identification) was
calculated for each injection. This fraction was called the classification rate and is defined in
Section 5.4. (Note that this approach does not take into consideration the match angle of each
agent alarm.) To present the data succinctly, the classification rates were divided into five levels,
which are shown in Table 6-9. Level 5 represents a classification rate of greater than 70%, Level
44
-------
4 between 31% and 69%, Level 3 between 1% and 30%, Level 2 indicates that the injected
contaminant was not correctly identified but other contaminants were identified, and Level 1 was
reserved for instances when no injections were detected. During this verification test, the Level 1
conditions were met only when two method blanks of pipe loop water were injected.
Contaminants with mostly Level 4 and 5 classification rates included dicamba, ferricyanide,
fluoroacetate, and lead nitrate. Those with mostly Level 4 classification rates included aldicarb,
colchicine, mercuric chloride, and methanol. Glyphosate, dichlorvos, and E. coli each had a
mixture of Level 2, 3, and 4 classification rates. Arsenic trioxide and nicotine had classification
rates of 2, which indicates that those two contaminants were not correctly identified during their
injection. As mentioned above, see the appendix to this report for additional data on glyphosate,
arsenic trioxide, and nicotine.
Table 6-9. Classification Rate Levels
Injected
Contaminant
Aldicarb
Arsenic trioxide
Colchicine
Dicamba
Dichlorvos
E. coli
Ferricyanide
Fluoroacetate
Glyphosate
Lead nitrate
Mercuric chloride
Methanol
Nicotine
Unit
4
2
4
4
4
3
5
5
4
5
4
4
2
Injection 1
1 Unit 2
4
2
4
5
3
2
5
5
3
5
4
4
2
Injection
Unitl
4
2
4
5
3
4
5
4
2
5
4
4
2
2
Unit 2
4
2
4
5
2
2
5
4
2
5
4
3
2
Level 5 = >70% correctly identified
Level 4 = 31-70% correctly identified
Level 3 = 1-30% correctly identified
Level 2 = 0%, other contaminants identified
Level 1 = 0%, no contaminant identified
Evaluating the differences between the performance of individual EMTS units in accurately
identifying injected contaminants is difficult because any differences between the two units are a
result of the monitoring data that is input to the algorithm. Repeatability of that data was
discussed previously. Presumably, because this is a software application, if identical data were
input, identical results would be generated.
6.7 Ease of Use and Data Acquisition
Hach Company staff performed all maintenance on the WDMP and EMTS units. They recorded
any maintenance activity they performed on either of the units in a logbook. The WDMPs did
45
-------
not require daily operator attention. Throughout the verification test, Hach Company staff
periodically adjusted the flows on the turbidity and total chlorine meters as needed to keep them
at the required levels and rebooted the EMTS when the real-time display was not displaying data
properly.
Reinitialization (i.e., rebooting the EMTS) occurred almost daily for Unit 2 for the first week or
so of Stage 3, but thereafter, it only was necessary one or two times. This was required when the
real-time display was not functioning properly. The sample cuvettes within the chlorine monitors
were cleaned four times throughout the verification test (twice during extended deployment) to
maintain accurate measurement. This process took approximately 15 minutes. The TOC
analyzers were calibrated three times throughout the test, the reagents were changed out once,
and the TOC manifold was cleaned two times: once after nitrogen flow had actually been
blocked and once after the nitrogen supply had run out. According to the maintenance records,
Hach Company staff cleaned the turbidimeter lines and checked its calibration two times
throughout the verification test. The conductivity data from one contaminant injection was lost
because of an air bubble. This was remedied by opening the conductance meter to release the
bubble.
The data were downloaded from the EMTS using a USB port. The data were in a comma-
delimited format that was easily opened into a spreadsheet. Overall, some of the regular
maintenance such as cleaning the chlorine meter cuvette and turbidimeter and calibrating the
TOC analyzer would have to be performed regularly if this system was placed in a remote
location, requiring periodic site visits.
46
-------
Chapter 7
Performance Summary
Evaluation Parameter
Stage 1—
Accuracy
Stage 2—
Response to
Injected
Contaminants
Stage 3 —
Accuracy During
Extended
Deployment
Stage 3—
Accuracy After
Extended
Deployment
Stage 3—
Response to
Injected
Contaminants
Injection
Summary
Inter-unit
Reproducibility
(Unit 2 vs. Unit 1)
Units 1 and 2, range
of %D (median)
Nicotine
Arsenic
trioxide
Aldicarb
Reference
WDMP
Reference
WDMP
Reference
WDMP
Units 1 and 2, range
of %D (median)
Unit 1, %D
Unit 2, %D
E. coli
Aldicarb
Reference
WDMP
Reference
WDMP
Total
Chlorine
-47.4 to 4.5
(-3.9)
-
-
-
-
-
-
-15.9 to 6.9
(-3.2)
-4.9
-4.9
-
-
-
-
Turbidity
-53.9 to -1.3
(-34.1)
(a)
+
(a)
+
(a)
+
-81.1 to
245.5 (-21.3)
-5.9
-11.8
_i_(b)
+
+(b)
+
Tem-
perature
-3.0 to 44.3
(-0.2)
NC
NC
NC
NC
NC
NC
-7.4 to 8.5
(-0.1)
-0.2
4.6
NC
NC
NC
NC
Conductivity
-15.5 to 8.1
(2.2)
NC
NC
+
+
NC
NC
-1.8 to 9.6
(4.8)
6.7
0.3
+
+
NC
NC
pH
-6.6 to 3.1
(0.9)
NC
NC
+
+
NC
NC
-2.7 to 0.5
(-0.9)
-2.2
0.2
-
-
-
-
TOC
-64.7 to 147.5
(-14.8)
+
+
NC
NC
+
+
-47.3 to 103.0
(-6.9)
-20.5
3.4
+
+
+
+
Total chlorine and TOC were dramatically affected by injections of nicotine, E. coli, and aldicarb; and
turbidity, pH, and conductivity were affected by some or all of the injections, but not as consistently as
total chlorine and TOC. Aldicarb altered the pH during Stage 3, but not Stage 2.
Slope (intercept)
r2
p-value
0.98
(0.03)
0.994
0.779
0.97
(0.005)(c)
0.88 l(c)
0.884(c)
0.72
(7.68)
0.758
5.5 x 10'6
0.92
(4.19)
0.961
0.006
1.06
(-0.40)
0.919
0.517
0.97
(0.31)
0.991
0.374
With the exception of temperature and conductivity, both units generated similar results.
Stage 4—
Contaminant
Identification
Each time a contaminant was injected, the EMTS detected a deviation in baseline conditions, causing a
"trigger event." Eleven of 13 contaminants were correctly identified at some point during the injection
time. Ferricyanide and lead nitrate were identified correctly 100% of the time. The rest of the injected
contaminants were identified as a contaminant other than themselves at some point throughout the
duration of the injection. Only nicotine and arsenic trioxide were never correctly identified.
Ease of Use and
Data Acquisition
Neither the WDMPs nor the EMTSs required daily operator attention. Hach Company staff adjusted the
flows on the turbidity and total chlorine meters as needed to keep them at the required levels and
rebooted the EMTS when it was not displaying data properly. The chlorine sensors and turbidimeters
needed periodic cleaning, and the TOC analyzer was calibrated three times.
-------
Chapter 8
References
1. Test/QA Plan for Verification of Multi-Parameter Water Monitors for Distribution Systems,
Battelle, Columbus, Ohio, August 2004.
2. U.S. EPA, EPA Method 150.1, pH, in Methods for Chemical Analysis of Water and Wastes,
EPA/600/4-79/020, March 1983.
3. American Public Health Association, et al, SM 2510, Conductivity, in Standard Methods for
the Examination of Water and Wastewater. 19th Edition, Washington, D.C., 1997.
4. American Public Health Association, et al., SM 4500-C1-G, Total Chlorine, in Standard
Methods for the Examination of Water and Wastewater, April 13, 2004.
5. U.S. EPA, EPA Method 415.1, Total Organic Carbon, in Methods for Chemical Analysis of
Water and Wastes, EPA/600/4-79/020, March 1983.
6. U.S. EPA, EPAMethod 170.1, Temperature, inMethodsfor Chemical Analysis of Water and
Wastes, EPA/600/4-79/020, March 1983.
7. U.S. EPA, EPA Method 180.1, Turbidity, in Methods for Chemical Analysis of Water and
Wastes, EPA/600/4-79/020, March 1983.
8. Quality Management Plan (QMP)for the ETV Advanced Monitoring Systems Center,
Version 5.0, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus,
Ohio, March 2004.
48
-------
Appendix A
Hach Company Review
The following text summarizes the results acquired by the Hach Company after review of the
ETV test results. This work was performed at its facility without EPA or Battelle QA oversight.
The results should not be considered part of the ETV testing. Questions about these results
should be directed to representatives of Hach Company.
To test whether the EMTS would identify pure glyphosate if it had been populated with data
attained during the injection of that chemical, the water quality parameter fingerprint of pure
glyphosate was added to the EMTS agent library; the contaminant previously called glyphosate
was renamed Roundup™; and the identification algorithm was reapplied to the original ETV
data. Table A-l shows that, during the original ETV evaluation, very few of the agent alarms
during the injection of glyphosate were reporting glyphosate; and, if they did, the match angles
were typically weak. After Hach's update, the vast majority of alarms were reported as
glyphosate, with mostly strong match angles.
In response to the results of the ETV test, The Hach Company updated the EMTS agent library
by including additional arsenic trioxide data and then reanalyzed the original ETV test data.
Table A-l shows that, during the ETV evaluation, arsenic trioxide was never identified during its
injection, while glyphosate (renamed Roundup™ for this reanalysis) was identified frequently.
The updated data from Hach show that some agent alarms were reported as arsenic trioxide, but
typically with low match angles. Also, Roundup™ was still identified frequently. The difficulty
in identifying arsenic trioxide may be due to its partial solubility in water, making it difficult to
maintain a consistent level during the injections into a pipe. The Hach Company noted that it
also had difficulty in maintaining a consistent suspension for its agent library development.
The Hach Company previously (during non-ETV testing) identified nicotine with the EMTS
rather successfully. It was noted that one difference between the nicotine solution used during
the ETV test and that used during the agent library development was how vigorous the stirring
had been during solution preparation. For ETV testing, the solution was stirred with a stirring
attachment on an electric drill as opposed to a small stir bar used by the Hach Company during
library development. The Hach Company performed an experiment both with and without
vigorous mixing and determined that the vigorous mixing caused the basic form of nicotine to
react with atmospheric carbon dioxide to form the neutralized form of nicotine, which had a
lower characteristic pH than the basic form of nicotine. The EMTS agent library signature was
updated to include the vigorously mixed nicotine, and the ETV data was reanalyzed as pre-
viously described for glyphosate and arsenic trioxide. The Hach Company did not provide the
raw data for these results, but they did indicate that nicotine was identified with strong match
angles during their independent testing.
A-l
-------
Table A-l. Comparison of ETV Data and Data Updated by Hach
Injected
Contaminant
Inj.
#
Unit
Total*
of IDs
Quality of angle match
Glyphosate
(ETV results)
Glyphosate
(Hach Update)
Arsenic
Trioxide (ETV
results)
Arsenic
Trioxide (Hach
Update)
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
9
41
13
24
40
44
60
59
19
13
10
18
24
20
25
18
Aldicarb
W
0
M
1
0
G
0
3
Arsenic
Trioxide
W
0
3
1
1
5
0
4
6
0
1
M
3
2
0
1
0
2
0
1
3
G
2
1
0
0
0
0
0
0
0
0
1
0
Dicamba
W
0
5
2
8
2
5
8
M
0
1
0
0
0
1
0
G
1
0
0
0
0
0
0
Glyphosate
W
1
4
0
0
3
4
4
2
16
9
6
2
1
M
1
0
0
4
3
7
7
3
3
3
6
5
G
1
0
0
17
20
9
20
0
0
1
4
0
Mercuric
Chloride
W
3
9
9
9
3
9
M
4
0
5
0
4
5
G
0
0
0
0
0
0
Malathion
W
4
3
2
1
4
2
2
1
1
6
1
M
0
10
0
1
0
0
9
1
0
0
0
G
0
0
0
0
0
0
0
0
0
0
0
Metha-
midiphos
W
1
2
1
2
M
0
1
0
2
G
0
7
0
6
Roundup
W
M
G
NA
0
2
2
0
1
0
NA
5
8
5
3
7
6
6
6
6
0
5
5
>
NA = Contaminant not injected during the ETV test.
------- |