-------
Table 4-2. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test
events
Calibration
information
(Sentinal™ 500
units and reference
methods)
Sentinal™ 500
units results
Where Recorded
ETV data sheets
and testing
notebook
ETV data sheets
and testing
notebook
Recorded
electronically by
How Often
Recorded
Start/end of test and
at each change of a
test parameter
Upon each
calibration
Recorded
continuously
By
Whom
Battelle
and T&E
Facility
Battelle
and T&E
Facility
Battelle
Disposition of
Data
Used to
organize/check test
results; manually
incorporated in
data spreadsheets
as necessary
Manually
incorporated in
data spreadsheets
as necessary
Excel files
Reference method
procedures
each monitor and
then downloaded to
computer daily
ETV laboratory
record books or
data recording
forms
Throughout sample T&E
analysis process Facility
Transferred to
spreadsheets or
laboratory record
book
11
-------
Chapter 5
Statistical Methods
The statistical methods presented in this chapter were used to verify the Sentinal™ 500 units'
accuracy, response to injected contaminants, and inter-unit reproducibility.
5.1 Accuracy
Throughout this verification test, results from the Sentinal™ 500 units were compared to the
results obtained from analysis of a grab sample by the reference methods. During Stage 1, the
percent difference (%D) between these two results was calculated using the following equation:
%D= ~
where CR is the result determined by the reference method and Cm is the result from a
Sentinal™ 500 unit; the Sentinal™ 500 unit results were recorded every 30 seconds, whereas
collecting the reference samples took only a few seconds. Therefore, Cm was the measurement
recorded closest to the time the reference sample was collected. Water quality stability, as well
as the stability of each sensor, was evaluated during the four-hour time period when reference
samples were analyzed every hour for each of the parameters. Ideally, if the result from a
Sentinal™ 500 unit and a reference method measurements were the same, there would be a
percent difference of zero. During Stages 2 and 3, the continuous data, graphed with the
reference method results, were visually examined to evaluate the response of the Sentinal™ 500
units to the injection of contaminants and their stability over an extended deployment. During
the accuracy and contaminant injection components of Stage 3, the data were evaluated as they
were for Stages 1 and 2, respectively.
5.2 Response to Injected Contaminants
To evaluate the response (i.e., the increase or decrease of water quality parameter measured by
the of the Sentinal™ 500 units) to contaminant injections, the pre- and post-injection reference
samples were graphed as individual data points, along with the continuous measurements. The
reference results showed the effect of each injection on the chemistry of the water in the pipe
loop, and the continuous results from the Sentinal™ 500 unit highlighted its response to such
changes.
12
-------
5.3 Inter-unit Reproducibility
The results obtained from two identical Sentinal™ 500 units were compared to assess inter-unit
reproducibility. Each time a reference sample was collected and analyzed (approximately
127 times throughout this verification test), the results from each Sentinal™ 500 unit were
compared to evaluate whether the two units were generating similar results. This was done in
two ways. First, the results from one unit were graphed against the results of the other unit. In
this evaluation, a slope of unity and a coefficient of determination (r2) of 1.0 would indicate ideal
inter-unit reproducibility. Slopes above 1.0 may indicate a high bias from Unit 2 (graphed on the
y-axis) or a low bias for Unit 1 with respect to each other. Similarly, slopes below 1.0 may
indicate a low bias for Unit 2 or a high bias for Unit 1, again with respect to each other. Second,
the data from each unit were included in a paired t-test, with the assumption that the data from
each unit had equal variances. The t-test calculated the probability of obtaining the subject
results from the two units if there was no significant difference between their results. Therefore,
probability values (p-values) of less than 0.05 (i.e., less than a 5% probability that this data set
would be generated if there actually was no difference between the two units) indicated a
significant difference between the two units. In addition, the results from both units were
graphed together for the Stages 2 and 3 results, allowing a visual comparison.
13
-------
Chapter 6
Test Results
As mentioned previously, this verification test was conducted in three stages that focused on
three different aspects of multi-parameter water monitors for distribution systems. The three
stages are summarized in Table 6-1. The first stage consisted of an evaluation (with varied pHs
and temperatures) of the accuracy of each sensor: free chlorine, temperature, conductivity, and
pH. ORP also was measured; but, because a laboratory reference measurement equivalent to the
on-line continuous measurement was not available, ORP was not included in the accuracy
evaluation. The second stage of the verification test consisted of an evaluation of the response of
the Sentinal™ 500 units to the injection of several contaminants into the pipe loop. The third
stage consisted of deploying the Sentinal™ 500 units for 52 consecutive days with minimal
intervention for maintenance. In addition, contaminant injections were performed at the close of
Stage 3 to confirm that the Sentinal™ 500 units were still responsive to contaminant injection
after the extended deployment. Two Sentinal™ 500 units were tested to evaluate inter-unit
reproducibility. In addition, required maintenance and operational characteristics were
documented throughout the verification test. This chapter provides the results of the three testing
stages, the inter-unit reproducibility data, and ease of use information.
Table 6-1. Summary of Test Stages and Type of Data Presentation
Stage Summary
Data Presentation
1 Accuracy when pH and temperature
were varied
Table of percent differences between
Sentinal™ 500 units and reference
measurements
Response to contaminant injection
Graphs of Sentinal™ 500 unit
measurements and reference measurements,
table showing the effect of injections on
both reference and Sentinal™ 500
measurements
Extended deployment with minimal
maintenance along with post-extended
deployment accuracy and response to
contaminant injections
Graphs of Sentinal™ 500 unit
measurements with reference measurements,
table showing average percent differences
throughout extended deployment, table
showing the effect of injections on both
reference and Sentinal™ 500 measurements
14
-------
6.1 Accuracy
Tables 6-2a-d list the data from the accuracy evaluation performed during the first stage of the
verification test. During four-hour periods, the water quality conditions were held stable, and
reference samples were collected and analyzed five times, once at the start of the designated test
period and four times at one-hour increments thereafter. Because reference sample collection
took just a few seconds and the results from the Sentinal™ 500 units were recorded every
30 seconds, the water quality parameter measurement at the time closest to reference sample
collection was compared to the reference sample. For each unit, this approach resulted in five
paired Sentinal™ 500 and reference results for each of nine sets of water conditions used to
simulate pH and temperature variations at a water utility. The average and standard deviations of
these five results are shown in the tables below, as well as the percent differences between the
average results from both Sentinal™ 500 units and the average of the reference results.
Table 6-2a. Accuracy Evaluation Under Various Conditions—Free Chlorine
Set
1
2
3
4
5
6
7
8
9
Conditions
ambient pH,
ambient temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
ambient pH, decreased
temperature
decreased pH, decreased
temperature
ambient pH, increased
temperature
decreased pH, increased
temperature
ambient pH, ambient
temperature
Reference
Average (SD)
[mg/L]
0.91 (0.08)
0.78 (0.02)
0.65 (0.01)
0.29 (0.02)
0.41 (0.08)
1.47 (0.06)
0.60 (0.04)
0.54 (0.05)
0.91 (0.03)
Unitl
Average (SD)
[mg/L]
1.00(0.00)
0.86 (0.05)
0.82 (0.00)
0.34 (0.01)
0.83 (0.05)
1.86(0.07)
0.96 (0.03)
0.72 (0.07)
1.20(0.02)
%D
9.9
10.3
26.2
17.2
102.4
26.5
60.0
33.3
31.9
Unit 2
Average (SD)
[mg/L]
0.97 (0.00)
1.05 (0.08)
0.82 (0.00)
0.34 (0.01)
0.89 (0.05)
1.52(0.02)
1.05 (0.02)
0.67 (0.05)
1.08(0.02)
%D
6.6
34.6
26.2
17.2
117.1
3.4
75.0
24.1
18.7
15
-------
Table 6-2b. Accuracy Evaluation Under Various Conditions—Temperature
Reference
Unit 1 Unit 2
Average (SD) Average (SD) Average (SD)
Set Conditions [°C] [°C] %D [°C] %D
1
2
3
4
5
6
7
8
9
ambient pH,
ambient temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
ambient pH, decreased
temperature
decreased pH,
decreased temperature
ambient pH, increased
temperature
decreased pH, increased
temperature
ambient pH, ambient
22.66 (0.33)
22.73 (0.23)
21.61 (0.16)
21.93(0.15)
13.82 (0.44)
12.63 (0.26)
26.60 (0.27)
26.69 (0.23)
22.79 (0.21)
21.80(0.11)
21.89(0.07)
21.05 (0.07)
21.72(0.04)
11.98(0.19)
10.52 (0.25)
27.31 (0.05)
27.34 (0.07)
22.41 (0.29)
-3.8
-3.7
-2.6
-1.0
-13.3
-16.7
2.7
2.4
-1.7
21.40(0.14)
21.46 (0.12)
21.05 (0.07)
21.11 (0.05)
11.64(0.22)
10.31 (0.21)
26.82 (0.02)
26.76 (0.06)
21.86(0.31)
-5.6
-5.6
-2.6
-3.7
-15.8
-18.4
0.8
0.3
-4.1
temperature
16
-------
Table 6-2c. Accuracy Evaluation Under Various Conditions—Conductivity
Reference
Unitl
Unit 2
Set
Conditions
Average (SD) Average (SD)
[US/cm]
Average (SD)
%D
%D
ambient pH, 451(1)
ambient temperature
decreased pH, ambient 484 (10)
temperature
decreased pH, ambient 503 (6)
temperature
decreased pH, ambient 694 (12)
temperature
ambient pH, decreased 412(1)
temperature
decreased pH, 501 (10)
decreased temperature
ambient pH, increased 447 (1)
temperature
decreased pH, increased 529 (2)
temperature
ambient pH, ambient 442(1)
temperature
334 (2)
360 (8)
380 (4)
515(8)
318(1)
380(10)
327 (2)
391 (4)
329(1)
-25.9
-25.6
-24.5
-25.8
-22.8
-24.2
-26.8
-26.1
-25.6
341 (1) .24.4
365 (8) -24.6
380 (4) -24.5
517(8)
319(1)
389 (9)
337 (1)
397 (2)
336 (0)
-25.5
-22.6
-22.4
-24.6
-25.0
-24.0
17
-------
Table 6-2d. Accuracy Evaluation Under Various Conditions—pH
Set
1
2
3
4
5
6
7
8
9
Conditions
ambient pH,
ambient temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
decreased pH, ambient
temperature
ambient pH, decreased
temperature
decreased pH, decreased
temperature
ambient pH, increased
temperature
decreased pH, increased
temperature
ambient pH, ambient
temperature
Reference
Average (SD)
[pH unit]
8.76 (0.02)
7.89 (0.09)
7.52 (0.04)
6.73 (0.12)
8.48 (0.02)
7.31 (0.08)
8.37 (0.05)
7.60 (0.06)
8.74 (0.01)
Unitl
Average (SD)
[pH unit]
8.62 (0.00)
7.72 (0.14)
7.32 (0.04)
6.38 (0.07)
8.52 (0.02)
7.06 (0.09)
8.30 (0.04)
7.32 (0.02)
8.65 (0.01)
Unit 2
%D
-1.6
-2.2
-2.7
-5.2
0.5
-3.4
-0.8
-3.7
-1.0
Average
(SD)
[pH unit]
8.80 (0.00)
7.77 (0.16)
7.32 (0.04)
6.32 (0.06)
8.51 (0.01)
7.09 (0.10)
8.34 (0.03)
7.27 (0.01)
8.67 (0.02)
%D
0.5
-1.5
-2.7
-6.1
0.4
-3.0
-0.4
-4.3
-0.8
Of the parameters that were evaluated for accuracy, the free chlorine sensor generated the largest
range of percent differences compared to the reference method (with the median shown in
parentheses): from 3.4 to 117.1 (26.2); for temperature, -18.4 to 2.7 (-3.7); for conductivity
-26.8 to -22.4 (-24.6); and for pH, -6.1 to 0.5 (-1.9).1 The chlorine sensor was calibrated by the
vendor before the verification test, but was not recalibrated throughout Stage 1. The tendency
evidenced by the range and median was for the sensor to drift high. As discussed in Section 6.2,
calibration of the free chlorine sensor was required to maintain accurate free chlorine measure-
ments. The temperature sensors generated very small percent differences (between -6% and 0%)
with respect to the reference method at ambient temperatures, larger negative percent differences
(between -13% and -18%) when the temperature of the water in the pipe loop was decreased, and
small positive percent differences (between 0% and 3%) when the temperature of the pipe loop
water was increased. This trend in percent differences is thought to be chiefly a result of the
reference sample collection and analysis procedure. Reference samples were carried to a
laboratory bench approximately 25 feet from the reference sample collection valve. Therefore,
upon sample collection, the reference sample immediately began equilibrating with the ambient
1 Throughout this report, median values are provided when a range of values is presented. The median of a set of
positive and negative numbers provides a good indicator of the overall direction of the percent differences in the
data set (i.e., whether most values were positive or negative). The disadvantage is that, unless the signs of all the
data are the same, information about the magnitude of change is not available from the median. In summary, the
medians in this report provide the direction, not magnitude, of difference information.
18
-------
air, causing the trends in percent differences with respect to the reference method. The
conductivity results generated a consistently negative percent difference throughout Stage 1, but
one recalibration of the conductivity sensor to match the reference result corrected the negative
percent difference. Stages 2 and 3 exhibited very small conductivity percent differences.
The standard deviations of the reference and continuous measurements collected during each test
period were, with few exceptions, very small with respect to the average result. In only a few
instances was the standard deviation greater than 5% of the average result. This shows both that
the water conditions during these test periods were very stable and that there was very little
variability in the sensors themselves.
6.2 Response to Injected Contaminants
Six injections of contaminants were performed during the second stage of this verification test;
i.e., duplicate injections of nicotine, arsenic trioxide, and aldicarb. Table 6-3 shows the
directional change of each reference and Sentinal™ 500 measurement in response to the
contaminant injections. In general, free chlorine and ORP were the only parameters clearly
affected (for both the reference and continuous measurements) by all six injections. Figures 6-1
through 6-4 show the responses of free chlorine, ORP, conductivity, and pH. The blue and
yellow lines on the graphs represent the measurements made by each Sentinal™ 500 unit, and
the magenta data points represent the results from the laboratory reference method. Because
accuracy was the focus of the first stage of verification testing, percent differences between the
Sentinal™ 500 units and the reference method results are not presented here; however, the
reference method results are included in these figures to confirm that the fluctuations in the
continuous results are due to changes in water chemistry as the result of the injected
contaminants. The figures are divided with vertical lines that define the approximate time period
for each injection. Each injection time period defined on the figures is approximately 24 hours,
but the times vary somewhat depending on when chlorine was added to restore the system to pre-
injection conditions. The contaminant that was injected and whether it was the first or second
replicate are shown at the top of each section of the figures. For each injection, at least four
reference sample results were collected and are included in these figures. The first occurred
within approximately one hour prior to contaminant injection during a period of stable water
quality conditions. The next three reference data points were from samples collected 3, 15, and
60 minutes after contaminant injection. For some of the injections, another reference sample was
collected the following day to show that the pipe loop system had recovered or was in the
process of recovering after the injection. This final reference data point also served as the first
reference sample collected for some of the injections, representing the stable baseline just prior
to injection.
19
-------
Table 6-3. Effect of Contaminant Injections Prior to Extended Deployment
Parameter
Free chlorine
Temperature
Conductivity
pH
ORP
Nicotine
Reference Sentinal™ 500
-
NC
NC
NC
-
NC
NC
NC
Arsenic
Trioxide
Reference Sentinal™ 500
-
NC
+
+
..
-
NC
+
+
Aldicarb
Reference Sentinal™ 500
-
NC
NC
NC
-
NC
NC
NC
+/- = Parameter measurement increased/decreased upon injection.
NC = No change in response to the contaminant injection.
2.5
Nicotne " Nicotine 2 Arsenic 1 Arsenic 2 Aldicart 1 Aldicarb 2
—Unitl
* Reference
Unit 2
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-1. Stage Contaminant Injection Results for Free Chlorine
20
-------
5UU
800 -
700 -
600 -
500 -
400 -
300 -
200 -
100 -
n
Nicotine 1
^
1
. /
M ' *
*^
*
Nicotine 2
^ \
*
i /
/
i /
y
Arsenic 1
* /
|
I
4
Arsenic 2
'
••
|
^
c
'
1
Aldicarb 1
1
Aldicarb 2
1 f T
i
4
i
\
1
I
1
1
1 "
— Unitl
• Reference
Unit 2
Eacli section (separated by vertical lines) represents approximately 24 hoLirs.
Figure 6-2. Stage 2 Contaminant Injection Results for ORP
460
Nicotirp 2 Arsenic 1 Arsenic 2
400 -"-
Each section (separated by vertical lines) represents approximately 24 hours
Figure 6-3. Stage 2 Contaminant Injection Results for Conductivity
21
-------
Each section (separated by vertical lines) lepresents approximately 24 hours
Figure 6-4. Stage 2 Contaminant Injection Results for pH
Figure 6-1 shows how the measurement of free chlorine was affected by the contaminant
injections. Prior to the injections, the free chlorine level was maintained at approximately 1 mg/L.
At the start of Stage 2, the Sentinal™ 500 unit's measurements had drifted significantly higher
than the reference measurement, a trend that had been observed during Stage 1. Nonetheless, in
each case, within one hour of contaminant injection, the free chlorine level, as measured by the
laboratory reference method, dropped to its low point. As evidenced by the vertical drop in the
line representing the free chlorine concentration, it was clear that the chlorine sensor on the
Sentinal™ 500 units responded to the decrease in free chlorine levels as a result of the presence
of the contaminant. In addition to the high measurements prior to the injections, the Sentinal™
500 units recovered from both nicotine injections to levels higher than for the reference method.
Note that just prior to the first arsenic injection, the chlorine measurements dropped to match the
reference measurement much more closely. At this time, the chlorine sensors were calibrated to
match the reference method measurements. Thereafter, the sensors seemed less prone to
recovering to a result biased high, but Unit 2 appeared to have drifted low after the arsenic
injection. Also, after the first aldicarb injection, Unit 2 recovered to a high measurement and
Unit 1 drifted low. Both sensors were calibrated prior to the final aldicarb injection, which is
evidenced by the convergence of their respective lines. Also, after the second aldicarb injection,
both chlorine sensors recovered to very high measurements, requiring calibration prior to the next
stage of the verification test. For each injection, the drop in free chlorine levels was followed by
the restoration of the pipe loop system to approximately pre-injection conditions through the
addition of sodium hypochlorite. This is shown in Figure 6-1 by the rapidly increasing free
chlorine concentration after the sensor reaches a low point in free chlorine concentration.
The ORP in water is dependent on the occurrence of oxidation-reduction chemical reactions
within the water. Therefore, when free chlorine is reacting with injected contaminants, it can be
expected that the ORP would be affected. Figure 6-2 shows that ORP tracked the concentration of
22
-------
free chlorine upon injection of the contaminants. The free chlorine reacted with the contaminants,
and the concentration dropped, as did the ORP. It is difficult to determine if the change in ORP is
in response to the drop in free chlorine or to the presence of the contaminant itself. Conductivity
and pH were affected (as measured by both the reference and continuous measurement) by the
injection of arsenic trioxide only. However, this may have been due to the pH adjustment required
to get it into solution.
6.3 Extended Deployment
Figures 6-5 through 6-9 show the continuous measurements from both Sentinal™ 500 units
during the 52-day extended deployment stage of the verification test. Those measurements are
represented by the blue and yellow lines, while the results of the reference samples, collected
once daily throughout this deployment, are represented by the magenta symbols. The x-axis on
each figure represents the period of time between September 1, 2004, and October 22, 2004, and
the y-axis gives the results of each water quality measurement. Data points were recorded every
30 seconds during the verification test; but, for the extended deployment figures, only data points
collected approximately every 2 minutes were depicted. This was done so that a standard spread-
sheet could be used to generate these figures. This approach was inconsequential to interpreting
the figures.
The objective of this stage of the verification test was to evaluate the performance of the
Sentinal™ 500 unit over an extended period of time with minimal intervention to simulate a
situation in which the units may be deployed at a remote location. The continuous trace was
visually evaluated to see whether any aspects of the data were noteworthy. A second, more
quantitative, evaluation was then performed to get an indication of the accuracy of the extended
deployment measurements. This evaluation, much like the accuracy evaluation conducted during
the first stage of testing, included calculating the percent differences between the average
continuous measurements and average reference sample results throughout the extended
deployment, as well as the standard deviation of each of those measurements. The standard
deviation of the results provided a means to evaluate the stability of the water conditions during
Stage 3, as well as how the standard deviations of the continuous measurements differed from the
standard deviations of the reference measurements. Similar relative standard deviations between
the continuous and reference measurements indicate that variability was mostly dependent on the
water conditions and not due to systematic variability in the Sentinal™ 500 unit results. (Note
that the reference results were only generated during business hours, so any fluctuations occurring
during off hours were not reflected in the standard deviation of the reference results. Because of
this, free chlorine, a parameter that varied at times during weekends when the supply of chlorine
ran low, might have been expected to have a larger variability than other more stable parameters.)
Table 6-4 lists the percent differences, along with the average and standard deviations of the
reference and continuous results during the extended deployment. The range and median (see the
footnote in Section 6.1 for direction on interpreting the median) percent difference for each water
quality parameter, as measured for each reference sample analyzed during the extended
deployment, are also given.
For free chlorine, visual inspection of the data in Figure 6-5 revealed that, at the start of Stage 3,
the Sentinal™ 500 units' measurements were dropping from the high reading to which they had
23
-------
2.5
1.5
B)
E
0.5
— Unitl
» Reference
Unit 2
Event
#3
Duration of Stage 3: 52 clays
Figure 6-5. Extended Deployment Results for Free Chlorine
12
11
10 -
9H
8 -
7 -
•Unitl
Reference
Unit 2
Duration of Stage 3: 52 days
Figure 6-6. Extended Deployment Results for pH
Event
#4
24
-------
800
700 -
600 -
500 -
400
300 -
200-
100-
0
— Unitl
» Reference
Unit 2
Duration of Stage 3: 52 days
Figure 6-7. Extended Deployment Results for ORP
— Unitl
1 Reference
Unit 2
Duration of Stage 3: 52 clays
Figure 6-8. Extended Deployment Results for Temperature
25
-------
500
450
E 400 -1
~
Q)
| 350 ^
iw
2
E 300 4
250 -
200
— Unitl
* Reference
Unit 2
Duration of Stage 3: 52 days
Figure 6-9. Extended Deployment Results for Conductivity
Table 6-4. Accuracy During Extended Deployment
Parameter
Free chlorine
Temperature
Conductivity
PH
Reference
average
(SD)(a)
0.95 (0.09)
22.83 (0.36)
333 (57)
8.72 (0.07)
Unitl
Average (SD)(a)
0.74 (0.22)
22.58 (0.08)
345 (55)
8.60 (0.27)
%D
-22.1
-1.1
3.6
-1.4
Unit 2
Average (SD)(a)
0.76 (0.22)
21.95 (0.08)
335 (54)
8.78 (0.08)
%D
-20.0
-3.9
0.6
0.7
Both Units
%D Range
(median)
-54.8 to 50.0 (-21.5)
-7.8 to 2.7 (-2.7)
-0.8 to 5.5 (2.1)
-7.2 to 1.6 (0.3)
(a) Free chlorine, mg/L; temperature, °C; conductivity, |_iS/cm; pH, pH units.
recovered after Stage 2. This drop was caused by the calibration of the sensors to match the
reference result. It is not clear why the sensors did not track the reference measurements better
thereafter; but, for the first approximately one-third of the extended deployment, the free chlorine
measurements were approximately 0.5 mg/L (with some variation) for both Sentinal™ 500 units,
while the reference method measurement was approximately 1 mg/L. At that point, Clarion
directed the verification staff to recalibrate the chlorine and pH sensors based on the reference
method result (free chlorine Event #1 in Figure 6-5). When this was done, both Sentinal™ 500
units tracked the free chlorine reference measurements rather well for several days until the
measured chlorine concentrations drifted high (to nearly 2 mg/L) for a three-day period (free
chlorine Event #2). The sensors recovered without recalibration and, with a few exceptions,
tracked the reference method results fairly well until the supply of sodium hypochlorite (used to
26
-------
maintain the chlorine concentrations in the pipe loop) ran low after a weekend and the chlorine
level dropped to less than 0.5 mg/L. After this drop, both chlorine sensors recovered to a
measurement somewhat lower than the reference method (free chlorine Event #3). This continued
until near the end of the extended deployment when the results from the two Sentinal™ 500 units
abruptly converged to a measurement very similar to that of the reference measurement. This
marked the time that Clarion again directed the verification staff to calibrate the chlorine sensors
to match the reference method result (free chlorine Event #4). During the extended deployment,
the percent differences for both Sentinal™ 500 units ranged from -54.8 to 50.0, with a median of
-21.5. The average free chlorine concentration, as measured by the reference method, was 0.95 ±
0.09 mg/L.
The pH results are presented in Figure 6-6. For the first approximately one-third of the extended
deployment, Unit 2 and the reference method were measuring the pH as approximately 8.8, while
Unit 1 was measuring it as approximately 8.0. The start of Stage 3 corresponded with a data
logging memory problem in Unit 1. The Clarion representative removed the memory module, had
it repaired, and replaced it a couple of days later. The large difference between Units 1 and 2
began the same day the memory module was reinstalled in Unit 1 (pH Event #1 in Figure 6-6).
For a reason that is not known, the pH sensor was either improperly calibrated or not calibrated
that day. Both pH sensors were calibrated several days later (at the direction of Clarion), which is
shown by the convergence of pH measurements for both units with the reference measurements
(pH Event #2). After that, both pH sensors maintained the accuracy of the results rather well. The
average pH, as measured by the reference method, was 8.72 ± 0.07, and the average pH
measurements for Units 1 and 2, respectively, were 8.60 ± 0.27 and 8.78 ± 0.08. Overall, during
the extended deployment, the percent difference for the pH sensor ranged from -7.2 to 1.6, with a
median of 0.3.
The other three water quality parameters were not affected by the pH adjustment. The ORP,
temperature, and conductivity sensors were allowed to operate without intervention throughout
the extended deployment. The measurements from these three sensors are shown in Figures 6-7
through 6-9. In Figure 6-7, the ORP results are shown along with a laboratory reference method
result. The reference method is not an accurate representation of water in a flowing pipe, but it
can be used to evaluate a trend in the decrease and increase in the ORP, as it was in the previous
section for the contaminant injections.
The Unit 1 and 2 conductivity results tracked the reference method results throughout the
extended deployment. The temperature results from both Units 1 and 2 had regular variability
because the test was conducted in a facility where the water temperature was heavily affected by
the outdoor temperature; therefore, the water temperature changed as a function of the high and
low for the day. Also, Unit 2 temperature results appeared to be biased low with respect to Unit 1
and the reference method.
6.4 Accuracy and Response to Injected Contaminants After Extended Deployment
After the 52-day deployment of the Sentinal™ 500 units with minimal intervention, their
performance was evaluated during a 4-hour period of ambient pH and temperature during which
reference samples were collected hourly. The results of this evaluation are given in Table 6-5.
The percent differences determined after the extended deployment for free chlorine, conductivity,
27
-------
and pH were considerably different from those determined during Stage 1. This is due to the fact
that each of these sensors was calibrated (as previously discussed) at least once between Stage 2
and Stage 3. In all three cases, the percent differences were closer to zero in the post-extended
deployment accuracy evaluation.
Table 6-5. Post-Extended Deployment Results
Parameter
Free chlorine
Temperature
Conductivity
PH
Reference
average (SD)(a)
0.92 (0.02)
22.66 (0.16)
356(1)
8.59 (0.01)
Unitl
average (SD)(a)
0.82 (0.05)
22.55 (0.01)
365 (2)
8.59 (0.00)
Unit 1 %D
-10.9
-0.5
2.5
0.0
Unit 2
average (SD)(a)
0.75 (0.10)
21.95 (0.05)
357 (1)
8.69 (0.01)
Unit 2 %D
-18.5
-3.1
0.3
1.2
(a) Free chlorine, mg/L; temperature, °C; conductivity, |_iS/cm; pH, pH units.
A second evaluation of the response to injected contaminants after the extended deployment used
four contaminants. Two were a repeat of the aldicarb injections performed during Stage 2 and two
were injections of E. coli, which was not available for injection during the earlier stage of the test.
Table 6-6 gives the directional change of each reference measurement and Sentinal™ 500
measurement in response to the contaminant injections. In general, free chlorine, ORP, and pH
were the three parameters that were affected (for both the reference and continuous measure-
ments) by all four injections. These parameters are shown in Figures 6-10 through 6-12. Conduc-
tivity is shown in Figure 6-13. The response of the chlorine sensor was consistent for all four
injections. The recovery of the chlorine sensors was not as consistent. After the first injection, the
recovery was difficult to evaluate because of a chlorine concentration increase resulting from
occurrences in the Cincinnati water system that could not be controlled. After the second and
third injections, the sensors both recovered to a concentration less than the reference method.
Therefore, the sensors were recalibrated prior to the final injection. After the final injection, the
chlorine sensors recovered fully. As during Stage 2, the ORP sensor tracked the chlorine response
for each injection. Again, it is difficult to determine whether the ORP change is due to the
reaction of chlorine or the presence of the contaminant itself. For pH, the reference results
indicated a decrease in response to all of the contaminant injections; this result was unexpected.
Aldicarb had not altered the pH during the Stage 2 injections. In addition, the conductivity
increased (in both the reference and continuous measurements) very slightly upon injection of the
E. coli.
28
-------
Table 6-6. Effect of Contaminant Injections After Extended Deployment
Parameter
Free chlorine
Temperature
Conductivity
pH
ORP
E. coli
Reference Sentinal™ 500 Reference
_ _ _
NC NC NC
+ + NC
- -
- - -
Aldicarb
Sentinal™ 500
-
NC
NC
-
-
+/- = Parameter measurement increased/decreased upon injection.
NC = No change in response to the contaminant injection.
z.u
2
1^
.5 -
J
Bi
E
1 -
0.5 -
n
E.coli 1
*
^s
E.coli 2
^
^
i
J
, r
I
j
Aldicarb 3
4
i
I
t
Aldicarb 4
|
iX/*"
i
I
J
*
— Unitl
Reference
Unit 2
^"**
i
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-10. Stage 3 Contaminant Injection Results for Free Chlorine
29
-------
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-11. Stage 3 Contaminant Injection Results for ORP
7.8 -L
Each section (separated by vertical lines) represents approximately 24 hours.
Figure 6-12. Stage 3 Contaminant Injection Results for pH
30
-------
420
410
340
Each section (separated by vertical lines.) lepresents approximately 24 hours.
Figure 6-13. Stage 3 Contaminant Injection Results for Conductivity
6.5 Inter-unit Reproducibility
Two Sentinal™ 500 units were compared throughout the verification test to determine whether
they generated results that were similar to one another. This was done using the Sentinal™ 500
data collected whenever a reference sample was collected throughout the verification test. Two
evaluations were performed to make this comparison. First, the results from Unit 2 were graphed
on the y-axis; those from Unit 1 were graphed on the x-axis; a line was fitted to the data; and the
slope, intercept, and coefficient of determination (r2) of this line were determined. Second, a t-test
assuming equal variances was performed on those same data. For the linear regression analysis, if
both Sentinal™ 500 units reported the identical result, the slope of such a regression would be
unity (1), the intercept zero (0), and the coefficients of determination (r2) 1.0. The slope can
indicate whether the results are biased in one direction or the other, while the coefficients of
determination provide a measure of the variability of the results. The t-test shows whether the
sensors generated statistically similar data. Small p-values (<0.05 at a 5% confidence level)
would suggest that the results from the two Sentinal™ 500 units are significantly different from
one another. Table 6-7 gives the slope, intercept, and coefficients of determination for the
inter-unit reproducibility evaluation and the p-value for the t-test performed for each sensor.
The temperature, conductivity, and pH sensors had coefficients of determination greater than 0.95
and slopes within 5% of unity, indicating that their results were very similar and repeatable.
Confirming that evaluation, the t-test p-value for those sensors was 0.23, 0.74, and 0.17,
respectively.
31
-------
Table 6-7. Inter-unit Reproducibility Evaluation
Parameter
Free chlorine
Temperature
Conductivity
pH
ORP
Slope
0.86
0.98
1.01
1.05
0.89
Intercept
0.10
-0.04
-4.13
-0.3
72
r2
0.87
1.00
0.98
0.95
0.98
t-test p-value
0.92
0.23
0.74
0.17
0.87
As can be seen from Table 6-7, the ORP sensors had a coefficient of determination of 0.98,
indicating that they were highly correlated with one another; but the slopes were approximately
11% less than unity, indicating that Unit 2 measurements were lower than Unit 1. This evaluation
was supported by the figures throughout Chapter 6, which show that Unit 2 measurements were
slightly less than Unit 1. However, based on the t-test, this difference was not significant.
The free chlorine sensor had a lower coefficient of determination and a slope that deviated from
unity by more than 10%. This lower correlation was observed in the figures for the extended
deployment when the Sentinal™ 500 units drifted by varying degrees. In addition, calibrating the
chlorine sensor several times during the verification test increased the degree of variability in the
free chlorine results. Because of this, the t-test indicated that the results from the free chlorine
sensors were statistically the same.
6.6 Ease of Use and Data Acquisition
Throughout the duration of the verification test, the verification staff was not required to perform
any routine maintenance. However, on several occasions, Clarion or verification test staff (at the
request of Clarion) adjusted the chlorine sensor reading to match the reference sample measure-
ment. Therefore, the accuracy of the free chlorine measurement was directly affected by this
adjustment. Based on the performance of the free chlorine sensors, they may have to be adjusted
periodically to maintain the accuracy of measurements. This would require a means of measuring
the chlorine content of the water, as well as a site visit, to make the adjustment. No other
maintenance was necessary during the test.
Data were saved onto memory modules mounted onto the Sentinal™ 500 units. With a 30-second
data collection frequency, the storage capacity of the modules was not reached during the three-
month test. One of the modules had to be removed for repair during the verification test. Also, in
two instances, the Sentinal™ 500 units failed to download properly. Both units were rebooted,
and the problem was resolved. This system may need to be reset periodically if the units are
deployed remotely.
32
-------
Chapter 7
Performance Summary
Evaluation Parameter
Stage 1—
Accuracy
Stage 2 — Response
to Injected
Contaminants
Stage 3 —
Accuracy During
Extended
Deployment
Stage 3 — Accuracy
After Extended
Deployment
Stage 3 — Response
to Injected
Contaminants
Injection Summary
Inter-unit
Reproducibility
(Unit 2 vs. Unit 1)
Ease of Use
and Data
Acquisition
Units 1 and 2, range of
%D (median)
Nicotine
Arsenic
trioxide
Aldicarb
Reference
Sentinal™ 500
Reference
Sentinal™ 500
Reference
Sentinal™ 500
Units 1 and 2, range of
%D (median)
Unit 1, %D
Unit 2, %D
E. colt
Aldicarb
Reference
Sentinal™ 500
Reference
Sentinal™ 500
Free
Chlorine
3.4 to 117.1
(26.2)
-
-
-
-
-
-
-54.8 to 50.0
(-21.5)
-10.9
-18.5
-
-
-
-
Tem-
perature
-18.4 to 2.7
(-3.7)
NC
NC
NC
NC
NC
NC
-7.8 to 2.7
(-2.7)
-0.5
-3.1
NC
NC
NC
NC
Conductivity
-26.8 to -22.4
(-24.6)
NC
NC
+
+
NC
NC
-0.8 to 5.5
(2.1)
2.5
0.3
+
+
NC
NC
pH
-6.1 to 0.5
(-1.9)
NC
NC
+
+
NC
NC
-7.2 to 1.6
(0.3)
0.0
1.2
-
-
-
-
ORP
(a)
-
-
-
-
-
-
(a)
(a)
(a)
-
-
-
-
For a reason that is not clear, aldicarb altered the pH, as measured by the reference method,
during the Stage 3 injections, but not during the Stage 2 injections.
Slope (intercept)
r2
p-value
0.86(0.10)
0.87
0.92
0.98 (-0.04)
1.00
0.23
1.01 (-4.13)
0.98
0.74
1.05 (-0.3)
0.95
0.17
0.89 (72)
0.98
0.87
All sensors generated results that were similar according to the results of the t-test. However,
the slopes of the ORP and free chlorine sensor data plotted against one another suggest that
the results from each unit were somewhat different from one another.
Based on the performance of the free chlorine sensors, they may have to be adjusted
periodically to maintain the accuracy of the measurements. The memory module in Unit 1 had
to be replaced and, twice, each unit had to be rebooted before data could be downloaded.
(a) ORP was not included in the accuracy evaluation because of the lack of an appropriate reference method.
+/- = Parameter measurement increased/decreased upon injection.
NC = No obvious change was noted through a visual inspection of the data.
33
-------
Chapter 8
References
1. Test/QA Plan for Verification of Multi-Parameter Water Monitors for Distribution Systems,
Battelle, Columbus, Ohio, August 2004.
2. Personal communication with John Hall, U.S. EPA, July 23, 2004.
3. U.S. EPA, EPA Method 150.1, pH, in Methods for Chemical Analysis of Water and Wastes,
EPA/600/4-79/020, March 1983.
4. American Public Health Association, et al, SM 2510, Conductivity, in Standard Methods for
the Examination of Water and Wastewater, 19th Edition, Washington, D.C., 1997.
5. American Public Health Association, et al., SM 4500-G, Residual Chlorine, in Standard
Methods for the Examination of Water and Wastewater, 19th Edition, Washington, D.C., 1997.
6. American Public Health Association, et al., SM 2580-B, Electrochemical Potential, in
Standard Methods for the Examination of Water and Wastewater. 19th Edition, Washington,
D.C., 1997.
7. U.S. EPA, EPAMethod 170.1, Temperature, inMethodsfor Chemical Analysis of Water and
Wastes, EPA/600/4-79/020, March 1983.
8. Quality Management Plan (QMP)for the ETV Advanced Monitoring Systems Center,
Version 5.0, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus,
Ohio, March 2004.
34
-------