September 2007
Environmental Technology
Verification Report
SENSICORE, INC.
WATERPOINT870
Prepared by
Battelle
Batteiie
"Ihe Business o/ Innovation
Under a cooperative agreement with
^& CrTF\ u.S. Environmental Protection Agency
ET1/ET1/ET1/
-------
September 2007
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
SENSICORE, INC.
WATERPOINT870
by
Ryan James
Raj Mangaraj
Zachary Willenberg
Amy Dindal
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at http://www.epa.gov/etv/
centers/center 1 .html.
in
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We acknowledge the contribution of
the Columbus, Ohio Department of Public Utilities Division of Power and Water in hosting this
verification at the Dublin Road Water Quality Laboratory and, in particular, the efforts of
Mr. Jeff Kauffman in support of all aspects of testing. We want to acknowledge the support of
Dr. Steven Holodnick and Mr. Uwe Michalak of Sensicore, Inc. during this verification test. We
also thank Mr. Ken Fritz of the U.S. EPA and Ms. Christine Kolbe of the Texas Commission on
Environmental Quality for taking the Sensicore WaterPOINT 870 on a field sampling trip and
providing the data they collected as well as qualitative information about the usability of the
technology in a true field setting. Finally, we would like to thank Mr. Kauffman (test/QA plan
and report), Mr. Fritz (test/QA plan), Ms. Kolbe (test/QA plan and report), Mr. John Hall (report)
of the U.S. EPA, and Ms. Lisa Olsen (test/QA plan) of the United States Geological Survey for
their review of the test/QA plan and/or verification report.
IV
-------
Contents
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations vii
Chapter 1 Background 1
Chapter 2 Technology Description 2
Chapters Test Design and Procedures 4
3.1 Test Overview 4
3.2 Experimental Design 5
3.2.1 Stage 1 Laboratory Testing of DI Water Samples 5
3.2.2 Stage 2 Laboratory and Field Testing of Drinking and Surface Water Samples 6
3.2.3 Stage 3 Remote Field Analysis 7
3.3 Laboratory Reference and Quality Control Samples 7
3.3.1 Reference Methods 7
3.3.2 Reference Methods Quality Control Samples 7
3.4 Qualitative Evaluation Parameters 8
Chapter 4 Quality Assurance/Quality Control 9
4.1 Audits 9
4.1.1 PE Audit 9
4.1.2 Technical Systems Audit 10
4.1.3 Data Quality Audit 10
4.2 QA/QC Reporting 10
4.3 Data Review 10
Chapters Statistical Methods 12
5.1 Accuracy 12
5.2 Precision 12
5.3 Inter-unit Reproducibility 12
Chapter 6 Test Results 13
6.1 Accuracy and Precision 13
6.1.1 Stage 1 Laboratory Testing of DI Water Samples 13
6.1.2 Stage 2 Results for Drinking and Source Water Samples 15
6.2 Inter-unit Reproducibility 18
Chapter 7 Performance Summary 22
Chapter 8 References 23
-------
Figures
Figure 2-1. Schematic of a WP870 Sensor and Photo of Handheld Unit 2
Figure 6-1. Figure 6-1 Screen Shot of EPA NERL WP870 Data on WaterNOW Software 21
Tables
Table 3-1. Test Sample Summary 5
Table 3-2. Stage 1 Test Sample Information 6
Table 3-3. Reference Methods 8
Table 4-1. Performance Evaluation Audit and Reference Method Duplicate Results 10
Table6-l. Stage 1 Accuracy and Precision Results for the WP870 14
Table 6-2. Stage 1 Performance Across Sensor Lifespan 15
Table 6-3. Stage 2 Finished Drinking Water (DW) Results 16
Table 6-4. Surface (SW) and "In-Process" (IPW) Drinking Water - Field and Laboratory
Results 17
Table 6-5. Surface with Significantly Different Results Between Sensors 19
Appendix
Sensicore, Inc. Comment on WP870 Improvements 24
VI
-------
List of Abbreviations
%D Percent Difference
AMS Advanced Monitoring Systems
ASTM American Society for Testing and Materials
CaCOs Calcium Carbonate
CDW Columbus, Ohio Public Utilities Division of Power and Water
cm Centimeter
D Absolute Difference
DI Deionized
DW Drinking Water
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification Program
IPW "In-Process" Drinking Water
mg/L Milligrams per liter
|j,S/cm MicroSiemens per centimeter
mm millimeters
NA Not Applicable
NERL U. S. EPA National Exposure Research Laboratory
NIST National Institute of Standards and Technology
ORP Oxidation Reduction Potential
PE Performance Evaluation
QA Quality Assurance
QC Quality Control
QMP Quality Management Plan
RSD Relative Standard Deviation
SM Standard Methods
SW Surface Water
TCEQ Texas Commission on Environmental Quality
TSA Technical Systems Audit
WP870 Sensicore WaterPOINT 870
vn
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Exposure Research Laboratory (EPA NERL) and its verification
organization partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under
ETV. The AMS Center recently evaluated the performance of the Sensicore WaterPOINT 870
(WP870), a multi-parameter water sensor. This test was carried out in collaboration with the
Columbus, Ohio Department of Public Utilities Division of Power and Water (CDW).
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This verification report provides
results for the verification testing of the WP870. Following is a description of the WP870, based
on information provided by the vendor. The information provided below was not verified in this
test.
Sensicore has developed a lab-on-chip micro-sensor array technology called the WaterPOINT
870 that incorporates chemical selective sensors and physical measuring devices on a single
silicon chip. This panel of tests is used to chemically profile drinking water (and/or other
liquids) in five minutes.
This handheld system was
designed for both
municipal and industrial
applications. It employs
Sensicore's platform sensor
chip with five membrane-
based ion selective
electrodes capable of
detecting light metal ions
and dissolved gases, two
micro amperometric arrays
for detecting free chlorine
and monochloramine
species, and electronic
sensors for measuring
oxidation-reduction potential (ORP), conductivity, and temperature. All of these sensors are
incorporated on a single silicon substrate that is 4 millimeters (mm) x 5 mm in size and
conveniently packaged in a semi-disposable unit that also contains its own reference electrode.
In all, with the direct measurements and calculated values that can be obtained from the direct
measurements, the system reports 16 different results as follows: pH, ORP, conductivity, total
dissolved solids, free chlorine, monochloramine, free and total ammonia, chlorine-ammonia
ratio, biocide-food ratio, carbon dioxide, total alkalinity, calcium, calcium hardness, total
hardness, and Langelier Saturation Index. Only the direct measurements including pH, ORP,
conductivity, free chlorine, monochloramine, free ammonia, calcium hardness, and total
alkalinity results were verified during this test.
Figure 2-1 Schematic of a WP870 sensor (left) and a photo
of the handheld unit (right)
-------
The WP870 handheld system includes several features:
• Incorporates a single point calibration/QC check into every measurement;
Calibrates all sensors via weekly two-point calibration;
Transfer of results and sensor diagnostic and calibration information to computer via
USB connection;
• Includes sample chain of custody information including time and date stamp, test
location (including an optional GPS recording if desired) and a barcode recorder for
identifying samples
• Software is menu driven and requires little training;
• Powered by rechargeable battery;
• Is compatible with WaterNOW software which is an online and secure data service
utilizing 128-bit data encryption that helps the user understand data they have collected
through unique visualization and comparison tools. It provides a means for the user to
combine data from a variety of locations. Datasets from the WP870 analyzer can be
uploaded through the internet or through email attachment. Subscriptions for the
WaterNOW service start at $400 per month;
• Has dimensions of approximately 16 centimeters (cm) x 22 cm and weighs 1.75
pounds;
• Completes full analysis within five minutes;
• One time cost of $2,495 for the handheld unit, and $295 for every additional sensor chip
(referred to as the "sensor" throughout this report) that is good for the analysis of 50
samples or for a duration of 30 days following the initial calibration, whichever comes
first. New sensors include all necessary calibration solutions and sample buffers and
conditioners required for the sample analyses. Note that each sample analysis provides
results for all the above listed water quality parameters.
While not evaluated during this test, the WP870 has an Optical Module that includes the
following capabilities:
• Turbidity measurements which meet US EPA Method 180.1
• 375 nm wavelength intrinsic color measurements following Standard Method 2120B
• Colorimetric measurements utilizing a red/green/blue light emitting diode and
corresponding photodetectors to measure a variety of ampouled chemistries, including
Total and Free Chlorine by Standard Method 4500 Cl-G.
• Total hardness ion selective electrode for the determination of hardness due to free
calcium and magnesium.
• One time cost of $2,995 for the optic-enabled handheld unit, with sensor kits for free and
monochloramine which are good for 90 analyses or up to 60 days ($295-$495), and ion
selective electrode sensor kits which are good for 90 tests and up to 60 days ($225-$410).
-------
Chapter 3
Test Design and Procedures
3.1 Test Overview
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Multi-parameter Water Sensors including amendments 1-4(1) and adhered to the
quality system defined in the ETV AMS Center Quality Management Plan (QMP).2 Multi-
parameter water sensor technologies consist of sensors that measure several different water
quality parameters from grab samples. Throughout this test, the WP870 was challenged with a
number of different types of water samples. For each sample, the WP870 generated all eight of
the water quality parameters verified during this test. Those types of water samples included: 1)
water samples that had been prepared in American Society for Testing and Materials Type II
deionized (DI) water so the water quality parameters would cover the range of response for each
parameter measured by the WP870, 2) finished drinking water samples, 3) surface water
samples, and 4) water samples collected from within the water treatment process. In addition,
the performance of the WP870 was evaluated over the lifetime (50 water samples or 30 days
following initial calibration) of an individual sensor and handheld unit by first analyzing 15
samples prepared in DI water, then six finished drinking water samples, followed by another 15
samples prepared in DI water again. Some analyses were performed at a field location as well as
in a laboratory.
The verification test for the WP870 was conducted from April through July 2007. This test was
coordinated by Battelle, but conducted at the CDW and at various field locations. Technicians
from both Battelle and CDW contributed to the testing effort. All reference measurements were
performed on-site at the CDW laboratories.
The WP870 was verified by evaluating the following parameters:
• Accuracy - comparison to results from standard laboratory reference analyses for DI water,
drinking water within the treatment process, finished drinking water, and untreated source
water test samples
• Precision - repeatability from sample replicates analyzed on the same day
• Inter-unit reproducibility - comparison of results from two identical sensors and handheld
units
• Field portability - operation during remote field site analysis
• Ease of use - general operation, data acquisition, set-up, consumables used, and purchase and
operational costs.
-------
3.2 Experimental Design
The verification test was organized into three stages that included: 1) samples prepared from DI
water, 2) samples consisting of finished drinking water, surface water, or water within the
drinking water treatment process, and 3) remote field analysis (qualitative testing only). Each
stage of testing is described below as well as summarized in Table 3-1. As a reminder, the key
component of the WP870 handheld unit is the sensor, a small chip that is inserted into the
handheld unit and contains the functionality required for water quality parameter measurement.
The software in the handheld unit keeps track and prevents more than 50 water samples to be
analyzed per sensor and also prevents any analyses after 30 days following initial calibration.
Therefore, the test sample matrix was designed to get the maximum amount of performance
information from each sensor that was used. Six different sensors were used throughout testing.
Table 3-1. Test Sample Summary
Stage of
Testing
Description and Number of Water Samples
Stage 1 Part 1
Accuracy
Sensors 1 and 2: Three levels of the eight water quality parameters analyzed
in triplicate using both sensors (72 individual results); as Table 3-2 shows,
several of the water quality parameters were grouped into a single solution
rather than requiring one solution for each water quality parameter, therefore,
only a total of 45 water samples were analyzed by each sensor. All samples
were compared to reference methods.
Stage 1 Part 2
Performance
Over Sensor
Lifespan
Sensor 3: First, triplicate analysis of one level of solution groupings 1, 2, and
3 from Table 3-2 (12 results on 9 analyses), then triplicate analysis of six
finished drinking water samples (18 analyses), and lastly, triplicate analysis of
one level of solution groupings 1, 2, and 3 from Table 3-2 (12 results on 9
analyses). All samples were compared to reference methods.
Sensor 4: Same as Sensor 3 only with solution groupings 4 and 5 from Table
3-2. All samples were compared to reference methods.
Stage 2 -
Drinking,
Surface, and
"in-process"
Water
Sensors 3 and 4: Six finished drinking water samples (analysis performed
during Stage 1 Part 2)
Sensors 5 and 6: Two surface water samples and two samples collected within
the drinking water treatment process were analyzed in triplicate and compared
to standard reference methods. Analyzed by the WP870 both at the collection
location and after returning to the laboratory.
Stage 3 - Field
Operation
Qualitative evaluation of operational performance when the WP870 is in use
during field measurement scenarios
3.2.1 Stage 1 Laboratory Testing ofDI Water Samples
Stage 1 consisted of two parts. The first part focused on testing the accuracy and precision of the
sensors with respect to accepted laboratory reference methods. This was done by preparing test
solutions with water quality measurements that spanned the working range of the sensor. Table
3-2 describes the levels of each water quality parameter that were analyzed. For example, the pH
component of the WP870 was tested by preparing test solutions that covered the pH ranges from
acidic (pH 5.4), to neutral (pH 7), to alkaline (pH 10). Table 3-2 also shows groupings of the
water quality parameters that were evaluated by analyzing a single solution prepared for that
-------
purpose. The alkalinity, monochloramine, and pH measurements were each evaluated in unique
solutions, but the other parameters were grouped in solution. The table also includes the key
components and/or critical aspects of each solution's preparation. Each of these solutions was
analyzed in triplicate on each of two sensors in order to thoroughly study the accuracy and
precision of two different sensors installed on separate WP870 handheld units.
The second part of Stage 1 focused on testing the performance of the sensors over the vendor-
specified lifetime of the sensors (30 days or 50 water samples) by testing one concentration level
(the middle level) of the solutions given in Table 3-2, in triplicate twice, once near the start of the
sensor's useful lifetime and once near the end. Between those analysis times, six finished
drinking water samples were analyzed in order to challenge the sensor with more realistic
samples between tests with samples prepared in DI water. If the sensors were susceptible to
fouling due to analysis of drinking water samples, the second set of samples that had been
prepared in DI water would be expected to exhibit diminished accuracy or precision. Because of
the limitation of 50 samples per sensor, and because of the rigor of accuracy testing during the
first part of Stage 1, this part of Stage 1 was split between two sensors in order to maintain the
ability to perform triplicate analyses across the 30 day time period.
Table 3-2. Stage 1 Test Sample Information
Parameter
Grouping
1
2
3
4
5
Water Quality
Parameter
PH
Alkalinity
Hardness
Ammonia
Conductivity
Free Chlorine
Oxidation / Reduction
Potential (ORP)
Monochloramine
Levels
5.4,7, 10 (pH units)
22, 130, and240mg/L
CaCO3
17.5, 125, and 225 mg/L
CaCO3
0.1,0.8, 1.5 mg/L
100, 1100, 1700nS/cm
0.2, 1.2, 2. 2 mg/L
Test Sample Preparation
(all samples in DI water)
Citrate, Phosphate, and Borate Buffers
Anhydrous sodium bicarbonate in DI
water
Calcium chloride
Ammonium chloride
Sodium chloride
Sodium hypochlorite
Use the free chlorine solutions to generate a range of oxidation
reduction potentials.
0.2, 1.2, 2.2 mg/L
Addition of ammonium chloride to Sol. #4
with a 15 minute reaction time (pH>9)
mg/L - milligram per liter
|j,S/cm - microSiemens per centimeter
3.2.2 Stage 2 Laboratory and Field Testing of Drinking and Surface Water Samples
The second stage of this verification test focused on the performance of the WP870 when
analyzing samples of finished drinking water, drinking water within the treatment process, and
untreated surface water. Throughout the verification test (including the drinking water samples
analyzed during the second part of Stage 1), six finished drinking water, two "in-process" water,
and two source water samples were analyzed. The in-process and surface water samples were
analyzed by the WP870 at a booster station within the CDW distribution system and then
returned to the laboratory for reference analysis. In addition to the field measurements, all 10
samples were analyzed in the laboratory using the WP870. Each of these samples were analyzed
in triplicate by the WP870 and then compared to the reference method results from these same
samples.
-------
3.2.3 Stage 3 Remote Field Analysis
The third stage of this verification test evaluated the ease of using the WP870 during a field
water quality study with two collaborators. The ETV program collaborated with 1) personnel
from EPANERL who conducted a short-term field analysis campaign during September 2006
that consisted of measuring temperature, pH, and conductivity throughout Shayler's Run, a
stream that flows into the East Fork of the Little Miami River in southern Ohio and 2) personnel
from the Texas Commission on Environmental Quality (TCEQ) who conducted a similar
sampling campaign in western Texas on the Rio Grande River in May of 2007. These studies
were independent from the ETV test, but EPA NERL and TCEQ agreed to take the WP870 with
them and perform single analyses at some of the measurement locations included in their studies.
No grab samples were transported for reference analysis during this stage of the testing.
Therefore, the focus of this part of the test was the evaluation of the practical aspects of using the
WP870 under non-laboratory, field analysis conditions.
3.3 Laboratory Reference and Quality Control Samples
The WP870 was evaluated by comparing its results with standard reference measurements. The
following sections provide an overview of the applicable procedures, analyses, and methods.
3.3.1 Reference Methods
The standard laboratory methods used for the reference analyses are shown in Table 3-3. Also
included in the table are method detection limits and quality control (QC) measurement
tolerances. CDW and Battelle technical staff performed the analyses for each of the water quality
parameters. Any required instrumentation was calibrated as required by the reference method
and those calibration activities were documented in the verification records. The CDW provided
reference sample results within one day of the analysis. The monochloramine on the WP870 was
measured directly while the monochloramine reference measurement was an indirect
measurement based on the difference of total chlorine and free chlorine.
3.3.2 Reference Methods Quality Control Samples
As shown in Table 3-3, duplicate reference samples were collected and analyzed once daily
during the verification test. Also, laboratory blanks consisting of DI water were analyzed with
the same frequency. These blank samples were most important for the chlorine, ammonia, and
monochloramine analysis because these were the only parameters that needed confirmation by
the reference method of the lack of contamination. The other parameters produced a detectable
result in DI water, so a blank sample could not be evaluated in a similar way. For those
parameters, the performance evaluation (PE) audit confirmed the accuracy of the method and the
absence of contamination. Tolerances for the PE audit comparisons and duplicate measurements
had to be within the acceptable tolerances provided in Table 3-3 or corrective actions (as
described in section B5 of the Test/QA plan) would be taken.
-------
Table 3-3. Reference Methods
Parameter
Ammonia
Hardness (CaCO3)
Conductivity
Free Chlorine
Monochloramine
ORP
pH
Alkalinity
Method
Standard Method
(SM) 4500-NH33
SM 3500 - Ca- B4
SM25105
SM 4500-C1-G6
SM 4500-C1-G6
SM2580-B7
SM 4500-H+-B8
SM 2320-B9
Instrument/
Description
Hach SENSion 1
Electrode
EDTA titration
YSI Datasonde
Hach Colorimeter
Hach Colorimeter
Myron L Model 6P
YSI Datasonde
Sulfuric acid
titration
Method Detection
Limits
0.03 mg/L
0.5 mg/L
2 (iS/cm
0.01 mg/L as C12
0.01 mg/L as NH2C1
NA
NA
20 mg/L
Acceptable
Duplicate and PE
Tolerance (%D)a
25%
25%
25%
25%
25%
25%
±0.3 pH units
25%
NA - not applicable due to nature of that water quality parameter
a: %D defined in Section 4.1.1
3.4 Qualitative Evaluation Parameters
Operational factors such as general operation, data acquisition, set-up, demobilization,
consumables used, purchase and operational costs, and ease of use were evaluated based on
observations by Battelle, CDW, U.S. EPANERL, and TCEQ staff. A laboratory record book
was maintained at the host facility and was used to enter daily observations on these factors.
Qualitative observations were made in logbooks during the field analyses.
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed in accordance with the QMP for the AMS Center(5) and the
test/QA plan for this verification test.(1) QA/QC procedures and results are described below.
4.1 Audits
Three types of audits were performed during the verification test: a PE audit of the reference
methods, a technical systems audit (TSA) of the verification test procedures, and a data quality
audit. Audit procedures are described further below.
4.1.1 PE Audit
A PE audit was conducted to assess the quality of the reference measurements made in this
verification test. Each type of reference measurement was compared with a National Institute of
Standards and Technology (NIST)-traceable standard reference water sample or a standard that
was obtained independently from the standard used to calibrated the reference instrument. The
NIST-traceable standard reference water samples had certified values of alkalinity, chlorine,
conductivity, hardness, and pH that were unknown to the analyst. The PE audit for ammonia was
evaluated with a second source of ammonia, and that for ORP was performed with separately
obtained stocks of Light solution, a solution that generates an ORP of approximately 450
millivolts. These samples were analyzed in the same manner as the rest of the reference analyses
to independently confirm the accuracy of the reference measurements. As Table 4-1 shows, all
PE audit results were within the acceptable differences provided in Table 3-3. The percent
difference (%D) was calculated using the following equation:
%D= N~ flxlOO% (1)
where CR was the reference method result and CN the NIST value for each respective water
quality parameter.
Other QC data collected during this verification test were reference method duplicate analysis
results, which are also shown in Table 4-1. With the exception of one duplicate measure of
monochloramine, all parameters were within the differences defined in Table 3-3. No corrective
action was taken for the one monochloramine measurement (45.2% D) that was outside the
acceptable difference because the absolute difference between the concentrations measured in
those two duplicate samples was very small (0.07 mg/L) and the rest of the duplicate
measurements were well within the acceptable tolerance of 25 %D. In addition, it should also be
9
-------
noted that because pH units are measured on a logarithmic rather than linear scale, the quality
control metric for that parameter was the absolute unit rather than percent difference. The pH PE
audit was completed after one attempt that was within, but nearly outside of, the acceptable
range, therefore it was repeated.
Table 4-1. Performance Evaluation Audit and Reference Method Duplicate Results
Parameter(a)
Alkalinity (mg/L)
Ammonia
Chlorine (mg/L C12)
Conductivity ((iS/cm)
Hardness (mg/L CaCO3)
Monochloramine (mg/L)
ORP (millivolts)
PHb
PE Audit Results
Standard
Value
36.2
0.60
4.29
665
136
NA
476
7.05
Reference
Method
Result
35.0
0.56
4.38
656
137
NA
447
7.00
%D
-3.3%
-6.7%
2.0%
-1.4%
0.7%
NA
6.1%
0. 05 pH units
Reference Method
Duplicate Analysis
Average of
Absolute
Values of %D
2.51
3.08
2.80
0.42
3.14
6.39a
1.19
0.02 pH units
Range of %D
0.0 to 5.7
1.5 to 6.9
0.0 to 6.2
0.0 to 1.9
O.Oto 11.8
0.0 to 18.2a
0.0 to 5.1
0.00 to 0.04 pH units
Removed outlier of 45.2%D because absolute difference was only 0.07 mg/L. %D was driven by small average concentration.
b Repeated measurement since original nearly failed acceptance criteria.
NA - No reliable traceable standard solution or method to compare reference method result.
4.1.2 Technical Systems Audit
The Battelle Quality Manager performed a TSA during the test to ensure that the verification test
was performed in accordance with the AMS Center QMP, the test/QA plan, and published
reference methods. The TSA noted no adverse findings. A TSA report was prepared, and a copy
was distributed to the EPA AMS Center Quality Manager.
4.1.3 Data Quality Audit
At least 10% of the data acquired during the verification test were audited. The data was traced
from the initial acquisition, through reduction and statistical analysis, to final reporting to ensure
the integrity of the reported results. All calculations performed on the data undergoing the audit
were checked.
4.2 QA/QC Reporting
Each audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the ETV
AMS Center.(5) Once the audit reports were prepared, the Battelle Verification Test Coordinator
ensured that a response was provided for each adverse finding or potential problem and
implemented any necessary follow-up corrective action. The Battelle Quality Manager ensured
that follow-up corrective action was taken. The results of the TSA were submitted to the EPA.
4.3 Data Review
Records generated in the verification test received a one-over-one review before these records
were used to calculate, evaluate, or report verification results. Data were reviewed by a Battelle
10
-------
technical staff member involved in the verification test. The person performing the review added
his/her initials and the date to a hard copy of the record being reviewed.
11
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the quantitative performance factors listed in Section 3.1
are presented in this chapter. Qualitative observations were also used to evaluate verification test
data.
5.1 Accuracy
Throughout this verification test, results from the WP870 were compared to the results obtained
from analyses by the reference methods. The %D between these two results was calculated from
the following equation:
C - C
% D = m R x 100% (2)
^R
where CR is the result determined by the reference method and Cm is the result from the WP870
units. Ideally, if the WP870 unit and reference method measurements were the same, there
would be a percent difference of zero. For pH, which is measured on a logarithmic scale, and in
cases when the water quality parameter levels were near the detection limit, the absolute
difference from the reference measurement was used to evaluate accuracy.
5.2 Precision
The precision of the WP870 was evaluated by calculating the percent relative standard deviation
(RSD) of each set of the triplicate samples that were measured during the verification test. The
RSD is defined as the standard deviation of the results of the three replicates divided by the
average result of the three replicates. Because pH is measured on a logarithmic scale, the RSD
of pH was not calculated.
5.3 Inter-unit Reproducibility
The results obtained from two identical WP870 sensors were compared to assess inter-unit
reproducibility. For each sample analysis during this verification test (119 samples), the triplicate
results from each WP870 sensors were compared to evaluate whether the two WP870 sensors
were generating similar results. This was done by performing a paired t-test with the assumption
that the data from each WP870 sensors had equal variances. A probability of less than 0.05
indicated a significant difference between the two WP870 sensors. Results found to be
statistically different from the two units were noted, in terms of the separate readings and
absolute difference of the two units.
12
-------
Chapter 6
Test Results
As mentioned previously, this verification test was conducted in three stages that focused on
three different aspects of multi-parameter water sensors. Stage 1 focused on the accuracy and
precision of the WP870 when test samples were prepared in a DI water matrix as well as the
performance of the WP870 throughout the expected lifetime of a sensor (30 days or 50 samples).
Stage 2 focused on the accuracy and precision of water samples that were either raw surface
water, water within the process of being treated within the water treatment system, or finished
drinking water. Some of these samples were also analyzed at a field location as well as at the
laboratory in order to evaluate their performance in both locations. Six different sensors were
used throughout Stages 1 and 2. The results are given so it is clear what sensor is being used.
Stage 3 was a qualitative evaluation of the operational aspects of the WP870 when it was used
during two field analysis trips.
6.1 Accuracy and Precision
6.1.1 Stage 1 Laboratory Testing ofDI Water Samples
Table 6-1 shows the water quality parameter levels that were prepared in each solution, the
laboratory reference method result, the average result of the triplicate analyses by the WP870,
the percent difference (%D) (or absolute difference (D) for pH only) for each parameter level,
and the relative standard deviation for each set of replicate samples. All samples were analyzed
by two WP870 units, identified as Sensor 1 and Sensor 2. For each level of alkalinity (except the
lowest level of Sensor 2), conductivity, hardness, and ORP, the %Ds were all less than 10%.
The %D for ammonia ranged from -19.9% to -23.8% for the higher two concentration levels and
was 47.1% for the lowest concentration level. The absolute difference between the average
WP870 result for ammonia and the reference measurement was only 0.05 mg/L, therefore, the
rather large percent difference was driven in part by the small concentrations being considered.
For free chlorine, the %Ds generated by each sensor were -41.2% and -29.4% for the lowest
concentration sample, 11.5% and 11.8% for the middle concentration, and 23.6% and 26.8% for
the highest concentration sample. For all the concentration levels of monochloramine across
both sensors, the %Ds ranged from 12.7% to 28.4%. As mentioned previously, because pH is on
a logarithmic scale, the absolute difference between the average result and the reference
measurement was determined and used to evaluate the performance of the WP870 as a pH
sensor. During Stage 1, the WP870 differed from the reference method by -0.05 and -0.08 for
the pH 5.4 sample, 0.13 and 0.21 for the pH 7.0 sample, and -0.25 and -0.01 for the pH 10
sample. The RSD for each set of replicate samples is also given in Table 6-1. With the
exception of the lowest free chlorine concentration, every triplicate set of samples exhibited
13
-------
Table 6-1. Stage 1 Accuracy and Precision Results for the WP870
Water Quality
Parameter
Alkalinity
(mg/L CaCO3)
Ammonia
(mg/L NH3)
Conductivity
(uS/cm)
Free chlorine
(mg/L C12)
Hardness
(mg/L CaCO3)
Mono-
chloramine
(mg/L NH2C1)
ORP
(millivolts)
pH
Test
Level
22
130
238
0.10
0.80
1.50
100
1100
1700
0.20
1.20
2.20
17.5
125
225
0.20
1.20
2.20
a
5.40
7.00
10.00
Reference
25
128
238
0.10
0.81
1.64
106
1096
1540
0.17
1.13
1.98
18
128
234
0.21
1.14
1.91
551
702
694
5.34
6.88
9.87
Sensor 1
Avg.
26
119
225
0.15
0.63
1.32
106
1126
1582
0.10
1.26
2.45
18
138
247
0.24
1.33
2.35
500
683
688
5.29
7.01
9.62
%D or D
4.0%
-7.3%
-5.6%
47.1%
-23.0%
-19.7%
0.3%
2.7%
2.7%
-41.2%
11.5%
23.6%
0.0%
7.8%
5.7%
12.7%
17.0%
23.0%
-9.2%
-2.8%
-0.8%
-0.05
0.13
-0.25
%RSD
3.8%
1.3%
8.7%
0.0%
4.0%
1.2%
0.5%
0.3%
0.2%
26.5%
2.9%
2.5%
0.0%
1.9%
2.9%
2.4%
3.0%
1.1%
7.6%
5.5%
6.3%
n/a
n/a
n/a
Sensor 2
Avg.
20
122
243
0.15
0.62
1.31
112
1126
1548
0.12
1.26
2.51
17
139
256
0.24
1.36
2.45
510
684
705
5.26
7.09
9.86
%D or D
-20.0%
-4.7%
2.1%
47.1%
-23.8%
-19.9%
5.7%
2.7%
0.5%
-29.4%
11.8%
26.8%
-5.6%
8.6%
9.3%
15.9%
19.3%
28.4%
-7.5%
-2.6%
1.6%
-0.08
0.21
-0.01
%RSD
5.0%
3.0%
7.2%
0.0%
4.3%
0.9%
0.9%
0.1%
1.4%
22.0%
4.4%
3.8%
0.0%
3.3%
3.0%
6.3%
1.3%
3.5%
8.4%
2.4%
4.0%
n/a
n/a
n/a
a ORP test levels not set, but allowed to vary with free chlorine concentration.
RSDs that were below 10%, and in most cases less than 5%. Presumably, the increased
variability for the lowest concentration of free chlorine is due to it being closer to the detection
limit of the WP870.
The second part of Stage 1 focused on the performance of the sensors over their 30 day (or 50
sample) lifespan while attempting to simulate a measurement scenario in which the WP870 may
be used for by a water utility or other end user. To do this, one concentration level of each water
quality parameter (solutions prepared in DI water) was analyzed near the start of the sensor's
lifetime, then six finished drinking water samples were analyzed to simulate how the sensor may
actually be used, and then the same concentration level of each water quality parameter (again
prepared in DI water) was analyzed again. These analyses covered approximately 30 days and
utilized almost all of the 50 samples that each sensor is able to measure.
Table 6-2 presents the %D from the reference method both before and after the analysis of six
finished drinking water samples. Instead of comparing the absolute results from before the
drinking water samples were analyzed, the %Ds were used for the basis of comparison because
the solutions were prepared once again prior to the second analysis and the reference analyses
were also performed again. Therefore, the appropriate comparison is with respect to the
14
-------
reference method result at that time and not the absolute water quality parameter measurement
performed at least several days prior using a different solution. The %Ds for alkalinity and ORP
changed by 20.4% and 15.3%, respectively, while the other water quality parameters did not
change any more than 10%. The results for alkalinity and ORP both indicated a smaller
difference after the drinking water analyses than before. The pH result also improved from a
difference of 0.18 from the reference method to a difference of just 0.04 during the second
analysis. In addition to evaluating the %Ds for each parameter, the RSDs can also be evaluated
in this way as results might be expected to become less repeatable as the sensor nears the end of
its lifespan. However, there was little consistent change in %RSDs from before to after the
analysis of the finished drinking water. In most cases, the %RSD either did not change more
than a few percent or decreased somewhat during the second analysis.
Table 6-2. Stage 1 Performance Across Sensor Lifespan
Water Quality
Parameter
Alkalinity
(mg/L CaCO3)
Ammonia
(mg/L NH3)
Conductivity
(uS/cm)
Free chlorine
(mg/L C12)
Hardness
(mg/L CaCO3)
Monochloramine
(mg/L NH2C1)
ORP
(millivolts)
pH
Sensor
3
3
4
4
3
4
4
3
Test Level
130
0.8
1100
1.2
125
1.2
700
7.01
Description
Pre-DW
Post-DW
Pre-DW
Post-DW
Pre-DW
Post-DW
Pre-DW
Post-DW
Pre-DW
Post-DW
Pre-DW
Post-DW
Pre-DW
Post-DW
Pre-DW
Post-DW
%D or D
-11.7%
8.7%
-20.6%
-19.6%
-0.3%
2.8%
20.9%
19.7%
-1.3%
1.6%
11.1%
19.3%
-11.4%
3.9%
0.18
0.04
%RSD
9.3%
3.6%
5.9%
4.7%
0.2%
0.9%
4.3%
1.5%
13.3%
0.0%
1.6%
5.0%
4.6%
6.8%
n/a
n/a
6.1.2 Stage 2 Results for Drinking and Source Water Samples
Table 6-3 shows the results for the finished drinking water samples that were referred to in the
second part of Stage 1. The six drinking water samples were collected at various locations
within the CDW distribution system. The amount of ammonia in these samples was below the
detectable level of the reference method (0.05 mg/L) so the results for ammonia are not shown.
Conductivity and ORP were the most accurate with average %Ds of mostly less than 5%.
Alkalinity generated average %Ds that ranged from -19.5% to 8.0% and hardness resulted in
average %Ds that ranged from -17.0% to 2.3%. The pH measurements were always within 0.22
pH units of that of the reference measurement. The Columbus, Ohio water system, from which
these samples were collected, is not a chloraminated system and therefore, there is only a small
concentration of monochloramines in the water (-0.2 mg/L). Because of this, even small
deviations from the reference concentration can cause %D to become artificially high due to the
small reference concentration (see Equation 2). Therefore, the difference between the
monochloramine concentration and the reference result is reported here as absolute concentration
15
-------
Table 6-3. Stage 2 Finished Drinking Water (DW) Results
Water Quality
Parameter
Alkalinity
(mg/L CaCO3)
Conductivity
(uS/cm)
Free chlorine
(mg/L C12)
Hardness
(mg/L CaCO3)
Monochloramine
(mg/L NH2C1)
ORP
(millivolts)
pH
Test
Matrix
DW1
DW2
DW3
DW4
DW5
DW6
DW1
DW2
DW3
DW4
DW5
DW6
DW1
DW2
DW3
DW4
DW5
DW6
DW1
DW2
DW3
DW4
DW5
DW6
DW1
DW2
DW3
DW4
DW5
DW6
DW1
DW2
DW3
DW4
DW5
DW6
DW1
DW2
DW3
DW4
DW5
DW6
Reference
29
48
45
43.5
50.5
35
257
455
481
479
501
335
1.14
1.23
1.47
1.19
0.77
0.92
67
106
101
102
75
77
0.16
0.26
0.22
0.27
0.15
0.26
604
674
676
685
622
646
7.68
7.62
7.75
7.68
7.7
7.73
Sensor 3
Avg.
31
46
39
38
47
31
260
451
481
483
498
336
0.98
1.00
1.05
0.93
0.55
0.74
63
88
95
92
65
71
0.06
0.19
0.36
0.22
0.22
0.19
603
669
692
646
654
670
7.70
7.69
7.80
7.90
7.80
7.82
%D or D
8.0%
-4.9%
-13.3%
-12.6%
-6.9%
-11.4%
1.4%
-0.9%
-0.1%
0.8%
-0.6%
0.2%
-14.3%
-18.7%
-28.3%
-22.1%
-28.1%
-19.9%
-6.0%
-17.0%
-5.6%
-10.1%
-13.8%
-8.2%
-0.10
-0.07
0.14
-0.05
0.07
-0.07
-0.1%
-0.8%
2.5%
-5.7%
5.1%
3.7%
0.02
0.07
0.05
0.22
0.10
0.09
%RSD
24.0%
3.3%
0.0%
2.6%
6.4%
3.2%
0.4%
0.4%
0.5%
0.2%
0.2%
0.2%
2.6%
5.0%
3.1%
3.3%
12.0%
6.7%
3.2%
4.1%
2.2%
5.4%
3.6%
2.2%
63.8%
21.7%
18.2%
25.4%
41.6%
22.9%
8.6%
1.7%
1.3%
5.0%
1.2%
0.8%
n/a
n/a
n/a
n/a
n/a
n/a
Sensor 4
Avg.
25
40
37
37
41
29
259
458
488
486
513
345
0.96
0.98
1.00
0.90
0.62
0.80
63
90
103
97
66
71
0.12
0.25
0.44
0.32
0.18
0.18
655
682
694
665
656
676
7.78
8
7.79
7.85
7.83
7.84
%D or D
-13.8%
-16.7%
-18.5%
-14.9%
-19.5%
-18.1%
1.0%
0.7%
1.5%
1.5%
2.4%
2.9%
-15.8%
-20.3%
-32.2%
-24.4%
-19.0%
-12.7%
-6.0%
-14.8%
2.3%
-4.6%
-11.6%
-7.4%
-0.04
-0.01
0.22
0.05
0.03
-0.08
8.4%
1.1%
2.7%
-2.9%
5.5%
4.6%
0.10
0.10
0.04
0.17
0.13
0.11
%RSD
10.6%
4.3%
1.6%
7.2%
3.8%
4.0%
0.7%
0.7%
0.9%
1.0%
0.4%
0.2%
3.8%
3.7%
3.2%
5.1%
9.8%
2.6%
0.0%
10.0%
1.5%
11.3%
3.1%
2.1%
44.7%
18.7%
11.6%
17.4%
31.0%
31.5%
3.4%
0.5%
1.0%
2.7%
1.4%
0.6%
n/a
n/a
n/a
n/a
n/a
n/a
units rather than a %D. All the monochloramine measurements were within 0.22 mg/L of the
reference measurement. Lastly, the free chlorine results were unique from the others because
they consistently exhibited a negative %D that ranged from -14.3% to -32.2%. As for the
precision of these finished drinking water results, with the exception of the monochloramine
results which were impacted by the low concentration, almost all of the %RSDs were below
16
-------
Table 6-4. Surface (SW) and "In-Process" (IPW) Drinking Water - Field and Laboratory
Results
Water Quality
Parameter
Alkalinity
(mg/L CaCO3)
Conductivity
(uS/cm)
Hardness
(mg/L CaCO3)
ORP
(millivolts)
pH
Test Matrix
IPW1 (Field)
IPW1 (Lab)
IPW2 (Field)
IPW2 (Lab)
SW1 (Field)
SW1 (Lab)
SW2 (Field)
SW2 (Lab)
IPW1 (Field)
IPW1 (Lab)
IPW2 (Field)
IPW2 (Lab)
SW1 (Field)
SW1 (Lab)
SW2 (Field)
SW2 (Lab)
IPW1 (Field)
IPW1 (Lab)
IPW2 (Field)
IPW2 (Lab)
SW1 (Field)
SW1 (Lab)
SW2 (Field)
SW2 (Lab)
IPW1 (Field)
IPW1 (Lab)
IPW2 (Field)
IPW2 (Lab)
SW1 (Field)
SW1 (Lab)
SW2 (Field)
SW2 (Lab)
IPW1 (Field)
IPW1 (Lab)
IPW2 (Field)
IPW2 (Lab)
SW1 (Field)
SW1 (Lab)
SW2 (Field)
SW2 (Lab)
Reference"
61
61
116
116
190
190
169
169
558
558
651
651
650
650
636
636
89
89
162
162
193
193
170
170
301
301
352
352
250
250
312
312
8.67
8.67
7.07
7.07
7.77
7.77
7.65
7.65
Sensor 5
avg
49
50
92
85
197
175
130
135
566
565
674
664
662
663
650
643
80
78
143
146
168
169
145
146
77
65
157
157
90
85
223
145
8.88
8.92
6.83
6.90
7.71
7.72
7.77
7.76
%D or D
-19.1%
-18.6%
-20.4%
-26.4%
3.5%
-7.7%
-23.3%
-20.1%
1.4%
1.3%
3.6%
2.0%
1.8%
2.1%
2.1%
1.2%
-10.5%
-12.4%
-11.5%
-9.9%
-13.1%
-12.6%
-14.9%
-13.9%
-74.4%
-78.5%
-55.5%
-55.5%
-64.1%
-65.9%
-28.5%
-53.5%
0.21
0.25
-0.24
-0.17
-0.06
-0.05
0.12
0.11
%RSD
12.9%
7.6%
13.4%
2.4%
7.2%
6.1%
6.2%
10.9%
0.2%
0.4%
0.2%
0.3%
0.2%
0.2%
0.5%
0.4%
0.7%
1.3%
2.2%
1.8%
3.6%
1.2%
3.4%
0.8%
3.4%
7.1%
7.1%
13.5%
5.0%
2.4%
34.5%
14.1%
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Sensor 6
avg
47
49
113
108
184
170
164
145
576
572
685
679
683
675
671
665
78
76
152
153
189
159
155
159
117
123
207
240
157
160
219
209
8.88
8.88
6.82
6.93
7.71
7.76
7.76
7.81
%D or D
-22.4%
-20.2%
-2.3%
-7.2%
-3.2%
-10.7%
-3.0%
-14.2%
3.3%
2.4%
5.3%
4.4%
5.0%
3.8%
5.5%
4.5%
-12.7%
-14.6%
-6.2%
-5.6%
-2.1%
-17.4%
-8.6%
-6.7%
-61.1%
-59.0%
-41.3%
-31.9%
-37.2%
-36.1%
-29.8%
-32.9%
0.21
0.21
-0.25
-0.14
-0.06
-0.01
0.11
0.16
%RSD
3.2%
4.3%
2.5%
10.0%
2.4%
0.3%
2.7%
3.6%
1.7%
1.4%
1.0%
1.3%
0.7%
0.4%
0.3%
0.5%
4.1%
7.0%
1.1%
4.1%
5.7%
4.8%
14.0%
5.6%
3.1%
15.1%
10.9%
10.3%
6.6%
4.2%
2.4%
27.5%
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
The reference value is from a single measurement for each water sample.
10%. Table 6-4 shows the results for the surface water as well as water samples collected from
within the water treatment process. Both surface water samples (SW1 and SW2) were collected
at the raw water (Scioto River) intake at a CDW treatment plant. The in-process waters (IPW 1
and IPW2) were taken from within that plant. IPW1 was collected at the end of the
recarbonation process and IPW2 was collected at the end of the coagulation process. Each of
17
-------
these samples was analyzed at a CDW booster station that was near where the samples were
collected as well as returned to the laboratory for analysis by the WP870 and the laboratory
reference method in order to compare the performance of the WP 870 when the measurements
were made at a field location and in the laboratory. The reference measurement results for free
chlorine, monochloramine, and ammonia were below the detection limits so those water quality
parameters were not included in this data set. Conductivity was most accurate with average %Ds
that ranged from 1.2% to 5.5%. Hardness generated average %Ds that ranged from -17.4% to
3.6% and alkalinity resulted in average %Ds that ranged from -26.4% to 3.5%. The pH
measurements were always within 0.25 pH units of that of the reference measurement. The ORP
results for all of the surface water samples and the in-process water samples had relatively high
average %Ds from the reference method compared with all of the other water quality parameters
and all of the other water samples. The %Ds ranged from -78.5% to -28.5%. The reason for
these relatively large differences was not clear, but it seems likely that some constituent of the
water sample that is removed during the water treatment process inhibited the ORP sensing
component of the WP870. The precision of the WP870 in these matrices as expressed in %RSD
for each set of replicates was 10% or less in 70 out of the 80 sets that were analyzed.
The difference in the average %D between the laboratory and field measurements was small in
most cases. In only four instances did the difference between average %Ds exceed 10% and in
each of those instances, the field result was closer to the reference measurement result than was
the laboratory result. Similarly, for pH, the average results determined at the laboratory did not
differ from those determined in the field by more than 0.11 pH units.
6.2 Inter-unit Reproducibility
Throughout the first part of Stage 1 and all of Stage 2 of this verification test, two WP870
sensors were used to analyze each test sample. Two types of data were used to compare the two
units equipped with separate sensors that have a lifespan of either 50 samples or 30 days. To do
this, the data were evaluated in two ways. First, a paired t-test was performed on each set of
replicate results to determine if there was a statistical difference between each of the sets of
replicate data. However, when t-tests are applied to very repeatable data (as was the case here
for some parameters), it is possible for extremely small differences to be considered significant.
Therefore, to provide some perspective to the t-test results, the absolute difference between the
averages from each unit was also reported. For example, in one instance during Stage 2, the
conductivity results for Sensors 3 and 4 were reported as 336 |iS/cm and 345 |iS/cm,
respectively, an absolute difference of only 9 jiS/cm. Because of the precision of this particular
replicate measurement, the results were reported as significantly different. Table 6-5 gives the
results for each of the samples for which the pairs of sensors were determined to generate
significantly different results. Overall, out of 106 pairs of triplicate results using separate units,
only 19 pairs were determined to be significantly different from one another by a paired t-test.
The reference method result, the average result from each sensor, and the absolute difference
between the average results from the two sensors are shown in the table. Replicate results for
samples with non-detectable reference measurement results were not included in this evaluation.
18
-------
Table 6-5. Surface with Significantly Different Results Between Sensors
Stage
Stage 1
(Sensors
1&2)
Stage 2
(Sensors
3&4)
Stage 2
(Sensors
5&6)
Water Quality
Parameter
pH
Alkalinity
(mg/L CaCO3)
Conductivity
(uS/cm)
Free chlorine
(mg/L C12)
ORP (low)
Alkalinity
(mg/L CaCO3)
Conductivity
(uS/cm)
Alkalinity
Conductivity
(uS/cm)
ORP
(millivolts)
pH
Matrix
DI Water
DI Water
DI Water
DI Water
DI Water
DI Water
DI Water
DW2
DW3
DW5
DW5
DW6
SW2 (Field)
SW1 (Field)
SW1 (Lab)
SW2 (Field)
SW2 (Lab)
IPW1 (Field)
IPW1 (Lab)
IPW2 (Field)
IPW2 (Lab)
SW1 (Field)
SW1 (Lab)
SW1 (Lab)
SW2 (Field)
SW2 (Lab)
Reference"
5.34
6.88
9.87
25
106
0.17
551
48
45
50.5
501
335
169
650
650
636
636
301
301
352
352
250
250
7.77
7.65
7.65
Sensor 1,3,
or 5 Avg.
5.29
7.01
9.62
26
106
0.10
500
46
39
47
498
336
130
662
663
650
643
77
65
157
157
90
85
7.72
7.77
7.76
Sensor 2,4,
or 6 Avg.
5.26
7.09
9.86
20
112
0.12
510
40
37
41
513
345
164
683
675
671
665
117
123
207
240
157
160
7.76
7.76
7.81
Absolute
Difference
0.03
0.08
0.25
6
6
0.02
10
6
2
6
15
9
34
21
12
21
22
40
58
50
83
67
75
0.04
0.01
0.05
The reference value is from a single measurement for each water sample.
6.3 Operational Factors
The verification staff found that the WP870 was easy to use both in the laboratory setting, where
most of the quantitative results were collected, and in the various field environments. The
WP870 procedure for calibration as well as measurement of samples includes the addition of
either calibration solutions or water samples to the sample tube, containing approximately 7-8
mL, attached to the handheld unit. All functions of the WP870 were controlled from the menus
displayed on the handheld unit. Full calibrations of the WP870, which took approximately 15-20
minutes, were required to be performed on a weekly basis or whenever a new sensor was
installed into the handheld unit. Upon selection of the calibration option on the menu, text
displayed on the screen guided the operators through the calibration process. The instructions
detailed which calibration and rinse solutions needed to be added to the sample reservoir and the
amount of time that should be allowed for each solution. The calibration solutions were
provided by Sensicore in disposable containers and were clearly labeled so that the on-screen
instructions directing the use of specific solutions were easily understood.
19
-------
The sensor was easily installed into the handheld unit by unscrewing the top of the sample tube
and inserting the sensor so that the electrical leads matched up with those on the handheld unit.
It is important to note that once a sensor is installed in the handheld unit and calibrated for the
first time (therefore hydrated) it is required to remain hydrated. This is done by filling the
sample tube with a storage solution provided by Sensicore between uses and 24 hours before a
sensor's first use. Each sensor is only able to be used for 50 sample analyses and within 30 days
after its first calibration. That total limit of samples does not include the calibration analyses that
are performed. The software will not allow any analyses to be performed after the 50 sample
maximum has been reached and the number of samples analyzed on a sensor is displayed
following each sample analysis. Also, the software will not allow any samples to be analyzed
after the 30 day time limit.
The WP870 also required daily calibrations (called "quick calibrations" in the software), that
took approximately 5 minutes, to ensure that the sensor was maintaining an appropriate
calibration. Similar to the full calibration, the quick calibration was guided by on-screen
instructions that were very easy to understand. For sample analysis, the operators selected the
measurement option and the menus guided the analysis of water samples in an identical fashion.
For each sample, approximately 6 mL of the water sample was added to the sample tube and then
3 drops of 3 different reagents (provided by Sensicore) were added to the sample tube throughout
the measurement cycle. Following the analysis of a sample, the operators were prompted for a
series of sample identification information in order to link the collected data with the appropriate
sample. The full analysis process took approximately 5 minutes for each sample analyzed.
Following the measurement of samples, the data (comma-delimited) were exported from the
handheld unit to a personal computer using a USB cable and software provided by Sensicore.
The data were then transferred to a spreadsheet for analysis.
Stage 3 of the verification test focused on the evaluation of the practical aspects of operating the
WP870 in a non-laboratory, field environment. As described in Section 3.2.3, WP870 units were
used on water sampling and analysis field studies by the TCEQ and EPA NERL. In addition,
Stage 2 included some samples that were analyzed near the surface water intake or within a
water treatment plant. The operators who completed this aspect of the testing documented
various aspects of the WP870 that stood out to them.
During the TCEQ sampling campaign, the WP870 battery was fully charged before leaving on
the trip and measurements were made at nine different locations over seven days. The low
battery indicator did not appear until the seventh day, which was three days longer than
Sensicore had informed the TCEQ to expect. Operators noted that both the calibration and
sample measurement screens on the handheld unit walked them right through the process. The
screens were easy to read in an outdoor environment and the on-screen directions were very
straightforward. Operators noted that the only thing that takes a bit of getting used to is setting
up the users and sample locations ahead of time using the software, so instead of taking some
time to learn, they just recorded that information in a sample log book and linked it with a
default sample number given by the WP870.
The EPA NERL operators reported that the carrying case was adequate in size to hold everything
needed for calibration and sample analysis and was relatively lightweight and easy to carry
across difficult terrain. They noted that, in general, the technology was easy to use in the field,
but would suggest that calibration would be more efficiently done in the lab so as to not waste
time in the field if on a day-long trip. The operators said that the WP870 was easily operated on
20
-------
the ground (e.g., on sloped banks along streams or uneven surfaces of streamside boulders and
bedrock) with only approximately one square foot required. Overall, both the TCEQ and EPA
NERL operators agreed that one drawback of the WP870 is the limited number of samples that
each sensor can measure before disposal.
Accouni USEPA-HERLCIN B mi HcCouNTS Q a w HcroiM e> LOCOUT
VtSUAUUTIDN ALCRTG RtPOflTING DATA ADMHrlSHHTtON ACCOUNT AOMIN1ETRATIDN HUP
Figure 6-1 Screen Shot of EPA NERL WP870 Data on WaterNOW Software
The EPA NERL operators worked with Sensicore to load their data onto Sensicore's WaterNOW
software which is an online and secure data service utilizing 128-bit data encryption that helps
the user understand their data through unique visualization and comparison tools. The sample
results from 80 locations were uploaded with the latitudinal and longitudinal coordinates to the
NERL online account. The software provides a graphical representation of selected parameters,
with a variety of contouring tools (area, pipeline, location) available to study the relationships
between locations and sample levels. Figure 6-1 shows a map of EPANERL's conductivity data.
In this example visualization, the sampling locations are shown with triangles and then arrows
indicate locations that are above a selected level (in this case, a conductivity measurement of 600
|iS/cm) that is entered by the user. Different sized arrows reflect the number of occurrences that
the selected level was exceeded at a location. The inset graphic shows that the rest of the sample
data is available by clicking on the location. This software tool was not rigorously evaluated
during this ETV test, but was utilized by the EPA NERL staff who found it to have a lot of
potential as a data evaluation tool for water quality data from several locations.
21
-------
Chapter 7
Performance Summary
The table below summarizes the results from the ETV testing of the WP870. The range of
accuracy results are given along with summaries of other verified performance parameters.
Water Quality
(WQ) Parameter
Alkalinity
Ammonia
Conductivity
Free Chlorine
Hardness
Monochloramine
ORP
PH
Overall Precision
Inter-unit
Reproducibility
Field Portability
Operational
Factors
Stage 1 WQ Levels
(test samples prepared
in DI water)
22, 130, and 240 mg/L CaCO3
0.1,0.8, 1.5 mg/L
100, 1100, 1700uS/cm
0.2, 1.2, 2.2 mg/L
17.5, 125, and 225 mg/L
0.2, 1.2, 2.2 mg/L
550 and 700 millivolts
5.4, 7,10 (pH units)
Stage 1 Accuracy -
%D from Ref.
-20 to 4.0
-23. 8 to 47.1
0.3 to 5.7
-41.2 to 26.8
-5.6 to 9.3
12.7 to 28.4
-9.2 to 1.6
-0.25 to 0.2 l(pH units)
Stage 2 Accuracy -
%D from Ref. (drinking,
surface, and "in-process" water)
-26.4 to 8.0
Ref. result below detection limit
-0.9 to 5.5
-32.2 to -12.7
-17.4 to 2.3
<0.22 mg/L from reference
-5.7 to 8.4 (DW)
-78.5 to -28.5 (SW and IPW)
-0.25 to 0.25 (pH units)
Excluding the monochloramine results for the DW, out of 216 triplicate measurements, 16
(7.4%) had %RSDs of greater than 10%.
Out of 106 pairs of triplicate results using separate units, 19 pairs were determined to be
significantly different from one another by a paired t-test. It seems most of these differences
were relatively small, driven mostly by extremely small variability
The difference between the average %D between the laboratory and field measurements in
Stage 2 was small in most cases. In only four instances did the difference between average
%Ds exceed 10% and in each of those occurrences, the field results were closer to the reference
measurement result than the laboratory result. Similarly, for pH, the results determined at the
laboratory did not differ by more than a pH of 0. 1 1 .
The verification staff found that the WP870 was easy to use both in the laboratory setting,
where most of the quantitative results were collected, and in the various field environments in
which the WP870 was used. The WP870 procedure for calibration as tube as measurement of
samples includes the addition of either calibration solutions or water samples to the sample tube
attached to the handheld unit. Overall, operators from both TCEQ and U.S. EPA NERL who
used the WP870 for field analysis considered it to be an easy to use instrument. However, the
operators also noted that the instrument is limited in that each sensor can analyze 50 samples
over 30 days.
22
-------
Chapter 8
References
1. Test/QA Plan for Test/QA Plan for Verification of Multi-parameter Water Sensors, Battelle,
Columbus, Ohio, April 4, 2007.
2. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version 6.0,
U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio,
December 2005.
3. American Public Health Association, et al., SM 4500-NH3 D Ammonia-Selective Electrode
Method in Standard Methods for the Examination of Water and Wastewater. 21st Edition,
Washington, D.C., 2005.
4. American Public Health Association, et al., SM-3500-Ca-B Calcium (Titrimetric, EDTA) in
Standard Methods for the Examination of Water and Wastewater. 21st Edition, Washington,
D.C., 2005.
5. American Public Health Association, et al., SM 2510 Conductivity in Standard Methods for
the Examination of Water and Wastewater. 21st Edition, Washington, D.C., 2005.
6. American Public Health Association, et al., SM 4500-G Residual Chlorine in Standard
Methods for the Examination of Water and Wastewater. 21st Edition, Washington, D.C.,
2005.
7. American Public Health Association, et al., SM 2580-B Electrochemical Potential in
Standard Methods for the Examination of Water and Wastewater. 21st Edition, Washington,
D.C., 2005.
8. American Public Health Association, et al., SM-4500- H+-B, pH in Standard Methods for the
Examination of Water and Wastewater. 21st Edition, Washington, D.C., 2005.
9. American Public Health Association, et al., SM-2320-B, Alkalinity in Standard Methods for
the Examination of Water and Wastewater. 21st Edition, Washington, D.C., 2005.
23
-------
Appendix
Sensicore Comment on WP870 Improvements
In an entirely separate effort, but ongoing simultaneously to the ETV test, Sensicore was
working to improve the Sensicore WP870. The following is a description of some of the
improvements that have been made. This information has not been independently verified.
Affect of Cleaning (Activation) Cycle on Precision and Accuracy of Free Chlorine and
Monochloramine Results on the WPS 70
Free available chlorine and monochloramine species are reduced at the surface of a platinum
array by applying a known potential difference, relative to the Ag/AgCl reference electrode
located on the WP870 sensor. Current generated in this process is proportional to the
concentration of chlorine species present. Calcium, magnesium and other dissolved species may
deposit on the array through the normal use of the instrument during repeated measurement
cycles. The calibration cycle of the WP870 has a built in activation cycle which uses the rinse
and conditioner solution of the sensor kit to electrochemically clean the array surfaces and leave
the array in a know state for calibration and measurements.
The activation cycle was modified by including the use of the sample buffer solution (already
contained in the sensor kit of the unit) to drop the pH to a value which more efficiently promotes
the removal of adverse deposits. Using the new activation cycle, the results of a single study on
free chlorine performed on laboratory prepared test solutions showed that the difference from the
reference method was 5.6% + 8.2% (N=48) compared with -7.1 + 15% (N=104) previously. The
most notable improvement was in the reproducibility. Contact Sensicore about details of this
and other possible improvements that may have been implemented since this testing was
completed.
24
------- |