Batreiie
lite. Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Test/QA Plan for
Verification of
Multi-Parameter Water Sensors
ETV ETV ET
-------
TEST/QA PLAN
for
Verification of
Multi-Parameter Water Sensors
April 4, 2007
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Verification of Multi-Parameter Water Sensors
Test/QA Plan
Page 2 of 29
Version 1
April 4, 2007
A2 TABLE OF CONTENTS
Section Page
A PROJECT MANAGEMENT
Al Title Page 1
A2 Table of Contents 2
A3 Distribution List 5
A4 Verification Test Organization 6
A5 Background 12
A6 Verification Test Description and Schedule 12
A7 Quality Objectives 13
A8 Special Training/Certification 13
A9 Documentation and Records 13
B MEASUREMENT AND DATA ACQUISITION
Bl Experimental Design 14
B2 Sample Collection 19
B3 Sample Handling and Custody Requirements 20
B4 Laboratory Reference Methods 20
B5 Quality Control Audits and Requirements 20
B6 Instrument/Equipment Testing, Inspection, and Maintenance 21
B7 Instrument Calibration and Frequency 22
B8 Inspection/Acceptance of Supplies and Consumables 22
B9 Non-Direct Measurements 22
BIO Data Management 23
C AS SES SMENT AND OVERSIGHT
Cl Assessments and Response Actions 25
C2 Reports to Management 26
D DATA VALIDATION AND USABILITY
Dl Data Review, Validation, and Verification Requirements 28
D2 Validation and Verification Methods 28
D3 Reconciliation with User Requirements 28
E REFERENCES 29
-------
Verification of Multi-Parameter Water Sensors
Test/QA Plan
Page 3 of 29
Version 1
April 4, 2007
Page
List of Figures
Figure 1 Organization Chart.
List of Tables
Table 1 Stage 2 Test Sample Information 16
Table 2 Summary of Test Sample Analysis 17
Table 3 Reference Methods 21
Table 4 Summary of Data Recording Process 24
-------
Verification of Multi-Parameter Water Sensors
Test/QA Plan
Page 4 of 29
Version 1
April 4, 2007
ETV Advanced Monitoring Systems Center
Test/QA Plan for Verification
of Multi-Parameter Water Sensors
April 4, 2007
APPROVAL:
Name
Company
Date
-------
A3 DISTRIBUTION LIST
Uwe Michalak
Sensicore, Inc.
745 Phoenix Drive
Ann Arbor, MI 48108
Jeff Kauffman
City of Columbus Division of Power and
Water (CDOPW)
910 Dublin Road
Columbus, OH 43215-9052
Ken Fritz
USEPA Facilities
26 West Martin Luther King Drive
Mail Code: 642
Cincinnati, OH 45268
Verification of Multi-Parameter Water Sensors
Test/QA Plan
Page 5 of 29
Version 1
April 4, 2007
Elizabeth A. Betz
U.S. Environmental Protection Agency -
HEASD
National Exposure Research Laboratory
E205-01 EPAMailroom
Research Triangle Park, NC 27711
Robert Fuerst
U.S. Environmental Protection Agency -
HEASD
National Exposure Research Laboratory
D205-05 EPA Mailroom
Research Triangle Park, NC 27711
Amy Dindal
Ryan James
Zachary Willenberg
Battelle
505 King Ave.
Columbus, OH 43201
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 6 of 29
SECTION A
PROJECT MANAGEMENT
A4 VERIFICATION TEST ORGANIZATION
The verification test will be conducted under the auspices of the U.S. Environmental
Protection Agency (EPA) through the Environmental Technology Verification (ETV) Program. It
will be performed by Battelle, which is managing the ETV Advanced Monitoring Systems (AMS)
Center through a cooperative agreement with EPA. The scope of the AMS Center covers
verification of monitoring technologies for contaminants and natural species in air, water, and
soil.
The day-to-day operations of this verification test will be coordinated and supervised by
Battelle personnel, with the participation of the vendor(s) who will be having the performance of
their multi-parameter water sensors verified. There will be a laboratory and field component to
this verification test. The laboratory testing will occur at the City of Columbus, Ohio Division of
Power and Water (CDOPW). Staff from the CDOPW will participate in this test by preparing
water samples with various levels of water quality parameters specified in this document,
collecting drinking and surface water samples to be used as test samples, analyzing verification
test samples with the technologies being verified, and conducting the reference analyses. The
field component of testing will be performed by staff from the EPA National Exposure Research
Laboratory (NERL) in Cincinnati, OH during an unrelated project which includes real-time water
quality measurements of surface water in southern Ohio. Vendor representatives will operate
their respective technologies throughout the test unless they provide training and give written
consent for the verification staff to carry out these activities. Quality assurance (QA) oversight
will be provided by the Battelle Quality Manager and the EPA AMS Center Quality Officer at
their discretion. The organization chart in Figure 1 identifies the responsibilities of the
organizations and individuals associated with the verification test. Roles and responsibilities are
defined further below.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 7 of 29
Battelle
Management
I
Amy Dindal
Battelle AMS
Center Manager
Zachary Willenberg
Battelle AMS Center
Quality Manager
Robert Fuerst
EPA AMS
Project Officer
Amy Dindal
Verification
Testing Leader
Columbus
Division of
Power and Water
(CDOPW), EPA
NERL, TCEQ
Elizabeth Betz
EPA AMS Center
Quality Assurance
Manager
Ryan James
Verification Test
Coordinator
Vendor
Representatives
Battelle
Technical Staff
Figure 1. Organization Chart
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 8 of 29
A4.1 Battelle
Dr. Ryan James is the AMS Center Verification Test Coordinator. In this role, Dr. James
will have overall responsibility for ensuring that the technical, schedule, and cost goals
established for the verification test are met. Specifically, he will:
• Assemble a team of qualified technical staff to conduct the verification test.
• Direct the team (Battelle, EPA, and CDOPW staff) performing the verification test in
accordance with the test/QA plan.
• Prepare the draft and final test/QA plan, verification reports, and verification
statements.
• Revise the draft test/QA plan, verification reports, and verification statements in
response to reviewers' comments.
• Respond to any issues raised in assessment reports and audits, including instituting
corrective action as necessary.
• Serve as the primary point of contact for vendor representatives.
• Coordinate distribution of the final test/QA plan, verification reports, and statements.
• Establish a budget for the verification test and manage staff to ensure the budget is not
exceeded.
• Ensure that confidentiality of sensitive vendor information is maintained.
Ms. Amy Dindal is a Verification Testing Leader for the AMS Center and is also
Battelle's manager for the AMS Center. Ms. Dindal will provide technical guidance and oversee
verification testing. Specifically, she will:
• Support Dr. James in preparing the test/QA plan and organizing the testing.
• Review the draft and final test/QA plan.
• Review the draft and final verification reports and verification statements.
• Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test. Ensure that confidentiality of sensitive vendor information is
maintained.
• Support Dr. James in responding to any issues raised in assessment reports and audits.
• Maintain communication with EPA's technical and quality managers.
• Facilitate a stop work order if Battelle or EPA QA staff discovers adverse findings that
will compromise test results.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 9 of 29
Battelle Technical Staff will work with the test collaborators to conduct the testing of the
multi-parameter water sensor technologies. Battelle staff will be on-site at the CDOPW at various
times throughout the verification test. The responsibilities of the technical staff will be to:
• Assist CDOPW in any and all aspects of performing the verification test as described
in the test/QA plan. This could include test sample analysis and/or reference sample
analysis.
Mr. Zachary Willenberg is Battelle's Quality Manager for the AMS Center. Mr.
Willenberg or his designee will:
• Review the draft and final test/QA plan.
• Conduct a technical systems audit at least once during the verification test.
• Audit at least 10% of the verification data.
• Prepare and distribute an assessment report for each audit.
• Verify implementation of any necessary corrective action.
• Notify Battelle's AMS Center Manager to issue a stop work order if audits indicate
that data quality is being compromised.
• Provide a summary of the QA/QC activities and results for the verification reports.
• Review the draft and final verification reports and verification statements.
• Ensure that all quality procedures specified in the test/QA plan and in the AMS Center
Quality Management Plan1 (QMP) are followed.
A4.2 Vendors
The responsibilities of the vendor representatives are as follows:
• Review the draft test/QA plan.
• Approve the test/QA plan prior to test initiation.
• Provide at lease two off-the-shelf multi-parameter water sensors for evaluation during
the verification test.
• Provide all other equipment/supplies/reagents/consumables needed to operate their
monitors for the duration of the verification test.
• Either supply a representative to operate their technology throughout the test, or
provide written consent and instructions for Battelle staff to carry out these activities.
• Review the draft verification report and statement.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 10 of 29
A4.3 EPA
EPA's responsibilities in the AMS Center are based in part on the requirements stated in
the "Environmental Technology Verification Program Quality Management Plan" (EPA QMP)2.
The roles of the specific EPA staff are as follows:
Ms. Elizabeth Betz is EPA's AMS Center Quality Assurance Manager. For the
verification test, Ms. Betz will:
• Review the draft test/QA plan.
• Perform at her option one external technical systems audit during the verification test.
• Notify the EPA AMS Center Manager of the need for a stop work order if external
audit indicates that data quality is being compromised.
• Prepare and distribute an assessment report summarizing results of external audit.
• Review draft verification reports and statements.
Mr. Robert Fuerst is EPA's Project Officer for the AMS Center. Mr. Fuerst will:
• Review the draft test/QA plan.
• Approve the final test/QA plan.
• Review the draft verification reports and statements.
• Oversee the EPA review process for the verification reports and statements.
• Coordinate the submission of verification reports and statements for final EPA
approval.
Dr. Ken Fritz is an ecologist within the EPA Office of Research and Development,
National Exposure Research Laboratory. Mr. Fritz will direct the field measurement component
of this ETV test and in doing that will:
• Provide EPA technical staff to perform verification testing in the context of real-time
field analyses.
• Provide data measured by the technologies being verified for all samples analyzed as
well as the corresponding data measured from other technologies used during field
analyses.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 11 of 29
• Review portions of the draft verification reports to assure accurate descriptions of the
field testing and to provide technical insights on verification results
A4.4 Texas Commission on Environmental Quality (TCEQ)
Ms. Christine Kolbe is an aquatic scientist at TCEQ. Ms. Kolbe will direct a second field
measurement component of this ETV test and in doing that will:
• Provide TCEQ technical staff to perform verification testing in the context of real-time
field analyses.
• Provide data measured by the technologies being verified for all samples analyzed as
well as the corresponding data measured from other technologies used during field
analyses.
• Review portions of the draft verification reports to assure accurate descriptions of the
field testing and to provide technical insights on verification results
A4.5 Columbus (OH) Division of Power and Water
Mr. Jeff Kauffman is a water analyst at CDOPW. Mr. Kauffman will be Battelle' s point
of contact for the laboratory measurement component of this ETV test to be conducted at the
CDOPW. In this capacity he will:
• Provide CDOPW technical staff to perform verification testing in the context of
laboratory analyses including test sample preparation and collection, and analyses
using the technologies to be verified.
• Perform reference analyses for the water quality parameters being measured during
this verification test.
• Provide data measured by the technologies being verified for all samples analyzed as
well as the data collected from the applicable laboratory reference methods.
• Document any unusual conditions that could substantively affect the outcome of the
verification test.
• Review portions of the draft verification reports to assure accurate descriptions of the
laboratory testing and to provide technical insights on verification results.
• Provide Battelle technical staff access to the CDOPW facilities in which testing is
ongoing.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 12 of 29
A5 BACKGROUND
The ETV Program's AMS Center does third-party verification testing of commercially
available monitoring technologies that detect natural species and contaminants in air, water, and
soil. A stakeholder committee of buyers and users of such technologies recommend the
technology categories and technologies within those categories as priorities for testing. Multi-
parameter water sensors were identified as a priority technology category through the AMS
Center stakeholder process.
A6 VERIFICATION TEST DESCRIPTION AND SCHEDULE
A6.1 Summary of Technology Category
The multi-parameter water sensor technologies to be tested consist of sensors that measure
several different water quality parameters from grab samples. This verification test will assess the
performance of each sensor relative to key verification parameters including accuracy, precision,
and inter-unit reproducibility. The accuracy measurements will include comparison to reference
measurements where possible. In performing the verification test, Battelle will follow the
technical and QA procedures specified in this test/QA plan and will comply with the data quality
requirements in the AMS Center QMP1.
A6.2 Verification Schedule
The verification test of multi-parameter water sensor technologies is planned to be
completed during April and May of 2007. Because the field testing by EPA NERL was planned
and scheduled well ahead of the development of the test/QA plan for this ETV test, the field
testing was performed during the second week of September 2006, prior to approval of this
test/QA plan. This was permitted because that portion of the verification test focused entirely on
the technologies' ease of field portability and not as an evaluation of the sensors' quantitative
performance. At the close of testing, individual reports will be drafted for each technology,
reviewed, and submitted to EPA for final signature. All documents will be submitted to EPA in
electronic (Microsoft Word and Adobe portable document format [PDF]) and hard copy formats.
-------
A7 QUALITY OBJECTIVES
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 13 of 29
This verification test will evaluate the performance of multi-parameter water sensor
technologies. This will include a comparison of the sensor results to the results of standard
laboratory reference methods. The quality of the reference measurements will be monitored by
the inclusion of blank, duplicate, and performance evaluation (PE) audit samples. The PE audit
samples will be analyzed by an instrument that is different from the one used for the rest of the
reference analyses or that was calibrated with a different set of calibration standards. These
samples are meant to independently confirm that the reference measurements are being performed
correctly with accurate results. Control limits on the duplicate and PE samples are given in
Section B4.
A8 SPECIAL TRAINING/CERTIFICATION
Documentation on training related to standard analytical chemistry methodology is
maintained for applicable Battelle or CDOPW technical staff in training files at the respective
locations. The Battelle Quality Manager will verify the presence of appropriate training records
prior to the start of testing. If the vendors request that Battelle or collaborator technical staff
operate and maintain their sensors during the verification test, the vendors will be required to train
the Battelle or collaborator technical staff prior to the start of testing. Battelle will document this
training with a consent form, signed by the vendor, which states which specific Battelle technical
staff have been trained to use their sensor. All technical staff will have a minimum of a bachelor's
degree in science/engineering or have equivalent work experience.
A9 DOCUMENTATION AND RECORDS
The records for this verification test will be contained in the test/QA plan, the protocols,
chain of custody forms, laboratory record books (LRB), data collection forms, electronic files
(both raw data and spreadsheets), and the final verification report. All of these records will be
maintained in the Verification Test Coordinator's office during the test and can be transferred to
permanent storage at Battelle's Records Management Office at the conclusion of the verification
test. All Battelle LRBs are stored indefinitely, either by the Verification Test Coordinator or
Battelle's Records Management Office. The results from the reference measurements made by
the CDOPW technical staff will be submitted to Battelle daily upon obtaining the results of the
analyses. Section BIO further details the data recording practices and responsibilities.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 14 of 29
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
This verification test will specifically address verification of multi-parameter water sensor
technologies by evaluating the accuracy of the water quality measurements made by each
technology in 1) ASTM Type II deionized (DI) water fortified with chemicals to provide various
water quality measurements across the working range of the sensor, 2) finished drinking water,
and 3) untreated source water samples. Precision of the sensors will be evaluated by analyzing
replicate samples on one day and repeating some of the analyses over a period of several days.
Two identical sensors will be tested simultaneously to compare their results. The drinking water
analyzed by the multi-parameter water sensors will be from the Columbus, Ohio distribution
system. Because only one geographical region is represented by the water system used for this
verification test, this is not intended to be an exhaustive study or to represent all possible water
types that could be tested.
The multi-parameter water sensors will be evaluated for the following parameters:
• Accuracy - comparison to results from standard laboratory reference analyses for DI
water, drinking water within the treatment process, finished drinking water, and
untreated source water test samples
• Precision - repeatability from sample replicates analyzed on the same day and across
several days
• Inter-unit reproducibility - comparison of results from two identical sensors
• Field portability - operation during remote field site analysis
• Ease of use - general operation, data acquisition, set-up, demobilization, required
maintenance
B1.1 Stage 1 - Field Analyses
The first stage of this verification test will evaluate the ease of using the multi-parameter
water sensor technologies during a field water quality study with two collaborators. The ETV
program is collaborating with 1) EPA NERL who conducted a short term field analysis campaign
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 15 of 29
during 2006 that consisted of measuring temperature, pH, and conductivity throughout Shayler's
Run, a stream that flows into the East Fork of the Little Miami River in southern Ohio and 2) the
TCEQ who is going to conduct a similar sampling campaign in western Texas on the Rio Grande
River. These studies are independent from the ETV test, but EPA, NERL, and TCEQ have agreed
to collaborate by taking the technologies to be verified with them and performing single analyses
at some of the measurement locations included in their studies. Because of the rugged terrain that
the analysis teams traverse to make these measurements, no grab samples will be transported for
reference analysis during this stage of the testing. However, the pH, conductivity, and temperature
measurements made by the EPA (using other portable sensors) and TCEQ may be compared with
the technologies being verified. However, the primary evaluation of sensor accuracy will take
place during Stages 2 and 3.
B1.2 Stage 2 - Laboratory Testing of DI Water Samples
The second stage of the verification test will test the accuracy and precision of the sensors
with respect to accepted laboratory reference methods. This will be done by preparing test
solutions with water quality measurements that span the working range of the sensors. Table 1
gives the levels of each water quality parameter that will be analyzed. For example, the pH
component of the sensors will be tested by preparing test solutions that cover the pH ranges from
acidic (pH 4-6), neutral (pH 7), and alkaline (pH 8-10). Table 1 also shows groupings of the
water quality parameters that will be evaluated by analyzing a single solution prepared for that
purpose. The monochloramine and pH measurements will each be evaluated in unique solutions,
but the other parameters will be grouped. The table also includes the key components and/or
critical aspects of each solution's preparation. Three replicates of each of these test solutions will
be analyzed blindly and in random order on each of three separate days to evaluate the short term
repeatability over minutes or hours and the repeatability of the sensors over a multi-day period.
Two identical sensors will be evaluated in the same way for the inter-unit reproducibility
evaluation. The first set of analyses will be performed on Day 1 or 2, the second on Day 6 or 7,
and the third on Day 12 or 13. The stability of the free chlorine solutions will require that they be
prepared on each analysis day, but the other solutions should be stable so they will be prepared
just once and then stored in closed containers in a refrigerator. These samples will be prepared
from National Institute of Standards and Technology (NIST) traceable standards or other primary
standards. However, to confirm that the solutions were prepared properly, at least one test sample
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 16 of 29
of each specific water quality parameter will be measured using the laboratory reference methods
described in Section B.4.
Table 1. Stage 2 Test Sample Information
Parameter
Grouping
1
2
3
4
5
Water Quality
Parameter
pH and temperature
Alkalinity and
Carbon Dioxide
Calcium
Ammonia
Conductivity
Free Chlorine
Oxidation /
Reduction Potential
Monochloramine
Levels
5,7, 10 (pH units)
22, 130, and240mg/L
CaCO3
7, 50, and 90 mg/L
0.1,0.8, 1.5 mg/L
100, 900, 1700 nS/cm
0.2, 1.2, 2.2 mg/L
Test Sample Preparation
(all samples in DI water)
Citrate, phosphate, and borate buffers,
respectively, prepared at appropriate pHs
Anhydrous sodium bicarbonate dissolved in DI
water
Calcium chloride (pH 8.3)
Ammonium chloride (pH 8.3)
Sodium chloride (pH 9.0)
Sodium hypochlorite (pH 9.0)
Use the free chlorine solutions to generate a range of oxidation reduction
potentials.
0.2, 1.2, 2.2 mg/L
Addition of ammonium chloride to Solution #4
with a 15 minute reaction time (pH>9)
mg/L - milligram per liter
|j,S/cm - microSiemens per centimeter
B1.3 Stage 3 - Laboratory and Field Testing of Drinking and Surface Water Samples
The third stage of this verification test will evaluate the performance of the multi-
parameter water sensors when analyzing samples of finished drinking water drinking water within
the treatment process, and untreated surface and ground source water. Throughout the
verification test, at least 10 samples (6 finished drinking water, 2 "in-process" water, and 2 source
waters) will be collected. At least four of these samples will be analyzed upon collection and then
immediately returned to the laboratory for reference analysis so that comparison between the two
measurements is valid. When possible, the reference measurement will be performed at the
location of the field measurement. In addition to these field measurements, all 10 samples will be
returned to the laboratory for analysis using the technologies being verified and all of the
reference analysis. Each of these samples will be analyzed in triplicate by the multi-parameter
water sensors and then compared to the reference method results from these same samples. The
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 17 of 29
finished drinking water samples will be analyzed during days between the Stage 2 analyses. This
will evaluate the degree to which the sensor response drifts throughout the lifetime of the sensor.
Table 2 summarizes the samples that will be analyzed in Stages 1-3 of this verification test
and what reference sample analyses are being performed.
Table 2. Summary of Test Sample Analysis
Testing Stage
1 - Field Analysis
2 - Laboratory
Testing of DI
Samples
3 - Laboratory
and Field Testing
of Drinking and
Untreated Source
Water Samples
Minimum
Number of
Samples
10 samples
with no
replicate
analysis
45 samples
analyzed in
triplicate
(7 blanks)
10 samples
analyzed in
triplicate
(2 blanks)
Summary of Test Sample Analysis
EPA will use the technologies for field analysis by analyzing
one sample collected from at least 10 different locations. No
reference measurements will be made, only a possible
comparison with a handheld sensor routinely used by EPA.
Each measurement level given in Table 1 will be analyzed in
triplicate on three different days over a multi-day period. The
first analysis will be performed on Day 1 or 2, the second on
Day 6 or 7, and the third on Day 12 or 13. Reference
measurements will be made on each solution on each day.
CDOPW will collect and analyze (in triplicate) at least 10 total
samples (6 finished drinking water, 2 "in-process" waters, and 2
source waters); reference measurements will be performed on
each sample. Analysis of 4 of these 10 samples will occur at the
field location, and samples will be returned to the laboratory
immediately for reference analysis. When possible reference
measurements will also occur at the field site.
B1.4 Statistical Analysis
B1.4.1 Percent Difference
Depending on the stage of testing, results from the analysis of drinking and source water
samples by the multi-parameter water sensors will be compared to the results obtained from
another sensor (Stage 1) or results obtained from a reference method (Stages 2 and 3). The results
for each sample will be recorded, and the accuracy will be expressed in terms of the percent
difference (%D), as calculated from the following equation:
%D =
C- s
100
(1)
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 18 of 29
where CR is the value from each stage (as described above) with which each sensor measurement
is being compared and Cs is the result from the sensor. This calculation will be performed for
each of the parameters measured by each sensor. Ideally, if the sensor result and value being
compared are the same, there would be a percent difference of zero. Percent difference will be
assessed independently for each sensor provided by the vendor to contribute to the inter-unit
reproducibility evaluation.
B'1.4.2 Precision
Precision will be evaluated across two different time periods during this verification test.
The first will evaluate the repeatability of the sensor across equivalent samples analyzed in
triplicate on the same day and the second will evaluate the repeatability of the sensor across
several days. In both cases, precision will be evaluated by calculating the standard deviations of
both sets of repeated measurements:
s =, „
M k=\
1/2
2
(2)
where n is the number of replicate samples, Cs is the water quality parameter measured for the kih
sample, and Cs is the average water quality parameter measurement of the replicate samples. The
multi-parameter water quality sensor precision at each concentration level will be reported in
terms of the relative standard deviation (RSD), shown in Equation 3:
RSD =
S
G
Bl. 4.3 Inter-Unit Reproducibility
xlOO
(3)
The results obtained from identical units of each sensor will be compiled independently
for each analyzer and compared to assess inter-unit reproducibility. The results will be interpreted
using a two-sided f-test, a linear regression of the unit 1 results plotted verses the unit 2 results, or
other appropriate comparison, to assess whether significant differences exist between the units
tested. This evaluation will be conducted for Stages 2 and 3.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 19 of 29
B1.5 Reporting
The statistical comparisons described above will be conducted separately for each of the
sensors being tested, and information on the additional performance parameters such as ease of
use, level of maintenance, calibration, and set-up/demobilization will be compiled and reported.
Separate verification reports will be prepared for each sensor that is tested. Each report will show
separate verification results from the duplicate sensors undergoing testing, along with calculations
of the inter-unit reproducibility of the technology. For each test, the verification report will
present the test procedures and test data, as well as the results of the statistical evaluation of those
data.
All interaction with the sensors (such as during maintenance, cleaning, and calibration)
will be documented at the time of the test and reported. In addition, descriptions of the data-
recording procedures, use of vendor-supplied software, consumables used, and required reagents
will be presented in the report. The verification report will briefly describe the ETV program, the
AMS Center, and the procedures used in verification testing. These sections will be common to
each verification report. The results of the verification test will then be stated quantitatively, as
described in section B1.4 without comparison to any other technology tested or comment on the
acceptability of the technology's performance.
B2 SAMPLE COLLECTION
As described above, Stage 1 testing of the multi-parameter water sensors will consist of an
evaluation of the field portability of the technologies. Because of the remote locations at which
these measurements will be made, no grab samples for reference method analyses will be
collected. During Stage 2, test solutions that intentionally alter the water quality parameters
across the working range of the sensor will be prepared. For confirmation of the accuracy of
solution preparation, at least one of the levels of each water quality parameter measured will be
analyzed by the laboratory reference method as well as by the technologies being verified. No
sample collection will be required as the analysis will be performed on the excess test solution
made from DI water. Stage 3 testing will use finished drinking water and source water samples
collected by CDOPW staff. The drinking water samples will be collected in SOOmL high density
polyethylene containers from pumping or tank locations throughout the Columbus, Ohio water
distribution system and the source water samples will be collected in a similar manner from the
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 20 of 29
source water intake at the Parsons Avenue (ground) and Dublin Road (surface) Columbus water
treatment plants.
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Sample custody will be documented throughout collection and analysis of the test
samples. A chain-of-custody form will include details about the sample such as the time, date,
location, and person collecting the sample. The chain-of-custody form will track sample release
from the sampling location to the analysis laboratory. Each chain-of-custody form will be signed
by the person relinquishing samples once that person has verified that the chain-of-custody form
is accurate. Upon arrival at the analysis laboratory, chain-of-custody forms will be signed by the
person receiving the samples (if different from the sample collector) once that person has verified
that all samples identified on the chain-of-custody forms are present. Copies of all chain-of-
custody forms will be delivered to the Verification Test Coordinator and maintained with the test
records.
B4 LABORATORY REFERENCE METHODS
Table 3 provides the standard laboratory methods that will be used for the reference
analyses during this verification test. Also included in the table is each method's detection limit
and the allowable percent difference (%D) for PE and duplicate reference samples. The analyses
will be performed by CDOPW or Battelle technical staff. For reference measurements that are
made at CDOPW, CDOPW is responsible for providing any necessary calibrated instrumentation,
performing all method QA/QC, and providing calibration records for any instrumentation used.
The CDOPW will provide Battelle all reference sample data daily upon obtaining results.
B5 QUALITY CONTROL AUDITS AND REQUIREMENTS
One duplicate sample analysis will be required by each standard reference method on each
day that sample analyses are performed. The percent relative standard deviation (%RSD) of the
duplicate measurements will be required to be within the acceptable tolerances provided in Table
3. Samples producing results not meeting these requirements shall be reanalyzed by the reference
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 21 of 29
method. If the results are still outside the required tolerance, the reference instrument will be
recalibrated (if applicable) and/or the reference samples reanalyzed. If the outlying results persist,
the repeat of the appropriate parts of the verification test or use of a different reference instrument
may be considered. DI water blanks will be required each day for the ammonia, calcium,
chlorine, and monochloramine reference
Table 3. Reference Methods
Parameter
Ammonia
Calcium
Conductivity
Free Chlorine
Monochloramine
Oxidation
Reduction Potential
pH
Temperature
Total Alkalinity
Method
SM 4500-NH33
EPA 21 5. 24
SM25105
SM 4500-C1-G6
SM 4500-C1-G6
SM2580-B7
EPA 150. 18
EPA 170. 19
EPA 3 10. 110
Method
Description
ion selective
electrode
titration
probe
colorimetric
colorimetric
probe
probe
probe
titration
Method Detection
Limits
0.03 mg/L
0.5 mg/L
2 limho
0.01mg/LasCl2
0.01 mg/L as NH2C1
NA
NA
NA
20 mg/L
Acceptable
Duplicate and PE
Tolerance
25%
25%
25%
25%
25%
25%
±0.3 pH units
±PC
25%
NA - not applicable due to nature of that water quality parameter
analyses. The blank reference method results for these water quality parameters are required to be
less than the method detection limit. For the other measurements, there is no conclusive blank
reading in DI water making the blank analysis of little use. One DI water blank will be analyzed
using the technologies being verified for every 20 samples analyzed to confirm the absence of
reagent contamination.
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The instruments used for the reference analyses (as allowed by the standard methods) will
be tested and inspected as per the instrument manuals, the standard operating procedures of the
CDOPW or the standard methods being used to make each measurement. If Battelle or CDOPW
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 22 of 29
staff operate and maintain the sensors during the verification test, it will be done as directed by
the vendor. Otherwise, operation and maintenance will be the responsibility of the vendor.
B7 INSTRUMENT CALIBRATION AND FREQUENCY
The instruments used for the reference analyses (as allowed by the standard methods listed
in Table 3) will be calibrated per the instrument manual, the standard reference methods being
used to make each measurement or the standard operating procedures of the analysis laboratory.
If the sensor being verified is being operated by Battelle or CDOPW, the vendor will provide the
verification staff with a checklist of tasks to be completed daily to properly maintain each sensor.
All calibrations performed will be documented by Battelle in the project LRB.
Calibration of the multi-parameter water sensors will be done for each measured
parameter as often as suggested by the vendor. Vendors will be required to supply the necessary
calibration solutions and devices specific to the sensors being verified. Balances and pipettes
used during test solution preparation will be maintained and calibrated as required by the Battelle
or CDOPW standard operating procedures which will be reviewed by the Battelle Quality
Manager prior to the verification test.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
All materials, supplies, and consumables will be ordered by the Verification Test
Coordinator or designee from Battelle or CDOPW. Where possible, Battelle will rely on sources
of materials and consumables that have been used previously as part of ETV verification testing
without problems. Battelle will also rely on previous experience or recommendations from
CDOPW technical staff to guide selection of manufacturers and materials. The manufacturer's
criteria for acceptance/purity will be required to be met.
B9 NON-DIRECT MEASUREMENTS
Data published previously in the scientific literature will not be used during this verification test.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 23 of 29
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by
verification staff during this verification test. Table 4 summarizes the type of data to be recorded.
All data and observations for the operation of the sensors will be documented by the vendors or
verification staff on data sheets or in LRBs. Results from the laboratory reference instruments
will be compiled by verification staff in electronic format and submitted to the Verification Test
Coordinator upon obtaining results.
Records received by or generated by any of the verification staff during the verification
test will be reviewed by a Battelle staff member within two weeks of receipt or generation,
respectively, before the records are used to calculate, evaluate, or report verification results. If a
Battelle staff member generated the record, this review will be performed by a Battelle technical
staff member involved in the verification test, but not the staff member that originally received or
generated the record. The review will be documented by the person performing the review by
adding his/her initials and date to the hard copy of the record being reviewed. In addition, data
calculations performed by verification staff will be spot-checked by Battelle technical staff to
ensure that calculations are performed correctly. Calculations to be checked include any
statistical calculations described in this test/QA plan. The data obtained from this verification test
will be compiled and reported independently for each multi-parameter water sensor. Results for
multi-parameter water sensors from different vendors will not be compared with each other.
During the course of any assessment or audit, the Battelle Quality Manager will inform the
technical staff of any immediate corrective action that should be taken. If serious quality
problems exist, the Battelle Quality Manager will notify the Battelle AMS Center Manager who is
authorized to stop work. Once the assessment report has been prepared, the Verification Test
Coordinator will ensure that a response is provided for each adverse finding or potential problem,
and will implement any necessary follow-up corrective action. The Battelle Quality Manager will
ensure that follow-up corrective action has been taken.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 24 of 29
Table 4. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test
events
Calibration
information
Multi-parameter
water sensor
results
Reference method
procedures
Where Recorded
ETV data sheets
and testing
notebook
ETV data sheets
and testing
notebook
Recorded
electronically by
each monitor and
then downloaded
and emailed to
Verification Test
Coordinator daily
for CDOPW and
weekly for EPA.
ETV laboratory
record books, or
data recording
forms
How Often
Recorded
Start/end of test, and
at each change of a
test parameter
Prior to sample
preparation
Recorded
continuously.
Throughout sample
analysis process;
sent to Battelle daily
upon completion of
analyses
By
Whom
EPA,
Battelle,
and
CDOPW
EPA,
Battelle,
and
CDOPW
EPA,
Battelle,
and
CDOPW
EPA,
Battelle,
and
CDOPW
Disposition of Data
Used to
organize/check test
results; manually
incorporated in data
spreadsheets as
necessary
Manually
incorporated in data
spreadsheets as
necessary
Comma delimited
text files.
Transferred to
spreadsheets or
laboratory record
book
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 25 of 29
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in this verification test to anticipate and resolve potential
problems before the quality of performance is compromised. One of the major objectives of the
test/QA plan is to establish mechanisms necessary to ensure this. Internal quality control
measures described in this test/QA plan, which is peer reviewed by a panel of outside experts,
implemented by the technical staff and monitored by the Verification Test Coordinator, will give
information on data quality on a day-to-day basis. The responsibility for interpreting the results
of these checks and resolving any potential problems resides with the Verification Test
Coordinator. Technical staff have the responsibility to identify problems that could affect data
quality or the ability to use the data. Any problems that are identified will be reported to the
Verification Test Coordinator, who will work with the Battelle Quality Manager to resolve any
issues. Action will be taken to control the problem, identify a solution to the problem, and
minimize losses and correct data, where possible. Battelle will be responsible for ensuring that
the following audits are conducted as part of this verification test.
C1.1 Performance Evaluation Audits
A PE audit will be conducted to assess the quality of the reference measurements made in
this verification test. Each type of reference measurement will be compared with an independent
instrument or a NIST-traceable standard that is independent of those used during the testing as
part of the PE audit. The results of the PE audit must be within the acceptable tolerances
provided in Table 3. If the results do not meet this requirement, they will be repeated. If the
outlying results persist, a change in reference instrument, and a repeat of the PE audit will have to
be performed. This audit will be performed once at the start of the verification test, and will be
the responsibility of the Verification Test Coordinator or designee.
C1.2 Technical Systems Audits
The Battelle Quality Manager will perform a technical systems audit (TSA) at least once
during this verification test. The purpose of this audit is to ensure that the verification test is
being performed in accordance with the AMS Center QMP1, this test/QA plan, published
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 26 of 29
reference methods, and any standard operating procedures (SOPs) used by the CDOPW. In this
audit, the Battelle Quality Manager, or designee, may review the reference methods used,
compare actual test procedures to those specified or referenced in this plan, and review data
acquisition and handling procedures. A ISA report will be prepared, including a statement of
findings and the actions taken to address any adverse findings. The EPA AMS Center Quality
Manager will receive a copy of Battelle's ISA report. At EPA's discretion, EPA QA staff may
also conduct an independent on-site TSA during the verification test. The TSA findings will be
communicated to technical staff at the time of the audit and documented in a TSA report.
C1.3 Data Quality Audits
The Battelle Quality Manager or designee will audit at least 10% of the verification data
acquired in the verification test. The Battelle Quality Manager will trace the data from initial
acquisition, through reduction and statistical comparisons, to final reporting. All calculations
performed on the data undergoing the audit will be checked.
C1.4 QA/QC Reporting
Each assessment and audit will be documented in accordance with Section 3.3.4 of the
AMS Center QMP1. The results of the TSA will be submitted to EPA. Assessment reports will
include the following:
• Identification of any adverse findings or potential problems
• Response to adverse findings or potential problems
• Recommendations for resolving problems
• Confirmation that solutions have been implemented and are effective
• Citation of any noteworthy practices that may be of use to others.
C2 REPORTS TO MANAGEMENT
The Battelle Quality Manager, during the course of any assessment or audit, will identify
to the technical staff performing experimental activities any immediate corrective action that
should be taken. If serious quality problems exist, the Battelle Quality Manager is authorized to
notify the Battelle AMS Center Manager who will issue the stop work. Once the assessment
report has been prepared, the Verification Test Coordinator will ensure that a response is provided
for each adverse finding or potential problem and will implement any necessary follow-up
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 27 of 29
corrective action. The Battelle Quality Manager will ensure that follow-up corrective action has
been taken. The test/QA plan and final report are reviewed by EPA AMS Center QA staff and the
EPA AMS Center program management staff. Upon final review and approval, both documents
will then be posted on the ETV website (www.epa.gov/etv).
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 28 of 29
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are the collection of QC samples
as outlined in the test/QA plan, a comparison of field data sheet comments against final data to
flag any suspect data, and a review of final data to resolve any questions about apparent outliers.
The QA audits, as described within this document are designed to assure the quality of this data.
D2 VALIDATION AND VERIFICATION METHODS
Section C of this test/QA plan provides a description of the validation safeguards
employed for this verification test. Data validation and verification efforts include the analysis of
QC samples as required in this document, and the performance of ISA and PE audits as described
in Section C.
D3 RECONCILIATION WITH USER REQUIREMENTS
This test/QA plan and the resulting ETV verification report(s) will be subjected to review
by the multi-parameter water sensor vendors, EPA, and expert peer reviewers. These reviews will
assure that this test/QA plan and the resulting verification report(s) meet the needs of potential
users of the multi-parameter water sensors. Performance data for the multi-parameter water
sensors, collected under conditions where the quality control requirements for the duplicate and
PE audit samples shown in Table 3 were met, will be presented in the final verification report
without any further comment. Performance data and reference measurements that do not meet
these criteria will be noted and a discussion of the possible impact of the failed requirements on
the performance evaluation will be presented in the final verification report. The final verification
report(s) will be submitted to EPA in MS Word and Adobe portable document format (PDF) and
subsequently posted on the ETV website.
-------
Verification of Multi-Parameter Water Sensors Test/QA Plan
Version 1
April 4, 2007
Page 29 of 29
SECTION E
REFERENCES
1. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version 6.0,
U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio,
December 2005.
2. "Environmental Technology Verification Program Quality Management Plan", December
2002, EPA/600/R-03/021.
3. American Public Health Association, et al., SM 4500-NH3 D Ammonia-Selective Electrode
Method in Standard Methods for the Examination of Water and Wastewater. 20th Edition,
Washington, D.C., 2005.
4. U.S. EPA, EPA Method 215.2, Calcium (Titrimetric, EDTA) m Methods for Chemical
Analysis of Water and Wastes, EPA/600/4-79/020, March 1983.
5. American Public Health Association, et al., SM 2510 Conductivity in Standard Methods for
the Examination of Water and Wastewater. 19th Edition, Washington, D.C., 1997.
6. American Public Health Association, et al., SM 4500-G Residual Chlorine in Standard
Methods for the Examination of Water and Wastewater. 19th Edition, Washington, D.C.,
1997.
7. American Public Health Association, et al., SM 2580-B Electrochemical Potential in Standard
Methods for the Examination of Water and Wastewater. 19th Edition, Washington, D.C.,
1997.
8. U.S. EPA, EPA Method 150.1, pR in Methods for Chemical Analysis of'Water andWastes,
EPA/600/4-79/020, March 1983.
9. U.S. EPA, EPA Method 170.1, Temperature in Methods for Chemical Analysis of Water and
Wastes, EPA/600/4-79/020, March 1983.
10. U.S. EPA, EPA Method 310.1, Alkalinity in Methods for Chemical Analysis of Water and
Wastes, EPA/600/4-79/020, March 1983.
------- |