Batreiie
The Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Test/QA Plan for Verification of
Coliform Detection Technologies for
Drinking Water
-------
TEST/ QUALITY ASSURANCE PLAN
for
Verification of
Coliform Detection Technologies for Drinking Water
Version 1.0
July 14, 2010
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Coliform Detection Technologies
Test/QA Plan
Page 3 of 3
Version 1.0
Date: 7/14/2010
SECTION A
PROJECT MANAGEMENT
Al VENDOR APPROVAL PAGE
ETV Advanced Monitoring Systems Center
Test/QA Plan for Verification of
Coliform Detection Technologies for Drinking Water
APPROVAL:
Name
Company
Date
-------
Coliform Detection Technologies
Test/QA Plan
Page 4 of 4
Version 1.0
Date: 7/14/2010
A2 TABLE OF CONTENTS
Section
SECTION A PROJECT MANAGEMENT 3
Al Vendor Approval Page 3
A2 Table of Contents 4
A3 List of Acronyms 6
A4 Distribution List 7
A5 Verification Test Organization 8
A6 Background 16
A7 Verification Test Description and Schedule 18
A8 Quality Objectives and Criteria for Measurement Data 20
A9 Special Training/Certification 21
A10 Documentation and Records 22
SECTION B MEASUREMENT AND DATA ACQUISITION 24
Bl Experimental Design 24
B2 Sampling Method Requirements 40
B3 Sample Handling and Custody Requirements 41
B4 Laboratory Reference Methods 41
B5 Quality Control Requirements 42
B6 Equipment Testing, Inspection, and Maintenance 43
B7 Calibration Verification 45
B8 Inspection/Acceptance of Supplies and Consumables 45
B9 Non-Direct Measurements 46
BIO Data Management 46
SECTION C ASSESSMENT AND OVERSIGHT 48
Cl Assessments and Response Actions 48
C2 Reports to Management 50
SECTION D DATA VALIDATION AND USABILITY 52
Dl Data Review, Validation, and Verification Requirements 52
D2 Validation and Verification Methods 52
D3 Reconciliation with User Requirements 52
SECTION E REFERENCES 54
El References 54
-------
Coliform Detection Technologies
Test/QA Plan
Page 5 of 5
Version 1.0
Date: 7/14/2010
List of Figures
Figure 1. Organization Chart for the Verification Test 9
Figure 2. Experimental Design for Sample Dilution and Analysis 31
FigureS. Flowchart describing confirmation analyses for both the technologies being verified
andSM9221B 37
List of Tables
Table 1. Proposed Verification Test Schedule 20
Table 2. Quality Control Strains 21
TableS. Summary of Data Recording Process 23
Table 4. Sample Preparation Timeline 25
Table 5. Volume of Drinking Water Required for Verification Testing 27
Table 6. Power Analysis Results for Determination of Replicates 33
Table 7. Sample Analysis Timeline 34
Table 8. False Negative Rate Comparison 39
Table 9. False Positive Rate Comparison 39
Table 10. Reference Methods 42
Table 11. Quality Control Samples for Reference and Technology Methods 44
-------
Coliform Detection Technologies
Test/QA Plan
Page 6 of 6
Version 1.0
Date: 7/14/2010
A3 LIST OF ACRONYMS
ADQ audit of data quality
AMS Advanced Monitoring Systems
ATCC American Type Culture Collection
BGLB brilliant green lactose bile
COC chain of custody
DANETV Danish ETV
DDW dechlorinated drinking water
DQA data quality audit
DW drinking water
E. coli Escherichia coll
EPA U.S Environmental Protection Agency
ETV Environmental Technology Verification
FN false negative
FP false positive
h hours
LRB laboratory record book
LTB lauryl tryptose broth
MB method blank
MCL Maximum Contaminant Level
MCLG Maximum Contaminant Level Goal
MUG 4-methyllumbelliferyl-p-D-glucorinide
N number
NA nutrient agar
PA presence/absence
PDF portable document format
PO project officer
QA quality assurance
QC quality control
QMP Quality Management Plan
RB reagent blank
RMO Records Management Office
SM Standard Methods
SSDW spiked, stressed drinking water
SWTP Southerly Wastewater Treatment Plant
TC total coliform
TCR Total Coliform Rule
TSA technical systems audit
-------
A4 DISTRIBUTION LIST
Vendors
James Berg
Colifast AS
Strandveien 33, P.O.Box 31
1324 Lysaker, Norway
Dr. Peter Gallant
Pathogen Detection Systems, Inc.
116 Barrie Street, Suite 4697
Kingston, Ontario, Canada K7L 3N6
EPA ETV
John McKernan
Michelle Henderson
U.S. Environmental Protection Agency
26 West Martin Luther King Drive
Cincinnati OH 45268
Battelle
Ryan James
Daniel Lorch
Amy Dindal
Battelle
505 King Ave.
Columbus, OH 43201
Stacy Pala
Rosanna Buhl
Battelle
397 Washington Street
Duxbury, MA 02332
Coliform Detection Technologies
Test/QA Plan
Page 7 of 7
Version 1.0
Date: 7/14/2010
Peer Reviewers
Jim Sinclair
Sandhya Parshionikar
Jennifer Best
Key a Sen
U.S. Environmental Protection Agency
Office of Water
26 West Martin Luther King Drive
Cincinnati OH 45268
Rick Sakaji
East Bay Municipal Water District
375 Eleventh St, MS 705
Oakland, CA 94607-4240
Mark Rodgers
U.S. Environmental Protection Agency
National Risk Management Research
Laboratory
26 West Martin Luther King Drive
Cincinnati OH 45268
Cooperating ETV Programs
John Neate
Mona El Hallak
ETV Canada
2070 Hadwen Road, Unit 201A
Mississauga, Ontario
L5K 2C9
Christian Gron
Mette Tjener Andersson
Claus J0rgensen
DANETV
Agern Alle 5
DK-2970 H0rsholm
Denmark
-------
Coliform Detection Technologies
Test/QA Plan
Page 8 of 8
Version 1.0
Date: 7/14/2010
A5 VERIFICATION TEST ORGANIZATION
The verification test will be conducted under the U.S. Environmental Protection Agency
(EPA) Environmental Technology Verification (ETV) Program. It will be performed by
Battelle, which is managing the ETV Advanced Monitoring Systems (AMS) Center through a
cooperative agreement with EPA. The scope of the AMS Center covers verification of
monitoring technologies for contaminants and natural species in air, water, and soil. This
verification testing is also being conducted in cooperation with the Canadian and Danish
(DANETV) ETV programs as a possible ETV verification in those countries. The criteria for
ETV cooperation are outlined in a cooperative verification process document prepared by the
respective cooperating ETV programs. At least one of the technology vendors has agreed to
participate in the verification with the possibility that their technology be considered for
verification. It should be noted however that neither U.S. ETV verification, nor the cooperation
with the Canadian or Danish ETV programs, represents an approval of methods for regulatory
compliance. This verification test is being conducted to provide a verification of technologies
that are not already approved for compliance monitoring in the U.S., or elsewhere, such that the
vendors have an opportunity to have their technologies tested under similar set of test conditions
that might be required for method approval.
The day to day operations of this verification test will be coordinated and supervised by
Battelle, with the participation of the vendors who will have the performance of their
technologies verified. Testing of the technologies being verified will be conducted at Battelle
laboratories in Columbus, Ohio. Each vendor will provide Battelle with their respective
technologies and will train the verification staff. Staff from Battelle will operate the
technologies during verification testing.
Quality Assurance (QA) oversight will be provided by the Battelle Quality Manager and
also by the EPA AMS Center Quality Manager, at her discretion. The organization chart in
Figure 1 identifies the responsibilities of the organizations and individuals associated with the
verification test. Roles and responsibilities are defined further below.
-------
Coliform Detection Technologies
Test/QA Plan
Page 9 of 9
Version 1.0
Date: 7/14/2010
This test/QA plan will be subjected to review by the vendors, EPA, and expert peer
reviewers referred to as verification test stakeholders in this test/QA plan. The reviews of this
test/QA plan will help to improve the design of the verification test and the resulting report(s)
such that they better meet the needs of potential users of these technologies. After all peer
review comments are received, revisions will be made to the test/QA plan, and it will be
submitted to EPA for approval.
AMS Center
Stakeholders
Battelle
Management
Battelle AMS Center
Quality Manager
Rosanna Buhl
Battelle AMS
Center Manager
Amy Dindal
Verification Test
Coordinator
Ryan James
Battelle Testing
Staff
EPA AMS Center
Project Officer
John McKernan
EPA AMS Center Quality
Assurance Manager
Michelle Henderson
Vendor
Representatives
Figure 1. Organization Chart for the Verification Test
A5.1 Battelle
Dr. Ryan James is the AMS Center's Verification Test Coordinator for this test. In this
role, Dr. James will have overall responsibility for ensuring that the technical, schedule, and cost
goals established for the verification test are met. Specifically, Dr. James will:
• Prepare the draft test/QA plan, verification reports, and verification statements
• Establish a budget for the verification test and manage staff to ensure the budget is
not exceeded
-------
Coliform Detection Technologies
Test/QA Plan
Page 10 of 10
Version 1.0
Date: 7/14/2010
• Revise the draft test/QA plan, verification reports, and verification statements in
response to reviewer comments
• Assemble a qualified technical staff to conduct the verification test
• Direct the staff in performing the verification test in accordance with this test/QA
plan
• Hold a kick-off meeting approximately one week prior to the start of the verification
test to review the critical logistical, technical, and administrative aspects of the
verification test. Responsibility for each aspect of the verification test will be
reviewed to ensure each participant understands his/her role
• Ensure that all quality procedures specified in this test/QA plan and in the AMS
Center Quality Management Plan1 (QMP) are followed
• Maintain real-time communication with the Battelle AMS Center Manager and EPA
AMS Center Project Officer and Quality Assurance Manager on any potential or
actual deviations from the test/QA plan
• Serve as the primary point of contact for vendor representatives
• Ensure that confidentiality of sensitive vendor information is maintained
• Assist vendors as needed during verification testing
• Become familiar with the operation and maintenance of the technologies through
instruction by the vendors
• Direct the 100% validation and verification of data by technical staff prior to submission for
data quality audit
• Respond to any issues raised in assessment reports, audits, or from verification staff
observations, and institute corrective action as necessary
• Coordinate distribution of the final test/QA plan, verification reports, and verification
statements.
Ms. Amy Dindal is Battelle's Manager for the AMS Center. As such, Ms. Dindal will
oversee the various stages of verification testing. Ms. Dindal will:
• Review the draft and final test/QA plan
• Review the draft and final verification reports and verification statements
-------
Coliform Detection Technologies
Test/QA Plan
Page 11 of 11
Version 1.0
Date: 7/14/2010
• Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test
• Ensure that confidentiality of sensitive vendor information is maintained
• Support Dr. James in responding to any issues raised in assessment reports and audits
• Maintain communication with EPA's technical and quality managers
• Issue a stop work order if Battelle or EPA QA staff discovers adverse findings that
will compromise test results.
Mr. Daniel Lorch and other technical staff in Battelle's microbiological laboratories will
support Dr. James in planning and conducting the verification test. They will:
• Assist in planning for the test, and making arrangements for the receipt of and
training on the technologies
• Attend the verification test kick-off meeting
• Assist vendor staff as needed during technology receipt and training
• Conduct verification testing using each participating technology, following all aspects
of the ETV AMS Center QMP as well as the test/QA plan for this verification
• Coordinate activities of Superior Laboratories, Inc., the laboratory performing the
reference measurements
• Confirm that Superior Laboratories, Inc. is completing all required QC samples and
corresponding documentation
• Support Dr. James in the preparation of the test/QA plan and reports, as necessary
• Support Dr. James in responding to any issues raised in assessment reports and audits
related to technical performance, statistics, or data reduction as needed.
Ms. Rosanna Buhl is Battelle's Quality Manager for the AMS Center. Ms. Buhl will:
• Review the draft and final test/QA plan
• Attend the verification test kick-off meeting and lead the discussion of the QA
elements of the kickoff meeting checklist
-------
Coliform Detection Technologies
Test/QA Plan
Page 12 of 12
Version 1.0
Date: 7/14/2010
• Prior to the start of verification testing, verify the presence of applicable training
records for Battelle and Superior Laboratories, Inc. staff, including any vendor
training on test equipment
• Contact Superior Laboratories, Inc. before the verification test to confirm adequate
quality practices are in place
• Conduct a technical systems audit at least once during the verification test, or
designate other QA staff to conduct the audit (Section Cl.l)
• Conduct audits of verification data quality or designate other QA staff to conduct the
data audit (Section C1.2).
• Prepare and distribute an assessment report for each audit
• Verify that audit responses for each audit finding and observation are appropriate and
that corrective action has been implemented effectively
• Communicate to the test coordinator and/or technical staff the need for immediate
corrective action if an audit identifies test/QA plan deviations or practices that
threaten data quality
• Review and approve test/QA plan amendments, deviations and audit reports
• Verify implementation of any necessary corrective action
• Request that Battelle's AMS Center Manager issue a stop work order if audits
indicate that data quality is being compromised
• Provide a summary of the QA/QC activities and results for the verification reports
• Review the draft and final verification reports and verification statements.
• Delegate QA activities to other Battelle quality staff as needed to meet project
schedules
A5.2 Technology Vendors
The responsibilities of the technology vendors are as follows:
• Review and provide comments on the draft test/QA plan
• Accept (by signature of a company representative) the final test/QA plan prior to test
initiation
-------
Coliform Detection Technologies
Test/QA Plan
Page 13 of 13
Version 1.0
Date: 7/14/2010
• Provide their technology for evaluation during the verification test
• Provide all other equipment/supplies/reagents/consumables needed to operate their
technology for the duration of the verification test
• Supply training on the use of the technology, and provide written consent and
instructions for verification staff to carry out testing, including written instructions for
routine operation of their technology
• Provide maintenance and repair support for their technology, on-site if necessary,
throughout the duration of the verification test
• Review and provide comments on the draft verification report and statement for their
respective technology
• Recognize that results in the verification report will be reviewed by the cooperating
ETV programs in order to determine if verification is applicable in those countries
• Allow for publication of the verification results by the cooperating ETV programs, if
vendors are considered for verification in those countries.
A5.3 EPA
EPA's responsibilities in the AMS Center are based on the requirements stated in the
"Environmental Technology Verification Program Quality Management Plan" (ETV QMP)2.
The roles of specific EPA staff are as follows:
Ms. Michelle Henderson is EPA's AMS Center Quality Manager. For the verification
test, Ms. Henderson will:
• Review the draft test/QA plan
• Perform at her option an external technical systems audit during the verification test
• Notify the EPA AMS Center Project Officer of the need for a stop work order if the
external audit indicates that data quality is being compromised
• Prepare and distribute an assessment report summarizing results of the external audit
• Perform at her option audits of data quality
-------
Coliform Detection Technologies
Test/QA Plan
Page 14 of 14
Version 1.0
Date: 7/14/2010
• Notify the EPA AMS Center Project Officer of the need for a stop or modify work
order if the audit of data quality indicates that data quality is being compromised
• Review draft verification reports and verification statements.
Dr. John McKernan is EPA's Project Officer for the AMS Center. Dr. McKernan or
designee will:
• Review the draft test/QA plan
• Approve the final test/QA plan
• Be available during the verification test to authorize any test/QA plan deviations by
phone and provide the name of a delegate to the Battelle AMS Center Manager
should he not be available during the testing period
• Review the draft verification reports and verification statements
• Oversee the EPA review process for the test/QA plan, verification reports, and
verification statements
• Coordinate the submission of verification reports and verification statements for final
EPA approval
• Post the test/QA plan, verification reports, and verification statements on the ETV
web site.
A5.4 Verification Test Stakeholders
This test/QA plan and the verification report(s) and verification statement(s) based on
testing described in this document will be reviewed by experts in the fields related to coliform
detection. The following experts have been providing input to this test/QA plan and have agreed
to provide a peer review:
• Rick Sakaji, East Bay Municipal Water District
• Jim Sinclair, Sandhya Parshionikar, Jennifer Best, and Keya Sen, EPA Office of
Water
-------
Coliform Detection Technologies
Test/QA Plan
Page 15 of 15
Version 1.0
Date: 7/14/2010
The responsibilities of verification test stakeholders (or expert peer reviewers) include:
• Participate in technical panel discussions (when available) to provide input to the test
design
• Review and provide input to the test/QA plan
• Review and provide input to the verification report(s)/verification statement(s).
In addition, the general approach for verification testing was reviewed with the broader AMS
Center Stakeholder Committee as a presentation during regular stakeholder teleconferences,
including the November 12, 2009 meeting, and input from the committee was solicited.
A5.5 Cooperating ETV Programs
This test/QA plan will be reviewed by the cooperating ETV programs to confirm that the
experimental design meets the requirements for possible verification in those countries. In
addition, the verification report(s) based on testing described in this document will be reviewed
to determine if verification is applicable. The following people from the cooperating ETV
programs will review these documents and provide input
• John Neate and Mona El Hallak, ETV Canada
• Mette Tjener Andersson, Claus J0rgensen, and Christian Gran, DANETV
Representatives of the cooperating ETV programs will:
• Participate in technical discussions during development of the test/QA plan to provide
input to the test design
• Review and provide input to the test/QA plan
• Review and provide input to the verification report(s)
• Draft the verification statements for their respective program if results are determined
to be acceptable.
-------
Coliform Detection Technologies
Test/QA Plan
Page 16 of 16
Version 1.0
Date: 7/14/2010
A6 BACKGROUND
A6.1 Technology Need
The ETV Program's AMS Center conducts third-party performance testing of
commercially available technologies that detect or monitor natural species or contaminants in air,
water, and soil. The purpose of ETV is to provide objective and quality assured performance
data on environmental technologies, so that users, developers, regulators, and consultants can
make informed decisions about purchasing and applying these technologies. The ETV Water
Stakeholder Committee, made up of buyers and users of such technologies recommend
technology categories, and technologies within those categories, as priorities for testing. Among
the technology categories recommended for testing are coliform detection technologies. In
particular, the use of coliform detection technologies for the monitoring of drinking water (DW)
was identified as an area of interest for technology verification. It is important to note that this
verification test is independent of any activities related to approval of technologies for use in
meeting regulatory requirements for the detection of total coliforms (TC) or Escherichia coll (E.
coli).
Fecal pollution can introduce disease-causing (pathogenic) bacteria, viruses, and parasites
into receiving waters, which may serve as private/public DW supplies. Through the
consumption of these waters, enteric and other pathogens can be transmitted to humans, resulting
in gastrointestinal illness. Utilities fully recognize the possibility of this waterborne pollution
and take every precaution (filtering, treatment with disinfectants such as chlorine and
chloramines, and regulatory compliance sampling and analysis) to avoid serving fecally polluted
water to consumers. Based on the 1989 Total Coliform Rule (TCR), assessment of this health
risk is based on the detection and enumeration of fecal indicator bacteria, such as TC and E. coli,
whose presence indicates that the water may be contaminated with sewage or animal wastes. In
this study, TC are defined as all facultative anaerobic, gram-negative, non-spore forming, rod-
shaped bacteria that ferment lactose with gas and acid formation within 48 hours (h) at 35°C, and
that develop red colonies with a metallic (golden) sheen within 24 h at 35°C on a Endo-type
medium containing lactose.3 E. coli is defined as a species of coliform bacteria that possesses the
enzyme p-glucuronidase and is capable of cleaving the fluorogenic substrate 4-
methyllumbelliferyl-p-D-glucorinide (MUG) with the release of fluorogen when grown in a
-------
Coliform Detection Technologies
Test/QA Plan
Page 17 of 17
Version 1.0
Date: 7/14/2010
MUG-based medium at 44.5°C within 24 ± 2 h or less, producing a blue fluorescence within the
media and/or around the periphery of the colony.3
The 1989 TCR sets both goals and legal limits for the presence of TC in DW. The rule
also details the type and frequency of testing that all public water systems must implement. The
Maximum Contaminant Level Goal (MCLG1) is no detectable TC in DW. The Maximum
Contaminant Level (MCL11) is no more than 5% of samples positive for TC in a month. For
water systems that collect fewer than 40 routine samples per month, no more than one sample
can be total coliform-positive per month. If a sample is positive for TC, each total coliform-
positive routine sample must be tested for the presence of fecal coliforms (fecal coliforms will be
removed from the revised TCR) or E. coli, and repeat samples must be collected and analyzed
for TC. As mentioned above, this verification test is not being conducted to provide data that
will be used to approve technologies for use in meeting regulatory requirements for the detection
of total coliform or E. coll as required by either the 1989 TCR or the upcoming revision to the
TCR. It is being conducted, based on feedback from ETV AMS Center stakeholders, to provide
a verification test that is similar in requirements to the current TCR approval process, such that
technologies that are not already approved have an opportunity to be tested under similar set of
test conditions.
Similar to the TCR in Europe, the Official Journal of the European Communities
published Council Directive 98/83/EC in 1998 that provided directives on the regulation of
public water systems within the European Union. Annex II and III within that document
includes the standard of zero E. coli per 100 mL and suggests the International Standards
Organization Method 9308 as the preferred method for regulatory E.coli analysis. More
recently, the Colilert-18 method has been shown to be equivalent to the suggested method and
therefore will be used to meet the DANETV requirement of European acceptance.
1 Maximum Contaminant Level Goal (MCLG) - Level of a contaminant in drinking water below which there is no
known or expected risk to health. MCLGs allow for a margin of safety and are non-enforceable public health goals.
"Maximum Contaminant Level (MCL) - The highest level of a contaminant that is allowed in drinking water. MCLs
are set as close to MCLGs as feasible using the best available treatment technology and taking cost into
consideration. MCLs are enforceable standards.
-------
Coliform Detection Technologies
Test/QA Plan
Page 18 of 18
Version 1.0
Date: 7/14/2010
A6.2 Technology Description
The coliform detection technologies to be tested use chromatogenic and fluorogenic
growth media to detect coliforms and E. coli based on the enzymatic activity of these organisms.
The systems consist of single-use sample containers that contain pre-measured reagents and can
be used in conjunction with a bench top incubator/analyzer/data logger system. In short, a 100
mL water sample is added to the containers and then incubated in and analyzed by the bench top
instrument. In order to accommodate the large number of samples required for this test, the
containers can be incubated in an external incubator, rather than in the bench top instrument, and
then returned to the instrument for analysis. The enzymes produced by TC and E. coli cleave the
chromatogenic or fluorogenic substrates in the growth media, resulting in the release of colored
or fluorescent products. The bench top instruments include optical sensors and/or
spectrophotometers that take readings of the sample at regular intervals to quantify the bacteria.
The sample data and instrument calibration are stored on the instrument's internal computer until
it is retrieved with a flash drive or some other portable data transfer device.
If no target bacteria are present in the sample, the sample color remains unchanged or
fluorescence is not produced. If the target organisms are present, there is a progressive change in
the color or fluorescence produced in the sample. For some technologies, results may be
available within 60 minutes after sample collection/test initiation, depending on the number of
bacteria present in the sample. However, the typical analyses are not completed until after 14 to
24 hours of incubation.
A7 VERIFICATION TEST DESCRIPTION AND SCHEDULE
This verification test will assess the performance of the coliform detection technologies
relative to key verification parameters, including specificity and sensitivity. In performing the
verification test, Battelle will follow the technical and QA procedures specified in this test/QA
plan and will comply with the data quality requirements in the AMS Center QMP (refer to Table
11 in Section B).
-------
Coliform Detection Technologies
Test/QA Plan
Page 19 of 19
Version 1.0
Date: 7/14/2010
A7.1 Verification Test Description
The objective of this verification test is to evaluate the technologies' performance
detecting the presence or absence of TC and E. coli against spiked concentrations of target
organism in DW. This verification test will take place at Battelle in Columbus, Ohio.
Technologies undergoing verification will be used to analyze spiked DW samples for TC and E.
coli. The presence/absence results from the technologies will be compared to the results from
the reference methods, Standard Methods (SM) 922IB and F3. These reference methods are
recognized in the U.S. and Canada as the standard for regulatory purposes. Each technology in
the test will be considered for possible verification in Canada. For those vendors that select the
optional verification in Denmark, their results will also be compared with Colilert -18 as this
product has recently been shown to be equivalent to the European Union approved method and
therefore will be used to meet the DANETV requirement of European acceptance.
Subsequent to the verification test, verification reports describing the test will be drafted.
These reports will be reviewed by the vendor and by peer reviewers, revised, and submitted to
EPA for final approval. In performing the verification test, Battelle will follow the technical and
QA procedures specified in this test/QA plan and will comply with the data quality requirements
in the AMS Center QMP. As a point of clarification, an ETV verification does not imply EPA
approval for use to meet regulatory requirements by current or future regulations (such as the
TCR).
A7.2 Proposed Verification Test Schedule
Table 1 shows the proposed schedule of testing, auditing, and data analysis/reporting
activities to be conducted during this verification. The verification of coliform detection
technologies is planned to be completed over the course of approximately three weeks after the
start of laboratory testing. The verification test is expected to be performed in July-August 2010.
The technical systems audit (TSA) will take place during testing; the data quality audit (DQAs)
will take place after the data are reviewed by the Test Coordinator, or designee. A daily
overview of the technical activities to be performed during the verification test is presented in
Section B 1.2.
-------
Coliform Detection Technologies
Test/QA Plan
Page 20 of 20
Version 1.0
Date: 7/14/2010
Table 1. Proposed Verification Test Schedule
Approximate Date(s)
January - June 2010
July and August 20 10
August 20 10
August 20 10
September 20 10
Testing Activities
Test/QA plan design and approval
Verification testing
Perform Technical Systems Audit
Audit of data (1st batch)
Reference analysis
Audit of data (remaining data)
Prepare draft verification reports and
statements
Coordinate reviews of draft verification
reports and statements
Prepare final verification reports and
statements
Data Analysis and Reporting
Not applicable
Compile data from technologies
Review and summarize testing staff
observations
Compile reference method results
Perform data analysis
Begin draft reports
Complete draft verification reports and
statements
Complete peer review and vendor review
of draft reports
Revise draft verification reports and
statements
Submit final reports for EPA approval1
Indicates EPA approval of ETV reports for release and does not imply approval for regulatory use
A8 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA
In performing the verification test, Battelle will follow the technical and QA procedures
specified in this test/QA plan and will comply with the data quality requirements in the AMS
Center QMP. QA Category II, Highly Visible Project, has been specified for this test by the
EPA Project Officer.
To ensure that this verification test provides suitable data for a robust evaluation of
performance, a variety of quality control (QC) samples will be incorporated for this test. Their
acceptance criteria are defined in Section B5. The acceptance criteria for the QC samples
indicate the minimum required performance to meet the objectives of the verification test and are
used as the QC criteria for laboratory measurements. The QC samples and acceptance criteria
for this verification test were established to assess the performance of the coliform detection
technologies relative to the reference method. In order to provide a suitable benchmark for
comparison, the reference measurements must meet the minimum requirements of the QC
sample acceptance criteria.
Method and reagent blank samples will be required to demonstrate freedom from
contamination, and positive and negative control cultures (American Type Culture Collection
-------
Coliform Detection Technologies
Test/QA Plan
Page 21 of 21
Version 1.0
Date: 7/14/2010
[ATCC]) will be processed to demonstrate the laboratories can accurately identify the target
organism and that the media and confirmation testing are providing appropriate results. ATCC
strains shown in Table 2, which are commonly used for water quality testing, will be acquired
from MicroBioLogics®. A Certificate of Analysis is provided with each strain that demonstrates
culture purity based on macroscopic, microscopic, and biochemical (phenotypic) testing.
Table 2. Quality Control Strains
Targeted Coliform
Total Coliform
Fecal Coliform
Positive Control
Enterobacter aerogenes
ATCC 13048
Escherichia coll
ATCC 8739
Negative Control
Pseudomonas aeruginosa
ATCC 10145
Enterobacter aerogenes
ATCC 13048
The Battelle Quality Manager or designee will perform a technical systems audit (TSA)
at least once during this verification test to augment these QC criteria. The EPA Quality
Manager also may conduct an independent TSA, at her discretion.
A9 SPECIAL TRAINING/CERTIFICATION
Documentation of training related to technology testing, data analysis, and reporting is
maintained for all Battelle technical staff in training files at their respective Battelle location.
Prior to the start of verification testing, the Battelle Quality Manager will verify the presence of
applicable training records, such as documentation, chain-of-custody, reference and technology
test methods, and equipment training. The vendors will be required to train the Battelle technical
staff prior to the start of testing. Battelle will document this training with a consent form, signed
and dated by the vendor, which states which Battelle technical staff have been trained to use their
technology and can train other staff. In the event that other staff members are required to use the
technologies, they will be trained by the operators that were trained by the vendors. All technical
staff will have a minimum of a bachelor's degree in science or equivalent work experience (e.g.,
experience performing bacterial analysis).
-------
Coliform Detection Technologies
Test/QA Plan
Page 22 of 22
Version 1.0
Date: 7/14/2010
Battelle will conduct all verification testing following the biosafety guidelines4
established at the Battelle laboratories (Battelle Building 20 Biosafety Manual, Version 4.0).
Exposure to biological organisms will be minimized, personal protective equipment will be
worn, and safe laboratory practices will be followed.
A10 DOCUMENTATION AND RECORDS
The documents for this verification test will include the test/QA plan, vendor instructions,
reference methods, verification reports, verification statements, and audit reports. The project
records will include laboratory record books (LRB) and data collection forms, supporting
laboratory records, training records, electronic files (both raw data and spreadsheets), and QA
audit files. Table 3 summarizes the types of data to be recorded. Documentation of Battelle staff
training by vendors and copies of other project specific training will also be included in the
project files. All of these records will be maintained in the Verification Test Coordinator's office
during the test and will be transferred to permanent storage at Battelle's Records Management
Office (RMO) at the conclusion of the verification test. All Battelle LRBs are stored indefinitely
with the project files by Battelle's RMO. Section BIO further details the data management
practices and responsibilities.
All data generated during the conduct of this project will be recorded directly, promptly,
and legibly in ink. All data entries will be dated on the date of entry and signed or initialed by
the person entering the data. Any changes in entries will be made so as not to obscure the
original entry, will be dated and signed or initialed at the time of the change and shall indicate
the reason for the change. Project-specific data forms will be developed prior to testing to ensure
that all critical information is documented in real time. The draft forms will be provided to the
Battelle QA Manager for review.
-------
Coliform Detection Technologies
Test/QA Plan
Page 23 of 23
Version 1.0
Date: 7/14/2010
Table 3. Summary of Data Recording Process
Data to Be
Recorded
Dates, times of test
events
Test parameters
Sample collection
data
Reference sample
data
Reference method
sample analysis,
chain of custody, and
results
Responsible
Party
Battelle
Battelle
Battelle
Battelle
Superior
Laboratories, Inc.
Where
Recorded
Laboratory record
books or data
collection sheets
Laboratory record
books or data
collection sheets
Laboratory record
books or data
collection sheets
Laboratory record
books or data
collection sheets
Laboratory record
books, chain of
custody forms,
data collection
sheets, or data
acquisition system,
as appropriate
How often
recorded
Start/end of test, and
at each change of a
test parameter
When set or changed,
or as needed to
document test
notable details during
testing
During each
sampling event
When test samples
are aliquotted for the
reference analysis
Throughout sample
handling and analysis
process
Disposition of
Data
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary
Used to document
collection, handling,
and storage of sewage
and drinking water
samples
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary
Transferred to
spreadsheets/agreed
upon report; project
files. Retained for
documentation of
reference method
performance
-------
Coliform Detection Technologies
Test/QA Plan
Page 24 of 24
Version 1.0
Date: 7/14/2010
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
Technologies undergoing verification will be used to analyze spiked DW samples for TC
and E. coli. Technology operation and sample handling and analysis will be performed
according to the vendor's instructions. The results from the coliform detection technologies will
be evaluated by comparing the proportion of positive results to the proportion of positive results
produced by SM9221B and F which includes the comparison of false positive rate (or
specificity) and false negative rate (or sensitivity. In addition, operational factors such as ease of
use, required reagents, analysis time, and laboratory space and utilities required will be
documented and reported.
Bl.l Verification Test Sample Preparation
The preparation of verification test samples includes the collection of raw sewage as the
source of the target organisms, collection of the DW sample, the fortification of the DW sample
with target organisms, and the chlorine stressing and dilution of samples for analysis. The
sample preparation steps and timeline are illustrated in Table 4 and described in detail in the
sections B1.1.1 through B1.1.3.
-------
Coliform Detection Technologies
Test/QA Plan
Page 25 of 25
Version 1.0
Date: 7/14/2010
Table 4. Sample Preparation Timeline
Day
Process
Battelle Key Activities
Collect Sewage
Collect sewage in early AM from wastewater treatment plant
Remove excess solids by filtration
Characterize sewage
Enumerate total coliform (TC) and E. coli via Standard Method (SM)
9222B/G
Enumerate total heterotrophic bacteria via standard spread plating
(R2A agar)
Measure pH (SOP GEN.V-003-10), free and total chlorine (HACH
Method 8021 & 8167)
Collect Drinking Water
Collect approximately 50 L drinking water from tap in Battelle laboratory
Measure pH (SOP GEN.V-003-10), free and total chlorine (HACH
Method 8021 & 8167)
Adjust 30 L DW to 2.0-2.5 mg/L free chlorine for chlorine stressing
Chlorine Stressing
(stress bacteria; reduce
TC/£. coli by 2-4
orders of magnitude)
(prepare spiked,
stressed TC or E. coli
DW)
Spike three 10 L DW samples with 200 mL filtered sewage (targeting 10 "
org./lOO mL for TC or E. coli)
Dechlorinate after 2.5, 5, and 10 minutes
Enumerate each spiked, stressed, DW sample (SSDW)
Refrigerate SSDW
Prepare dilutions for
sample analysis
Based on Day 1 enumeration results, prepare three 10 L dilutions of
SSDW targeting 0.1, 1, and 10 TCorE1. coli/WO mL from one of the three
SSDW 10 L aliquots
Complete Sample
Preparation; Initiate
Verification Testing
Dispense 20 replicate 100 mL aliquots of each dilution into sterile
reservoirs (total of 60 samples plus 1 negative matrix control, 1 negative
TC control,, 1 positive TC/negative E. coli, and 1 positive E. coli for a
total of 64 samples per method)
Promptly transfer required samples to the reference laboratory3 on wet ice
to begin Reference testing (SM 9221B/F) and Colilert®-18 testing
Initiate vendor analysis at Battelle
Superior Laboratories, Inc., 1075 Amity Road, Galloway, OH 43119-9304, (614- 793-2172). Attn: James Gossard
B1.1.1 Sewage and Drinking Water Sample Collection
A single raw sewage sample (approximately 1 L), will be collected in the morning hours
from the influent sewage line entering the Southerly Wastewater Treatment Plant (SWTP) in
Columbus, OH by SWTP staff. The sewage will be immediately stored on wet ice, and
transported by Battelle staff to Battelle laboratories. Upon receipt, the sewage will be filtered
through a Whatman No. 2 pleated filter under vacuum using a Buchner funnel to remove excess
solids, shaken vigorously for 1 minute to insure homogeneity, and then immediately
characterized as follows:
• Measure pH using calibrated pH meter (SOP GEN.V-003-10)5
-------
Coliform Detection Technologies
Test/QA Plan
Page 26 of 26
Version 1.0
Date: 7/14/2010
• Measure free chlorine and total chlorine using a HACK Chlorine test kit [(HACK
Method 8021 (free) and Method 8167 (total); procedures are equivalent to U.S.
EPA Standard Method 4500-CL G for drinking water]
• Quantify the total, culturable heterotrophic bacteria by enumerating onto R2A
agar using a standard spread plating method as described in the AOAC's
Bacteriological Analytical Manual6.
• Quantify TC and E. coli (organisms per 100 mL) using quantitative methods for
TC (SM 9222B - m-Endo) and E. coli (SM 9222G - NA-MUG)3.
- Based on recent analysis of sewage samples collected from SWTF, the
concentration of total coliform and E. coli is expected to be in the range of
106"7 organisms/100 mL.
- To obtain readable filters, serial dilutions of the sewage will be prepared
- Total coliform concentrations will be based on the number of typical
coliform colonies (pink to dark-red with metallic surface sheen) observed
on m-Endo.
- To determine E. coli concentrations, the filter membranes will be
transferred from m-Endo onto NA-MUG, incubated for 4 hours at
35±0.5°C, and then held under a long-wave length ultraviolet light.
Colonies that fluoresce blue are counted as E. coli.
A single DW sample (approximately 62 L) will be collected into multiple, sterile carboys
the same day the sewage sample is collected. This volume will accommodate the preparation of
a complete set of 100 mL aliquots for one round of verification testing for TC and E. coli. Table
5 shows that approximately 62 L of DW will be required to prepare three 10.5 L volumes for
chlorine stressing (32 L) and to prepare the three dilutions for sample analysis from one of the
chlorine stressed samples (30 L). These diluted samples will be the source of the 100 mL
samples that will be analyzed by the technologies being tested as well as the reference methods.
The DW sample will be collected as follows:
(1) Remove faucet screen if present, and surface decontaminate with 70% isopropanol.
(2) Purge line for 2 to 3 minutes with cold water.
-------
Coliform Detection Technologies
Test/QA Plan
Page 27 of 27
Version 1.0
Date: 7/14/2010
(3) Collect approximately 62 L DW from the tap into multiple sterile (autoclaved)
carboys equipped with a spigot and containing large stir bars.
Once collected, a 100 mL aliquot will be collected from the carboy's spigot and used to
characterize the DW by measuring the pH (SOP GEN.V-003-10) and the concentration (mg/L)
of free and total chlorine using a HACK Chlorine kit (HACK Methods 8021 and 8167). Since
recent chlorine tests of the tap water indicate the water generally contains 0.2 to 1.5 mg/L free
chlorine, sodium thiosulfate will not be added prior to the chlorination step. The DW sample
will be used the same day as collected for the chlorine stressing and stored at 2°C to 8°C and
used the following day to prepare the diluted samples for analysis. DW used to dilute the stress
samples will be dechlorinated prior to use.
Table 5. Volume of Drinking Water Required for Verification Testing
Step
Chlorine Stressing
Dilution and Sample
Preparation
Description
Prepare three 10.5 L spiked aliquots for
three timed exposures
Prepare three 10.5 L dilutions from one
of the 10.5 L Chlorinated aliquots
above ; 2 L required for each method
(60 x lOOmL);
-control preparation
-replicate 100 ml samples will be
generated from these three dilutions
Total=
DW Volume Needed
32L
Approx. 30 L
(Volume required for prep, of
three dilutions depends on
concentration of chlorinated
sample selected for sample
prep.)
62L
Bl.1.2 Chlorine Stressing and Preparation of Samples for Verification Testing
The goal of the chlorine stressing step is to reduce the number organisms in the spiked
DW samples by two to four orders of magnitude, starting at approximately 105 target
organisms/100 mL. In addition, this step also stresses the organisms with chlorine. Three spiked
DW samples will be chlorinated for 2.5, 5, or 10 minutes, after which time they will be
dechlorinated with sodium thiosulfate and subsequently enumerated to determine the reduction
of target organisms in each spiked, stressed drinking water (SSDW) sample. Chlorine stressing
will be initiated the day the sewage and DW samples are collected in order to minimize the
storage time between sewage and DW collection and the start of sample analysis. Following
-------
Coliform Detection Technologies
Test/QA Plan
Page 28 of 28
Version 1.0
Date: 7/14/2010
chlorine stressing, one of the three 10 L dilutions of SSDW will be selected to prepare a set of
three 10 L dilutions prepared into dechlorinated drinking water targeting a range of 0.1 to 10
target organisms/100 mL. The chlorine stressing and sample preparation steps are detailed
below:
Chlorine Stressing Procedure:
(1) Dispense 10.5 L of the original DW sample into each of three, sterile 10 L
carboys containing stir bars. Place carboys on stir plates and mix at medium
speed.
(2) Measure the total residual and free residual chlorine concentration using a HACK
Chlorine test kit (approved for SM 4500-C1- G3) in each carboy. If necessary,
adjust the free residual chlorine level to 2.0-2.5 mg/L using a 4% solution (40,000
mg/L) of sodium hypochlorite. Since the spiked DW sample will generally have a
low chlorine demand, overstressing or killing the organisms by prolonged
exposure to free chlorine must be avoided. The total residual chlorine level
should not exceed 2.5 mg/L.
(3) Measure pH by collecting a 10 mL aliquot from each carboy and measure using a
calibrated pH meter, following SOP GEN.V-003-105.
(4) Measure temperature of water in each carboy using a calibrated thermometer.
Thermometers are calibrated according to SOP GEN.V-013-047. [Surface
decontaminate thermometer with 70% isopropanol before each use].
(5) Start chlorine stressing by spiking each 10.5 L DW sample with 200 mL filtered
sewage to achieve a concentration of approximately 105 target organisms/100 mL.
The concentration of TC or E. coli will be assumed to be 104-105 org./mL (as
indicated by analysis of sewage collected over the past several weeks leading up
to the verification study).
(6) At each of three time points (2.5, 5, and 10 minutes after spiking with sewage):
a. Add 8.6 mL 10% (w/v) sodium thiosulfate solution (i.e., 0.8 mL per L) to
stop chlorine oxidation (dechlorinate). Allow to mix for at least 1 minute.
b. Measure pH and temperature of sample.
c. Measure free residual chlorine and total chlorine.
-------
Coliform Detection Technologies
Test/QA Plan
Page 29 of 29
Version 1.0
Date: 7/14/2010
(7) Enumerate the target organism densities of each SSDW via SM 9222B for TC and
SM9222Gfor£. coli. .
a. Enumerate triplicate 100 mL aliquots of undiluted sample, and triplicate
100 mL aliquots of 10"1 and 10"2 dilutions of the sample; using phosphate
buffered water as the diluent.
b. Calculate TC and E. coli/100 mL in each 10 L sample.
(8) Calculate the reduction of TC and E. coli based on the estimated spiking (starting)
concentration and the concentrations after each exposure time.
If none of the exposure times used was adequate to achieve the desired reduction of two
to four orders of magnitude, the entire process will be repeated using a freshly collected sewage
sample and DW. The measured density of chlorine-stressed target organisms confirms that a two
to four order of magnitude reduction has been attained, and can then be used to prepare three
dilutions that target a density of 0.1 to 10 target organisms/100 mL. While it is difficult to
determine if a single target organism is present in 100 mL of water, when approximately half of
the analyzed replicates are positive and half are negative, the density of the organism has become
adequately low so that a positive result can be considered single organism detection. Therefore,
the ultimate objective is to have one of the prepared dilution sets provide 50 ±25% positive
results for TC or E. coli (depending on which is being targeted) with the reference method(s).
Based on the results of the chlorine stressing step, three 10.5 L volumes of DW with TC
or E.coli concentrations of 0.1 org./lOO mL, 1 org./lOO mL, and 10 org./lOO mL, respectively,
will be prepared, and replicate 100 mL aliquots of each dilution will be subsampled for
verification testing. Preparation of the dilution samples will be performed as follows:
(1) Calculate the volume of the original DW needed to prepare 10.5 L of each of three
dilutions, and dispense this volume into one or more carboys. Measure the free
chlorine concentration, and dechlorinate if the free chlorine concentration is in the
range of 0.2 - 0.5 mg/L by adding 0.8 mL 10% sodium thiosulfate per 1 L. Use this
dechlorinated DW (DDW) to prepare the dilutions.
(2) Accurately dispense the appropriate volume of DDW to each of three sterile carboys
having spigots and containing large stir bars.
-------
Coliform Detection Technologies
Test/QA Plan
Page 30 of 30
Version 1.0
Date: 7/14/2010
(3) Spike the DDW with the appropriate volume of SSDW sample to each of the three
carboys to generate suspensions of 0.1 org./lOO mL, 1 org./lOO mL, and 10 org./lOO
mL.
(4) Each dilution will be treated as follows:
a. Mix on stir plate for 5-10 minutes.
b. During continuous mixing on the stir plate, dispense 100 mL aliquots into
sterile 100 mL bottles using 50 mL and/or 100 mL graduated pipets. All
bottles will be pre-labeled with a unique sample ID.
c. Store bottles at 2 °C to 8 °C as they are generated.
d. Immediately after all samples have been dispensed for the reference methods,
transport them in coolers packed with ice packs to Superior Laboratories, Inc.,
which is approximately a 15 to 20 minute drive from the sample preparation
laboratory. Reference method analysis following SM9221B/F and Colilert -
18 will commence the same day as arrival at the laboratory.
Once all 100 mL aliquots are dispensed for technology verification (20 at each dilution level for
a total of 60 replicates per technology), verification testing will begin. Figure 2 illustrates the
experimental design for the sample dilution. Not shown in Figure 2 are the QC samples which
will be prepared and analyzed at the same time.
-------
Coliform Detection Technologies
Test/QA Plan
Page 31 of 31
Version 1.0
Date: 7/14/2010
Reference
Methods
Dilution 1
(0.1 org./100ml_)
20 replicates SM9221B
(LIB)
20 replicates
Colilert-18
Spiked, Stressed
Sample
(TC or E. coll)
Dilution 2
org./100mL)
20 replicates SM 9221B
(LIB)
20 replicates
Colilert-18
Dilutions
(10org./100ml_)
20 replicates SM 9221B
(LIB)
20 replicates
Colilert-18
TC&
E co//
E co//
only
Technologies
Being Verified
20 replicates
Each Verified Technology
20 replicates
Each Verified Technology
20 replicates
Each Verified Technology
TC&
E co//
Figure 2. Experimental Design for Sample Dilution and Analysis
-------
Coliform Detection Technologies
Test/QA Plan
Page 32 of 32
Version 1.0
Date: 7/14/2010
B1.2 Sample Analysis
The number of replicates to be analyzed by each technology and reference method was
selected following a power analysis based on a Pearson chi-square test for equality of
proportions associated with two independent populations, where the test is two-sided (i.e., test
for equality versus any type of non-equality). The populations being compared are the two test
types (reference and technologies being verified), and the proportion corresponds to the
proportion of positive results out of the total tests performed. Conducted using the POWER
procedure in the SAS System, the power analysis determined the number of replicate tests
(across both test types) that is necessary to detect a specified difference in proportions of a
specified size with 80% power, given a specified value of the proportion for the reference test
(the acceptable range of reference test positive proportions will be 0.25 to 0.75 for this test), and
a significance level of 0.05 for the test. Table 6 gives the power analysis results for reference
method positive proportions of 0.25, 0.5, and 0.65. The results indicate that approximately 20
replicate samples of the technologies being tested will be adequate to determine significant
differences in the proportion of positive results generated by the technologies being tested and
the reference method. The absolute differences in proportion relative to the reference test that is
shown through this power analysis to be detectable as significantly difference were 0.45, 0.4, and
0.325 (shaded results) for the three reference method positive proportions of 0.25, 0.50, and 0.65,
respectively. Differences in positive proportion less than those may be detectable, but not with
80 percent power under the assumptions of the test.
As shown in Table 6, the use of approximately 20 replicates provides a likelihood that a
significant difference can be determined between the reference method and the technologies
being tested. If a significant difference is determined based on the results of this ETV test, the
result means that the reference method and technology being tested are at least different by the
shaded ratios shown in the table. Upon discussion with the peer reviewers for this verification
test, it was determined that use of 20 replicates would provide data that was useful to the end
user of these technologies. The reviewers determined that, because this test does not have
regulatory implications, the result will be valuable as a screening tool to help with purchase
decisions.
-------
Coliform Detection Technologies
Test/QA Plan
Page 33 of 33
Version 1.0
Date: 7/14/2010
Table 6. Power Analysis Results for Determination of Replicates
Reference
Method Ratio of
Positive Results
0.25
0.50
0.65
Reference
Method
N
71
152
1,251
1,251
152
58
31
19
15
20
39
93
388
388
93
39
20
22
39
189
804
189
39
22
Tested
Technology
Ratio of
Positive
Results
0.025
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.05
0.10
0.20
0.30
0.40
0.60
0.70
0.80
0.90
0.325
0.390
0.520
0.715
0.780
0.910
0.975
Difference in
Positive
Proportion
Between
Reference and
Technology
0.225
0.15
0.05
0.05
0.15
0.25
0.35
0.45
0.45
0.4
0.3
0.2
0.1
0.1
0.2
0.3
0.4
0.325
0.26
0.13
0.065
0.13
0.26
0.325
Required Tested
Technology N
for Significant
Difference
71
152
1,251
1,251
152
58
31
19
15
20
39
93
388
388
93
39
20
22
39
189
804
189
39
22
Test
Method
Positives
2
15
251
375
61
29
19
13
1
2
8
28
233
155
28
31
18
7
15
98
575
147
36
21
Test
Method
Negatives
69
137
1,001
876
91
29
12
6
15
18
31
65
155
233
65
8
2
15
24
91
229
42
o
J
1
The ability of each coliform detection technology to determine the presence of TC and E.
coli will be challenged using spiked stressed DW samples. Positive/negative control samples
spiked with quality control cultures listed in Table 2 will be included during testing. The
technologies will be operated according to the manufacturer's instructions and vendor will
provide the necessary equipment and supplies to analyze up to sixty 100 mL samples at a given
time. The dilution samples generated in Section Bl.l will be assayed by the reference methods
and the technology methods concurrently. The reference analyses will be conducted as described
-------
Coliform Detection Technologies
Test/QA Plan
Page 34 of 34
Version 1.0
Date: 7/14/2010
in Section B4. Table 7, which is continued from Table 4, presents an overview of the daily
verification test activities during days 3 through 7. The details of the activities are described in
sections B 1.2.1 and B 1.2.2.
Table 7. Sample Analysis Timeline
Day
Battelle and Superior Key Activities
9221B
(Superior)
9221F
(Superior)
Colilert®18
(Superior)
Vendor
Technologies
(Superior)
Vendor
Technologies
(Battelle)
1
Sewage and DW collection, and chlorination (refer to Table 4)
2 (Dilutions
already prepared)
100 mL SSDW into LTB
Start analysis on
replicate 100 mL
samples
Replicate 100 mL
samples into
vendor supplied
containers; start
analysis
24h LTB into
BGLB /EC-MUG
24 h LTB into
EC-MUG
Complete tests
Receive
Vendor
analyzed
samples; start
Confirmed
phase by
inoculating
LTB
Vendor Results;
transport all
samples to
Superior Labs
48h LTB into
BGLB /EC-MUG
48 h LTB into
EC-MUG
24h LTB into
BGLB /EC-
MUG
Complete tests
from Day 3
48h LTB into
BGLB /EC-
MUG
Complete tests
from Day 4
Complete
tests from
Day 4
(Optional)
Additional
completed tests if
deemed
appropriate
Complete
tests from
Day 5
(Optional)
Additional
completed
tests if
deemed
appropriate
The data used for verification will be from the dilution(s) which produces results closest
to an equal number of positive (50%) and negative (50%) results for the reference method. A 50
± 25% split in responses (in either direction; i.e., 25:75 or 75:25% split) will be sought. For
comparability, the evaluated results for the coliform detection technologies will be from the same
dilution level as the 50 ± 25% positive reference method results. If none of the dilutions
-------
Coliform Detection Technologies
Test/QA Plan
Page 35 of 35
Version 1.0
Date: 7/14/2010
produces an acceptable split in positive and negative results for the reference method, the process
may be repeated starting with the collection of fresh sewage and DW. The results from the
dilution sets not used for the data evaluation will be documented, but the samples will be
discarded. Confirmation tests will be conducted on the dilutions to be used for the data
evaluation.
B1.2.1 Confirmation of Results
All sample results must be confirmed with more definitive tests in order to adequately
compare results between the reference method and the technologies being tested. Confirmation
for both the SM 922IB and F reference methods, as well as the technology methods, is described
below. Figure 3 illustrates the process by which all positive and negative samples from the
verification technologies and SM 922IB and F are confirmed. Although all samples will be
processed through the confirmation phase, only the results from the dilution of reference samples
that gives the 50 ± 25% positive result will be used to compare the technologies to the reference
methods.
Confirmation of reference method (SM 922 IB and F) results
1) All lauryl tryptose broth (LTB) bottles showing acidic reaction (distinct yellow
color) within 24 ± 2 hours of incubation will be submitted for confirmatory
testing according to SM 922IB using inoculation into brilliant green lactose bile
(BGLB) broth. Negative LTB samples will continue to incubate.
o If acidic reaction in LTB is displayed at the end of a 48 ± 3 hour
incubation period, these samples will also be submitted to the confirmed
phase by simultaneous inoculation into BGLB and EC-MUG broth
o The absence of acidic reaction in LTB at the end of 48 ± 3 hours of
incubation constitutes a negative test for TC and E. coli.
2) Formation of gas in the BGLB broth fermentation tube at anytime within a 48 ± 3
hour incubation period constitutes a positive confirmed phase for TC.
3) After incubation in EC-MUG, the appearance of a blue fluorescence under long-
wave (approximately 366 nm) UV light after 24 h incubation at 44.5°C constitutes
a positive confirmed phase for E. coli.
-------
Coliform Detection Technologies
Test/QA Plan
Page 36 of 36
Version 1.0
Date: 7/14/2010
4) As additional, optional confirmation, a complete test for TC may be performed by
inoculating MacConkey media and then selecting suspected TC colonies and
inoculating into LIB, as described by SM 9221B.
No confirmation testing is required for Colilert®-18. Positive Colilert®-18results for TC
are indicated by presence of yellow color equal to or greater than the Colilert®-18comparator
control after 18 h at 35°C. The appearance of blue fluorescence when exposed to long-wave UV
light represents a positive reaction for E. coli.
Confirmation of the samples analyzed by the vendor technologies will be performed as
follows:
(1) Transport all vendor samples to Superior Labs after all vendor analyses have been
performed.
(2) .For each vendor samples following proper incubation time and result determination
(regardless if positive or negative): inoculate 10 mL LTB into test tubes containing
Durham tubes with 1 mL of vendor sample to initiate confirmation tests.
(3) Confirm results as illustrated in Figure 3 and following SM 922IB and SM9221F.
-------
-~-~ " —
f TC Dilution \ /^^ AH c /w,- "^X
/ _n ,_0/ \ f All E. co// X
V D ^ M % 7 V Dilutions J
\ Positive:Negative / ^^^^^~-'^^^f
^--^ ^--^
Inoculate
BGLB
Confirm TC
( \ Inoculat*
LTB-K) L^ eriunir ^
Replicates EC MUG
(SMS^IB) T .....
L J Test for £. co/i
i SM9"'"11F > UV366
SM Reference (EC MUGI Fluorescence
Method
Technologies
Being Verified
rosmve I*) uontirm (.+.) results |nrtrui^iTR
Rpplicat^ False + Specificity ^ Inoculate LTB
"
Negative H _^ uonnrm (-) «esuns inoculate I TB
Rpplirates FsA« - .
-------
Coliform Detection Technologies
Test/QA Plan
Page 38 of 38
Version 1.0
Date: 7/14/2010
B1.3 Statistical Analysis
Bl.3.1 False Positive Rates, False Negative Rates, Sensitivity, and Specificity
False positive (FP) and false negative (FN) rates of reference methods will be evaluated
when assessing comparability. During this test, true positives are those positive results from the
technologies being tested that are confirmed, and false positives are those positive results from
the technologies being tested that are not confirmed as directed by the reference method.
Conversely, true negative results are those negative results that are confirmed as negative, and
false negative results are those negative results that were shown to be positive by the
confirmatory method. Performance of the coliform detection technologies will be tested by
comparing the proportion of true positive results from those technologies to the proportion of
positive results from the SM 9221B and F and Colilert®-18.
Specificity is defined as the percent of negative samples correctly identified as negative,
and sensitivity is defined as the percent of positive samples correctly identified as positive.
Estimates of sensitivity, specificity, false positive rates, and false negative rates as percentages
for the two methods will be calculated as follows:
jp.
Sensitivity; = Vv x 100%
I"r ?i»
Specificity, = T^ ^pp; x 100%
PP.- f 77v, \
False positive rate.- = -=^-, — Vw x 100% = 1 - -=^-, — LF-=r x 100% 1 - Specificitr.-
T^i + rPi \ T^'i + rP.J
f TP.- \
False negative rateL - •== - J^T- x i°°°'c = U ~ ^p — V^rl x: 1°°% = 1 ~ Sensiti
Where,
F = False
N = Negative
P = Positive
T = True
i = specified method (i = 1 for technology being tested, i = 2 for reference
method)
Bl.3.2 Method Comparability
The chi-square test will be used to compare the percent of false positives and false
negatives in the coliform detection technologies with the percent of false positives and false
-------
Coliform Detection Technologies
Test/QA Plan
Page 39 of 39
Version 1.0
Date: 7/14/2010
negatives produced by the reference method. To perform the chi-square test, the data will first
be arranged in the format shown in Tables 8 and 9:
Table 8. False Negative Rate Comparison
Result
True +
False -
Total
Method
Tested
Technology
TPj
FNi
TPj + FNi
Reference
TP2
FN2
TP2 + FN2
Total
TPj + TP2
FNi + FN2
TPJ+TP2+FNJ + FN2
Table 9. False Positive Rate Comparison
Result
True +
False -
Total
Method
Tested
Technology
FPi
TNi
FPi + TNi
Reference
FP2
TN2
FP2 + TN2
Total
FPi + FP2
TNi + TN2
FPi + FP2+TNi + TN2
In order to assess whether the false positive and false negative rates differ between
methods, chi-square tests will be run over all samples. For false negative rates, the chi-square
test(s) indicate whether the proportions of negative samples correctly identified as negative by
the two methods are significantly different, and for false positive rates, the chi-square test(s)
indicate whether the proportions of true positive results correctly identified as positive for the
two methods are significantly different.
Bl.3.3 Operational Factors
Operational factors such as maintenance needs, calibration frequency, data output,
consumables used, ease of use, repair requirements, waste production, and sample throughput
will be documented based on operator and Verification Test Coordinator observations.
B1.4 Additional Concentration Levels
An optional component of the ETV test will be performed to verify the capability of each
technology to detect E. coli ATCC 8739 at various concentration levels. Four target inoculations
will be prepared in dechlorinated DW. A stock solution will be prepared that contains
-------
Coliform Detection Technologies
Test/QA Plan
Page 40 of 40
Version 1.0
Date: 7/14/2010
approximately 104 E. coli per 100 mL, and then a serial dilution (1:10, 1:100, and 1:1,000) of
the stock will be prepared to obtain four separate samples for testing (10, 100, and 1,000 E. coli
per 100 mL). Four replicates of each sample will be analyzed to establish kinetic data and "early
warning" capabilities of these technologies. The data from these tests will identify: 1) whether
or not each technology detects the presence ofE.coli and 2) the time required for detection.
Triplicate aliquots at each will also be analyzed using a quantitative method for E. coli (SM
9222G-NA-MUG).
B1.5 Reporting
Separate verification reports and verification statements are planned for each vendor that
is participating in the verification testing. The statistical comparisons described above will be
conducted separately for each of the technologies being tested, and information on the
operational factors will also be compiled and reported separately for each technology. The
verification reports will present the test procedures, test data as statistical evaluation of those
data, and discuss any deviations from the approved test/QA plan.
Operational aspects of the technologies will be recorded by the testing staff at the time of
observation during the verification test, and summarized in the verification reports. The
verification report and verification statements will briefly describe the ETV program, the AMS
Center, and the procedures used in verification testing. The results of the verification test
regarding coliform detection technology performance will be stated quantitatively. Each draft
verification report and verification statement will be subjected to review by the vendor, EPA, and
other peer reviewers. The resulting review comments will be addressed in a subsequent revision
of the report or statement, and the peer review comments and responses will be tabulated to
document the peer review process and submitted to EPA. The reporting and review process will
be conducted according to the requirements of the AMS Center QMP. The cooperating ETV
programs will deem whether the results are acceptable. If acceptable, a separate verification
statement will be prepared by the cooperating ETV program.
B2 SAMPLING METHOD REQUIREMENTS
Sampling method requirements are described above in Section B 1.1.1.
-------
Coliform Detection Technologies
Test/QA Plan
Page 41 of 41
Version 1.0
Date: 7/14/2010
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Sample custody will be documented for the DW, SWTP water, and reference test samples
following Battelle SOP ENV-ADM-009 for Chain of Custody8. A chain-of-custody (COC) form
will include details about the sample such as the time, date, location, and person collecting the
sample. The COC form will track sample release from the sampling location to the testing
laboratory. The COC form will be signed by the person relinquishing samples once that person
has verified that the COC form is accurate. Upon arrival at the testing laboratory, COC forms
will be signed by the person receiving the sample once that person has verified that all samples
identified on the COC forms are present. Copies of all COC forms will be delivered to the
Verification Test Coordinator and maintained with the test records.
B4 LABORATORY REFERENCE METHODS
Technology verification will involve comparison of the results from each coliform
detection technology being verified to the results obtained from appropriate reference methods
which are described further in Table 10. Superior Laboratories, Inc. will perform the reference
analysis, including all QA/QC procedures, and will comply with this test/QA plan and all
requirements of the ETV QMP. Prior to testing, the Battelle QA Manager will contact them and
confirm that adequate quality systems are in place and then a TSA will include that laboratory.
Any options described in the method will be considered and determined prior to testing and
documented in the laboratory records. All laboratory records required by the method will be
maintained. The reference methods chosen for the U.S. verification test are presence/absence
methods for TC and E. coli per SM 922IB (TC) and 922IF (E. coif). These methods utilize
selective and/or chromatogenic liquid growth media to detect TC and E. coli. In addition, an
optional verification of these technologies for the detection of E. coli will be performed using
Colilert®-18. The two reference methods (SM and Colilert®-18) will be run in parallel during the
verification test, and the results for both will be reported as presence/absence. Reference data
from the Colilert -18 method will be compared to only those technologies for which the vendor
has agreed to verification of E. coli results.
-------
Coliform Detection Technologies
Test/QA Plan
Page 42 of 42
Version 1.0
Date: 7/14/2010
Table 10. Reference Methods
Analyte
Total coliforms
E. coli
E. coli
Method Format
P/A
P/A
P/A
Approving Agency
U.S. EPA
U.S. EPA
European Union
Reference Method
SM9221B
SM9221F
Colilert-18
Citation
40 CFR 14 1.2 1111
40 CFR 141.2F
Drinking Water
Directive 98/83/EC
(3 November 1998)
P/A = presence/absence
Colilert®-18method is an optional reference method and will only be included in the verification test for each of the
technologies upon the vendor's agreement.
B5 QUALITY CONTROL REQUIREMENTS
Quality control procedures will follow the requirements described in this test/QA plan,
ETV QMP, and any vendor specified requirements for the coliform detection technologies.
Some of the technologies may require calibration before testing begins. If so, the calibration
method and results will be documented.
The reference method requires the use of method blanks (MB), positive and negative
control organisms, and result confirmation (Table 11). One MB will be performed during the
analysis for every 20 samples analyzed. The MB will consist of 100-mL dechlorinated tap water
processed as a sample. MB samples will be exposed to identical handling and analysis
procedures as the DW samples, including the addition of all reagents. These samples will be
used to help ensure that no sources of contamination are introduced in the sample handling and
analysis procedures. The MB should be rejected if the target organisms are detected.
Three positive and negative control samples will also be analyzed using each method.
Positive and negative control samples will be obtained from ATCC. Control organisms include
the total coliform Enterobacter aerogenes (ATCC 13048), E. coli (ATCC 8739), and the non-
coliform Pseudomonas aeruginosa (ATCC 10145). All control cultures will be prepared at
Battelle following the manufacturer's instructions. In short, freeze dried cultures will be
rehydrated using the medium and incubation conditions specified in the manufacture's catalog.
The QC control samples will be prepared by diluting the cultures in sterile deionized water for
11140 CFR 141.21 - Coliform sampling. Code of Federal Regulations - Title 40: Protection of Environment
(December 2005)
-------
Coliform Detection Technologies
Test/QA Plan
Page 43 of 43
Version 1.0
Date: 7/14/2010
analysis. Control samples will be used to determine the specificity and sensitivity of the
coliform detection technologies and reference methods.
In addition, reference results for one of the three dilutions sets for each target organism
must result in a ratio of 50 ± 25% positive results. If these requirements are not met, a new set of
samples may be analyzed by the reference method. If the results are still outside the required
limits, the repeat of the appropriate parts of the verification test may be considered.
B6 EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The equipment used for the reference analyses will be tested and inspected as per the
equipment manuals, the standard operating procedures of the analysis laboratory, or the methods
being used to make each measurement; results will be documented. Operation of the
technologies during the verification test will be performed by Battelle as directed by the vendor.
Temperatures for the refrigerators and freezers will be logged on a daily basis while test
samples are in storage. If found to be outside of the specified range above, the samples will be
-------
Table 11. Quality Control Samples for Reference and Technology Methods
Coliform Detection Technologies
Test/QA Plan
Page 44 of 44
Version 1.0
Date: 7/14/2010
Performance
Test
Method Blank
TC Positive
control
E. coli Positive
control
Negative Control
Method of
Assessment
Analysis of filter-
sterilize,
dechlorinated tap
water
Analysis of ATCC
E. aerogenes
Analysis of ATCC
E. coli
Analysis of ATCC P.
aemgenes
Dilution
N/A
N/A
N/A
N/A
Applicable
Method
Reference and
Technology
Reference and
Technology
Reference and
Technology
Reference and
Technology
Frequency
Once every 6-0
samples
Once every 60
samples
Once every 60
samples
Once every 60
samples
Total Number
per Analysis
and Target
Organism
o
J
3
o
3
3
Acceptance
Criteria
No growth
Target
organisms
detected
Target
organisms
detected
Target
organisms not
detected
Corrective Action
Review data and analysis for
possible sources of
contamination. Re-analyze QC
sample and document corrective
action.
Recheck controls for evidence
of cross contamination or
deterioration.
Recheck controls for evidence
of cross contamination or
deterioration.
Recheck controls for evidence
of cross contamination or
deterioration.
Note: Sterility control testing on every lot of reagents (buffers and media) used will be performed during each batch (iteration) of testing.
-------
Coliform Detection Technologies
Test/QA Plan
Page 45 of 45
Version 1.0
Date: 7/14/2010
transferred to an acceptable refrigerator or freezer and the deviation will be noted in the LRB, in
a deviation report, and in the final verification reports.
B7 CALIBRATION VERIFICATION
Balances, thermometers, pH meters, autoclaves, incubators, pipettes, etc. used during
sample preparation and analysis will be maintained and calibrated as required by the Battelle or
reference laboratory SOPs. A list of relevant SOPs is attached to this test/QA plan. Equipment
maintenance logs and calibration records will be reviewed by the Battelle Quality Manager prior
to the verification test.
One laboratory clock will be used as the standard for all time recording. Any other time
pieces must be calibrated to it. In particular, the sewage and DW sample collection time must be
determined based on a calibrated time piece.
If necessary, the technologies undergoing testing will be calibrated initially by the
respective vendors at the time of set-up, at the vendor's discretion.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
All materials, supplies, and consumables will be ordered by the Verification Test
Coordinator or designee. Where possible, Battelle will rely on sources of materials and
consumables that have been used previously as part of ETV verification testing without
problems. Battelle will also rely on previous experience or recommendations from its technical
staff to guide selection of manufacturers and materials. The manufacturer's criteria for
acceptance/purity/sterility will be required to be met. Supplies must meet the following criteria:
• Solvent and reagent grades are based on the intended use. All materials must
meet the purity requirements of the method.
• Equipment used to generate data must provide appropriate sensitivity.
• A certificate of analysis must be retained for reagents and standards.
• A certificate of sterility must be retained for sample containers and growth media.
• The quality and purity of expendable materials must be adequate to meet the data
quality objectives of the client.
-------
Coliform Detection Technologies
Test/QA Plan
Page 46 of 46
Version 1.0
Date: 7/14/2010
These same requirements for quality must be implemented by the reference laboratory. The
laboratory manager is responsible for verifying the quality of supplies used for this test.
B9 NON-DIRECT MEASUREMENTS
Data published previously in the scientific literature will not be used during this
verification test.
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by
verification staff during this verification test. Table 3 summarizes the type of data to be
recorded. All data and observations for the operation of the coliform detection technologies will
be documented by the verification staff on data sheets or in LRBs. Results from the laboratory
analyses will be compiled by laboratory staff in electronic format and submitted to the
Verification Test Coordinator or other verification staff upon obtaining results within 7 days of
test completion. A dedicated shared folder within the ETV AMS Center SharePoint site will be
established for all project records.
Records received by or generated by any of the verification staff during the verification
test will be reviewed by a Battelle staff member within two weeks of receipt or generation,
respectively, before the records are used to calculate, evaluate, or report verification results. The
review will be documented as the dated initials of the reviewer. Some of the checks that will be
performed include:
• QC samples and calibration standards were analyzed according to the test/QA
plan and the acceptance criteria were met. Corrective action for exceedances was
taken
• 100% hand-entered and/or manually calculated data were checked for accuracy
• Calculations performed by software are verified at a frequency sufficient to
ensure that the formulas are correct, appropriate, and consistent
• For each cut and paste function, the first and last data value was verified vs. the
source data
-------
Coliform Detection Technologies
Test/QA Plan
Page 47 of 47
Version 1.0
Date: 7/14/2010
• Data are reported in the units specified in the test/QA plan.
• Results of QC samples are reported.
If a Battelle staff member generated the record, this review will be performed by a
Battelle technical staff member involved in the verification test, but not the staff member that
originally received or generated the record. The review will be documented by the person
performing the review by adding his/her initials and date to the hard copy of the record being
reviewed. In addition, data calculations performed by verification staff will be spot-checked by
Battelle technical staff to ensure that calculations are performed correctly. Calculations to be
checked include any statistical calculations described in this test/QA plan. The data obtained
from this verification test will be compiled and reported for each technology. Results for
technologies from different vendors will not be compared with each other.
Battelle will provide technology test data and associated reference data (including
records, data sheets, notebook records) from the first day of testing within one day of receipt to
EPA for simultaneous review. The goal of this data delivery schedule is prompt identification
and resolution of any data collection or recording issues. These data will labeled as preliminary
and will not have had a QA review before their release.
-------
Coliform Detection Technologies
Test/QA Plan
Page 48 of 48
Version 1.0
Date: 7/14/2010
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in this verification test to anticipate and resolve potential
problems before the quality of performance is compromised. One of the major objectives of the
test/QA plan is to establish mechanisms necessary to ensure this. Internal quality control
measures described in this test/QA plan, which is peer reviewed by a panel of outside experts,
implemented by the technical staff and monitored by the Verification Test Coordinator, will give
information on data quality on a day-to-day basis. The responsibility for interpreting the results
of these checks and resolving any potential problems resides with the Verification Test
Coordinator, who will contact the Battelle AMS Center Manager, Battelle AMS Center Quality
Manager, EPA AMS Center Project Officer, and EPA AMS Center QA Manager if any
deviations from the test/QA plan are observed. The Verification Test Coordinator will describe
the deviation in a teleconference or by email, and once a path forward is determined and agreed
upon with EPA, the deviation form will be completed. Technical staff have the responsibility to
identify problems that could affect data quality or the ability to use the data. Any problems that
are identified will be reported to the Verification Test Coordinator, who will work with the
Battelle Quality Manager to resolve any issues. Action will be taken by the Verification Test
Coordinator and Battelle testing staff to identify and appropriately address the issue, and
minimize losses and correct data, where possible. Independent of any EPA QA activities,
Battelle will be responsible for ensuring that the following audits are conducted as part of this
verification test.
Cl.l Technical Systems Audits
Battelle Quality Manager will perform a technical systems audit (TSA) at least once
during this verification test. The purpose of this audit is to ensure that the verification test is
being performed in accordance with the AMS Center QMP and this test/QA plan. The primary
focus of the audit will be the reference test method and specifically the determination of positive
and negative results. During this audit, the Battelle Quality Manager, or designee, will compare
-------
Coliform Detection Technologies
Test/QA Plan
Page 49 of 49
Version 1.0
Date: 7/14/2010
actual test procedures to those specified or referenced in this plan, and review data acquisition
and handling procedures. The audit will include a review of the testing facility and equipment
(calibration, maintenance, operation), and observation of testing and records (including custody
forms). She will also check data acquisition procedures and may confer with the vendor and
Battelle technical staff. The TSA will be guided by a project-specific checklist based on the
test/QA plan and reference methods.
The Battelle Quality Manager will prepare an initial TSA report and will submit the
report to the Verification Test Coordinator within 10 business days after completion of the audit.
A copy of this initial TSA report (with no corrective actions documented) will be provided to the
EPA AMS Center Project Officer. A copy of the final TSA report (with corrective actions
documented) will be provided to the EPA AMS Center Project Officer within 20 business days
after completion of the audit. At EPA's discretion, EPA QA staff may also conduct an
independent on-site TSA during the verification test. The TSA findings will be communicated to
technical staff at the time of the audit and documented in a TSA report.
C1.2 Data Quality Audits
The Battelle Quality Manager, or designee, will audit at least 25% of the sample results
acquired in the verification test and 100% of the calibration and QC data vs. the test/QA plan
requirements. A checklist based on the test/QA plan will guide the audit. An initial data quality
audit will be conducted on the first batch of test data within 10 business days of when data were
collected to identify errors early in the data reduction process. Given the short duration of this
test, the first batch is defined as the first set of reference test data released by the Verification
Test Coordinator. The remaining data will be audited once all data for a technology or method
has been posted on the project SharePoint site and once all statistical analyses are complete. The
primary focus of the audit will be the reference test method. The Battelle Quality Manager, or
designee, will trace the data from initial acquisition, through reduction and statistical
comparisons, to final reporting. All formulae applied to the data will be verified. Data for the
technologies will be reviewed for calculation and transcription errors and data traceability.
Results of each audit of data quality (ADQ) will be documented using the checklist and reported
to the Verification Test Coordinator within 10 business days after completion of the audit. The
EPA AMS Center Project Officer will be provided a copy of the initial ADQ report (with no
-------
Coliform Detection Technologies
Test/QA Plan
Page 50 of 50
Version 1.0
Date: 7/14/2010
corrective actions documented). A final ADQ that assesses overall data quality, including
accuracy and completeness of the verification report, will be prepared as a narrative and
distributed to the Verification Test Coordinator within 10 business days of completion of the
audit. The EPA AMS Center Project Officer will be provided a copy of the final ADQ report
(with corrective actions documented).
C1.3 QA/QC Reporting
Each assessment and audit will be documented in accordance with Section 3.3.4 of the
AMS Center QMP. The results of the TSA and DQA will be submitted to EPA. Assessment
reports will include the following:
• Identification of Findings and Observations
• Recommendations for resolving problems
• Response to adverse findings or potential problems
• Confirmation that solutions have been implemented and are effective
• Citation of any noteworthy practices that may be of use to others.
C2 REPORTS TO MANAGEMENT
The Battelle Quality Manager, during the course of any assessment or audit, will identify
to the technical staff performing experimental activities any immediate corrective action that
should be taken. If serious quality problems exist, the Battelle Quality Manager is authorized to
notify the Battelle AMS Center Manager who will issue the stop work. Once the TSA or data
quality audit report has been prepared, the Verification Test Coordinator will respond to each
Finding and Observation following the timeline defined in Section Cl and will implement any
necessary corrective action. The Battelle Quality Manager will verify that corrective action has
been implemented effectively.
In addition to this test/QA plan, a final report and a verification statement for each
technology verified will be prepared and reviewed. The final report is a comprehensive
document describing the verification test. The verification statement is a two-to-three page
summary of the technology, the test procedures, and the test results. Each draft report and
verification statement will be submitted to the respective vendor for review. They are then
-------
Coliform Detection Technologies
Test/QA Plan
Page 51 of 51
Version 1.0
Date: 7/14/2010
reviewed by EPA AMS Center Quality Manager and the EPA AMS Center program Project
Officer. Upon approval by EPA, each verification statement will be signed by a senior manager
of Battelle and by an EPA laboratory director. Original signed verification statements will be
provided to the respective vendors for use in marketing their technology. Upon final review and
approval, the final verification report(s) will be submitted to EPA in MS Word and Adobe
portable document format (PDF) and subsequently posted on the ETV website
(www.epa.gov/etv).
-------
Coliform Detection Technologies
Test/QA Plan
Page 52 of 52
Version 1.0
Date: 7/14/2010
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are the analysis of QC samples
as outlined in the test/QA plan, a comparison of laboratory data sheet comments against final
data to flag any suspect data, and a review of final data to resolve any questions about apparent
outliers. The QA audits, as described within this document, are designed to assure the quality of
this data.
D2 VALIDATION AND VERIFICATION METHODS
Section C of this test/QA plan provides a description of the validation safeguards
employed for this verification test. Data validation and verification efforts include the analysis
of QC samples as required in this document, and the performance of a ISA and DQAs as
described in Section C.
D3 RECONCILIATION WITH USER REQUIREMENTS
The purpose of this verification test is to evaluate the performance of coliform detection
technology for use in DW monitoring. In part, this evaluation will include demonstrations of the
monitoring ability of coliform detection technologies to detect the presence of TC and E. coli.
To meet the requirements of the user community, input on the tests described in this test/QA plan
has been provided by external experts. Additional performance data regarding operational
characteristics of the coliform detection technologies will be collected by verification test
personnel. To meet the requirements of the user community, these data will include thorough
documentation of the performance of the technologies during the verification test. The data
review, verification, and validation procedures described above will assure that data meeting
these requirements are accurately presented in the verification reports generated from this test,
and will assure that data not meeting these requirements will be appropriately flagged and
discussed in the verification reports.
-------
Coliform Detection Technologies
Test/QA Plan
Page 53 of 53
Version 1.0
Date: 7/14/2010
This test/QA plan and the resulting ETV verification report(s) will be subjected to review
by the vendor(s), EPA, and expert peer reviewers. The reviews of this test/QA plan will help to
improve the design of the verification test and the resulting report(s) such that they better meet
the needs of potential users of these technologies.
-------
Coliform Detection Technologies
Test/QA Plan
Page 54 of 54
Version 1.0
Date: 7/14/2010
SECTION E
REFERENCES
El REFERENCES
1. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version 7.
U.S. Environmental Technology Verification Program, Battelle, November 2008.
2. Environmental Technology Verification Program Quality Management Plan. January,
2008 (EPA/600/R-08/009).
3. American Public Health Association, American Water Works Association, and Water
Environment Federation. 2005. Standard Methods for the Examination of Water and
Wastewater. 21st Edition.
4. Battelle Building 20 Biosafety Manual, Version 4.0.
5. SOP GEN.V-003-10. Standard Operating Procedure for the Use ofpH meters to
Measure pH. Battelle.
6. AO AC International. 1995. Bacteriological Analytical Manual, 8th Edition.
1. SOP GEN.V-013-04. SOP for the Calibration and Maintenance of Thermometers.
Battelle.
8. SOP ENV-ADM-009, Standard Operating Procedure for Sample Chain-of-Custody
Battelle, September 2007.
------- |