March 2011
Environmental Technology
Verification Report
COLIFAST
ALARM AT-LlNE AUTOMATED REMOTE MONITOR
Prepared by
Baireiie
The Business of Innovation
Under a cooperative agreement with
^& CrTr\ U.S. Environmental Protection Agency
-------
March 2011
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
COLIFAST
ALARM AT-LlNE AUTOMATED REMOTE MONITOR
by
Ryan James, Dan Lorch, Stacy Pala, Amy Dindal, Battelle
John McKernan, U.S. EPA
-------
Notice
The U.S. Environmental Protection Agency, through its Office of Research and Development,
funded and managed, or partially funded and collaborated in, the research described herein. It
has been subjected to the Agency's peer and administrative review. Any opinions expressed in
this report are those of the author(s) and do not necessarily reflect the views of the Agency,
therefore, no official endorsement should be inferred. Any mention of trade names or
commercial products does not constitute endorsement or recommendation for use.
11
-------
Foreword
The EPA is charged by Congress with protecting the nation's air, water, and land resources.
Under a mandate of national environmental laws, the Agency strives to formulate and implement
actions leading to a compatible balance between human activities and the ability of natural
systems to support and nurture life. To meet this mandate, the EPA's Office of Research and
Development provides data and science support that can be used to solve environmental
problems and to build the scientific knowledge base needed to manage our ecological resources
wisely, to understand how pollutants affect our health, and to prevent or reduce environmental
risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http ://www. epa.gov/etv/centers/centerl .html.
in
-------
Acknowledgments
The authors wish to thank Jim Sinclair, Sandhya Parshionikar, Mark Rodgers, Jennifer Best, and
Keya Sen of the U.S. EPA, and Rick Sakaji of the East Bay Municipal Water District, for their
review of the test/QA plan and/or this verification report. Quality assurance (QA) oversight was
provided by Michelle Henderson, U.S. EPA, and Rosanna Buhl, Battelle.
IV
-------
Contents
Page
Foreword iii
Acknowledgments iv
List of Abbreviations vii
Chapter 1 Background 8
Chapter 2 Technology Description 9
Chapters Test Design and Procedures 11
3.1 Introduction 11
3.2 Test Overview 11
3.3 Experimental Design 13
3.3.1 Verification Test Sample Preparation 13
3.3.2 Sample Analysis 15
3.3.3 Analysis in "At-line" Mode 16
Chapter 4 Quality Assurance/Quality Control 18
4.1 Quality Control Samples 18
4.2 Audits 19
4.2.1 Technical Systems Audit 19
4.2.2 Data Quality Audit 20
Chapters Statistical Methods 21
5.1 False Positive Rates, False Negative Rates, Sensitivity, and Specificity 21
5.2 Method Comparability 21
Chapter 6 Test Results 23
6.1 TCData 23
6.2 EC Data 24
6.3 Method Comparability 25
6.4 Analysis in "At-line" Mode 26
6.5 Operational Factors 27
Chapter 7 Performance Summary 29
Chapter 8 References 32
-------
Figures
Figure 2-1. Colifast ALARM 9
Figure 2-2. Colifast ALARM Display 9
Figure 3-1. Flowchart describing confirmation analyses for both the Colifast ALARM and
SM9221BandF 17
Tables
Table 3-1. Methods, Equipment, and Results for the Characterization of Sewage and Drinking
Water Samples 14
Table 3-2. Quality Control Strains 15
Table 3-3. Replicate Samples by each Analysis Method 16
Table 6-1. TC Positive Results 23
Table 6-2. TC Data Summary -Positives 23
Table 6-3. TC Data Summary -Negatives 23
Table 6-4. TC Data Summary - Confirmations a 24
Table 6-5. EC Positives 24
Table 6-6. EC Summary -Positives 24
Table 6-7. EC Summary - Negatives 24
Table 6-8. EC Data Summary- Confirmations a 25
Table 6-9. TC-SM9221B 25
Table 6-10. EC-SM9221F 26
Table 6-11. EC-Colilert-18 26
Table 6-12. Results of Analysis in "At-line" Mode 27
Table 7-1. Results Summary for Positive Colifast ALARM Results for TC and EC 29
Table 7-2. Results Summary of Colifast ALARM 29
Appendix Raw Data from Reference Methods, Colifast ALARM, and Confirmation
Analyses 33
VI
-------
List of Abbreviations
AMS Advanced Monitoring Systems
ATCC American Type Culture Collection
ATP Alternate Test Procedures
BGLB brilliant green lactose bile
cm centimeter(s)
COC chain of custody
DDW dechlorinated drinking water
DQA data quality audit
DW drinking water
EC Escherichia coll
EPA U.S Environmental Protection Agency
ETV Environmental Technology Verification
FN false negative
FP false positive
h hour(s)
in inch(es)
ISO International Standards Organization
LTB lauryl tryptose broth
MB method blank
min minute(s)
mL milliliter(s)
MUG 4-methyllumbelliferyl-p-D-glucorinide
N number of results
n/a not applicable
NA nutrient agar
NRMRL National Risk Management Research Laboratory
org organism(s)
ppm parts per million
QA quality assurance
QC quality control
QMP Quality Management Plan
SM Standard Methods
SSDW spiked, stressed drinking water
SWTP Southerly Wastewater Treatment Plant
TQAP Test Quality Assurance Plan
TC total coliforms
TCR Total Coliform Rule
TN true negative
TP true positive
TSA technical systems audit
TSB trypticase soy broth
USB Universal Serial Bus
vn
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Risk Management Research Laboratory (NRMRL) and its verification
organization partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under
ETV. The AMS Center recently evaluated the performance of the ALARM At-Line Automated
Remote Monitor by Colifast (Colifast ALARM), a bench top sample collector/analyzer/data
logger system for the analysis of total coliforms (TC) and Escherichia coll (EC).
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This report provides results for
the verification testing of the Colifast ALARM At-Line Automated Remote Monitor (hereafter
referred to as the Colifast ALARM). The following is a description of the Colifast ALARM,
based on information provided by the vendor.
••• •_ O
The Colifast ALARM is an automated system for detection of TC
or EC in 100 mL water samples. The Colifast ALARM
automatically collects the water sample at programmed intervals
for the analysis of TC or EC. The Colifast ALARM method is
based on an enzymatic reaction. The Colifast TC medium
contains the substrate 4-metylumbelliferyl (MU)-P-D-
galactoside, and this substrate is hydrolyzed by the enzyme P-
galactosidase that is present in TC. The Colifast EC medium
contains the substrate 4-metylumbelliferyl (MU)-P-D-
glucuronide, and this substrate is hydrolyzed by the enzyme P-
D-glucuronidase that is present in EC. The fluorescent product
MU is produced as a result of the hydrolysis reactions. The
media contains inhibitors to hinder growth of non-coliforms.
Figure 2-1. Colifast
ALARM
A 100 mL water sample is added to a sample bottle,incubated,
and analyzed by the Colifast ALARM. The main components of
the Colifast ALARM are the incubator reaction chamber, a flow
injection pump system for liquid handling and a
detector system including wavelength specific
emitters combined with a spectrometer. The
bacterial detection results are based on measured
concentrations of the fluorescent product. An
increase in the number of EC means an increase in
the amount of p-D-glucuronidase (enzyme). This
leads to an increase in the production of MU (the
fluorescent product) that yields a higher
fluorescence signal on the Colifast ALARM. The
Colifast ALARM is shown in Figure 2-1 and the
instrument software displayed during a run in the
embedded Colifast ALARM touch screen
computer is shown in Figure 2-2.
a I a i •, a I S
*•*. W<4, MlriMAVd. MllfMVuI mj [X* | *MOl
Figure 2-2. Colifast ALARM Display
-------
When the Colifast ALARM is operated in "at-line" mode, 100 mL samples are collected
automatically using the pump system. The sample can be collected from any source (flowing or
static) reachable by the flow injection pump system. The analysis is then peformed autmatically
and the software within the Colifast ALARM automatically interprets the fluorescent
measurements hourly throughout the incubation time and a positive result is reported on the
screen and by audio/ visual alarms when the presence of TC or EC is detected, regardless of the
amount of time that has passed. In addition, Colifast has the capability to provide results by
industrial interface (relays) and mobile networks. The results are stored on the computer
provided with the Colifast ALARM and can be downloaded with a universal serial bus (USB)
drive or accessed via local area network remote control. In "at-line" mode, the Colifast ALARM
can collect and analyze TC samples in approximately 15 hours (h) and EC samples in 14 h, and
have a total capacity of 20 analyses per run.
The "at-line" operation mode for the Colifast ALARM allows for one sample every 16 hours, but
for the convenience of the staff that was spiking the samples it was programmed to do one
sample analysis every 24 hours. Colifast provided one unit for testing which limited the sample
capacity to one sample per 24 h. The large number of samples required for this verification test
exceeded that capacity. Therefore, the Colifast ALARM was used primarily in manual mode. In
manual mode, following the addition of water sample to the sample bottles containing the growth
medium, the bottles were incubated in laboratory water baths for the specified timeframes (15-17
h for TC and 14-16 h for EC) before being inserted into the Colifast ALARM for a 30 second
fluorescent measurement. While there is no exact time specified at which the samples must be
read, the TC samples were removed from the water baths after 15.5 h and the EC samples were
removed from the water baths after 14.5 h. It is possible that analysis in manual mode may not
be entirely representative of the "at-line" mode because of the pre-warming of sample and
growth medium and optimal incubation temperature control and during "at-line" analysis within
the Colifast ALARM. However, in order for this test to be accomplished in a reasonable
timeframe primary use of manual mode was necessary. The results were displayed on the screen
in the same way they were for the continuous measurements.
The Colifast ALARM has dimensions of 42 centimeters (cm) wide x 36 cm deep x 64 cm high
(17 inches (in) wide x 14 in deep x 26 in high) and weighs approximately 31 kilograms (68
pounds). The Colifast ALARM has dust and water resistant enclosure and is equipped with
transport handles to facilitate moving around and installation at various locations.
The Colifast growth media are sold as bottles with 20 tests for the "at-line" mode or as single
sample cartridges. The Colifast ALARM is self contained and does not require any additional
equipment or materials to perform analyses.
10
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
The ETV AMS Center Water Stakeholder Committee identified the use of coliform detection
technologies for the monitoring of drinking water (DW) as an area of interest for technology
verification. Fecal pollution can introduce disease-causing (pathogenic) bacteria, viruses, and
parasites into receiving waters, which may serve as private/public DW supplies. Utilities fully
recognize the possibility of this waterborne pollution and take every precaution (filtering,
treatment with disinfectants such as chlorine and chloramines, and regulatory compliance
sampling and analysis) to avoid fecal contamination in DW. Based on the U.S. EPA's 1989
Total Coliform Rule (TCR)1, assessment of this health risk is based on the detection and
enumeration of fecal indicator bacteria, such as TC and EC, whose presence indicates a potential
pathway for contamination (e.g., sewage or animal waste) of the distribution system which is
designed to provide a physical barrier to contamination of DW. It is important to note that this
verification test was not being conducted to provide data to be used to approve technologies for
use in meeting regulatory requirements for the detection of TC or EC as required by either the
1989 TCR or the anticipated revision to the TCR. It was conducted, based on feedback from
ETV AMS Center stakeholders, to provide a verification test that is similar in requirements to the
current TCR approval protocol (referred to as the Alternative Testing Procedures (ATP)
protocol)2, such that technologies that are not already approved have an opportunity to be tested
under a similar set of test conditions.
This verification testing was also conducted in cooperation with ETV programs in Canada (ETV
Canada) and Denmark (DANETV) as a possible ETV verification by those programs. The
criteria for ETV cooperation are outlined in a cooperative verification process document
prepared by the respective cooperating ETV programs. It should be noted however that neither
U.S. ETV verification, nor the cooperation with the ETV Canada or DANETV programs,
represents an approval of methods for regulatory compliance.
3.2 Test Overview
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Coliform Detection Technologies for Drinking Water3 (TQAP) and adhered to
the quality system defined in the ETV AMS Center Quality Management Plan (QMP)4. As
indicated in the test/QA plan, the testing conducted satisfied EPA QA Category II requirements.
The test/QA plan and/or this verification report were reviewed by:
• Rick Sakaji, East Bay Municipal Water District
• John Neate, Strategies for Change
11
-------
• Mona El Hallak, OCETA
• Claus J0rgensen, DHI Group
• Jim Sinclair, U.S. EPA
• Sandhya Parshionikar, U.S. EPA
• Jennifer Best, U. S. EPA
• Keya Sen, U.S. EPA
• Mark Rodgers, U.S. EPA (test/QA plan only).
The TCR sets both goals and legal limits for the presence of TC and EC in DW. To summarize,
the TCR states that the objective is for zero TC organisms in DW samples and the rule (for large
water systems) is that no more than 5% of all DW samples collected by a utility can be positive.
In order to comply with the TCR, water utilities need coliform detection technologies that are
able to detect TC and EC at concentrations of one organism (org) per 100 milliliters (mL).
While it is difficult to determine if a single target organism is present in 100 mL of water, the
ATP protocol suggests that when approximately half of the analyzed replicates are positive and
half are negative, the density of the organism has become adequately low so that a positive result
can be considered single organism detection. Therefore, for the purpose of this verification test,
the objective was to use the ATP protocol as a guide to prepare spiked DW dilution sets that
provided 50 ±25% positive results for both TC and EC with the reference method(s) and then
compare the results from the reference method to those from the tested technology.
Similar to the TCR in Europe, the Official Journal of the European Communities published
Council Directive 98/83/EC in 1998 that provided directives on the regulation of public water
systems within the European Union. Annex IA within that document includes the requirement
for zero EC per 100 mL. The European Drinking Water Directive prescribes minimum sampling
and analysis frequencies with the use of ISO 9308-15, the ISO procedure for measurement of
coliforms, or an equivalent method for compliance monitoring of EC. In order to be accepted as
equivalent, the alternative method must be compared to ISO 9308-1 according to ISO 179946,
the ISO procedure for showing method equivalence. ISO 9308-1 provides both a measurement
of the presence or absence of E-coli or total coliform bacteria, and number of bacteria. Colifast
ALARM only provides a test of presence or absence of EC and TC. Recently, the Colilert-18
combined with QuantiTray for quantification has been shown to be equivalent7 with ISO 9308-1.
Colilert-18 is considered a relevant reference method (since it is the presence/absence version of
the identical test) and was considered adequate to meet the DANETV requirement of use of a
European accepted reference method to potentially grant DANETV verification following this
test.
In this report, results from the Colifast ALARM were compared to the results obtained from the
reference method analyses which were presence/absence methods for TC and EC, specifically,
Standard Methods (SM)8 922IB (TC) and 922IF (EC). In addition, the EC results were also
compared to the Colilert-18 presence/absence method. The SM and Colilert methods utilize
selective and/or chromatogenic liquid growth media to detect TC and EC. The verification test
of the Colifast ALARM was conducted from August 31 through September 8, 2010 at Battelle in
Columbus, Ohio with the reference method analyses being performed at Superior Laboratories in
Galloway, Ohio (which is a 20 minute drive from Battelle). Technology operation and sample
handling and analysis were performed according to the vendor's instructions. Both reference
method and Colifast ALARM sample analysis results were reported as presence/absence.
12
-------
Sample analysis results from the Colifast ALARM were evaluated by comparing the proportion
of positive and negative results to the proportion of positive and negative results produced by the
reference methods which includes the comparison of false positive rate and false negative rate.
In addition, sustainable operational factors such as ease of use, required reagents, analysis time,
and laboratory space and utilities required are reported.
3.3 Experimental Design
3.3.1 Verification Test Sample Preparation
The preparation of verification test samples included the collection of raw sewage as the source
of the target organisms, collection of the DW sample, the fortification of the DW sample with
target organisms, and the chlorine stressing and dilution of samples for analysis. A detailed
description of the sample preparation steps is provided in the TQAP. A summary of the sample
preparation activities and timeline is provided below.
3.3.1.1 Sewage and Drinking Water Sample Collection
A single raw sewage sample (approximately 0.6 liter (L)), was collected at 7 A.M. August 31,
2010 at the Southerly Wastewater Treatment Plant (SWTP) in Columbus, Ohio. The sewage
sample was a 24 h composite sample collected automatically over a 24 h period (midnight
August 30 - midnight August 31). The SWTP automated system collects 50-100 mL aliquots at
approximately 5 minutes intervals directly into a refrigerated carboy. The sewage sample was
collected from this carboy. The sampling approach was a deviation from the TQAP, which had
implied that the sample would be collected without compositing. Battelle believes that there was
not an adverse impact to the results of the evaluation due to this deviation because the coliform
levels were adequate for the purposes of testing.
Upon sampling, the sewage sample was immediately stored on wet ice, and transported by
Battelle staff to Battelle laboratories. Upon receipt, the sewage sample was filtered through a
Whatman No. 2 filter (11 micron pore-size) under vacuum using a Buchner funnel to remove
excess solids, shaken vigorously for 1 minute to insure homogeneity, and then immediately
characterized for total culturable heterotrophic bacteria, TCs, and EC.
A single DW sample was collected from the tap at the Battelle laboratory the same day the
sewage sample was collected. The DW sample was collected by first removing the faucet screen
and decontaminating the surface with 70% isopropanol. Next, the line was purged for 3 minutes
with cold water and 80 L of DW were collected from the tap into multiple sterile (autoclaved)
carboys equipped with a spigot and containing large stir bars. Once collected, aliquots from each
carboy were pooled and then used to characterize the DW using the methods and standard
operating procedures provided in Table 3-1. Table 3-1 also gives the results of the initial
characterization of the sewage and DW samples.
13
-------
Table 3-1. Methods, Equipment, and Results for the Characterization of Sewage and
Drinking Water Samples
Parameter
pH
temperature
free chlorine
total chlorine
total,
culturable
heterotrophic
bacteria
TC
EC
Units
n/a
°C
mg/L
mg/L
org/lOOmL
org/lOOmL
org/lOOmL
Equipment/Media
calibrated pH meter
calibrated thermometer
HACH Chlorine test
kit
HACH Chlorine test
kit
R2A agar
m-Endo
NA-MUG
SOP/Method
SOP GEN.V-003-109
SOPGEN.V-013-
04710
HACH Method 8021
HACH Method 8 167
AOAC's
Bacteriological
Analytical Manual11
SM 9222B
SM 9222G
Sewage
n/a
n/a
n/a
n/a
5.6 x 106
1.6 x 107
2.7 x 106
DW
7.7
24
1.2
1.2
n/a
n/a
n/a
n/a - not measured
NA - Nutrient agar
MUG4-methyllumbelliferyl-p-D-glucorinide
3.3.1.2 Chlorine Stressing and Preparation of Samples for Verification Testing
The Colifast ALARM was tested with chlorine stressed TC and EC. The chlorination stressing
step was started within 3 h from the time Battelle received the sample, or approximately 10 h
from the time the last automated sample was collected and 34 h from the time the first automated
sample was collected. This multi-step stressing process was accomplished on the same day as
DW sample collection by adding approximately 40 L of the unspiked DW sample to one 50 L
carboy. The DW was adjusted to a free chlorine concentration of 2 parts per million (ppm) using
a 4% hypochlorite solution, after which 10.5 L was dispensed into three 10L aliquots containing
stir bars. Each aliquot was then spiked with TC and EC by adding 200 mL of filtered sewage
(amount of sewage providing enough TC and EC to bring the DW sample to a starting
concentration of approximately 10 TC org/100 mL and 104EC org/100 mL). Based on pre-
testing range finding experiments, the three aliquots were chlorinated for 20, 40, and 60 seconds,
respectively, after which time the samples were dechlorinated with sodium thiosulfate and
subsequently enumerated using SM9222 B and G. The results determined the log reduction of
TC and EC due to the chlorine stressing that had occurred in each aliquot. This chlorine
stressing step was considered adequate if the number of organisms in the spiked DW samples
was reduced by two to four orders of magnitude.
During the testing of the Colifast ALARM, the 20 second chlorine stressing attained a two log
reduction in both TC and EC so after having been refrigerated overnight, that aliquot of spiked,
stressed drinking water (SSDW) was used to prepare the diluted samples for analysis. To test the
coliform technologies, separate SSDW samples of TC and EC containing concentrations of
approximately 1 org/100 mL needed to be prepared. Based on preliminary work with similar
sewage samples, EC concentrations were approximately 10 times less than the TC
concentrations. To ensure that concentrations of approximately 1 org/100 mL would be attained
for both TC and EC, a range of concentrations were prepared. Three separate aliquots,
approximately 10 L each, of dechlorinated DW (DDW) were added to carboys and spiked with a
calculated volume of SSDW sample to generate target suspensions of 5 TC/100 mL, 10 TC/100
mL, and 50 TC/100 mL. Each dilution was mixed on a stir plate for 5 to 10 minutes, and then, as
14
-------
mixing continued, 100 mL aliquots were dispensed into sterile 100 mL bottles using 50 mL
and/or 100 mL graduated pipettes. Twenty replicate samples were prepared at each
concentration level. Once all 100 mL aliquots were dispensed for technology verification (20 at
each dilution level for a total of 60 replicates), verification testing was initiated. All samples
were stored refrigerated during the day of preparation until the analysis was initiated that same
day. These concentration levels were changed through a deviation in the TQAP because these
concentration levels offered an increased likelihood that the targeted ratio of positive and
negative results would be obtained from the reference method.
In addition to the samples to be used for Colifast ALARM verification, a set of twenty 100 mL
aliquots were prepared for the reference method analysis. Immediately after being dispensed, all
reference samples were transported by car in coolers packed with ice packs to Superior
Laboratories, Inc. Sample custody for all samples transferred to Superior Laboratories were
documented using a chain-of-custody (COC) form following Battelle SOP ENV-ADM-009 for
Chain of Custody12. The COC form was signed once receipt of all samples had been confirmed.
Reference method analysis was initiated on the same day as arrival at the laboratory, within 2 h
of initiation of the Colifast ALARM sample analysis.
3.3.2 Sample Analysis
The ability of the Colifast ALARM to determine the presence of TC and EC was challenged
using 20 replicates of the three concentrations of SSDW samples. The number of replicates was
determined after performing a power analysis with a fixed 80% power (described more
thoroughly in Section 5.2). Positive/negative control samples spiked with quality control (QC)
cultures listed in Table 3-2 as well as method blank samples were included during testing. One
Colifast ALARM was provided to perform all of the replicate samples shown in Table 3-3.
Because of the large number of concurrent samples analyses required during this verification
test, the samples were incubated apart from the Colifast ALARM and then analyzed in the
Colifast ALARM one at a time after incubation periods (15.5 h TC and 14.5 h EC) for
fluorescent measurement. All of the samples were assayed by the reference methods and the
Colifast ALARM concurrently.
Table 3-2. Quality Control Strains
Targeted Coliform
TC
EC
Method Blank
Sterilized DW
Sterilized DW
Positive Control
Enterobacter aerogenes
ATCC 13048
Escherichia coll
ATCC 8739
Escherichia coli
ATCC 8739
Negative Control
Pseudomonas
aeruginosa
ATCC 10145
Enterobacter aerogenes
ATCC 13048
Pseudomonas
aeruginosa
ATCC 10145
ATCC - American Type Culture Collection
15
-------
Table 3-3. Replicate Samples by each Analysis Method
Sample Description
Dilution A - approx. 50 TC/100 mL
Dilution B - approx. 10 TC/100 mL
Dilution C - approx. 5 TC/100 mL
Method Blank
TC Positive control
EC Positive control
Negative control
Total Replicate Analyses
Replicate
Analyses by
Colifast
ALARM
20
20
20
3
3
3
3
72
Replicate
Analyses by
SM9221B
20
20
20
3
3
3
3
72
Replicate
Analyses by
SM9221F
20
20
20
3
3
3
3
72
Replicate
Analyses by
Colilert-18
20
20
20
3
3
3
3
72
3.3.2.1 Confirmation of Results
The SM 9221B and 9221F reference methods and Colifast ALARM results were confirmed with
more definitive tests in order to adequately verify the Colifast ALARM. Confirmation for the
SM 922IB and 922IF reference methods, as well as the Colifast ALARM, is described in detail
in the TQAP. In summary, for the Colifast ALARM analyses, 1 mL of each 100 mL sample
resulting from the 15.5 h (for TC) and 14.5 h (for EC) Colifast ALARM incubation was
inoculated into 9 mL of lauryl tryptose broth (LTB) and analyzed using SM 9221B and 9221F.
Following the LTB step, TCs were confirmed using brilliant green lactose bile (BGLB) broth,
and EC were confirmed using EC-MUG. Figure 3-1 illustrates the process by which all positive
and negative samples from the Colifast ALARM and SM 922IB and 922IF were confirmed. As
an additional, optional confirmation, a complete test for TC was performed for several samples
by inoculating MacConkey media and then selecting suspected TC colonies and inoculating into
LTB, as described by SM 9221B.
3.3.3 Analysis in "At-line" Mode
An optional component of the ETV test was performed to verify the capability of the Colifast
ALARM to detect EC ATCC 8739 in "at-line" mode which provides positive results as soon as
determined by the Colifast ALARM. The "at-line" operation mode for the Colifast ALARM
allows for one sample every 16 hours, but for the convenience of the staff that was spiking the
samples it was programmed to do one sample analysis every 24 hours. Therefore, only four
analyses containing EC were performed. Two analyses of EC ATCC 8739 at a concentration of
approximately 30 EC per 100 mL each followed by 1-3 filter sterilized water samples. More
than one filter sterilized water sample was analyzed when these experiments were set to run over
the weekend. Following the initial intake of an EC sample on Friday, the filter-sterilized water
sample was connected to the Colifast ALARM and was analyzed repeatedly until the sample was
switched at the start of the following week. This procedure was repeated with EC at a similar
concentration level, but using EC from sewage water rather than from a pure culture. This was a
deviation from the TQAP because the details of the approach to this testing for the Colifast
ALARM had been inadvertently omitted. The EC ATCC 8739 samples were also analyzed by
using a quantitative method for EC (SM 9222G - Na-MUG).
16
-------
_ !?! r~. " / v Dilutions
Posrtive:Negative / ^-~^___
•--— ___I---^ — --
i^Anficm Tf^
/" "\ r* ^bUIlb
LTB(+) •-
Replicates
(SM9221B) ....,._ ..
^ l J Test for E colj
"^ ^fv4QJT(O 'I LT
SM Reference (EC MUG)
Method
Technology
Being Verified
Positive (•*•) Confirm (+) Results
Replicates
Negative (-) Confirm (-) Results
Replicates
-^ Sample Continual
Inoculate
^ BGLB
Inoculate UV366
> EC MUG ^ Fluorescence
I IV^«« Rernrrl S- f
Fluorescence Reference Re
Inoculate -.
BGLB
In An ilato 1 TR
^ EC MUG
^ If +. Inoculate
•> D/ll D
In ArnlssifA 1 TR
if 4- In nn i I^tP
^ EC MUG
tion
1 IV
^ ^ "3^i
Fluorescence
rc
1 IW
^ ^ * 366
Fluorescence
E, co//
E. co//
Figure 3-1. Flowchart describing confirmation analyses for both the Colifast ALARM and SM9221B and F
17
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed in accordance with the TQAP for this verification test1 and
the QMP for
sub chapters.
9
the QMP for the AMS Center . QA/QC procedures and results are described in the following
During testing, there were three deviations from the TQAP. The first was described in Section
3.3.1.1 and involved a change in collection method for the sewage sample. The TQAP had
implied that the sewage sample would be sampled directly and not composited over two days.
The second deviation was described in Section 3.3.1.2 and changed the concentrations of the test
samples in order to provide a better likelihood that the target ratio of positive and negative results
be obtained from the reference method. The third deviation was described in Section 3.3.3 and
clarified the approach to the additional optional testing of the "at-line" feature of the Colifast
ALARM as it had been previously omitted. These deviations were judged by the Battelle
Verification Test Coordinator to not result in any adverse impacts on the quality of the data
generated. The deviations were reviewed and approved by the EPA ETV AMS Center Project
Officer and EPA ETV AMS Center Quality Manager.
4.1 Quality Control Samples
The reference method required the use of method blanks (MB), positive and negative control
organisms, and result confirmation. One MB was performed during the analysis for every 20
samples analyzed. The MB consisted of 100-mL dechlorinated, sterilized tap water processed as
a sample. MB samples were exposed to identical handling and analysis procedures as the rest of
the test samples, including the addition of all reagents. These samples were used to help ensure
that no sources of contamination were introduced in the sample handling and analysis
procedures. All three MB samples analyzed by the Colifast ALARM as well as the reference
methods were negative, indicating the absence of TC and EC.
Three positive and negative control samples were also be analyzed using the Colilert-18 and SM
9221B/F reference methods. Positive and negative ATCC control cultures were purchased from
MicroBioLogics. Control organisms included the TC Enterobacter aerogenes (ATCC 13048),
EC (ATCC 8739), and the non-coliform Pseudomonas aeruginosa (ATCC 10145). All control
cultures were prepared onto tryptic soy agar and incubated overnight. The QC control samples
were then prepared by inoculating triplicate 100 mL filter sterilized DDW aliquots each with 1
mL of a slightly turbid culture suspension prepared from the agar cultures in DDW. Control
samples were used to confirm the accurate response (positive response for positive control and
negative response for the negative controls) of the Colifast ALARM and reference methods at
relatively high concentrations. The control cultures were approximately 106 org/100 mL.
18
-------
All three TC positive controls were determined to be positive using the reference methods and
the Colifast ALARM (and confirmed to be positive during the applicable confirmation analyses).
In addition, all three EC positive controls were determined to be positive (for both TC and EC)
using the reference methods and the Colifast ALARM (and confirmed to be positive during the
confirmation analysis). All three TC negative control samples were found to be negative for TC
during the Colifast ALARM and reference analyses. All three EC negative control samples were
found to be negative for EC during the Colifast ALARM and reference analyses.
4.2 Audits
Two types of audits were performed during the verification test; a technical systems audit (TS A)
of the verification test procedures, and a data quality audit (DQA). Audit procedures for the
TSA and the data quality audit are described further below.
4.2.1 Technical Systems Audit
The Battelle AMS Center Quality Manager performed a TSA on July 20, 21, and 22, 2010 at
Battelle's microbiology laboratory in Columbus, OH and at the reference laboratory, Superior
Laboratories in Galloway, OH during the initial verification tests. The EPA AMS Center Quality
Manager participated in the Battelle and Superior Laboratories audits on July 21. The TSA
consisted of interviews with Battelle and Superior Laboratories personnel, observations of test
sample preparation and testing at Battelle and Superior Laboratories, and observation of sample
analysis. The purpose of the audit was to verify that:
• Sample preparation procedures were performed by Battelle according to the TQAP
requirements
• Reference laboratory methods for analyzing test samples conformed to the TQAP and
reference method requirements
• Technology testing was performed according to the TQAP and vendor instructions
• Test documentation provided a complete and traceable record of sample preparation and
analysis
• Equipment used in the test was calibrated and monitored according to TQAP
requirements and standard laboratory procedures.
Seven (7) Findings, six (6) Observations, and three (3) Remarks were identified during the TSA.
The findings involved training records, reference method requirements, sewage sample
collection, sample custody, and traceability of critical reagents. It was determined by Battelle
that none of these had an adverse impact on the test results and all findings have received a
satisfactory response.
In response to this audit report, the following actions were taken:
• Documentation of reference laboratory microbiology training was provided;
• Generation of a deviation to more accurately describe the collection of the sewage water
sample;
• Clarified and added detail to the documentation of sewage sample collection on the
custody form.
A TSA report was prepared and distributed to EPA.
19
-------
4.2.2 Data Quality Audit
Records generated in the verification test received a one-over-one review before these records
were used to calculate, evaluate, or report verification results. Data were reviewed by a Battelle
technical staff member involved in the verification test. The person performing the review added
his/her initials and the date to a hard copy of the record being reviewed.
In addition, audits of data quality (ADQs) were conducted on October 6 and 7, 2010. During the
audits, laboratory data generated at the reference laboratory, Superior Laboratories, Inc. and data
generated by the Colifast ALARM were reviewed and verified for completeness, accuracy and
traceability. The verification of coliform detection technologies was determined by the EPA
AMS Center Project Officer to be Category II test. Accordingly, at least 25% of the results for
each of the testing scenarios were verified versus the raw data, and 100% of the QC sample
results were verified. The data were traced from the initial acquisition, through reduction and
statistical analysis, to final reporting to ensure the integrity of the reported results. All
calculations performed on the data undergoing the audit were checked.
Three (3) Findings and three (3) Observations were identified during the ADQs. The three
findings involved sample custody, missing test data, and changes in the design of the optional
"at-line" analysis. Battelle believes that none of these had an adverse impact on the test results
and all have received a satisfactory response.
In response to these audit reports, the following actions are anticipated:
• Custody forms amended to accurately reflect sample transfers;
• Laboratory documentation provided to verify missing data point;
• "At-line" procedure clarified.
A data audit report was prepared and distributed to EPA.
20
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the quantitative performance factors are presented in this
chapter. Qualitative observations were also used to evaluate verification test data.
5.1 False Positive Rates, False Negative Rates, Sensitivity, and Specificity
False positive (FP) and false negative (FN) rates of the Colifast ALARM were evaluated when
assessing comparability. During this test, true positives (TP) were those positive results from the
Colifast ALARM that were confirmed, and false positives were those positive results from the
Colifast ALARM that were not confirmed by the reference method. Conversely, true negative
(TN) results were those negative results that were confirmed as negative, and false negative
results were those negative results that were shown to be positive by the confirmatory method.
Performance of the Colifast ALARM was tested by comparing the proportion of true positive
results from those technologies to the proportion of positive results from the SM 922 IB and F
reference methods.
Sensitivity is defined as the percent of positive samples correctly identified as positive and
specificity is defined as the percent of negative samples correctly identified as negative.
Estimates of sensitivity, specificity, false positive rates, and false negative rates as percentages
for the two methods were calculated as follows:
TP
Sensitivity = - x 100%
J TP+FN
TN
Specificity = x 100%
FP TN
False positive rate = - x 100% = (1 - ) x 100% = 1 - Specificity
FN TN
False negative rate = - x 100% = (1 -- ) x 100% = 1 - Sensitivity
TP+FN TN+FPJ J
5.2 Method Comparability
In order to assess whether the proportion of positive and negative samples were significantly
different between the Colifast ALARM and the reference method, chi-square tests for
independence were conducted. The chi-squared test was modeled in SAS® (ver. 9.1.3), using
the FREQ procedure. Because of the small sample size (some dilutions had less than five
21
-------
positive or negative results); the Yates continuity correction was applied. If the corrected chi-
square value was less than the critical value, the sample results between the two methods were
not considered significantly different (95% confidence, alpha = 0.05, p-value > 0.05). If the
corrected Chi-square value is greater than the critical value (p-value < 0.05), the results between
the two methods were considered to be significantly different. It should be noted that the Yates
continuity correction is a more conservative statistical approach, making it less likely that a
significant difference would be determined when it does not exist.
Prior to testing, a power analysis was conducted to determine the number of replicates required
to determine possible significant differences between the technologies being tested and the
reference method. The power analysis was done assuming that the total number of tests, while
not limited, would be the same for both the technology being tested and the reference method
and that the standard deviations of each would be equal. Conducted using the POWER
procedure in the SAS System, the power analysis determined the number of replicate tests
(across both test types) that would be necessary to detect a specified difference in proportions of
a specified size with fixed 80% power, given a specified value of the proportion for the reference
test (the acceptable range of reference test positive proportions was 25% to 75% for this test),
and a significance level of 0.05 for the test. To summarize, the power analysis shows that for
approximately 20 replicates, if the reference method was 25% positive (5 positive results and 15
negative results), then the technology being tested would be required to be 65% positive (13
positives and 7 negative results) to have a significant difference. Colifast ALARM results with a
lesser percentage of positive results out of 20 replicates would be considered similar to the
reference method. Similarly, if the reference method was 50% positive, then a significant
difference could be determined with Colifast ALARM results that were less than 2 positives and
18 negatives or more than 18 positives and 2 negatives. Finally if the reference method was 65%
positive, then a significant difference could be determined with at most a 32% positive result (6
positives and 14 negatives). The Colifast ALARM results are discussed in the context of this
power analysis.
In summary, the smallest difference that is able to be determined with 20 replicates is a relative
difference of 6-8 positive results. The power analysis revealed that differences of 1 or 2 positive
results could be determined, but between 150 and 1,250 replicates may be required.
22
-------
Chapter 6
Test Results
As mentioned previously, this verification test included both quantitative and qualitative
evaluations. The quantitative evaluation was conducted to assess the comparability of results
generated by the presence/absence results for the Colifast ALARM with those generated by the
presence/absence result from the reference methods. The qualitative evaluation was performed
to document the operational aspects of the Colifast ALARM when it was used during verification
testing. The following sections provide the results of the quantitative and qualitative evaluations.
Tables presenting the raw data presence/absence results for the reference methods, the Colifast
ALARM, and the confirmation analyses are provided in the Appendix.
6.1 TCData
The positive TC test results for the Colifast ALARM and reference method (SM 922IB) are
presented in Table 6-1. One of the three dilutions (Dilution B) yielded the target 50 ± 25% split
in responses for SM9221B.
Table 6-1. TC Positive Results
Dilution
(target TC cone.)
B(10org/100mL)
Colifast ALARM
+
Results
7
% of total
samples
35%
SM 9221B
+
Results
13
% of total
samples
65%
N - Number of results
Tables 6-2 and 6-3 summarize the TP (confirmed) and TN (confirmed) TC results for the
Colifast ALARM. The reference method data are also presented.
Table 6-2. TC Data Summary - Positives
Dilution
(target TC cone.)
B(10org/100mL)
Colifast ALARM
+ Results
7
Confirmed
TP
7
Difference
(FP)
0
SM 9221B
+ Results
13
Table 6-3. TC Data Summary - Negatives
Dilution
(target TC cone.)
B(10org/100mL)
Colifast ALARM
- Results
13
Confirmed
TN
13
Difference
(FN)
0
SM 9221B
- Results
7
23
-------
The sensitivity, specificity, false-positive, and false-negative rates for the Colifast ALARM TC
with respect to SM 9221B were determined as described in Section 5.1 and are presented in
Table 6-4.
Table 6-4. TC Data Summary - Confirmations a
Parameter
Sensitivity
Specificity
False Positive
False Negative
Dilution B
100%
100%
0%
0%
Results calculated with respect to SM 922IB as the reference method (see Section 5.1).
6.2 EC Data
Table 6-5 summarizes the positive EC test results for the Colifast ALARM analyzed according to
the manufacturer's directions. The positive EC test results for the reference methods (SM 922IF
and Colilert-18) are also presented. One of the three dilutions (Dilution A) yielded 45% positive
results for SM9221F, which was within the target range of 50 ± 25%, so only results from
Dilution A are reported.
Table 6-5. EC Positives
Dilution
(target TC cone.)
A (50 org/100 mL)
Colifast ALARM
+ Results
3
% of total
samples
15%
SM 9221F
+ Results
9
% of total
samples
45%
Colilert-18
+ Results
14
% of total
samples
70%
Tables 6-6 and 6-7 summarize the confirmed TP and TN EC results for the Colifast ALARM.
The reference method data are also presented.
Table 6-6. EC Summary - Positives
Dilution
(target TC cone.)
A(50org/100mL)
Colifast ALARM
+ Results
3
Confirmed
TP
3
Difference
(FP)
0
SM 9221F
+ Results
9
Colilert-18
+ Results
14
Table 6-7. EC Summary - Negatives
Dilution
(target TC cone.)
A(50org/100mL)
Colifast ALARM
- Results
17
Confirmed
TN
16
Difference
(FN)
1
SM 9221F
- Results
11
Colilert-18
- Results
6
The sensitivity, specificity, false-positive, and false-negative rates for the Colifast ALARM EC
results with respect to SM 922IF were determined as described in Section 5.1 and are presented
in Table 6-8.
24
-------
Table 6-8. EC Data Summary - Confirmations a
Parameter
Sensitivity
Specificity
False Positive
False Negative
Colifast ALARM
75%
100%
0%
25%
Results calculated with respect to SM 9221F as the reference method (see Section 5.1)
6.3 Method Comparability
Table 6-9 shows the results from the chi-square test for independence that was performed to
compare the TC results from the Colifast ALARM for each incubation time period against the
reference method SM 9221B. Because of the small number of replicates the Yates continuity
correction was performed on the chi-square results. When comparing the Colifast ALARM
results to the SM 922IB reference method, the corrected chi-square value for the TC dilution
was less than the critical limit; therefore, the chi-square test did not detect any differences
between the results of the Colifast ALARM and SM 9221B. The p-value was greater than 0.05,
indicating that the data did not show a statistically significant difference between the two
methods for the detection of TCs at the 95% confidence level.
Table 6-9. TC-SM9221B
Dilution
(target TC cone.)
B(10org/100mL)
Colifast
ALARM
+
7
_
13
SM9221B
+
13
_
7
Chi-
Square
2.500
Degrees
of
freedom
1
p-Value
0.114
Critical Limits
(p=0.05)
3.841
These results are consistent with the power analysis performed before testing and described in
Section 5.2. For TC, the SM 922IB reference method generated an observed result of 13
positive results out of 20 for Dilution B. According to the power analysis, when 22 replicates are
included in the experiment (we included 20), an observed result of 6 or less positive results out
of 20 from the Colifast ALARM would be required for the result to indicate a significant
underlying difference between the reference method and the Colifast ALARM. Therefore, the
observed 7 positive results out of 20 from the Colifast ALARM indicated there was not a
significant underlying difference indicated.
Tables 6-10 and 6-11 show the results from the Yates corrected chi-square test for independence
that was performed to compare the EC results from the Colifast ALARM against both reference
methods (SM 9221F and Colilert-18). When comparing the Colifast ALARM results to the SM
922IF reference method, the chi-square value was also less than the critical limits; therefore, the
chi-square test did not detect any differences between the results of the Colifast ALARM and the
reference method SM 9221F. The calculated p-values were also greater than 0.05, indicating that
the data did not show a statistically significant difference between the two methods for detection
of EC.
25
-------
When comparing to the Colilert-18 reference method, the corrected chi-square value was greater
than the critical limits; therefore, the chi-square test determined a significant difference between
the results of the Colifast ALARM and Colilert-18. In addition, calculated p-values were less
than 0.05, indicating a statistically significant difference between the two methods for detection
of EC.
Table 6-10. EC-SM9221F
Dilution
(target TC cone.)
A(50org/100mL)
Colifast
ALARM
+
3
_
17
SM9221F
+
9
_
11
Chi-
Square
2.976
Degrees
of
freedom
1.000
p- Value
0.0845
Critical
Limits
(p=0.05)
3.841
Table 6-11. EC - Colilert-18
Dilution
(target TC cone.)
A(50org/100mL)
Colifast
ALARM
+
3
-
17
Colilert-18
+
14
-
6
Chi-Square
10.23
Degrees of
freedom
1.000
p- Value
0.0014
Critical
Limits
(p=0.05)
3.841
As was the case for TC, the EC results are consistent with the power analysis performed before
testing. The proportion of observed positive results from the SM 922 IF reference method was
45% (9 positive and 11 negative). According to the power analysis, an observed result of
approximately 5% positive result (1 positive or less out of 20 results) would be required from the
Colifast ALARM for a significant underlying difference to be determined between the reference
method and the Colifast ALARM. The observed result from the Colifast ALARM was 3 positive
results out of 20 so there was not a significant underlying difference determined.
In comparing the EC result from the Colifast ALARM to the Colilert-18 reference method, the
observed results from Dilution A resulted in 70% positive results from the Colilert-18 reference
method. According to the power analysis, when the reference method has an observed rate of
75% positive (15 positive and 5 negative), the observed results from the Colifast ALARM would
require 13 or more negative results out of 20 to indicate a significant underlying difference from
the reference method. The Dilution A Colifast ALARM observed result was 85% negative (3
positive and 17 negative) so a significant underlying difference was indicated.
6.4 Analysis in "At-line" Mode
The objective of this component of the testing was to verify the Colifast ALARM capability of
collecting a sample from a reservoir (which in practice could be almost any container or flowing
pipe) and perform the analysis and report results as soon as determined by the Colifast ALARM
rather than waiting for the end of an incubation time period. Table 6-12 gives the results for the
26
-------
Table 6-12. Results of Analysis in "At-line" Mode
Sample Description
EC ATCC 8739
FS-DDW
FS-DDW
EC ATCC 8739
FS-DDW
EC from sewage
FS-DDW
FS-DDW
FS-DDW
EC from sewage
FS-DDW
m-Endo plate
counts
(EC/lOOmL)
38
n/a
n/a
33
n/a
36
n/a
n/a
n/a
28
n/a
SM 9222G-
NA-MUG
(EC/lOOmL)
0
n/a
n/a
0
n/a
36
n/a
n/a
n/a
28
n/a
EC
X=Presence
O=Absence
O
0
0
O
O
X
O
O
O
X
O
Incubation
Time/EC
Detected
14 h
14 h
14 h
14 h
14 h
llh
14 h
14 h
14 h
llh
14 h
X=Presence; O= Absence; n/a = not analyzed
FS-DDW - filter sterilized dechlorinated drinking water
analysis of approximately 30 org/lOOmL of EC ATCC 8739 and EC from sewage separated by
uncontaminated filter sterilized water. Both the reference method (SM 9222G-Na-MUG) and
the Colifast ALARM did not generate positive EC responses (as evidence by fluorescent
colonies) for the ATCC 8739 samples after 4 hr but they did exhibit slight fluorescence after 24
hr, however, the presence of ATCC 8739 organisms in each solution was confirmed with m-
Endo plate counts. The sewage samples were determined accurately by the reference method
and the Colifast ALARM. When testing the "at-line" mode, adjacent samples with
contamination and clean water were analyzed to test the issue of cross-contamination. All of the
FS-DDW water samples were negative for EC. The sewage EC samples were determined by the
Colifast ALARM to be positive after approximately 11 hours of incubation.
6.5 Operational Factors
The verification staff found that the Colifast ALARM was easy to use. A Colifast ALARM
representative came to Battelle to set up the equipment and train the verification staff in the
operation of the Colifast ALARM. The Colifast ALARM was set up by plugging the Colifast
ALARM and powering up. For operation in continuous mode, no special laboratory facilities
were required. In manual mode, laboratory water baths were required. Following an
approximately 30 minute training session, the operators (consisting of Battelle microbiology
technicians) were comfortable operating the Colifast ALARM without assistance.
As previously described, the Colifast ALARM was operated in manual measurement mode for
the measurement of TC and EC. In manual mode, 100 mL of the water sample were dispensed
into each sample bottle containing growth media (separate bottles for TC and EC) and the lid to
the bottle was tightened and then swirled to dissolve the contents. The cartridges were then
placed in a water bath that was held between 37-37.5 °C. The vendor instructions call for the TC
sample bottles to be incubated for 15-17 hours and the EC sample bottles for 14-16 hours.
During this test, the TC samples were incubated for approximately 15.5 hours and 14.5 hours,
respectively. After the appropriate incubation time, the bottles were removed from the water
27
-------
bath and inserted one at a time into the Colifast ALARM for fluorescent measurement. The
bottles were analyzed by clicking on a "start" button on the computer screen. The measurement
of each sample took approximately 30 seconds.
Incubation of the samples at the correct temperature was critical to obtaining accurate results
from the Colifast ALARM. The complete procedure described in the test/QA plan was
performed initially with water bath temperatures ranging from 35-36°C. The positive control
samples included as part of the ETV test provided negative results suggesting a problem. Upon
consultation with Colifast, it was determined that water bath temperatures needed to be in the
range of 37-37.5°C. Because of this, the testing was repeated. The results in this report were
obtained during the repeated testing and the previous results were not reported since the
incubation temperatures utilized were not correct due to a miscommunication with the vendor.
In "at-line" mode (which was demonstrated for EC only because of limited time), at least 2.5 L
of sample was prepared in order to accommodate for the various rinse cycles that took place for
each sample collection. Tubing from the Colifast ALARM was connected to the sample
reservoir placed on the bench top next to the Colifast ALARM. The sample analysis was started
with a tap on the start button on the touch screen computer and the sample was drawn into a
sample bottle within the Colifast ALARM and the appropriate growth media was added to the
sample bottle. This sample bottle was incubated for 14 hours. Every hour throughout that
incubation, the fluorescence was measured from the sample bottle to determine if the sample was
positive or not. Positive results were indicated by a red light on the outside of the Colifast
ALARM, by audio alarm, on the screen, and recorded in a text-delimited data file. A positive
result could have been reported at any point during the incubation time, while a negative result
would not occur until the end of the 14 h incubation. The automated at-line mode eliminates the
need for a technician to be present to collect the water sample, analyze and read the sample
result. Also, the Colifast ALARM method calls for a 14 h analysis, shortening the analysis time
from the 48 to 72 required by the standard methods, increasing the efficiency and decreasing the
amount of reagents and manpower expended performing the reference methods.
The Colifast ALARM has dimensions of 42 centimeters (cm) wide x 36 cm deep x 64 cm high
(17 inches (in) wide x 14 in deep x 26 in high) and weighs approximately 31 kilograms (68
pounds). The Colifast growth media are sold as bottles with 20 tests for the "at-line" mode or as
single sample cartridges. The Colifast ALARM is self contained and does not require any
additional equipment or materials to perform analyses. The Colifast ALARM costs
approximately $35,000. Sample cartridges can be purchased for approximately $10-15 per
sample bottle.
28
-------
Chapter 7
Performance Summary
In order to comply with the TCR, water utilities need coliform detection technologies that are
able to detect TC and EC at concentrations of one organism per 100 mL samples. This ETV test
verified the performance of the Colifast ALARM at that level of detection. While it is difficult
to determine if a single target organism is present in 100 mL of water, when approximately half
of the analyzed replicates are positive and half are negative, the density of the organism has
become adequately low so that a positive result can be considered near single organism
detection. Therefore, for the purpose of this verification test, spiked DW dilution sets were
prepared that provided 50 ±25% positive results for TC and EC with the reference methods and
then the results from the reference methods were compared with the Colifast ALARM. The
results of the verification of the Colifast ALARM are summarized below:
Positive Results. Table 7-1 summarizes the positive TC test results for the Colifast ALARM.
Table 7-1. Results Summary for Positive Colifast ALARM Results for TC and EC
TC
or
EC
TC
EC
Dilution
B(10org/100mL)
A(50org/100mL)
Colifast ALARM
+
Results
7
3
% of total
samples
35%
15%
SM 9221B/F
+
Results
13
9
% of total
samples
65%
45%
Colilert-18
+
Results
NA
14
%of
total
samples
NA
70%
NA - Colilert-18 analyses were only applicable to the EC samples
Specificity, Sensitivity, FPrate, and FN rate. Table 7-2 summarizes the specificity, sensitivity,
FP rate, and FN rate for TC and EC with respect to SM 9221B and 9221F. Sensitivity is defined
as the percent of positive samples correctly identified as positive and specificity is defined as the
percent of negative samples correctly identified as negative.
Table 7-2. Results Summary of Colifast ALARM
Parameter
Sensitivity
Specificity
False Positive Rate
False Negative Rate
TC
Dilution B
100%
100%
0%
0%
EC
Dilution A
75%
100%
0%
25%
Results calculated with respect to SM 922IB and F as the reference methods for TC and EC, respectively.
29
-------
Comparability. In addition, a chi-square test for independence with a Yates correction for
continuity (because of the small sample size) was performed to compare the Colifast ALARM
against the reference methods (SM 9221B for TC, 9221F for EC, Colilert-18 for EC). For the
Colifast ALARM TC results being compared to the SM 922IB, the chi-square value was less
than the critical limit in each case. Therefore, the chi-square test did not detect any differences
between the results of the Colifast ALARM and the reference method for TC. In addition, the
calculated p-values were also greater than 0.05, indicating that the data did not show a
statistically significant difference between the two methods for the detection of TC at the 95%
confidence interval.
For the Colifast ALARM EC results being compared to SM 9221 F, the corrected chi-square
value for the EC dilution was less than the critical limit. Therefore, the chi-square test did not
detect any differences between the results of the Colifast ALARM and SM 9221F for EC. In
addition, the calculated p-values were also greater than 0.05, indicating that the data did not
show a statistically significant difference between the two methods for the detection of EC at the
95% confidence interval. When comparing with Colilert-18, the corrected chi-square value for
the EC dilution was more than the critical limit and the calculated p-values were less than 0.05,
indicating that the data did show a statistically significant difference between the two methods
for the detection of EC at the 95% confidence interval.
Overall, these results were consistent with the power analysis performed before testing and
described in Section 5.2 in that the results confirmed that 20 replicates was adequate to
determine significant differences (differences of 6-8 positive results out of 20 replicates) between
the methods at 80% power. The determination of smaller differences (< 2 positive results out of
20 replicates) would require additional replicates.
Analysis in "At-line" Mode. The objective of this component of the testing was to verify the
Colifast ALARM capability of collecting a sample from a reservoir (which in practice could be
almost any container or flowing pipe) and perform the analysis and report results as soon as
determined by the Colifast ALARM rather than waiting for the end of the 14-hour incubation time
period. Duplicate analysis of approximately 30 org/lOOmL of EC ATCC 8739 and EC from
sewage separated by uncontaminated filter sterilized water were performed. Both the reference
method and the Colifast ALARM did not generate positive EC responses for the ATCC 8739
samples after 4 hr (some fluorescence after 24 hr), however, the presence of ATCC 8739
organisms was confirmed by counting on m-Endo plates. The sewage samples were determined
accurately by the reference method and the Colifast ALARM. The sewage EC samples were
determined to be positive after approximately 11 hours of incubation.
Operational Factors. The verification staff found that the Colifast ALARM was easy to use. A
Colifast ALARM representative came to Battelle to set up the equipment and train the
verification staff in the operation of the Colifast ALARM. In manual mode, 100 mL of the water
sample were dispensed into each sample bottle containing growth media (separate bottles for TC
and EC) and the lid to the bottle was tightened and then swirled to dissolve the contents. The
bottles were then placed in a water bath that was held between 37-37.5 °C. The vendor
instructions called for the TC sample bottles to be incubated for 15-17 hours and the EC sample
bottles for 14-16 hours. During this test, the TC samples were incubated for approximately 15.5
hours and 14.5 hours, respectively. After the appropriate incubation time, the bottles were
removed from the water bath and inserted one at a time into the Colifast ALARM for fluorescent
30
-------
measurement. The bottles were analyzed by clicking on a "start" button on the computer touch
screen. The measurement of each sample took approximately 30 seconds.
Incubation of the samples at the correct temperature was critical to obtaining accurate results
from the Colifast ALARM. The complete procedure described in the test/QA plan was
performed initially with water bath temperatures ranging from 35-36°C. The positive control
samples included as part of the ETV test provided negative results suggesting a problem. Upon
consultation with Colifast, it was determined that water bath temperatures needed to be in the
range of 37-37.5°C. Because of this, the testing was repeated. The results in this report were
obtained during the repeated testing and the previous results were not reported since the
incubation temperatures utilized were not correct due to a miscommunication with the vendor.
In "at-line" mode, at least 2.5 L of an EC sample was prepared in order to accommodate for the
various rinse cycles that took place for each sample collection. Tubing from the Colifast
ALARM was connected to the sample reservoir placed on the bench top next to the Colifast
ALARM. The sample analysis was started by clicking on a "start" button on the computer touch
screen and the sample was drawn into a sample bottle within the Colifast ALARM and the
appropriate growth media was added to the sample bottle. This sample bottle was incubated for
14 hours and the fluorescence was measured from the sample bottle every hour throughout the
incubation to determine if the sample was positive or not. Positive results were immediately
indicated by a red light on the outside of the Colifast ALARM, on the screen, and recorded in a
text-delimited data file. A positive result could have been reported at any point during the
incubation time, while a negative result would not occur until the end of the 14 h incubation.
The automated "at-line" mode eliminates the need for a technician to be present to collect the
water sample, analyze and read the sample result. Also, the Colifast ALARM method calls for a
14 h analysis, shortening the analysis time from the 48 to 72 hr required by the standard
methods, increasing the efficiency and decreasing the amount of reagents and manpower
expended performing the reference methods.
The Colifast ALARM has dimensions of 42 centimeters (cm) wide x 36 cm deep x 64 cm high
(17 inches (in) wide x 14 in deep x 26 in high) and weighs approximately 31 kilograms (68
pounds). The Colifast growth media are sold as bottles with 20 tests for the "at-line" mode or as
single sample cartridges. The Colifast ALARM is self contained and does not require any
additional equipment or materials to perform analyses. The Colifast ALARM costs
approximately $35,000. Sample cartridges can be purchased for approximately $10-15 per
sample bottle.
31
-------
Chapter 8
References
1. Total Coliform Rule, United States Federal Register, 54 FR 27544-27568, June 29, 1989,
Vol. 54, No. 124
2. EPA Microbiological Alternate Test Procedure Protocol for Drinking Water, Ambient
Water, and Wastewater Monitoring Methods, EPA 821-B-03-004, April 2004.
3. Test/QA Plan for Verification of Coliform Detection Technologies for Drinking Water,
Battelle, Version 1.0, July 14, 2010.
4. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version 7.
U.S. Environmental Technology Verification Program, Battelle, November 2008.
5. International Standards Organization, 9308-1 Water quality - Detection and Enumeration
of Escherichia coli and Coliform Bacteria, Geneva: International Organization for
Standardization, 2009.
6. International Standards Organization, 17994 Water Quality - Criteria for the
Establishment of Equivalence Between Microbiological Methods, Geneva: International
Organization for Standardization, 2001.
7. A comparison of the International Standards Organisation reference method for the
detection of coliforms and Escherichia coli in water with a defined substrate procedure,
Journal of Applied Microbiology, 2003, 95, 1285-1292.
8. American Public Health Association, American Water Works Association, and Water
Environment Federation. 2005. Standard Methods for the Examination of Water and
Wastewater. 21st Edition.
9. SOP GEN.V-003-10. Standard Operating Procedure for the Use of pH meters to Measure
pH. Battelle.
10. SOP GEN. V-013-04. SOP for the Calibration and Maintenance of Thermometers.
11. Bacteriological Analytical Manual, 8th Edition. 1995. "Chapter 3: Aerobic Plate Count".
AOAC International.
12. SOP ENV-ADM-009, Standard Operating Procedure for Sample Chain-of-Custody
Battelle, September 2007.
32
-------
Appendix
Raw Data from Reference Methods, Colifast ALARM, and Confirmation
Analyses
33
-------
Dilution
A
(50org/100ml)
Sample
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Percent Positive=
B
(lOorg/lOOml)
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Percent Positive=
SM 9221B/F
(2100 series)
TC
X
X
X
X
X
o
X
X
X
X
X
X
X
X
X
X
X
X
X
X
95%
O
o
X
X
o
X
X
X
X
X
X
o
X
o
X
o
X
o
X
X
65%
EC
X
X
o
o
X
o
o
o
X
X
o
X
o
X
X
o
o
X
o
o
45%
O
o
o
X
o
o
o
o
o
o
X
o
o
o
o
o
o
o
X
o
15%
Colilert
EC
X
X
X
o
X
o
X
X
X
o
X
X
X
X
o
X
X
o
o
X
70%
o
o
o
o
o
o
o
o
o
o
o
o
o
X
o
o
X
o
o
o
10%
Colifast
TC
(2400 series)
X
O
X
X
O
X
X
X
O
X
X
X
X
X
X
X
X
X
X
X
85%
O
o
X
o
X
o
o
o
X
o
o
o
X
X
X
o
o
o
X
o
35%
EC
(2500 series)
O
O
O
O
O
O
O
X
O
X
O
o
X
o
o
o
o
o
o
o
15%
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
0%
Colifast TC
Comfirmation
via SM9221B/F
(2400 series)
TC
X
X
X
X
X
X
X
X
o
X
X
X
X
X
X
X
X
X
X
X
95%
O
o
X
o
X
o
o
o
X
o
o
o
X
X
X
o
o
o
X
o
35%
EC
O
O
o
o
o
o
o
X
o
o
o
o
X
X
X
o
X
o
o
o
25%
O
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
0%
Colifast EC
Comfirmation
via SM9221B/F
(2500 series)
TC
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
100%
o
o
o
X
o
o
X
o
X
o
X
o
o
X
X
o
X
X
o
o
40%
EC
o
o
o
o
o
o
o
X
o
X
X
o
X
o
o
o
o
o
o
o
20%
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
0%
34
-------
Dilution
Sample
No.
Percent Positive=
C
(5org/100ml)
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Precent Positive=
^-^-^^^^»
Controls^
Method Blank
TC Positive (Ea)
EC Positive (Ec)
TCNeg/EcNeg
(Pa)
64
69
72
61
65
70
62
67
71
63
66
68
SM 9221B/F
(2100 series)
TC
65%
O
O
O
X
X
O
O
O
O
X
O
O
O
O
O
O
X
O
O
O
20%
•
O
O
O
X
X
X
X
X
X
O
O
O
EC
15%
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0%
•
O
O
O
O
O
O
X
X
X
O
O
O
Colilert
EC
10%
O
O
O
O
O
O
O
O
X
X
X
O
O
O
O
O
O
O
O
O
15%
M
O
O
O
O
O
O
X
X
X
O
O
O
Colifast
TC
(2400 series)
35%
O
O
O
O
O
X
X
O
O
O
O
O
O
X
O
X
O
O
O
X
25%
M
O
O
O
X
X
X
X
X
X
O
O
O
EC
(2500 series)
0%
O
O
O
O
O
O
X
O
O
O
O
O
O
O
O
O
O
O
O
O
5%
M
O
O
O
O
O
O
X
X
X
O
O
O
Colifast TC
Comfirmation
via SM9221B/F
(2400 series)
TC
35%
O
O
X
X
O
X
X
O
O
O
O
O
O
X
X
O
O
O
O
X
35%
•
O
O
O
X
X
X
X
X
X
O
O
O
EC
0%
O
O
O
O
O
X
O
O
O
O
O
O
O
O
O
O
O
O
O
O
5%
•
O
O
O
O
O
O
X
X
X
O
O
O
Colifast EC
Comfirmation
via SM9221B/F
(2500 series)
TC
40%
X
O
X
X
O
X
X
O
X
O
X
O
O
O
X
O
X
O
O
O
45%
•
O
O
O
X
X
X
X
X
X
O
O
O
EC
0%
X
O
O
O
O
O
X
O
O
O
O
O
O
O
O
O
O
O
O
O
10%
•
O
O
O
O
O
O
X
X
X
O
O
O
X= Presence
O= Absence
35
------- |