December 2004
Environmental Technology
Verification Report
Applied Biosystems
TaqMan® E. COLI 0157:H7
Detection System
Prepared by
Battel le
Baireiie
I he Business of Innovation
Under a cooperative agreement with
&EPA U.S. Environmental Protection Agency
ElV ElV ElV
-------
December 2004
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Applied Biosystems
TaqMan® E. coli0157:H7
Detection System
by
Stephanie Buehler
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six verification centers. Information about
each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/centers/centerl.html.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We sincerely appreciate the
contribution of drinking water samples from the New York City Department of Environmental
Protection (Paul Bennett), the City of Orlando (Terri Slifko), and the Metropolitan Water
District of Southern California (Paul Rochelle). Also, thanks go to the Metropolitan Water
District of Southern California for concentrating each drinking water sample. We would also
like to thank Myriam Medina-Vera, U.S. Environmental Protection Agency National Exposure
Research Laboratory; Jorge Santo Domingo, U.S. Environmental Protection Agency National
Risk Management Research Laboratory; Kerri Alderisio, New York City Department of
Environmental Protection; Ricardo DeLeon, Metropolitan Water District of Southern
California; and Stanley States, Pittsburgh Water and Sewer Authority, for their careful review of
the test/quality assurance plan and this verification report
IV
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments vii
List of Abbreviations x
1 Background 1
2 Technology Description 2
3 Test Design and Procedures 4
3.1 Introduction 4
3.2 Test Samples 6
3.2.1 Performance Test Samples 6
3.2.2 Drinking Water Samples 7
3.2.3 Quality Control Samples 8
3.3 Reference Methods 9
3.3.1 Plate Enumeration 9
3.3.2 Drinking Water Analysis 9
3.4 Test Procedure 9
3.4.1 Sample Handling 9
3.4.2 Sample Preparation and Analysis 10
3.4.3 Drinking Water Characterization 12
4 Quality Assurance/Quality Control 13
4.1 Sample Chain-of-Custody Procedures 13
4.2 Equipment Calibration 13
4.3 Characterization of Contaminant Stock Solution 13
4.4 Quality Control Samples 14
4.5 Audits 15
4.5.1 Technical Systems Audit 15
4.5.2 Audit of Data Quality 15
4.6 QA/QC Reporting 15
4.7 Data Review 15
5 Data Analysis 17
5.1 Accuracy 17
5.2 Specificity 17
5.3 False Positive/Negative Responses 17
5.4 Precision 18
v
-------
5.5 Interferences 18
5.6 Other Performance Factors 18
6 Test Results 19
6.1 Accuracy 21
6.2 Specificity 23
6.3 False Positive/Negative Responses 23
6.4 Precision 26
6.5 Interferences 26
6.5.1 Interferent PT Samples 26
6.5.2 Drinking Water Samples 27
6.6 Other Performance Factors 27
7 Performance Summary 29
8 References 31
Figures
Figure 2-1. Applied Biosystems' ABI Prism® 7000 Sequence Detection System 2
Figure 6-1. ARn vs. Cycle Number ABI Prism® 7000 Sequence Detection System
Amplification Plot for E. coli Samples and Controls 20
Tables
Table 3-1. Infective/Lethal Dose of Target Contaminant 6
Table 3-2. Performance Test Samples 6
Table 3-3. Drinking Water Samples 8
Table 3-4. ATEL Water Quality Characterization of Drinking Water Samples 12
Table 4-1. E. coli Triplicate Plate Enumeration Data 14
Table 4-2. Data Recording Process 16
Table 6-1. E. coli Contaminant-Only PT Sample Results 22
Table 6-2. E. coli Specificity Results 23
Table 6-3. E. coli False Positive/Negative Results 25
Table 7-1. E. coli Summary Table 29
VI
-------
List of Abbreviations
AMS Advanced Monitoring Systems
ASTM American Society of Testing and Materials
ATEL AquaTech Environmental Laboratories, Inc.
ATCC American Type Culture Collection
BSL Biosafety Level
Ca calcium
cfu colony forming unit
cm centimeter
DI deionized water
DNA deoxyribonucleic acid
DW drinking water
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
ID identification
IPC internal positive control
L liter
LOD limit of detection
MB method blank
Mg magnesium
mg milligram
|_iL microliter
mL milliliter
MWD Metropolitan Water District
NAC No Amplification Control
NTC No Template Control
PBS phosphate buffered saline
PCR polymerase chain reaction
PT performance test
QA quality assurance
QC quality control
QMP Quality Management Plan
SOP standard operating procedure
TSA technical systems audit
Vll
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative tech-
nologies by developing test plans that are responsive to the needs of stakeholders, conducting
field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance
(QA) protocols to ensure that data of known and adequate quality are generated and that the
results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
recently evaluated the performance of Applied Biosystems' TaqMan® E. coli 0157:H7
Detection System for the detection of Escherichia coli 0157:H7 (E. coli).
1
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of environ-
mental monitoring technolo^es for air, water, and soil. This veri fication report provides results
for the verification testing of the TaqMan° E. coli 0157:H7 Detection System, which includes
the TaqMan° E coli 0157:H7 Detection Kit, the Prep Man™ Ultra Sample Preparation Reagent,
and the ABI Prism0 7000 Sequence Detection System and associated software. The following is
a description of the TaqMan° E. coli 0157:H7 Detection System based on information provided
by the vendor. The information provided below was not subjected to verific ation in this test.
The TaqMan° E. coli 0157: H7 Detection Kit is part of an integr ated system that includes
polymerase chain reaction (PCR) chemistry, instrumentation, and data analysis software. Each
TaqMan° Detection Kit contains TaqMan° probes and prirrers in the PCR mix along with
MgCl2, dNTPs, and an internal positive control (IPC) system The ABI Prism0 7 000 Sequence
Detection System is a real-time PCR fluorescence detection instrument that combines a thermal
cycler with an optical detection system. During amplification, it can distinguish between the
fluorogenic labels for the assay target (i.e., E. coli 0157:H7) and an IPC. The ABI Prisrn° 7000
uses a rnicr opiate capable of running 96 samples. The PrepMan™ Ultra Sample Preparation
Reagent removes the inhibitory and interfering substances that potentially affect PCR
amplification and lead to inconclusive results.
The basis for TaqMan°pathogen detection assays
is the specific amplification of a deoxyribonucleic
acid (DMA) target located within abacterial
^nome and unique to the targpt organism
ArnpliTaq Gold0 DMA polymerase, which is used
exclusively in the detection kit, prevents the
formation of non-specific PCR products that
decrease performance of the enzyme and assay. A
fluorogenic signal is not generated unless both the
PCR primers and the 5' nuclease probe hybridize
to the PCR target. The Sequence Detection System
software offers an Absolute Quantitation (real-
time) and Plus/Minus (endpoint) assay for PCR and data analysis. The Absolute Quantitation
assay can provide qualitative and quantitative information, while the PlusTvlinus assay provides
a "yesfao" result for each sample. The Absolute Quantitation assay was used in this test.
Figure2-1. Applied Biosystems' ABI
Prism0 7000 Sequence Detection System
2
-------
All three components of the TaqMan® E. coli 0157:H7 Detection System were used in this
verification test for the analysis of test samples. The ABI Prism® 7000 Sequence Detection
System, excluding the computer, measures approximately 20 inches [51 centimeters (cm)] deep
by 21 inches (53 cm) high by 15 inches (39 cm) wide, and weighs 75 pounds (34 kilograms).
The ABI Prism® 7000 Sequence Detection System costs $47,250. The T ® E. coli
0157:H7 Detection Kit costs $800 for approximately 100 assays, and the PrepMan™ Ultra
Sample Preparation Reagent costs $100 for 50 to 100 sample DNA extractions.
3
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
The purpose of this verification test of rapid PCR technologies was to evaluate the ability of
these technologies to detect the presence of specific bacteria in water and to determine the
technologies' performance when specific interferents were added to pure water and when
interferents were inherently present in several drinking water matrices. The technologies for this
verification test operate based on the PCR process, which involves enzyme-mediated reactions
that allow for target DNA (that from the bacteria of interest) replication and amplification
through a series of temperature cycles. Before the target DNA can be amplified, however, it
must first be extracted from the bacteria and then purified.
Because rapid PCR technologies are anticipated to serve mostly as screening tools in water
monitoring scenarios, providing rapid results as to whether or not a pathogen or biological agent
is present in the water, this verification test involved only qualitative results. This verification
test of the TaqMan® E. coli 0157:H7 Detection System was conducted according to procedures
specified in the Test/QA Plan for Verification of Rapid PCR Technologies.(1) The performance
of the TaqMan® E. coli 0157:H7 Detection System was verified in terms of the following
parameters:
¦ Accuracy
¦ Specificity
¦ False positive/negative responses
¦ Precision
¦ Interferences
¦ Other performance factors.
The performance of the TaqMan® E. coli 0157:H7 Detection System was verified by
challenging it with various concentration levels of E. coli 0157:H7 in American Society of
Testing and Materials (ASTM) Type II deionized (DI) water, ASTM Type IIDI water spiked
with various interferents, and concentrated drinking water (DW) samples obtained from four
water utilities from different geographical locations in the United States. Each source of DW
represented a unique water treatment process. In addition, the interferent and DW samples were
analyzed without adding any contaminant to evaluate the potential for false positive results.
4
-------
Contaminant concentrations included the infective/lethal dose concentration given in Table 3-1
for E. coli and approximately 2, 5, 10, and 50 times the vendor-reported system limit of
detection (LOD) for this technology. The infective/lethal dose of E. coli was determined by
calculating the concentration at which ingestion of 250 milliliters (mL) of water is likely to
cause the death of a 70-kilogram (approximately 154 pounds) person based on human LD50 or
ID50 data.(2) The results from quadruplicate analysis of the contaminant performance test (PT)
samples and comparison with the known sample compositions provided information on the
accuracy and precision of the TaqMan® E. coli 0157:H7 Detection System. The interferent PT
samples contained humic and fulvic acids at two concentrations, both spiked and unspiked with
E. coli. Each was analyzed in quadruplicate and provided information on potential matrix
interferences. As necessary, additional concentration levels were analyzed to more thoroughly
evaluate the performance of the TaqMan® E. coli 0157:H7 Detection System.
For the purposes of this test, 10 colony forming units (cfu)/mL were used to calculate the
concentration levels spiked in the PT samples per the original test/QA plan design. This
vendor-provided concentration level was anticipated to be the level at which quantifiably
reproducible positive results could be obtained from a raw water sample using the TaqMan®
E. coli 0157:H7 Detection System. This concentration level is referred to in this report as the
"system LOD." The system LOD incorporates the sensitivities and uncertainties of the entire
TaqMan® E. coli 0157:H7 Detection System, in particular the PrepMan™ Ultra Sample
Preparation Reagent, as well as the TaqMan® E. coli 0157:H7 Detection Kit itself; and, as such,
the system LOD is a method detection limit rather than an instrument or reagent-specific
detection limit. As mentioned previously, the system LOD provided by the vendor was used
specifically as a guideline in calculating sample concentration ranges per the original test/QA
plan (2, 5, 10, and 50 times the system LOD for PT samples) for use with the TaqMan® E. coli
0157:H7 Detection System in this verification test. It should be noted that the concentration
level provided by the vendor was one that was known for the system in detecting E. coli in
pre-enriched food samples (see Section 6.1). Applied Biosystems does not claim that this is the
true LOD of the TaqMan® E. coli 0157:H7 Detection System. Detection limits for individual
components of the TaqMan® E. coli 0157:H7 Detection System and the system as a whole may
differ and were not verified in this test.
The verification test was conducted at Battelle from June 2, 2004, through June 24, 2004. Aqua
Tech Environmental Laboratories, Inc. (ATEL) of Marion, Ohio, performed physicochemical
characterization for each DW sample, including turbidity, dissolved and total organic carbon,
specific conductivity, alkalinity, pH, magnesium (Mg), calcium (Ca), hardness, total organic
halides, trihalomethanes, and haloacetic acids. Battelle cultured the bacteria, provided the stock
solutions used in this test, and then confirmed the presence and quantity of E. coli bacteria in
the stock solutions using plate enumeration. The stock solutions of E. coli were stored frozen as
1 mL aliquots. A new 1 mL vial of stock solution was thawed and used for each day of testing.
All test samples were prepared from the stock solutions on the day of analysis. All purified DNA
was used the same day it was extracted and purified. Each set of replicates for a sample came
from the same batch of purified DNA.
5
-------
Table 3-1. Infective/Lethal Dose of Target Contaminant
Contaminant
Infective/Lethal Dose Concentration
E. coli 0157:H7
0.2 cfu/mL
3.2 Test Samples
Test samples used in this verification test included PT samples, DW samples, and quality control
(QC) samples. Each type of test sample, including QC samples, is described further below.
3.2.1 Performance Test Samples
Table 3-2 lists the PT samples analyzed in this verification test for E. coli. PT samples were
prepared in ASTM Type II DI water. The first type of PT sample consisted of ASTM Type IIDI
water spiked at various concentration levels of E. coli. In accordance with the test/QA plan,
contaminant PT samples with concentrations ranging from the infective/lethal dose
concentration to 50 times the vendor-stated system LOD were analyzed using the TaqMan®
E. coli 0157:H7 Detection System. The infective/lethal dose concentration was analyzed to
document the response of the TaqMan® E. coli 0157:H7 Detection System at that important
concentration level. Four concentration levels at 2, 5, 10, and 50 times the vendor-reported
system LOD in addition to the infective/lethal dose concentration were analyzed. Each
concentration level for the PT samples was analyzed in quadruplicate. Preliminary data for these
samples indicated that E. coli was not detectable or that results were highly inconsistent at these
levels. To more thoroughly assess the system and determine the best contaminant level to use in
the interferent PT and DW samples, a dilution series of contaminant-only PT samples was
analyzed, ranging from lxlO6 cfu/mL to 10 cfu/mL of E. coli in order of magnitude increments.
This testing and the subsequent results are fully described in Section 6.1.
Table 3-2. Performance Test Samples
Type of PT
Sample
Sample Characteristics
Approximate Concentrations
(cfu/mL)
Contaminant-only
E. coli in DI water
0.2 to lxlO6
Interferent
E. coli in 0.5 milligram per liter
(mg/L) humic acid and 0.5 mg/L
fulvic acid
lxlO4
E. coli in 2.5 mg/L humic acid
and 2.5 mg/L fulvic acid
lxlO4
The second type of PT sample was potential interferent samples. Four replicates of each
interferent PT sample were analyzed to determine the performance of the TaqMan® E. coli
0157:H7 Detection System in the presence of humic and fulvic acids. The interferent PT
6
-------
samples contained humic and fulvic acids isolated from Elliot Soil near Joliett, IL, (obtained
from the International Humic Substances Society) spiked into ASTM Type IIDI water. Each of
these interference mixtures was prepared at two concentration levels. One concentration was
near the upper limit of what would be expected in DW (5 mg/L) and one was at a mid-low range
of what would be expected (1 mg/L). The 1 mg/L interferent mixture was prepared as 0.5 mg/L
humic acid and 0.5 mg/L fulvic acid. Similarly, the 5 mg/L interferent solution was prepared as
2.5 mg/L humic acid and 2.5 mg/L fulvic acid. These interferent levels were confirmed through
analysis of aliquots by ATEL. Also, E. coli was added to these samples, along with the potential
interferent, at a concentration level (lxlO4 cfu/mL) as determined in the dilution series analysis
described above. This concentration level was set at approximately 10 times the lowest level in
the dilution series where consistent results were obtained for all replicates. The samples were
analyzed in quadruplicate.
In all cases, four replicates for each PT sample, DW sample, and QC sample were taken from the
extracted and purified product (unspiked) or DNA (spiked) of one sample solution. That is, only
one spiked or unspiked sample solution was prepared for each set of replicates and taken
through the DNA extraction and purification procedure. Four replicates were then taken from
the same purified product or DNA. In an effort to characterize the efficacy of the extraction and
purification procedure in the presence of inhibitory substances (humic and fulvic acids), four
solutions of humic and fulvic acids each at 0.5 mg/L spiked with E. coli at lxlO4 cfu/mL were
prepared in addition to the samples listed in Table 3-2. Each solution was put through the DNA
extraction and isolation procedure, and then four replicates from each of the four purified DNA
solutions were analyzed on the TaqMan® E. coli 0157:H7 Detection System.
3.2.2 Drinking Water Samples
Table 3-3 lists the DW samples analyzed for E.coli in this test. DW samples were collected from
four geographically distributed municipal sources (Ohio, California, Florida, and New York) to
evaluate the performance of the TaqMan® E. coli 0157:H7 Detection System with various
sample matrices. These samples varied in their source and treatment and disinfection process.
All samples had undergone either chlorination or chloramination prior to receipt. Samples were
collected from utility systems with the following treatment and source characteristics:
¦ Chlorinated filtered surface water
¦ Chloraminated filtered surface water
¦ Chlorinated filtered groundwater
¦ Chlorinated unfiltered surface water.
All samples were collected in pre-cleaned high density polyethylene containers. After sample
collection, to characterize the DW matrix, an aliquot of each DW sample was sent to ATEL to
determine the following water quality parameters: turbidity, organic carbon, conductivity,
alkalinity, pH, Ca, Mg, hardness, total organic halides, concentration of trihalomethanes, and
haloacetic acids. The DW samples were dechlorinated with sodium thiosulfate pentahydrate to
prevent the degradation of some of the contaminants by chlorine. Because real-world
applications of PCR technologies to screen water samples rely on pre-concentration of the water
7
-------
sample to be analyzed, approximately 100 L of each of the above sources of DW were
dechlorinated and then concentrated through ultrafiltration techniques to a final volume of
250 mL by the Metropolitan Water District (MWD) of Southern California. As shown in
Table 3-3, each DW sample was analyzed without adding any contaminant (i.e., unspiked), as
well as after fortification with E. coli at a single concentration level (the same as determined for
the interferent PT samples).
Table 3-3. Drinking Water Samples
Drinking Water Sample Description
Approximate
Contaminant
Concentrations
(cfu/mL)
Water Utility
Water Treatment
Source Type
E. coli
Columbus, Ohio
(OH)
chlorinated filtered
surface
unspiked and
lxlO4
MWD of Southern California
(CA)
chloraminated filtered
surface
unspiked and
lxlO4
Orlando, Florida
(FL)
chlorinated filtered
ground
unspiked and
lxlO4
New York City, New York
(NY)
chlorinated unfiltered
surface
unspiked and
lxlO4
3.2.3 Quality Control Samples
QC samples included method blank (MB) samples consisting of ASTM Type IIDI water and
positive and negative controls, as provided by the vendor. A positive control reaction was also
analyzed in quadruplicate with each batch of samples in a given day. All of the MB QC samples
were exposed to sample preparation and analysis procedures identical to the test samples. The
positive control reaction sample was simply DI water spiked with E. coli at lxl06cfu/mL, per
the PrepMan™ Ultra Sample Preparation Reagent protocol. External positive and negative
controls were prepared and used according to the protocol provided by the vendor. At least two
positive controls and 12 negative controls were prepared with each 96-well plate of samples
placed in the ABI Prism® 7000. The negative controls consisted of six No Amplification Control
(NAC) reactions and six No Template Control (NTC) reactions. The MB samples were used to
confirm negative responses in the absence of any contaminant and to ensure that no sources of
contamination were introduced into handling and analysis procedures. At least 10% of the test
samples (eight replicates) were MB samples. The vendor-provided control samples indicated to
the technician whether the TaqMan® E. coli 0157:H7 Detection System was functioning
properly. If the controls failed for any reason, that batch of samples would be discarded and the
extracts re-analyzed. To the extent practicable, the test samples were analyzed blindly by having
the technician label the vials with only a sample number prior to the DNA purification step, such
8
-------
that the samples were tracked through the purification, PCR, and detection steps by only a
sample number.
3.3 Reference Methods
3.3.1 Plate Enumeration
Plate enumeration was used to quantify E. coli to confirm the concentration of the aliquoted
stock solution of this contaminant. The Battelle standard operating procedure (SOP) followed
was SOP No. MREF X-054, Standard Operating Procedure (SOP) for the Enumeration of
BSL-2 and BSL-3 Bacteria Samples Via the Spread Plate Technique.
Prior to testing, the E. coli were grown and then suspended in phosphate buffered saline (PBS).
Twenty-five or more individual 1 mL aliquots of stock solution were prepared from the original
PBS stock solution. Three 1 mL aliquots were randomly taken for enumeration, while the others
were frozen for later use in sample preparation. The enumerations were done on each of the three
selected 1 mL aliquot to confirm the determined concentration.
3.3.2 Drinking VV
Because E. coli can occur naturally in water, and because rapid PCR technologies cannot
distinguish between live and dead organisms, each unspiked concentrated DW sample was plate
enumerated to verify, to the extent practicable, the presence or absence of the contaminant of
interest. The samples were plated onto tryptic soy agar plates with 5% sheep blood and
incubated at 30 to 35°C. After 20 hours of incubation, the unspiked OH, CA, and NY DW
samples produced lawns of bacteria with a level of contamination estimated to be greater than
lxlO3 cfu/mL. The unspiked FL DW sample showed only 10 to 100 cfu/mL estimated
concentration levels after 20 hours. After further incubation, the FL DW sample produced
bacteria at a concentration estimated to be greater than lxlO3 cfu/mL. Each DW sample had at
least three distinct types of bacteria growing. Gram stains were performed on any distinct colony
types visible in each sample to gain further insight into the colony morphology. For OH and CA
DW, three Gram negative bacteria colonies were identified. For NY, four Gram negative
colonies were identified; and, for FL, both Gram negative and positive colonies were present.
Because no confirmed positive responses for E. coli were detected in any of the unspiked DW
samples, further identification tests were not conducted.
3.4 Test Procedure
3.4.1 Sample Handling
All testing for this verification test was conducted within Battelle laboratories staffed with
technicians trained to safely handle E. coli bacteria. The technician operating the TaqMan®
E. coli 0157:H7 Detection System had prior PCR experience. E. coli samples were tested in a
9
-------
Biosafety Level 2 (BSL-2) laboratory. Appropriate safety guidelines associated with the
laboratory were followed throughout the verification test. Each day, fresh samples were prepared
in either DI water, an interferent matrix, or a DW matrix from a thawed aliquot of frozen stock
solution. Concentration levels for spiked samples from the original test/QA plan design at
various multiples of the TaqMan® E. coli 0157:H7 Detection System's LOD (2, 5, 10, and 50
times the system LOD for PT samples) were calculated from the system LOD provided by the
vendor. Sample solutions were prepared to these concentrations, and other concentrations used
in the verification test, based on the concentration of the bacteria stock solution, which was
determined through triplicate plate enumeration prior to testing. Each sample was prepared in its
own container and labeled only with a sample identification (ID) number that also was recorded
in a laboratory record book along with details of the sample preparation. Samples were diluted to
the appropriate concentration using volumetric pipettes and glassware. Each sample was
prepared in 1 mL quantities. The entire 1 mL quantity was taken through the DNA extraction
and purification step. For the dilution series contaminant-only PT samples used in the expanded
testing discussed in Section 3.2.1, 900 microliters (|_iL) of each sample from lxlO6 cfu/mL to
100 cfu/mL were used to accommodate the processing of the order of magnitude dilution
samples and ensure that each sample was within the series.
Despite rigorous sample preparation efforts, solutions consisting of low bacterial concentrations,
such as the E. coli infective/lethal dose, may have no DNA present in a given sample or
aliquot.(3'4) The rationale for this is based on the Poisson statistical distribution, where there is
some probability that a sample taken will contain no particles (i.e., bacteria or target DNA) and
thus yield a negative result.(3'4) As a practical example, assume that 1 mL contains exactly five
particles (i.e., bacteria or target DNA) of interest. If one takes ten 0.1 mL samples and analyzes
them, the maximum number of positives will be five out of the ten samples. From this it follows
that there will be at least five negatives. Random variation in the sampling will cause this ratio
to change. This verification test was not designed to differentiate between the stochastic nature
of the low concentration samples and the capabilities of the assays, but this phenomenon should
be noted.
3.4.2 Sample Preparation and Analysis
Three steps were carried out to test a liquid sample for the presence of E. coli bacteria: (1) DNA
extraction using the PrepMan™ Ultra Sample Preparation Reagent, (2) PCR setup using the
TaqMan® E. coli 0157:H7 Detection Kit, and (3) PCR and analysis using the ABI Prism® 7000
Sequence Detection System and software. To perform these steps, the laboratory work area was
separated into three distinct areas: DNA extraction and plating the purified DNA onto the
96-well plate was done in one area; preparing the Pathogen PCR Cocktail and putting it on the
96-well plate was performed in a separate, DNA-free area; and loading the instrument (the ABI
Prism® 7000) was done in another area. These steps are described below.
First, the DNA was isolated and purified from the sample. The entire 1 mL sample was taken
through this isolation procedure. The PrepMan™ Ultra Sample Preparation Reagent general
extraction procedures for pathogens from food samples were followed. This procedure calls for a
pre-enrichment step that was not followed because the samples used in this test were already
10
-------
prepared to be at levels specified by the vendor (or determined through additional testing) to be
detectable. After the extraction step was complete, the PCR reaction setup using the TaqMan®
E. coli 0157:H7 Detection Kit was performed in the DNA-free area. This process involved first
making the Pathogen PCR Cocktail, which consists of the TaqMan E. coli 0157:H7 mix and the
AmpliTaq Gold® DNA Polymerase fas provided in the TaqMan® E. coli 0157:H7 Detection
Kit). Appropriate quantities of the PCR Cocktail were prepared for each day of testing as it
should be made within one hour of use. A volume of 45 \iL of PCR Cocktail mix was added to
the wells on a 96-well plate to be used for samples, positive controls, and NTCs. A volume of
44 \xL of TaqMan E. coli 0157:H7 mix only was added to the six NAC wells. A volume of 5 |iL
of Negative Control was then added to the NTC wells, and the wells were capped with optical
caps. The remaining wells were lightly capped and moved to another area.
In a separate area, 5 \iL of purified DNA were added to the appropriate sample wells and then
those wells were capped. A volume of 5 \iL of Positive Control was added to the two positive
control wells and 5 \iL of Negative Control were added to each NAC well. Once all wells were
filled and capped, the 96-well PCR plate was spun down to ensure that everything was in
solution and to remove any bubbles from the bottom of the well. The plate was then loaded onto
the ABI Prism® 7000 Sequence Detection System per the instructions provided in the TaqMan®
Pathogen Detection Kits User Manual with the following vendor-made modifications: an
Absolute Quantitation Assay (a real-time assay) instead of a Plus/Minus Assay (an endpoint
assay) was used, a VIC detector (fluorescent reporter dye) was used for the IPC in the Sequence
Detection System software (version 1.0), and no pre-read or post-read procedures were
conducted. A FAM detector (fluorescent reporter dye) was used for the sample wells. After the
instrument had completed its PCR program run, which consisted of 40 cycles on the thermal
cycler, the results were analyzed using the Sequence Detection System 7000 software. The
resulting amplification plots (plots of relative fluorescence units versus the thermal cycler cycle
number) were used to determine the results for each sample. Whether the sample was positive or
negative was based on the sample fluorescence crossing a set threshold. A default threshold of
0.2 ARn (relative unit of fluorescence proportional to the amplification of the target template)
was set by the software for each PCR run. The threshold is defined as a point above the mean
background fluorescence signal present in all wells on the plate for a particular PCR run. As
needed, this threshold value can be adjusted by the operator to be more specific to the data being
analyzed. The adjustment of the threshold is based on the operator's experience with and
understanding of PCR amplification plots. The threshold was set based on the location of the
exponential phase data (amplification curves) to fall within the linear portion of the plots and
generally be set above the noise in the amplification plot. For this test, the default threshold
value of 0.2 ARn was appropriate and was used with the FAM detector for all sample wells. Any
amplification for a given sample above this threshold indicated a positive result (E. coli is
present in the sample) and was assigned a Ct value or threshold cycle value by the software. The
Ct value is the fractional cycle number where the exponential phase fluorescence or
amplification curve crosses the threshold. When the amplification did not cross the threshold,
the sample was considered negative (no E. coli detected in the sample) and no Ct value was
assigned by the software. The negative controls were considered successful if no amplification
on the FAM detector was present. The positive controls were considered successful if
amplification was noted in the FAM detector. The IPC was monitored using the VIC detector
11
-------
screen to ensure that amplification of the IPC was present in each sample. As with the FAM
detector, a default threshold of 0.2 AR^ was set by the software for the IPC determination. This
threshold was incorrect for the IPC amplification results and was set to 0.08 ARn in all cases for
the determination of IPC amplification. The technician recorded the sample identification
number on a sample data sheet along with the qualitative results (positive or negative) for each
sample.
3.4.3 Drinking Water Characterization
An aliquot of each DW sample, collected as described in Section 3.2.2, was sent to ATEL prior
to concentration to determine the following water quality parameters: turbidity; concentration of
dissolved and total organic carbon; conductivity; alkalinity; pH; concentration of Ca and Mg;
hardness; and concentration of total organic halides, trihalomethanes, and haloacetic acids.
Table 3-4 lists the methods used to characterize the DW samples, as well as the characterization
data from the four water samples used in this verification test. Water samples were collected and
water quality parameters were measured by ATEL in January 2004. Some of the water quality
parameters may have changed slightly prior to verification testing.
Table 3-4. ATEL Water Quality Characterization of Drinking Water Samples
Sources of Drinking Water Samples
Parameter
Unit
Method
Columbus,
Ohio
(OH DW)
MWD,
California
(CA DW)
Orlando,
Florida
(FL DW)
New York
City, New
York
(NY DW)
Turbidity
NTU
EPA ISO. 1(5)
0.2
0.1
0.5
1.3
Dissolved organic
mg/L
SM 5310(6)
1.9
2.3
1.7
1.5
carbon
Total organic
mg/L
SM 5310(6)
1.6
2.1
1.8
2.1
carbon
Specific
microSiemens
SM 2510(6)
357
740
325
85
conductivity
Alkalinity
mg/L
SM 2320(6)
55
90
124
4
pH
EPA 150.1™
7.33
7.91
7.93
6.80
Ca
mg/L
EPA 200.8(8)
42
35
41
5.7
Mg
mg/L
EPA 200.8(8)
5.9
1.5
8.4
19
Hardness
mg/L
EPA 130.2™
125
161
137
28
Total organic
M-g/L
SM 5320(6)
360
370
370
310
halides
Trihalomethanes
|ig/L/analyte
EPA 524.2(9)
26.9
79.7
80.9
38.4
Haloacetic acids
[ig/L/analyte
EPA 552.2(10)
23.2
17.6
41.1
40.3
NTU = nephelometric turbidity unit
|ig = microgram
12
-------
Chapter 4
Quality Assurance/Quality Control
Quality assurance/quality control procedures were performed in accordance with the quality
management plan (QMP) for the AMS Center(11) and the test/QA plan for this verification test.(1)
4.1 Sample Chain-of Custody Procedures
Sample custody was documented throughout collection, shipping, and analysis of the samples.
Sample chain-of-custody procedures were generally those provided in the guidelines in
ASAT.II-007, Standard Operating Procedure for Chain of Custody for Dioxin/Furan Analysis.
The chain-of-custody forms summarized the samples collected and analyses requested and were
signed by the person relinquishing samples once that person had verified that the custody forms
were accurate. The original sample custody forms accompanied the samples; the shipper kept a
copy. Upon receipt at the sample destination, sample custody forms were signed by the person
receiving the samples once that person had verified that all samples identified on the custody
forms were present in the shipping container.
4.2 Equipment Calibration
The TaqMan® E. coli 0157:H7 Detection System, and all associated reagents and supplies
specific for the detection of E. coli were provided to Battelle by the vendor. This system
required no calibration. The performance of the system was monitored through positive controls,
IPC, and negative controls. For DW characterization and confirmation of the possible
interferent, analytical equipment was calibrated by ATEL according to the procedures specified
in the appropriate standard methods. Pipettes used during the verification test were calibrated
according to Battelle SOP VI-025, Operation, Calibration, and Maintaining Fixed and
Adjustable Volume Pipettes.
4.3 Characterization of Contaminant Stock Solution
E. coli was grown and prepared by Battelle. The bacteria were plate enumerated in triplicate for
confirmation of the concentration of the 1 mL aliquots of stock solution. The Battelle SOP No.
MREF X-054, Standard Operating Procedure (SOP) for the Enumeration of BSL-2 and BSL-3
Bacteria Samples Via the Spread Plate Technique were followed for the plate enumerations of
13
-------
E. coli. The results of the plate enumerations are presented in Table 4-1. The plate enumeration
was conducted prior to testing. The average of triplicate enumerations was used to calculate and
prepare all spiked sample solutions.
Table 4-1. E. coli Triplicate Plate Enumeration Data
Plate 1
Plate 2
Plate 3
Relative
Concentration
Concentration
Concentration
Average
Standard
Bacteria
(cfu/mL)
(cfu/mL)
(cfu/mL)
(cfu/mL)
Deviation
E. coli
7.3xl08
5.0xl08
9.0xl08
7.1xl08
28%
4.4 Quality Control Samples
MB samples consisting of ASTM Type II DI water, and positive and negative control (NTC and
NAC) samples, as provided in the TaqMan® E. coli 0157:H7 Detection System, were analyzed
to help identify potential cross-contamination issues as well as verify that the PCR process was
functioning properly. Positive control reaction samples as prepared by the Battelle technician
were also analyzed as part of each day of testing. Positive control reaction samples were DI
water samples spiked with E. coli at lxl06cfu/mL and put through the same analysis procedures
(DNA extraction, PCR setup, etc.) as regular samples. These positive control reaction samples
were also handled blindly, as the regular test samples were. IPCs were part of each sample that
was analyzed and provided further checks on the performance of the system, especially in
identifying the presence of potential inhibitory substances. Two positive control samples,
12 negative control samples (six NTC and six NAC), and four positive control reaction samples
were run with each set of samples placed on the 96-well plate. Eight MB replicates were
analyzed over the course of the verification test.
All eight MB replicates for this verification test returned negative results. All positive control
reaction samples returned positive results. A threshold of approximately 0.8 ARn was chosen for
the IPC samples based on the amplification plots. IPC peaks were present in all samples except
for one positive control, one positive control reaction replicate, and one unspiked NY DW
replicate. The suppression of the IPC in the unspiked NY DW likely indicates the presence of
inhibitors in the sample. For the positive control samples, all other IPCs within the sample set
produced clear amplification, including all other positive controls for that plate. No IPC
amplification was present in any NAC samples, as expected. No positive or negative controls
failed.
14
-------
4.5 Audits
4.5.1 Technical Systems Audit
The Battelle Quality Manager conducted a technical systems audit (TSA) on June 23, 2004, to
ensure that the verification test was performed in accordance with the test/QA plan(1) and the
AMS Center QMP.(11) As part of the audit, the Battelle Quality Manager reviewed the standards
and methods used, compared actual test procedures to those specified in the test/QA plan, and
reviewed data acquisition and handling procedures. Observations and findings from this audit
were documented and submitted to the Verification Test Coordinator for response. No findings
were documented that required any significant action. The records concerning the TSA are
stored for at least seven years with the Battelle Quality Manager.
4.5.2 Audit of Data Quality
At least 10% of the data acquired during the verification test was audited. Battelle's Quality
Manager traced the data from the initial acquisition, through reduction and statistical analysis,
to final reporting, to ensure the integrity of the reported results. All calculations performed on
the data undergoing the audit were checked.
4.6 QA/QC Reporting
Each assessment and audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the
QMP for the ETV AMS Center/11* Once the assessment report was prepared, the Verification
Test Coordinator ensured that a response was provided for each adverse finding or potential
problem and implemented any necessary follow-up corrective action. The Battelle Quality
Manager ensured that follow-up corrective action was taken. The results of the TSA were sent to
the EPA.
4.7 Data Review
Records generated in the verification test were reviewed before these records were used to
calculate, evaluate, or report verification results. Table 4-2 summarizes the types of data
recorded. The review was performed by a Battelle technical staff member involved in the
verification test, but not the staff member that originally generated the record. The person
performing the review added his/her initials and the date to a hard copy of the record being
reviewed.
15
-------
Table 4-2. Data Recording Process
Data to Be
Recorded
Where Recorded
How Often Recorded
Disposition of Data(a)
Dates and times of
test events
ETV data sheets
Start/end of test and at
each change of a test
parameter
Used to organize/check test
results; manually incorporated
in data spreadsheets as
necessary
Sample collection
and preparation
information,
including chain-of-
custody
ETV data sheets
and chain-of-
custody forms
At time of sample
collection and
preparation
Used to organize/check test
results; manually incorporated
in data spreadsheets as
necessary
TaqMan® E. coli
0157:H7 Detection
System procedures
and sample results
ETV data sheets
and data
acquisition system
Throughout test
duration
Manually incorporated in data
spreadsheets
Enumeration data
Enumeration data
forms and ETV
data sheets
With every
enumeration
Used to organize/check test
results
Reference method
procedures and
sample results
Data acquisition
system, as
appropriate
Throughout sample
analysis process
Transferred to spreadsheets
(a) All activities subsequent to data recording were carried out by Battelle, except for the reference method analyses
(DW characterization), which were carried out by ATEL.
16
-------
Chapter 5
Data Analysis
The TaqMan® E. coli 0157:H7 Detection System was evaluated for qualitative results (i.e.,
positive/negative responses to samples) based on the expected application of rapid PCR
technologies as rapid screening tools. All data analyses were based on these qualitative results.
QC and MB samples were not included in any of the analyses.
5.1 Accuracy
Accuracy was assessed by evaluating how often the TaqMan® E. coli 0157:H7 Detection
System results were positive in the presence of a concentration of contaminant above the system
LOD. Contaminant-only PT samples were used for this analysis. An overall percent agreement
was determined by dividing the number of positive responses by the overall number of analyses
of contaminant-only PT samples above the system LOD.
5.2 Specificity
The ability of the TaqMan® E. coli 0157:H7 Detection System to provide a negative response
when the contaminant was absent was assessed. The specificity rate was determined by dividing
the number of negative responses by the total number of unspiked samples.
5.3 False Positive/Negative Responses
A false positive response was defined as a detectable or positive TaqMan® E. coli 0157:H7
Detection System response when the ASTM Type IIDI water (including interferent samples) or
DW samples were not spiked. A false positive rate was reported as the frequency of false
positive results out of the total number of unspiked samples.
A false negative response was defined as a non-detectable response or negative response when
the sample was spiked with a contaminant at a concentration greater than the system LOD.
Spiked PT (contaminant and interferent) samples and spiked DW samples were included in the
analysis. A false negative rate was evaluated as the frequency of false negative results out of the
total number of spiked samples for a particular contaminant.
17
-------
5.4 Precision
The precision of the four replicates of each sample set were assessed. Responses were
considered consistent if all four replicates gave the same result. The precision of the TaqMan® E.
coli 0157:H7 Detection System was assessed by calculating the overall number of consistent
responses for all the sample sets.
5.5 Interferences
The potential effect of the DW matrix on the TaqMan® E. coli 0157:H7 Detection System
performance was evaluated qualitatively by comparing the results for the spiked and unspiked
DW samples to those for the PT samples. Similarly, the potential effect of interferent PT samples
containing humic and fulvic acids at two levels, both spiked and not spiked with bacteria, were
evaluated.
5.6 Other Performance Factors
Aspects of the TaqMan® E. coli 0157:H7 Detection System performance such as ease of use and
sample throughput are discussed in Section 6. Also addressed are qualitative observations of the
technician pertaining to the performance of the TaqMan® E. coli 0157:H7 Detection System.
18
-------
Chapter 6
Test Results
The results for the TaqMan® E. coli 0157:H7 Detection System were evaluated based on the
responses provided by the Sequence Detection System software Absolute Quantitation Assay
amplification plot output. An example amplification plot for E. coli is presented in Figure 6-1.
The plot displays the change in relative fluorescence versus the thermal cycler cycle number,
which is relative to time. In this example, all of the samples are displayed at once and assigned a
different color. Using the Sequence Detection System 7000 software, the number of samples
displayed on the amplification plot can be controlled by the operator. Only qualitative
(positive/negative) responses were recorded for each sample. To determine the results of each
sample, the threshold cycle or Ct value for each curve on the amplification plot was monitored
for the FAM detector. To do this, a threshold had to be defined. This threshold is meant to help
the user distinguish between noise and actual DNA amplification. A default threshold is set by
the software at 0.2 AR^. Based on the amplification plots for the samples from this test, the
default value was determined to be appropriate and was used as the threshold when analyzing all
samples. Amplification curves that crossed this threshold were assigned a Ct value by the
software at the fractional cycle at which the amplification curve crossed the threshold. E. coli
was considered present in the sample if the amplification curve crossed the threshold and
showed actual amplification (i.e., the crossing of the threshold was real amplification and not
simply a spike in the curve). The software reports an "Undetected" in place of a Ct value when
amplification does not cross the defined threshold, meaning that no amplification occurred
within the 40 cycles of the PCR. The E. coli was considered not present, and thus a negative
response was recorded, if no amplification was apparent for the FAM detector for that sample
and the curve did not cross the threshold. For the purposes of this test, amplification curves that
crossed the threshold AR^ value were considered positive, regardless of the cycle number at
which they crossed. In some cases, many Ct values were quite high (i.e., close to 40 cycles).
Often, such high Ct values, even those as low as 37 cycles, might lead to suspect amplification
and might not necessarily be considered by some to be a positive sample without further
analyses. Often, comparison to the results of the other replicate samples can help in determining
if the amplification is real or not. Positive controls and negative controls were monitored with
each day's sample set. No controls were unsuccessful throughout the testing process.
19
-------
Delia Rn vs Cycle
1 0e+001
ok
V
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 30 37 38 39 40
Cycle Number
Figure 6-1. ARn\s. Cycle Number ABI Prism® 7000 Sequence Detection System Amplification Plot for E. coli Samples and
Controls
-------
6.1 Accuracy
The results for the TaqMan® E. coli 0157:H7 Detection System using the contaminant-only PT
samples containing E. coli are discussed in this section. The infective/lethal dose samples are
included in the contaminant-only PT samples. The infective dose for E. coli (see Table 3-1) was
below the vendor-stated system LOD. Although the results for E. coli at the infective/lethal dose
are presented in Table 6-1, they are not included in the overall accuracy calculations.
The results obtained for the PT samples containing E. coli are given in Table 6-1. The first five
concentration levels listed reflect the original test/QA plan samples (infective/lethal dose, 2, 5,
10, and 50 times the system LOD) that were initially analyzed. The results indicated large incon-
sistencies among replicates in samples that were at levels above the system LOD originally
supplied by the vendor. In most instances where a positive result was obtained, the amplification
curve crossed the threshold at a very high cycle number (usually around 38 or 39) and barely
crossed above the threshold line on the amplification plot. Only the 500 cfu/mL sample gave
consistent positive results (4/4) for all replicates. Because the positive control reaction replicates
(lxlO6 cfu/mL) were all positive, the positive and negative controls did not fail, and the IPC
curves in the VIC detector appeared to amplify correctly, operator error or failure of the TaqMan®
E. coli 0157:H7 Detection System to function properly were not suspected. After discussions
with Applied Biosystems, a possible explanation for these results was considered. The system
LOD originally provided by Applied Biosystems was one that was known for E. coli in food
samples. Because the TaqMan® E. coli 0157:H7 Detection System had not been tested
extensively with water samples, it was possible that the LOD for food was not appropriate for this
test with water matrices. Given this, and that the concentrations of the original set of PT samples
were low-bacterial concentrations, the vendor suggested that Poisson distribution effects were
playing a part in the results for these samples.
Additional testing beyond that described in the test/QA plan was performed to gain more
information about the performance of the TaqMan® E. coli 0157:H7 Detection System. For the
purposes of this verification test, a system LOD is anticipated to be the level at which
quantifiably reproducible positive results are obtained. It represented more of a method detection
limit than an instrument detection limit. Results from the additional testing were meant to help
estimate the system LOD per this definition. Results from this expanded testing were also used to
determine the appropriate level to spike the interferent PT and DW samples. To do this, a dilution
series of contaminant-only PT samples were tested from lxlO6 cfu/mL to 10 cfu/mL, with each
solution being 10 times less than the one before (see Table 6-1). A concentration of lxlO6 cfu/mL
was chosen as the highest level analyzed because it is the same concentration as the positive
control reaction sample, which was consistently positive when analyzed with the original set of
test/QA plan PT samples. A concentration of 10 cfu/mL was chosen as the lowest level in the
dilution series because it was the original system LOD provided by the vendor. Four positive
results out of four replicates were found for the samples containing lxlO6, 1x10s, lxlO4, and lxlO3
cfu/mL of E. coli. All of the replicates from each sample showed clear and strong amplification.
Three out of four samples were positive for the 100 cfu/mL replicates, and two out of four
samples were positive for the 10 cfu/mL replicates. The crossing point for the positive samples at
the lowest dilutions were around 39 cycles and were generally only slightly above the threshold.
21
-------
Clearly the TaqMan® E. coli 0157:H7 Detection System is capable of detecting E. coli in DI
water at 10 cfu/mL, but consistent results are not obtained below samples at 500 cfu/mL. Based
on these results, a system LOD (as defined above) of 500 cfu/mL was used for all subsequent
calculations concerning samples above the system LOD. An overall accuracy was calculated
using the replicate results from the 500 cfu/mL, lxl03 cfu/mL, and lxl04 cfu/mL samples to best
replicate the contaminant-only PT samples defined in the test/QA plan (2, 5, 10, and 50 times the
system LOD). Additional test samples above lxl04 cfu/mL were considered outside of the original
sample range and not included in any calculations.
Table 6-1. E. coli Contaminant-Only PT Sample Results
Sample Type
Concentration^
(cfu/mL)
Positive Results Out of
Total Replicates
0.2(b)
0/4
Original test/QA plan
PT samples
20
50
100
1/4
2/4
2/4
500(d)
4/4
lxlO6
4/4
lxlO5
4/4
Additional dilution series PT
samples(c)
lxl04(d)
lxl O3^
4/4
4/4
lxlO2 3/4
lxlO1 2/4
Overall Accuracy 100% (12/12)(d)
-------
6.2 Specificity
Specificity assesses the TaqMan® E. coli 0157:H7 Detection System's ability to provide a
negative response when the contaminant was absent. The results from all unspiked interferent PT
samples and unspiked DW samples are presented in this section. Negative results out of total
replicates are presented in each table.
The results obtained for the unspiked samples are given in Table 6-2. For the unspiked interferent
PT samples, all of the 2.5 mg/L humic and fulvic acid replicates showed all negative responses.
One of the four 0.5 mg/L humic and fulvic acid replicates showed a positive response. This
replicate had a Ct value of 38.88. OH, CA, FL, and NY unspiked DW samples showed all negative
responses also, indicating that the bacteria were not present in these samples, as would be
expected.
An overall specificity rate was determined by dividing the number of negative responses to the
overall number of analyses of unspiked samples. This resulted in 96% agreement for the overall
specificity of the TaqMan® E. coli 0157:H7 Detection System.
Table 6-2. E. coli Specificity Results
Negative Results Out of
Sample Type
Sample
Total Replicates
0.5 mg/L humic acid and
0.5 mg/L fulvic acid,
3/4
Interferent PT Samples
unspiked
2.5 mg/L humic acid and
2.5 mg/L fulvic acid,
4/4
unspiked
OH DW, unspiked
4/4
DW Samples
CA DW, unspiked
FL DW, unspiked
4/4
4/4
NY DW, unspiked
4/4
Overall Specificity
96% (23/24)
6.3 False Positive/Negative Responses
Contaminant-only PT samples (as determined with additional testing, see Section 6.1 above),
interferent PT samples, and DW samples were evaluated to determine false positive and false
negative results for the TaqMan® E. coli 0157:H7 Detection System. Included in the calculations
were the 16 additional interferent samples (0.5 mg/L humic and fulvic acids) tested to determine
23
-------
the effects of the DNA extraction and isolation on the results. A false positive response was
defined as a positive result when bacteria were not spiked into the sample. A false negative
response was defined as a negative result when the sample was spiked with a contaminant at a
concentration greater than the TaqMan® E. coli 0157:H7 Detection System LOD for that bacteria.
The contaminant-only PT samples at 10 times the vendor-supplied system LOD showed
inconsistent results among the replicates. For the interferent and DW samples, a level that showed
consistent positive results in spiked DI water needed to be evaluated to test the ability of the
TaqMan® E. coli 0157:H7 Detection System to perform in the presence of inhibitors. Since the DI
water samples at 10 times the vendor-provided system LOD were inconsistent, results from any
interferent or DW samples spiked at this level that might be attributable to inhibitors present in
these samples could not be distinguished from the normal performance of the system at that
concentration level. Based on the results of the additional dilution series contaminant-only PT
samples that were tested, lxlO4 cfu/mL was chosen as an appropriate spiking level for the
interferent and DW samples. This concentration provided consistent positive results for all
replicates. Furthermore, because 500 cfu/mL was the lowest level tested to provide consistent
positive results, and given that data were available for the performance of the system at lxlO4
cfu/mL, lxlO4 cfu/mL served as a good surrogate for the 10 times the system LOD called for in
the test/QA plan, assuming that the system LOD in this case was one that provided consistent
positive responses for spiked samples.
It should be noted that false positive responses cannot be absolutely confirmed as false because
there is a possibility of cross-contamination. All appropriate steps were taken throughout the
verification test to avoid this issue, such as using three work areas and following daily clean-up
procedures. However, cross-contamination is always a possibility in any PCR process.(12) No
appropriate reference method was available to cross-check the amplified PCR product to confirm
the TaqMan® E. coli 0157:H7 Detection System responses.
Table 6-3 presents the results for the E. coli samples. The number of positive samples out of the
total replicates analyzed is presented in the table. No false negative samples were found in any of
the sample matrices. One replicate for unspiked 0.5 mg/L humic acid and 0.5 mg/L fulvic acid did
show a positive result for the presence of E. coli. The crossing point for this replicate was 38.88
cycles (out of 40 total).
24
-------
Table 6-3. E. coli False Positive/Negative Results
Concentration(a)
Positive Results Out of
Sample Type
Sample
(cfu/mL)
Total Replicates
DI water
500
4/4
Contaminant-Only
PT Samples
DI water
lxlO3
4/4
DI water
lxlO4
4/4
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
Blank
1/4
Interferent PT
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
lxlO4
20/20
Samples
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
Blank
0/4
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
lxlO4
4/4
OH DW
Blank
0/4
OH DW
lxlO4
4/4
CA DW
Blank
0/4
DW Samples
CA DW
FLDW
lxlO4
Blank
4/4
0/4
FLDW
lxlO4
4/4
NY DW
Blank
0/4
NY DW
lxlO4
4/4
False Positive Rate
1/24
False Negative Rate
0/52
(a) Sample solutions were prepared at these levels from stock solutions based on the enumeration data (see Table 4-1).
25
-------
6.4 Precision
The performance of the TaqMan® E. coli 0157:H7 Detection System within sample sets of four
replicates was consistent for the DW and interferent PT samples. Only one set of replicates, that
for unspiked 0.5 mg/L humic and fulvic acids, was inconsistent, with one of the replicates
showing a positive result while the other samples were negative. All other DW and interferent PT
samples showed the same results within a set of replicates.
For the contaminant-only PT samples, all samples at or above 500 cfu/mL showed consistent
results within each set or replicates. For those samples below 500 cfu/mL, the results for sample
sets were inconsistent, with mixed positive and negative results for all five sets of replicates. The
one exception to the inconsistent results for sample sets below 500 cfu/mL was the sample set for
E. coli at the infective/lethal dose, which showed negative results for all replicates. Out of the
27 sample sets analyzed, six were inconsistent, resulting in 78% (21/27) of the sample sets
showing consistent results.
6.5 Interferences
6.5.1 Interferent PT Samples
In both the 0.5 mg/L and 2.5 mg/L humic and fulvic acid solutions spiked with E. coli and the
2.5 mg/L unspiked sample, the TaqMan® E. coli 0157:H7 Detection System provided expected
results. In the absence of the bacteria, the samples tested negative; in the presence of the bacteria,
the samples tested positive. For the 0.5 mg/L unspiked humic and fulvic acid sample, one of the
four replicates tested positive for the presence of E. coli. In all cases for the interferent PT
samples, the IPC was present and showed clear amplification in the amplification plot for the VIC
detector. This would indicate that the humic and fulvic acids were not acting as inhibitory
substances for the TaqMan® system.
As discussed in section 3.2.1, four solutions of humic and fulvic acids each at 0.5 mg/L spiked
with E. coli at lxl04cfu/mL were prepared in addition to the initial 0.5 mg/L and 2.5 mg/L humic
and fulvic acid solutions. Each solution was put through the DNA extraction and isolation
procedure, and then four replicates from each of the four purified DNA solutions were analyzed
using the TaqMan® E. coli 0157:H7 Detection System. These samples were included in the
verification test in an effort to evaluate the efficacy of the DNA extraction and isolation procedure
in the presence of inhibitory substances. These samples also contribute to the precision
evaluations of the TaqMan® E. coli 0157:H7 Detection System. All of the samples tested resulted
in positive responses. Thus, 20 out of the 20 spiked 0.5 mg/L each humic and fulvic acid samples
tested resulted in positive responses. No IPC suppression was apparent in any of these samples.
26
-------
6.5.2 Drinking Water Samples
The TaqMan® E. coli 0157:H7 Detection System DW sample results are presented in Table 6-3.
The TaqMan® E. coli 0157:H7 Detection System showed positive results for each set of replicates
for the spiked DW samples and negative results for each set of replicates for the unspiked samples.
IPC suppression was unapparent in all but one of the DW samples, indicating that the DW
matrices were generally not acting as inhibitors for the TaqMan® system. Only one replicate for
unspiked NY DW did not have a Ct value for the IPC. All PT samples spiked with lxlO4 cfu/mL of
E. coli showed consistent positive responses and no IPC suppression.
6.6 Other Performance Factors
The TaqMan® E. coli 0157:H7 Detection System was operated by the same Battelle technician
throughout the verification test. This technician had prior PCR experience as well as prior ABI
Prism® 7000 Sequence Detection System experience and was trained by Applied Biosystems in
the operation of the TaqMan® E. coli 0157:H7 Detection System before testing began. The
Battelle technician was already familiar with general DNA extraction and isolation techniques,
PCR 96-well plating techniques, and ABI Prism® 7000 Sequence Detection System operation, as
well as general PCR theory, prior to training. The overall operation of the TaqMan® E. coli
0157:H7 Detection System was straightforward, and the experienced technician found the system
easy to use with slight difficulties in data interpretation. The PrepMan™ Ultra Sample Preparation
Reagent procedure was short and simple, requiring very little reagent. The PCR setup was also
straightforward. The aseptic procedure while handling the PCR Cocktail and adding the sample
DNA is essential to ensure reliable sample results. The TaqMan® E. coli 0157:H7 Detection Kit
required very little reagent to make the PCR Cocktail and was straightforward. The amount of
sample DNA needed to run the PCR was also small. Loading the 96-well plate into the ABI
Prism® 7000 Sequence Detection System and starting the instrument were very straightforward.
Setting up the real-time PCR run and conducting data analysis, however, were a little more
complicated and would require either prior experience or training to do properly. The user's
manual is straightforward in helping to set the thermal cycler conditions for a run, but it might be
difficult to optimize an assay without further training on the instrument and software. Data
analysis using the Absolute Quantitation Assay can be somewhat complicated and relies on the
operator to make appropriate choices. Though the software produced the amplification plots and
gave Ct values for all amplification curves above the threshold, it was up to the technician to
determine the appropriate threshold and then further determine if the sample was positive or
negative, even if the curve crossed the threshold. An understanding of the software, what the Ct
value represents, and a general understanding of the amplification plots were necessary for the
data interpretations. In some instances, even the experienced technician had trouble determining if
a sample was positive and showing true amplification when the sample had spikes at the last few
temperature cycles. Overall, the Sequence Detection System software was easy to navigate
through, and the amplification plot analysis section controls were user friendly.
All testing was performed in a laboratory setting because the TaqMan® E. coli 0157:H7 Detection
System is not field portable. Three distinct and separate testing areas were required in each
27
-------
laboratory to operate the TaqMan® E. coli 0157:H7 Detection System. The PrepMan™ Ultra
Sample Preparation Reagent components were stable at room temperature. The TaqMan® E. coli
0157:H7 Detection Kit had to be stored at -15 to -25°C and thawed before use. The Pathogen
PCR Cocktail must be made within one hour of use and can be stored at room temperature until its
use. The PCR reactions were assembled at room temperature. The Prism® has a footprint of
20 inches by 15 inches and stands 21 inches high. It comes with a laptop.
E. coli samples were tested in a BSL-2 laboratory. Because live bacteria were being handled,
special safety requirements and protocols had to be implemented in the laboratory. Some of these
requirements may have impacted the analysis time for the TaqMan® E. coli 0157:H7 Detection
System and are inherently present in any throughput estimations for this verification test. Thus,
such performance factors mentioned here also incorporate the safety and facility requirements
necessary for this test.
More than 128 samples (including method blanks and positive control reactions) were tested for
E. coli using the TaqMan® E. coli 0157:H7 Detection System. The DNA extraction and isolation
step for between seven and nine sample solutions took 40 minutes to an hour. The PCR steps,
including the Pathogen PCR Cocktail set up, took 30 minutes to an hour, depending on the
number of samples. The loading of the ABI Prism® 7000 Sequence Detection System and the
completion of the thermal cycle program to amplify the sample DNA took a little over two hours.
For this study, the technician analyzed on average 28 to 36 replicate samples in a day. The plates
used in the instrument can hold up to 96 samples and controls.
For the purposes of this test, the Absolute Quantitation Assay was used to determine the results for
each sample. This assay is a real-time PCR assay, meaning that results can be monitored in real
time because the DNA present in each sample is amplified. Although not used in this test, this
assay can also be used in conjunction with standards of known concentration to determine the
concentration of the unknown samples being analyzed. The Sequence Detection System 7000
software also offers the use of a Plus/Minus Assay, which is an endpoint assay that identifies
samples as positive or negative. In this assay, the software sets the threshold for the for data
analysis using the IPC and negative control results and then assigns a positive (+), negative (-), or
undetermined (?) call to the sample based on this analysis. An undetermined or inconclusive result
is returned if neither the target nor the IPC is detected in the sample. A ARn value is also given for
each sample. Using this assay, no manual data interpretation is needed because the software sets
and uses consistent criteria to produce positive and negative results for each sample.
28
-------
Chapter 7
Performance Summary
Table 7-1. E. coli Summary Table
Parameter
Sample Information
Number Detected/
Concentration Number of Samples
Contaminant-
Original test/QA plan
samples— DI water
0.2 cfu/mL(a)
20 cfu/mL
50 cfu/mL
100 cfu/mL
500 cfu/mL
0/4
1/4
2/4
2/4
4/4
Qualitative
results
only PT
samples
Additional testing
samples— DI water(b)
lxlO6 cfu/mL
lxlO5 cfu/mL
lxlO4 cfu/mL
lxlO3 cfu/mL
lxlO2 cfu/mL
lxlO1 cfu/mL
4/4
4/4
4/4
4/4
3/4
2/4
Interferent
PT samples
Humic and fulvic acids
lxlO4 cfu/mL
24/24
DW samples
Concentrated DW
lxlO4 cfu/mL
16/16
Accuracy
100% (12 out of 12) of the contaminant-only PT samples between (and
including) 500 cfu/mL and lxlO4 cfu/mL were positive.
Specificity
96% (23 out of 24) of the unspiked interferent and DW samples were
negative. One unspiked humic and fulvic acids replicate at 0.5/mg/L of each
acid returned a positive result.
False positives
One false positive resulted from the analysis of the unspiked 0.5 mg/L humic
acid and 0.5 mg/L fulvic acid. No false positives resulted for any other
interferent or DW samples.
False negatives
No false negative results were obtained from the analysis of the interferent
and DW samples spiked with detectable levels of E. coli.
Precision
78% (21 out of 27) of the sample sets showed consistent results among the
individual replicates within that set.
(a) Infective/lethal dose. Below the vendor-stated system LOD.
(b) Additional testing samples only contained 900 jaL of samples, except for 10 cfu/mL, which contained 1 mL of
sample.
29
-------
Other performance factors: A technician with prior PCR and ABI Prism® Sequence Detection
System experience operated the TaqMan® E. coli 0157:H7 Detection System at all times. The
PrepMan™ Ultra Sample Preparation Regent and TaqMan® E. coli 0157:H7 Detection Kit were
straightforward and easy to use. The ABI Prism® Sequence Detection System data analysis and
PCR setup were more complicated and required experience and understanding to properly use.
Three separate work areas were needed for testing. Small amounts of reagents were required for
DNA extraction and PCR setup and small amounts of sample DNA were needed for PCR.
Reagents for the various steps of the system had different storage requirements. Sample
throughput for this verification test was 28 to 36 replicate samples per day. The approximate
operational times were less than one hour for DNA extraction/purification, less than one hour for
PCR setup, and two hours for PCR. The cost is around $100 for the PrepMan™ Ultra Sample
Preparation Reagent (50 to 100 DNA extractions), $800 for the TaqMan® E. coli 0157:H7
Detection Kit (100 assays) and approximately $47,250 for the ABI Prism® Sequence Detection
System (20 inches x 21 inches xl5 inches, 75 pounds).
30
-------
Chapter 8
References
1. Test/QA Plan for Verification of Rapid PCR Technologies, Battelle, Columbus, Ohio,
May 2004.
2. Burrows, W. D.; Renner, S. E. "Biological Warfare Agents as Threats to Potable Water,"
Environmental Health Perspectives, 107, 975-984, 1999.
3. Stenman, J.; Orpana, A. "Accuracy in amplification," Nature Biotechnology, 19, 1011-
1012, 2001.
4. Hughes, J.; Totten, P. "Estimating the accuracy of polymerase chain reaction-based tests
using endpoint dilution," University of Washington Biostatistics Working Paper Series,
Working Paper 196, 2003.
5. U.S. EPA Method 180.1, "Turbidity (Nephelometric)" in Methods for the Determination
of Inorganic Substances in Environmental Samples, EPA/600/R-93-100, August 1993.
6. American Public Health Association, et al. Standard Methods for Examination of Water
and Wastewater. 19th Edition. 1997. Washington D.C.
7. U.S. EPA, Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79-020,
March 1983.
8. U.S. EPA Method 200.8, "Determination of Trace Elements in Waters and Wastes by
Inductively-Coupled Plasma Mass Spectrometry" in Methods for the Determination of
Organic Compounds in Drinking Water, Supplement I, EPA/600/R-94/111, October 1994.
9. U.S. EPA Method 524.2, "Permeable Organic Compounds by Capillary Column GC/Mass
Spectrometry" in Methods for the Determination of Organic Compounds in Drinking
Water, Supplement III, EPA/600/R-95-131. August 1995.
10. U.S. EPA Method 552.2, "Haloacetic Acids and Dalapon by Liquid-Liquid Extraction,
Derivatization and GC with Electron Capture Detector" in Methods for the Determination
of Organic Compounds in Drinking Water, Supplement III, EPA/600/R-95-131, August
1995.
31
-------
Quality Management Plan (QMP)for the ETV Advanced Monitoring Systems Center,
Version 5.0. EPA Environmental Technology Verification Program, prepared by Battelle,
Columbus, Ohio, March, 2004.
Shapiro, D. S. "Quality Control in Nucleic Acid Amplification Methods: Use of
Elementary Probability Theory," Journal of Clinical Microbiology, 37, 848-851, 1999.
32
------- |