THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
EX
RAM^ft
T
EPA Bairene
^^Hl M^ m T7 T7 • /T i-
TT£. T, . , , n , ,. . J/jj l_>nsincss of Innovation
U.S. h,nvironmental Froteetion Agency
ETV Joint Verification Statement
TECHNOLOGY TYPE: Rapid Polymerase Chain Reaction
APPLICATION: DETECTING BIOLOGICAL AGENTS AND
PATHOGENS IN WATER
TECHNOLOGY NAME: PathAlert™ Detection Kit
COMPANY: Invitrogen Corporation
ADDRESS: 7335 Executive Way PHONE: 240-379-4209
Frederick, Maryland 21704
WEB SITE: www.invitrogen.com
E-MAIL: willem.folkerts@invitrogen.com
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology Verification (ETV)
Program to facilitate the deployment of innovative or improved environmental technologies through performance
verification and dissemination of information. The goal of the ETV Program is to further environmental protection
by accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal
by providing high-quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies. Information and ETV
documents are available at www.epa.gov/etv.
ETV works in partnership with recognized standards and testing organizations, with stakeholder groups
(consisting of buyers, vendor organizations, and permitters), and with individual technology developers. The
program evaluates the performance of innovative technologies by developing test plans that are responsive to the
needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and pre-
paring peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA)
protocols to ensure that data of known and adequate quality are generated and that the results are defensible.
The Advanced Monitoring Systems (AMS) Center, one of six verification centers under ETV, is operated by
Battelle in cooperation with EPA's National Exposure Research Laboratory. The AMS Center has recently
evaluated the performance of rapid polymerase chain reaction (PCR) systems to detect biological agents and
pathogens in water. This verification statement provides a summary of the test results for the Invitrogen
Corporation's PathAlert™ Detection Kits for the detection of Francisella tularensis (F. tularensis), Yersinia pestis
(Y. pestis), and Bacillus anthracis (B. anthracis).
-------
VERIFICATION TEST DESCRIPTION
The PathAlert™ Detection Kits were evaluated between June 9 and June 30, 2004, using F. tularensis LVS
[American Type Culture Collection (ATCC)# 29684], Y. pestis CO92, and B. anthracis Ames strain. The
performance of each PathAlert™ Detection Kit was verified in terms of its accuracy, specificity, number of false
positive/negative responses, precision, interferences, ease of use, and sample throughput. Performance test (PT)
samples, drinking water (DW) samples, and quality control (QC) samples were used in the verification test for
each bacteria. PT samples included individual bacteria spiked into American Society of Testing and Materials
(ASTM) Type II deionized (DI) water at 2, 5, 10, and 50 times the vendor-stated method limit of detection (LOD),
as well as the infective/lethal dose for each contaminant. PT samples also included potential interferent samples
containing a single concentration (10 times the method LOD) of the contaminant of interest in the presence of
fulvic and humic acids [at 0.5 milligram (mg)/liter (L) each and 2.5 mg/L each] spiked into ASTM Type II DI
water. Interferent samples also were analyzed without the addition of any bacteria. DW samples consisted of
chlorinated filtered surface water, chloraminated filtered surface water, chlorinated filtered groundwater, and
chlorinated unfiltered surface water collected from four geographically distributed municipal sources. DW samples
were analyzed without adding contaminant and after fortification with each individual bacteria at a single
concentration level (10 times the vendor-stated method LOD). QC samples included method blank samples and
positive (both internal and external) and negative controls, as supplied with each PathAlert™ Detection Kit. For all
contaminants, plate enumerations were performed in triplicate to confirm the concentrations of the stock solutions
of each bacteria prior to testing.
For the purposes of this test, IxlO4 colony forming units per milliliter (cfu/mL) were used to calculate the
concentration levels of F. tularensis and B. anthracis spiked into the PT and DW samples; 100 cfu/mL were used
to calculate levels of Y. pestis spiked in the PT and DW samples. These vendor-provided concentration levels were
anticipated to be the levels for the entire experimental process at which quantifiably reproducible positive results
could be obtained from a raw water sample. These concentration levels are referred to as the "method LOD" for a
particular assay. The method LOD incorporates the sensitivities and uncertainties of not only the PathAlert™
Detection Kit, but also the deoxyribonucleic acid (DNA) purification step; and, as such, it is an experimental
detection limit rather than an instrument or reagent-specific detection limit. As mentioned previously, the method
LOD provided by the vendor was used specifically as a guideline in calculating sample concentration ranges for
use with the PathAlert™ Detection Kit and all other components used in this verification test to analyze a sample,
and it should be noted that Invitrogen Corporation does not claim this to be the true LOD of the PathAlert™
Detection Kit alone. The vendor claims the absolute LOD (the least amount of target DNA that would generate a
positive result) for the PathAlert™ Detection Kit alone is as low as 1 to 10 copies of DNA, depending on the
assay. This information was not verified in this test.
Samples were spiked with F. tularensis and B. anthracis at 2xl04 colony-forming units (cfu)/milliliter (mL),
5xl04 cfu/mL, lxl05cfu/mL, and 5xl05cfu/mL for PT samples and lxl05cfu/mL for interferent and DW samples.
Samples were spiked with Y. pestis at 2xl02 cfu/mL, 5xl02 cfu/mL, IxlO3 cfu/mL, and 5xl03 cfu/mL for PT
samples and IxlO3 cfu/mL for interferent and DW samples. The infective/lethal dose of each contaminant was
determined by calculating the concentration at which ingestion of 250 mL of water is likely to cause the death of a
70-kilogram person based on human LD50 or ID50 data. The infective/lethal doses for F. tularensis, Y. pestis, and
B. anthracis were 4xl05cfu/mL, 0.28 cfu/mL, and 200 cfu/mL, respectively. Samples were prepared in 1 mL
quantities and tested blindly by trained Battelle operators who had prior PCR experience. To test a 1 mL liquid
sample for the presence or absence of F. tularensis, Y. pestis, or B. anthracis, DNA was extracted and purified
from the sample using the Roche High Pure PCR Template Preparation Kit, assays were prepared using the
PathAlert™ Detection Kit reagents, PCR was performed using a MJ Research DNA Engine® (PTC-200™) Peltier
Thermal Cycler, and the amplified products were analyzed using the Agilent 2100 Bioanalyzer instrument along
with the 2100 Bioanalyzer DNA 500 chips and reagent kit and associated 2100 Bioanalyzer software. The kit was
only tested for one bacteria at a time. All samples were analyzed in quadruplicate from the same batch of purified
DNA. The PathAlert™ Detection Kit was evaluated for qualitative results only by monitoring the internal positive
control (IPC) along with the bacteria-specific peaks in the 2100 Bioanalyzer electropherogram output. Only
-------
positive, negative, and inconclusive results were recorded. Inconclusive results occurred when not all of the
bacteria-specific peaks were present in the electropherogram.
QA oversight of verification testing was provided by Battelle and EPA. Battelle QA staff conducted a technical
systems audit and a data quality audit of 10% of the test data. This verification statement, the full report on which
it is based, and the test/QA plan for this verification are all available at www.epa.gov/etv/centers/centerl.html.
TECHNOLOGY DESCRIPTION
The following description of the PathAlert™ Detection Kit was provided by the vendor and was not subjected to
verification in this test.
The PathAlert™ Detection Kit is a multiplex PCR reagent system capable of detecting F. tularensis, Y. pestis,
B. anthracis, or smallpox in individual assays. The PathAlert™ Detection Kit comprises an optimized PCR
SuperMix specific to the pathogen of interest, as well as an external positive control (EPC) template for system
validation. The kit includes Taq polymerase, pre-complexed with antibodies to maintain "hot start" PCR (for
specificity and sensitivity); uracil DNA glycosylase and deoxyuridane triphosphate to eliminate post-PCR
cross-contamination; and an IPC to identify potential PCR inhibition from sample contaminants or environmental
samples. Included in the kit is an EPC that has been engineered to produce different amplicon sizes than either the
IPC or the pathogen-specific loci. As a result, pathogen-specific results can be read with minimal interference if
contamination by the external control should occur.
The PathAlert™ Detection Kit is an endpoint assay; post-amplification products may be analyzed using any
platform capable of distinguishing amplicon size, such as the Agilent Bioanalyzer 2100, Agilent ALP (high
throughput), transgenomic WAVE high-performance liquid chromatography, gel electrophoresis, and Caliper
AMS 90. The Agilent Bioanalyzer 2100 and ALP are recommended by Invitrogen Corporation for use with the
PathAlert™ Detection Kit because all field testing to date has been performed with these systems, and Agilent and
Invitrogen Corporation have established a co-marketing relationship for the complete system. The list price of each
PathAlert™ Detection Kit assay is $12 to $16. Additional discounts based on volume and concept of operations
are available through the vendor. PathAlert™ Detection Kits can perform up to 320 assays per kit.
VERIFICATION OF PERFORMANCE
Accuracy: Accuracy was assessed by evaluating how often the PathAlert™ Detection Kit results were positive in
the presence of a concentration of contaminant above the method LOD. Contaminant-only PT samples were used
for this analysis. An overall percent agreement was determined by dividing the number of positive responses by
the overall number of analyses of contaminant-only PT samples above the method LOD. The results are presented
in the table below.
Concentration Range of Samples
Used in Accuracy Calculations Overall Accuracy
Bacteria (cfu/mL) (Positive Results Out of Total Replicates)
F. tularensis
Y. pestis
B. anthracis
2xl04 to 5xl05
2xl02 to 5xl03
2xl04 to 5xl05
100% (20/20)
100% (16/16)
100% (16/16)
For F. tularensis, Y. pestis, and B. anthracis, all samples at concentration levels above the vendor-stated method
LOD generated positive responses for each set of replicates, resulting in 100% agreement for the overall accuracy
of the PathAlert™ Detection Kit for each bacteria. The infective/lethal doses for Y. pestis (0.28 cfu/mL) and
-------
B. anthracis (200 cfu/mL) were below the method LOD and not included in the accuracy calculations for those
bacteria.
Specificity: The ability of the PathAlert™ Detection Kit to provide a negative response when the contaminant was
absent was assessed. The specificity rate was determined by dividing the number of negative responses by the total
number of unspiked samples. Unspiked interferent PT samples and unspiked DW samples were used to assess
specificity. The results are presented in the table below. For F. tularensis and Y. pestis, one unspiked DW replicate
for each bacteria produced an inconclusive response.
Overall Specificity
Bacteria (Negative Results Out of Total Replicates)
F. tularensis
Y. pestis
B. anthracis
96% (23/24)
96% (23/24)
100% (22/22)
False positive/negative responses: A false positive response was defined as a detectable or positive PathAlert™
Detection Kit response when the interferent PT samples or DW samples were not spiked. The false positive rate
was reported as the frequency of false positive results out of the total number of unspiked samples. The false
negative response was defined as a negative response when the sample was spiked with a contaminant at a
concentration greater than the method LOD. Spiked PT (contaminant and interferent) samples and spiked DW
samples were included in the analysis. The false negative rate was reported as the frequency of false negative
results out of the total number of spiked samples for a particular contaminant. The results are presented in the table
below. No false positives or false negatives were found for any of the sample matrices for any bacteria. One
replicate for unspiked DW in two different DW samples showed an inconclusive result for F. tularensis and one
for Y. pestis. Two inconclusive results were reported (one for F. tularensis in one DW sample and one for Y. pestis
in a different DW sample).
Bacteria False Positive Rate False Negative Rate
F. tularensis
Y. pestis
B. anthracis
0/24
0/24
0/22
0/60
0/56
0/56
Precision: The precision of the PathAlert™ Detection Kit was assessed by calculating the overall percentage of
consistent responses for all the sample sets. Responses were considered consistent if all responses of the four
replicates were the same.
For F. tularensis replicates, 95% of the sample sets (20 out of 21) showed consistent results. Similarly, 95% of the
sample sets (20 out of 21) showed consistent results for Y. pestis. For both bacteria, the inconsistency resulted from
an inconclusive result for a DW replicate for each bacteria. As with F. tularensis and Y. pestis, 95% of the sample
sets for B. anthracis (20 out of 21) showed consistent results among the replicates. For this bacteria, the one
sample set with inconsistent results was the infective dose PT sample. In this set of replicates, three of the four
samples had inconclusive results, while the fourth sample was positive for B. anthracis. The infective dose of B.
anthracis was below the method LOD for this bacteria.
Other performance factors: A Battelle technician with prior PCR experience who was trained by the vendor
operated the PathAlert™ Detection Kit. The kit was straightforward and easy to use. All components necessary for
-------
the PCR process (excluding the sample DNA) were contained in one solution, which had to be stored at -20°C.
Three separate work areas were needed for testing: a "clean" area free of DNA, a "medium" area with moderate
DNA presence, and a "dirty" area, with high DNA presence. SuperMix prep took approximately 15 minutes and
loading the sample DNA into the PCR tubes took between 15 and 30 minutes for each set of samples. Sample
throughput for this verification test (from DNA purification to amplified product detection) was 36 to 48 samples
per day.
Original signed by Gabor J. Kovacs
Gabor J. Kovacs
Vice President
Energy and Environment Division
Battelle
12/2/04
Date
Original signed by E. Timothy Oppelt 12/29/04
E. Timothy Oppelt Date
Director
National Homeland Security Research Center
U.S. Environmental Protection Agency
NOTICE: ETV verifications are based on an evaluation of technology performance under specific, predetermined
criteria and the appropriate quality assurance procedures. EPA and Battelle make no expressed or implied
warranties as to the performance of the technology and do not certify that a technology will always operate as
verified. The end user is solely responsible for complying with any and all applicable federal, state, and local
requirements. Mention of commercial product names does not imply endorsement.
------- |