THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
EX
RAM^ft
T
EPA Bairene
^^Hl M^ m T7 T7 • /T i-
TTŁ. T, . , , n , ,. . J/jj l_>nsincss of Innovation
U.S. h,nvironmental Froteetion Agency
ETV Joint Verification Statement
TECHNOLOGY TYPE: Rapid Polymerase Chain Reaction
APPLICATION: DETECTING BIOLOGICAL AGENTS AND
PATHOGENS IN WATER
TECHNOLOGY NAME: R.A.P.I.D.® System
COMPANY: Idaho Technology Inc.
ADDRESS: 390WakaraWay PHONE: 801-736-6354
Salt Lake City, Utah 84108 FAX: 801-588-0507
WEB SITE: www.idahotech.com/rapid
E-MAIL: matts@idahotech.com
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology Verification (ETV)
Program to facilitate the deployment of innovative or improved environmental technologies through performance
verification and dissemination of information. The goal of the ETV Program is to further environmental protection
by accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal
by providing high-quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies. Information and ETV
documents are available at www.epa.gov/etv.
ETV works in partnership with recognized standards and testing organizations, with stakeholder groups
(consisting of buyers, vendor organizations, and permitters), and with individual technology developers. The
program evaluates the performance of innovative technologies by developing test plans that are responsive to the
needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and pre-
paring peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA)
protocols to ensure that data of known and adequate quality are generated and that the results are defensible.
The Advanced Monitoring Systems (AMS) Center, one of six verification centers under ETV, is operated by
Battelle in cooperation with EPA's National Exposure Research Laboratory. The AMS Center has recently
evaluated the performance of rapid polymerase chain reaction (PCR) systems to detect bacteria in water. This
verification statement provides a summary of the test results for the Idaho Technology Inc.'s (ITI's) Ruggedized
Advanced Pathogen Identification Device (R.A.P.I.D.®) System for the detection of Francisella tularensis (F.
tularensis), Yersinia pestis (Y. pestis), Bacillus anthracis (B. anthracis), Brucella suis, and Escherichia coli
(E. coli).
-------
VERIFICATION TEST DESCRIPTION
The R.A.P.I.D.® System was evaluated between May 27 and July 8, 2004, using F. tularensis LVS [American
Type Culture Collection (ATCC)# 29684], Y. pestis CO92, B. anthracis Ames strain, Brucella suis
(ATCC#23444), and E. coli O157:H7. The performance of the R.A.P.I.D.® System was verified in terms of its
accuracy, specificity, number of false positive/negative responses, precision, interferences, ease of use, and sample
throughput. Performance test (PT) samples, drinking water (DW) samples, and quality control (QC) samples were
used in the verification test for each bacteria. PT samples included individual bacteria spiked into American
Society of Testing and Materials (ASTM) Type II deionized (DI) water at 2, 5, 10, and 50 times the vendor-stated
system limit of detection (LOD), as well as the infective/lethal dose for each contaminant. PT samples also
included potential interferent samples containing a single concentration (10 times the system LOD) of the
contaminant of interest in the presence of fulvic and humic acids [at 0.5 milligram (mg)/liter (L) each and 2.5 mg/L
each] spiked into ASTM Type II DI water. Interferent samples were also analyzed without the addition of any
bacteria. DW samples consisted of chlorinated filtered surface water, chloraminated filtered surface water,
chlorinated filtered groundwater, and chlorinated unfiltered surface water collected from four geographically
distributed municipal sources. DW samples were analyzed without adding contaminant and after fortification with
each individual bacteria at a single concentration level (10 times the vendor-stated system LOD). QC samples
included method blank samples and positive and negative controls, as supplied with the R.A.P.I.D.® System. For
all contaminants, plate enumerations were performed in triplicate to confirm the concentration of the stock solution
of each bacteria prior to testing.
For the purposes of this test, 1,000 colony forming units per milliliter (cfu/mL) were used to calculate the
concentration levels spiked into the PT and DW samples.. This vendor-provided concentration level was
anticipated to be the level at which quantifiably reproducible positive results could be obtained from a raw water
sample using the R.A.P.I.D.® System. This concentration level is referred to as the "system LOD." The system
LOD incorporates the sensitivities and uncertainties of the entire R.A.P.I.D.® System, in particular the ITI1-2-3
Flow Kit deoxyribonucleic acid (DNA) purification step, as well as the ITI freeze-dried reagents; and, as such, it is
a method detection limit rather than an instrument or reagent-specific detection limit. As mentioned previously, the
system LOD provided by the vendor was used specifically as a guideline in calculating sample concentration
ranges for use with the R.A.P.I.D.® System in this verification test, and it should be noted that Idaho Technology
Inc. does not claim this to be the true LOD of the R.A.P.I.D.® System. Detection limits for individual components
of the R.A.P.I.D.® System and the system as a whole may differ and were not verified in this test.
Samples were spiked individually with each bacteria (F. tularensis, Y. pestis, B. anthracis, Brucella suis, and
E. coli) at 2xl03 cfu/mL, 5xl03 cfu/mL, lxl04cfu/mL, and 5xl04cfu/mLfor PT samples and lxl04cfu/mLfor
interferent and DW samples. The infective/lethal dose of each contaminant was determined by calculating the
concentration at which ingestion of 250 mL of water is likely to cause the death of a 70-kilogram person based on
human LD50 or ID50 data. The infective/lethal doses for F. tularensis, Y. pestis, B. anthracis, Brucella suis, and E.
coli were 4xl05cfu/mL, 0.28 cfu/mL, 200 cfu/mL, 40 cfu/mL, and 0.2 cfu/mL respectively. Samples were
prepared in 5 mL quantities and tested blindly by trained Battelle operators who had prior PCR experience. To test
a liquid sample for the presence of F. tularensis, Y. pestis, B. anthracis, Brucella suis, or E. coli, DNA was
extracted and purified using the R.A.P.I.D.® System's ITI 1-2-3 Flow Kit, samples were prepared using the
R.A.P.I.D.® System's freeze-dried reagents, and the samples were analyzed using the R.A.P.I.D.®7200 instrument
and software. Each reconstituted, freeze-dried reagent sample was split into two glass capillaries for analysis in
duplicate on the R.A.P.I.D.® 7200 instrument. For F. tularensis, Y. pestis, and B. anthracis, more than one freeze-
dried reagent target was used for the analysis of each bacteria. The system was only tested for one bacteria at a
time. All samples were analyzed in quadruplicate from the same batch of purified DNA. The R.A.P.I.D.® System
was evaluated for qualitative results only by monitoring the positive and negative controls along with the
amplification curves and crossing points for each sample using the R.A.P.I.D.® 7200 software Second Derivative
Maximum method. Only positive, negative, and inconclusive results were recorded. Inconclusive results occurred
when the results from the two capillaries for the sample didn't agree with each other (i.e., one was positive and one
was negative).
-------
QA oversight of verification testing was provided by Battelle and EPA. Battelle QA staff conducted a technical
systems audit and a data quality audit of 10% of the test data. This verification statement, the full report on which
it is based, and the test/QA plan for this verification are all available at www.epa.gov/etv/centers/centerl.html.
TECHNOLOGY DESCRIPTION
The following description of the R.A.P.I.D.® System was provided by the vendor and was not subjected to
verification in this test.
The R.A.P.I.D.® System is an integrated three-part system for the timely detection and identification of pathogens
and biowarfare agents, including anthrax, plague, salmonella, and botulism, in water. The system allows for rapid
and specific presumptive identification of threat pathogens in hours rather than days. The system components
consist of the ITI1-2-3 Flow Kit for the purification of DNA, ITI target-specific freeze-dried reagents containing
all the necessary ingredients for specific pathogen DNA detection, and the R.A.P.I.D.® 7200 instrument. The
operator prepares the DNA from the environmental sample, hydrates the freeze-dried reagents with the DNA
sample, runs the R.A.P.I.D.® 7200 instrument, and then reads the auto-analyzed results using the R.A.P.I.D.®
software.
The ITI 1 -2-3 Flow Kit is a three-step DNA extraction and purification kit that contains the components for
purifying DNA from water or other environmental matrix. It removes inhibitors from a sample that would
adversely affect a reaction and has been optimized for purifying DNA from difficult-to-process anthrax spores, as
well as non-spore forming bacteria. Each kit is optimized and validated for the R.A.P.I.D.® 7200 instrument and
contains all the ingredients necessary for DNA purification (one 30 mL, one 20 mL, and one 25 mL buffer;
50 bead tubes with lysis buffer; 50 spin filters; 200 receiver tubes; and 50 swabs). The ITI freeze-dried reagents
(hybridization probe reagents) are freeze-dried in a single tube and require no refrigeration or freezing. There are
multiple gene targets for assay confirmation, with the additional backup of melting curve analysis to confirm any
suspected positive test.
The R.A.P.I.D.® 7200 instrument is a field-hardened, air-driven, real-time thermocycler with concurrent
fluorescence monitoring that is capable of automatically analyzing samples for the presence of any given DNA
sequence. The R.A.P.I.D.® 7200 instrument is based on LightCycler technology and is capable of 45 PCR cycles in
30 minutes. It has three color optics, can run on either 110 or 220 volt power, and is watertight in its case.
R.A.P.I.D.® software allows the user to automatically collect and interpret data and report results. The software has
two user levels, with simple push-button software and auto-analysis or with all the features of the original
laboratory instrument for advanced real-time analysis. The R.A.P.I.D.® 7200 instrument is a 50-pound (22.7-kg),
portable commercial off-the-shelf system. It operates in various environmental conditions (heat, humidity, salt
spray) and has passed a one-meter drop test. The R.A.P.I.D.® instrument includes a backpack, laptop computer,
microcentrifuge, and sample capillaries. Its dimensions are 19.4 inches [49.3 centimeters (cm)] by 14.3 inches
(36.3 cm) by 10.5 inches (26.7 cm), and its cost is $55,000 U.S., including the centrifuge and laptop. Sample
preparation using the ITI 1-2-3 Flow Kit costs approximately $8 per sample, and testing the samples using the
freeze-dried reagents costs approximately $17 per sample.
VERIFICATION OF PERFORMANCE
Accuracy: Accuracy was assessed by evaluating how often the R.A.P.I.D.® System results were positive in the
presence of a concentration of contaminant above the system LOD. Contaminant-only PT samples were used for
this analysis. An overall percent agreement was determined by dividing the number of positive responses by the
overall number of analyses of contaminant-only PT samples above the system LOD. The results are presented in
the table below.
-------
Bacteria
F. tularensis Target 1
F. tularensis Target 2
F. pestis Target 1
F. pestis Target 2
B. anthracis Target 1
B. anthracis Target 2
B. anthracis Target 3
Brucella suis
E. coli
Concentration Range of Samples
Used in Accuracy Calculations
(cfu/mL)
2xl03to4xl05
2xl03to4xl05
2xl03to5xl04
2xl03to5xl04
2xl03to5xl04
2xl03to5xl04
2xl03to5xl04
2xl03to5xl04
2xl03to5xl04
Overall Accuracy
(Positive Results Out of Total
Replicates)
100% (20/20)
100% (20/20)
100% (16/16)
100% (16/16)
94% (15/16)
94% (15/16)
100% (16/16)
88% (14/16)
100% (16/16)
For F. tularensis (both targets), F. pestis (both targets), B. anthracis Target 3, and E. coli, all samples at
concentration levels above the vendor-stated system LOD generated positive responses for each set of replicates,
resulting in 100% agreement for the overall accuracy of the R.A.P.I.D.® System for each bacteria. One replicate at
2xl03 cfu/inL for B. anthracis Target 1 reagents resulted in an inconclusive response; one replicate at 2xl03
cfu/mL for B. anthracis Target 2 reagents resulted in a negative response; and two replicates at 2xl03 cfu/inL for
Brucella suis reagents resulted in inconclusive responses; all other replicates at each level for each bacteria
produced positive results. The infective/lethal doses for F. pestis (0.28 cfu/mL), B. anthracis (200 cfu/mL),
Brucella suis (40 cfu/mL), and E. coli (0.2 cfu/inL) were below the system LOD and not included in the accuracy
calculations for those bacteria.
Specificity: The ability of the R.A.P.I.D.® System to provide a negative response when the contaminant was absent
was assessed. The specificity rate was determined by dividing the number of negative responses by the total
number of unspiked samples. Unspiked interferent PT samples and unspiked DW samples were used to assess
specificity. The results are presented in the table below. For F. tularensis Target 1, three of the replicates for
unspiked 2.5 mg/L humic and fulvic acids produced inconclusive results. For F. tularensis Target 2, one unspiked
DW replicate produced an inconclusive response. The results for all other bacteria were negative for all unspiked
samples.
Overall Specificity
Bacteria (Negative Results Out of Total Replicates)
F. tularensis Target 1 88% (21/24)
F. tularensis Target 2 96% (23/24)
F. pestis Target 1 100% (24/24)
F. pestis Target 2 100% (24/24)
B. anthracis Target 1 100% (24/24)
B. anthracis Target 2 100% (24/24)
B. anthracis Target 3 100% (24/24)
Brucella suis 100% (24/24)
E. coli 100% (24/24)
-------
False positive/negative responses: A false positive response was defined as a detectable or positive R.A.P.I.D.®
System response when the interferent PT samples or DW samples were not spiked. The false positive rate was
reported as the frequency of false positive results out of the total number of unspiked samples. The false negative
response was defined as a negative response when the sample was spiked with a contaminant at a concentration
greater than the system LOD. Spiked PT (contaminant and interferent) samples and spiked DW samples were
included in the analysis. The false negative rate was reported as the frequency of false negative results out of the
total number of spiked samples for a particular contaminant. Inconclusive results were considered neither positive
nor negative and were thus not counted as false positives or false negatives. The results are presented in the table
below.
Bacteria
F. tularensis Target 1
F. tularensis Target 2
Y. pestis Target 1
Y. pestis Target 2
B. anthracis Target 1
B. anthracis Target 2
B. anthracis Target 3
Brucella suis
E. coli
False Positive Rate
0/24
0/24
0/24
0/24
0/24
0/24
0/24
0/24
0/24
False Negative Rate
0/60
4/60
0/56
0/56
2/56
2/56
0/56
0/56
0/52
No false positives were found for any of the sample matrices for any bacteria. Three unspiked interferent PT
sample replicates showed inconclusive results for F. tularensis Target 1. Inconclusive results were also found for
one replicate in unspiked DW using F. tularensis Target 2.
Four false negatives were found for F. tularensis Target 2: one in a set of replicates for spiked interferent PT
samples, and three in a set of replicates for spiked DW. Inconclusive results were also found for one replicate in
spiked DW for this target. One replicate for spiked DW in each of two different DW samples produced a false
negative result for B. anthracis Target 1. One replicate each for two different spiked DW samples and spiked DI
water at 2xl03 cfu/mL produced inconclusive results for B. anthracis Target 1. Samples analyzed using
B. anthracis Target 2 also had two false negatives: one replicate of 2x103 cfu/mLB. anthracis in DI water and one
replicate of spiked DW. Two inconclusive results were also found for spiked DW replicates using B. anthracis
Target 2 reagents. B. anthracis Target 3 reported no false negative results, although four spiked DW replicates
from one DW sample generated inconclusive results. Two inconclusive results were obtained for DI water spiked
at 2xl03 cfu/mL for Brucella suis.
Precision: The precision of the R.A.P.I.D.® System was assessed by calculating the overall percentage of
consistent responses for all the sample sets. Responses were considered consistent if all responses of the four
replicates were the same.
For F. tularensis Target 1 replicates, 95% of the sample sets (20 out of 21) showed consistent results; and for
F. tularensis Target 2 replicates, 86% of the sample sets (18 out of 21) showed consistent results. The
inconsistencies for the F. tularensis Target 1 resulted from inconclusive results for three interferent PT sample
replicates; Target 2 inconsistencies resulted from an inconclusive DW replicate and a negative response for an
-------
interferent PT sample replicate. For Y. pestis Targets 1 and 2, 100% of the samples sets (21 out of 21) showed
consistent results among the replicates.
For B. anthracis, 76% of the Target 1 sample sets (16 out of 21) showed consistent results, 86% of the Target 2
sample sets (18 out of 21) showed consistent results, and 95% of the Targets sample sets (20 out of 21) showed
consistent results. For B. anthracis Target 1, the five sample sets with inconsistent results included two PT
samples (one inconclusive replicate at 2xl03 cfu/inL and three inconclusive and one negative result at the
infective/lethal dose) and three different spiked DW samples (one inconclusive replicate in each of two different
DW samples and one negative replicate in each of two different DW samples). For B. anthracis Target 2, the three
sample sets with inconsistent results included two PT samples (one negative replicate at 2xl03 cfu/inL and two
inconclusive and two negative results at the infective/lethal dose) and one spiked DW sample (two inconclusive,
one positive, and one negative replicate). For B. anthracis Target 3, the only inconsistent sample set was the
infective/lethal dose, with two inconclusive and two negative results. The infective dose of B. anthracis was below
the system LOD for the R.A.P.I.D.® System for this bacteria.
For Brucella suis, 90% (19 out of 21) of the sample sets showed consistent results. For this bacteria, two PT
sample sets showed inconsistent results (two positive and two inconclusive replicates at 2xl03 cfu/mL and one
inconclusive and three negative replicates at the infective/lethal dose). The infective dose of Brucella suis was
below the system LOD for the R.A.P.I.D.® System for this bacteria. For E. coli, 100% of the sample sets showed
consistent results among the replicates.
Other performance factors: A Battelle technician with prior PCR experience who was trained by the vendor
operated the R.A.P.I.D.® System at all times. All three components of the R.A.P.I.D.® System (the ITI1-2-3 Flow
Kit, the freeze-dried reagents, and the R.A.P.I.D.® 7200 instrument) were straightforward and easy to use. Three
separate work areas were needed for testing to minimize cross-contamination. The freeze-dried reagents were color
coded, contained all of the necessary components for PCR in one vial, and were reconstituted in the same vial,
making PCR setup easy. Reagents for the DNA purification and PCR setup had room temperature storage
requirements. The glass capillaries used on the R.A.P.I.D.® 7200 instrument were problematic when the rotors on
the carousel were dirty, but posed no problems once the rotors were properly cleaned. The sample throughput for
this verification test was 36 to 72 samples per day. Approximate operational times were two hours for DNA
extraction/purification, 30 minutes for reconstituting the reagents and loading the capillaries for one carousel
batch, and 30 minutes/carousel batch for PCR. The R.A.P.I.D.® 7200 instrument can hold up to 32 capillaries
(14 split samples, plus controls). The R.A.P.I.D.® software was easy to use, and additional software analysis tools
other than those used in this test are available.
Original signed by Gabor J. Kovacs
Gabor J. Kovacs
Vice President
Energy and Environment Division
Battelle
12/2/04
Date
Original signed by E. Timothy Oppelt 12/29/04
E. Timothy Oppelt Date
Director
National Homeland Security Research Center
U.S. Environmental Protection Agency
NOTICE: ETV verifications are based on an evaluation of technology performance under specific, predetermined
criteria and the appropriate quality assurance procedures. EPA and Battelle make no expressed or implied
warranties as to the performance of the technology and do not certify that a technology will always operate as
verified. The end user is solely responsible for complying with any and all applicable federal, state, and local
requirements. Mention of commercial product names does not imply endorsement.
------- |