December 2004
Environmental Technology
Verification Report
IDAHO TECHNOLOGY INC.
R. A. P. I. D.® SYSTEM
FOR THE DETECTION OF
FRANCISELLA TULARENSIS,
YERSINIA PESTIS, BACILLUS ANTHRACIS,
BRUCELLA suis, AND ESCHERICHIA COLI
Prepared by
Battelle
Baireiie
7/7C Business of Innovation
Under a cooperative agreement with
y>EIPA U.S. Environmental Protection Agency
-------
December 2004
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Idaho Technology Inc.
R.A.RI.D.® System
for the detection of
Francisella tularensis,
Yersinia pestis, Bacillus anthracis,
Brucella suis, and
Escherichia coli
by
Stephanie Buehler
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six verification centers. Information about
each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/centers/centerl.html.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We sincerely appreciate the
contribution of drinking water samples from the New York City Department of Environmental
Protection (Paul Bennett), the City of Orlando (Terri Slifko), and the Metropolitan Water
District of Southern California (Paul Rochelle). Also, thanks go to the Metropolitan Water
District of Southern California for concentrating each drinking water sample. We would also
like to thank Myriam Medina-Vera and Jeanette Van Emon, U.S. Environmental Protection
Agency National Exposure Research Laboratory; Jorge Santo Domingo, U.S. Environmental
Protection Agency National Risk Management Research Laboratory; Kerri Alderisio, New York
City Department of Environmental Protection; Ricardo DeLeon, Metropolitan Water District of
Southern California; and Stanley States, Pittsburgh Water and Sewer Authority, for their careful
review of the test/quality assurance plan and this verification report.
IV
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations viii
1 Background 1
2 Technology Description 2
3 Test Design and Procedures 4
3.1 Introduction 4
3.2 Test Samples 6
3.2.1 Performance Test Samples 6
3.2.2 Drinking Water Samples 7
3.2.3 Quality Control Samples 8
3.3 Reference Methods 9
3.3.1 Plate Enumeration 9
3.3.2 Drinking Water Analysis 9
3.4 Test Procedure 10
3.4.1 Sample Handling 10
3.4.2 Sample Preparation and Analysis 11
3.4.3 Drinking Water Characterization 12
4 Quality Assurance/Quality Control 14
4.1 Sample Chain-of-Custody Procedures 14
4.2 Equipment Calibration 14
4.3 Characterization of Contaminant Stock Solutions 15
4.4 Quality Control Samples 15
4.5 Audits 16
4.5.1 Technical Systems Audit 16
4.5.2 Audit of Data Quality 16
4.6 QA/QC Reporting 17
4.7 Data Review 17
5 Data Analysis 18
5.1 Accuracy 18
5.2 Specificity 18
5.3 False Positive/Negative Responses 18
5.4 Precision 19
-------
5.5 Interferences 19
5.6 Other Performance Factors 19
6 Test Results 20
6.1 Accuracy 22
6.1.1 F. tularensis 22
6.1.2 Y.pestis 22
6.1.3 B. anthracis 23
6.1.4 Brucella suis 25
6.1.5 E. coli 25
6.2 Specificity 26
6.2.1 F. tularensis 26
6.2.2 Y. pestis 27
6.2.3 B. anthracis 28
6.2.4 Brucella suis 28
6.2.5 E. coli 29
6.3 False Positive/Negative Responses 30
6.3.1 F. tularensis 31
6.3.2 Y. pestis 31
6.3.3 B. anthracis 31
6.3.4 Brucella suis 31
6.3.5 E. coli 39
6.4 Precision 40
6.5 Interferences 41
6.5.1 Interferent PT Samples 41
6.5.2 Drinking Water Samples 42
6.6 Other Performance Factors 42
7 Performance Summary 46
8 References 56
Figures
Figure 2-1. Idaho Technology Inc.'s R.A.P.I.D.® 7200 Instrument 2
Figure 6-1. LCD A Amplification Plot Samples Analyzed Using B. anthracis Target 2 21
Figure 6-2. R.A.P.I.D.® Detector View Output for Spiked (Sample 7) and Unspiked
(Samples 8 and 9) Brucella suis Samples 45
VI
-------
Tables
Table 3-1. Infective/Lethal Dose of Target Contaminants 6
Table 3-2. Performance Test Samples 7
Table 3-3. Drinking Water Samples 8
Table 3-4. ATEL Water Quality Characterization of Drinking Water Samples 13
Table 4-1. Contaminant Triplicate Plate Enumeration Data 16
Table 4-2. Data Recording Process 17
Table 6-la. F. tularensis Contaminant-Only PT Sample Results 23
Table 6-lb. Y. pestis Contaminant-Only PT Sample Results 23
Table 6-lc. B. anthracis Contaminant-Only PT Sample Results 24
Table 6-Id. Brucella suis Contaminant-Only PT Sample Results 25
Table 6-le. E. coli Contaminant-Only PT Sample Results 26
Table 6-2a. F. tularensis Specificity Results 27
Table 6-2b. Y. pestis Specificity Results 28
Table 6-2c. B. anthracis Specificity Results 29
Table 6-2d. Brucella suis Specificity Results 29
Table 6-2e. E. coli Specificity Results 30
Table 6-3a. F. tularensis Target 1 False Positive/Negative Results 32
Table 6-3b. F. tularensis Target 2 False Positive/Negative Results 33
Table 6-3c. Y. pestis False Positive/Negative Results 34
Table 6-3d. B. anthracis Target 1 False Positive/Negative Results 35
Table 6-3e. B. anthracis Target 2 False Positive/Negative Results 36
Table 6-3f. B. anthracis Target 3 False Positive/Negative Results 37
Table 6-3g. Brucella suis False Positive/Negative Results 38
Table 6-3h. E. coli False Positive/Negative Results 39
Table 7-1. F. tularensis Target 1 Summary Table 47
Table 7-2. F. tularensis Target 2 Summary Table 48
Table 7-3. Y. pestis Target 1 Summary Table 49
Table 7-4. Y. pestis Target 2 Summary Table 50
Table 7-5. B. anthracis Target 1 Summary Table 51
Table 7-6. B. anthracis Target 2 Summary Table 52
Table 7-7. B. anthracis Target 3 Summary Table 53
Table 7-8. Brucella suis Summary Table 54
Table 7-9. E. coli Summary Table 55
vn
-------
List of Abbreviations
AMS
ASTM
ATEL
ATCC
BSL
Ca
cfu
cm
DI
DNA
DW
EPA
ETV
ID
m
L
LCDA
LOD
MB
Mg
mg
mL
MWD
PBS
PCR
PT
QA
QC
QMP
R.A.P.I.D.
SOP
TSA
Advanced Monitoring Systems
American Society of Testing and Materials
AquaTech Environmental Laboratories, Inc.
American Type Culture Collection
Biosafety Level
calcium
colony forming unit
centimeter
deionized water
deoxyribonucleic acid
drinking water
U.S. Environmental Protection Agency
Environmental Technology Verification
identification
Idaho Technology Inc.
liter
LightCycler Data Analysis
limit of detection
method blank
magnesium
milligram
milliliter
Metropolitan Water District
phosphate buffered saline
polymerase chain reaction
performance test
quality assurance
quality control
Quality Management Plan
Ruggedized Advanced Pathogen Identification Device
standard operating procedure
technical systems audit
Vlll
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative tech-
nologies by developing test plans that are responsive to the needs of stakeholders, conducting
field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance
(QA) protocols to ensure that data of known and adequate quality are generated and that the
results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
recently evaluated the performance of Idaho Technology Inc.'s R.A.P.I.D.® (Ruggedized
Advanced Pathogen Identification Device) System for the detection of Francisella tularensis
(F. tularensis), Yersinia pestis (Y. pestis), Bacillus anthracis (B. anthracis), Brucella suis, and
Escherichia coli (E. coli).
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of environ-
mental monitoring technologies for air, water, and soil. This verification report provides results
for the verification testing of Idaho Technology Inc.'s (m's) R.A.P.I.D.® System. The following
is a description of the R. A.P.I.D.® System based on information provided by the vendor. The
information provided below was not subjected to verification in this test.
The R.A.P.I.D.® System is an integrated three-part system for the timely detection and
identification of pathogens and biowarfare agents, including anthrax, plague, salmonella, and
botulism, in water. The system allows for rapid and specific presumptive identification of threat
pathogens in hours rather than days. The system components consist of the ITI1-2-3 Flow Kit
for the purification of deoxyribonucleic acid (DNA), ITI target-specific freeze-dried reagents
containing all the necessary ingredients for specific pathogen DNA detection, and the
R. A.P.I.D.® 7200 instrument. The operator prepares the DNA from the environmental sample,
hydrates the freeze-dried reagents with the DNA sample, runs the R.A.P.I.D.® 7200 instrument,
and then reads the auto-analyzed results using the R.A.P.I.D.® software.
The ITI 1-2-3 Flow Kit is a three-step DNA extraction and purification kit that contains the
components for purifying DNA from water or other environmental matrix. It removes inhibitors
from a sample that would adversely affect a reaction and has been optimized for purifying DNA
from difficult-to-process anthrax spores, as well as non-spore forming bacteria. Each kit is
optimized and validated for the R.A.P.I.D.®
7200 instrument and contains all the
ingredients necessary for DNA purification
[one 30 milliliter (mL), one 20 mL, and one
25 mL buffer; 50 bead tubes with lysis
I iii& Iv~B buffer; 50 spin filters; 200 receiver tubes;
1 .i -
and 50 swabs].
The ITI freeze-dried reagents (hybridization
probe reagents) are freeze-dried in a single
tube and require no refrigeration or freezing.
There are multiple gene targets for assay
confirmation, with the additional backup of
Figure 2-1. Idaho Technology Inc.'s R.A.P.I.D.® melting curve analysis to confirm any
7200 Instrument suspected positive test.
-------
The R. A.P.I.D.® 7200 instrument is a field-hardened, air-driven, real-time thermocycler with
concurrent fluorescence monitoring that is capable of automatically analyzing samples for the
presence of any given DNA sequence. The R.A.P.I.D.®7200 instrument is based on LightCycler
technology and is capable of 45 polymerase chain reaction (PCR) cycles in 30 minutes. It has
three color optics, can run on either 110 or 220 volt power, and is watertight in its case.
R. A.P.I.D.® software allows the user to automatically collect and interpret data and report results.
The software has two user levels, with simple push-button software and auto-analysis or with all
the features of the original laboratory instrument for advanced real-time analysis.
The R.A.P.I.D.®7200 instrument is a 50-pound (22.7-kilogram), portable commercial off-the-
shelf system. It operates in various environmental conditions (heat, humidity, salt spray) and has
passed a one-meter drop test. The R.A.P.I.D.® instrument includes a backpack, laptop computer,
microcentrifuge, and sample capillaries. Its dimensions are 19.4 inches (49.3 centimeters [cm])
by 14.3 inches (36.3 cm) by 10.5 inches (26.7 cm), and its cost is $55,000 U.S., including the
centrifuge and laptop. Sample preparation using the ITI1-2-3 Flow Kit costs approximately $8
per sample, and testing the samples using the freeze-dried reagents costs approximately $17 per
sample.
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
The purpose of this verification test of rapid PCR technologies was to evaluate the ability of
these technologies to detect the presence of specific bacteria in water and to determine the
technologies' performance when specific interferents were added to pure water and when
interferents were inherently present in several drinking water matrices. The technologies for this
verification test operate based on the PCR process, which involves enzyme-mediated reactions
that allow for target DNA (that from the bacteria of interest) replication and amplification
through a series of temperature cycles. Before the target DNA can be amplified, however, it
must first be extracted from the bacteria and then purified.
Because rapid PCR technologies are anticipated to serve mostly as screening tools in water
monitoring scenarios, providing rapid results as to whether or not a pathogen or biological agent
is present in the water, this verification test involved only qualitative results. This verification
test of the R.A.P.I.D.® System was conducted according to procedures specified in the Test/QA
Plan for Verification of Rapid PCR Technologies^ The performance of the R.A.P.I.D.® System
was verified in terms of the following parameters:
Accuracy
Specificity
False positive/negative responses
Precision
Interferences
Other performance factors.
The performance of the R. A.P.I.D.® System was verified by challenging it with various
concentration levels of F. tularensis, LVS [American Type Culture Collection (ATCC# 29684)]
Y. pestis CO92, B. anthracis Ames strain, Brucella suis (ATCC#23444), and E. coli O157:H7
(E. coli) in American Society of Testing and Materials (ASTM) Type n deionized (DI) water;
ASTM Type n DI water spiked with various interferents; and concentrated drinking water (DW)
samples obtained from four water utilities from different geographical locations in the United
States. Each source of DW represented a unique water treatment process. In addition, the
interferent and DW samples were analyzed without adding any contaminant to evaluate the
potential for false positive results. The system was only tested for one bacteria at a time.
-------
Contaminant concentrations included the infective/lethal dose concentrations given in Table 3-1
for each contaminant and approximately 2, 5, 10, and 50 times the vendor-reported system limit
of detection (LOD). The infective/lethal dose of each contaminant was determined by
calculating the concentration at which ingestion of 250 mL of water is likely to cause the death
of a 70-kilogram (approximately 154 pounds) person based on human LD50 or ID50 data.(2) The
results from quadruplicate analysis of the contaminant performance test (PT) samples and
comparison with the known sample compositions provided information on the accuracy and
precision of the R. A.P.I.D.® System. The interferent PT samples contained humic and fulvic
acids at two concentrations, both spiked and unspiked with contaminants. Each was analyzed in
quadruplicate and provided information on potential matrix interferences.
For the purposes of this test, 1,000 colony forming units per milliliter (cfu/mL) were used to
calculate the concentration levels spiked into the PT and DW samples. This vendor-provided
concentration level was anticipated to be the level at which quantifiably reproducible positive
results could be obtained from a raw water sample using the R.A.P.I.D.® System. This
concentration level is referred to as the "system LOD." The system LOD incorporates the
sensitivities and uncertainties of the entire R.A.P.I.D.® System, in particular the ITI1-2-3 Flow
Kit DNA purification step, as well as the ITI freeze-dried reagents; and, as such, it is a method
detection limit rather than an instrument or reagent-specific detection limit. As mentioned
previously, the system LOD provided by the vendor was used specifically as a guideline in
calculating sample concentration ranges for use with the R. A.P.I.D.® System in this verification
test, and it should be noted that Idaho Technology Inc. does not claim this to be the true LOD of
the R. A.P.I.D.® System. Detection limits for individual components of the R. A.P.I.D.® System
and the system as a whole may differ and were not verified in this test.
The verification test was conducted at Battelle's Medical Research and Evaluation Facility in
West Jefferson, Ohio, as well as Battelle headquarters in Columbus, Ohio, from May 27, 2004,
through July 8, 2004. Aqua Tech Environmental Laboratories, Inc. (ATEL) of Marion, Ohio,
performed physicochemical characterization for each DW sample, including turbidity, dissolved
and total organic carbon, specific conductivity, alkalinity, pH, magnesium (Mg), calcium (Ca),
hardness, total organic halides, trihalomethanes, and haloacetic acids. Battelle cultured the
bacteria, provided the stock solutions of each bacteria used in this test, and then confirmed the
presence and quantity of F. tularensis, Y. pestis, B. anthracis, Brucella suis, and E. coli bacteria
in the stock solutions using plate enumeration. The stock solutions of F. tularensis, Y. pestis,
Brucella suis, and E. coli were stored frozen as 1 mL aliquots. The B. anthracis stock solutions
were refrigerated as 1 mL aliquots. A new 1 mL vial of stock solution was thawed and used for
each day of testing. All test samples were prepared from the stock solutions on the day of
analysis. All purified DNA was used the same day it was extracted and purified. Each set of
replicates for a sample came from the same batch of purified DNA.
-------
Table 3-1. Infective/Lethal Dose of Target Contaminants
Contaminant
F. tularensis
Y. pestis
B. anthracis
Brucella suis
E. coli
Disease
Caused by
Contaminant
Tularemia
Plague
Anthrax
Brucellosis
NA
Infective/Lethal Dose Concentration
(cfu/mL)
4xl05
0.28
200
40
0.2
NA = not applicable
3.2 Test Samples
Test samples used in this verification test included PT samples, DW samples, and quality control
(QC) samples. Each type of test sample, including QC samples, is described further below.
3.2.1 Performance Test Samples
Table 3-2 lists the PT samples analyzed in this verification test for each bacteria. The bacteria
were added individually to each spiked sample. PT samples were prepared in ASTM Type n DI
water. The first type of PT sample consisted of ASTM Type n DI water spiked at five
concentration levels of each individual contaminant. The contaminant PT sample concentrations
ranged from the infective/lethal dose concentration to 50 times the vendor-stated system LOD.
The infective/lethal dose concentration was analyzed to document the response of the
R.A.P.I.D.® System at that important concentration level. Four concentration levels at 2, 5, 10,
and 50 times the vendor-reported system LOD, in addition to the infective/lethal dose
concentration, were analyzed. Each concentration level for the PT samples was analyzed in
quadruplicate.
The second type of PT sample was potential interferent samples. Four replicates of each
interferent PT sample were analyzed to determine the performance of the R. A.P.I.D.® System in
the presence of humic and fulvic acids. The interferent PT samples contained humic and fulvic
acids isolated from Elliot Soil near Joliett, IL, (obtained from the International Humic
Substances Society) spiked into ASTM Type n DI water. Each of these interferent mixtures was
prepared at two concentration levels. One concentration was near the upper limit of what would
be expected in DW [5 milligrams per liter (mg)/L] and one was at a mid-low range of what
would be expected (1 mg/L). The 1 mg/L interferent mixture was prepared as 0.5 mg/L humic
acid and 0.5 mg/L fulvic acid. Similarly, the 5 mg/L interferent solution was prepared as 2.5
mg/L humic acid and 2.5 mg/L fulvic acid. These interferent levels were confirmed through
analysis of aliquots by ATEL. Also, each bacteria was added separately to these samples, along
with the potential interferent, at a concentration of 10 times the system LOD and analyzed in
quadruplicate.
-------
Table 3-2. Performance Test Samples
Type of PT
Sample
Sample Characteristics
Approximate Concentrations
(cfu/mL)
Contaminant-only
F. tularensis
Y. pestis
B. anthracis
Brucella suis
E. coli
2xl03to5xl04
0.28 to 5xl04
200to5xl04
40to5xl04
0.2to5xl04
Interferent
Contaminants in 0.5 mg/L humic
acid and 0.5 mg/L fulvic acid
F. tularensis—IxlO4
Y. pestis— IxlO4
B. anthracis—IxlO4
Brucella suis—IxlO4
E. coli—IxlO4
Contaminants in 2.5 mg/L humic
acid and 2.5 mg/L fulvic acid
F. tularensis—IxlO4
Y. pestis— IxlO4
B. anthracis—IxlO4
Brucella suis—IxlO4
E. coli—IxlO4
In all cases, four replicates for each PT sample, DW sample, and QC sample were taken from the
extracted and purified product (unspiked) or DNA (spiked) of one sample solution. That is, only
one spiked or unspiked sample solution was prepared for each set of replicates and taken
through the DNA extraction and purification procedure. Four replicates were then taken from
the same purified product or DNA. In an effort to characterize the efficacy of the extraction and
purification procedure in the presence of inhibitory substances (humic and fulvic acids), four
solutions of humic and fulvic acids at 0.5 mg/L spiked with each contaminant at 10 times the
system LOD, were prepared in addition to the samples listed in Table 3-2. Each solution was put
through the DNA extraction and purification procedure, and then four replicates from each of
the four purified DNA solutions were analyzed on the R. A.P.I.D.® System.
3.2.2 Drinking Water Samples
Table 3-3 lists the DW samples analyzed for each bacteria in this test. Drinking water samples
were collected from four geographically distributed municipal sources (Ohio, California,
Florida, and New York) to evaluate the performance of the R. A.P.I.D.® System with various
sample matrices. These samples varied in their source and treatment and disinfection process.
All samples had undergone either chlorination or chloramination prior to receipt. Samples were
collected from utility systems with the following treatment and source characteristics:
• Chlorinated filtered surface water
• Chloraminated filtered surface water
• Chlorinated filtered groundwater
• Chlorinated unfiltered surface water.
-------
All samples were collected in pre-cleaned high density polyethylene containers. After sample
collection, to characterize the DW matrix, an aliquot of each DW sample was sent to ATEL to
determine the following water quality parameters: turbidity, organic carbon, conductivity,
alkalinity, pH, Ca, Mg, hardness, total organic halides, concentration of trihalomethanes, and
haloacetic acids. The DW samples were dechlorinated with sodium thiosulfate pentahydrate to
prevent the degradation of some of the contaminants by chlorine. Because real-world
applications of PCR technologies to screen water samples rely on pre-concentration of the water
sample to be analyzed, approximately 100 L of each of the above sources of DW were
dechlorinated and then concentrated through ultrafiltration techniques to a final volume of
250 mL by the Metropolitan Water District (MWD) of Southern California. As shown in
Table 3-3, each DW sample was analyzed without adding any contaminant (i.e., unspiked), as
well as after fortification with each individual contaminant at a single concentration level (10
times the vendor-stated system LOD).
Table 3-3. Drinking Water Samples
Drinking Water Sample Description
Water
Utility
Columbus,
Ohio (OH)
MWD of
Southern
California (CA)
Orlando,
Florida (FL)
New York
City, New
York (NY)
Water
Treatment
chlorinated
filtered
chloraminated
filtered
chlorinated
filtered
chlorinated
unfiltered
Source
Type
surface
surface
ground
surface
Approximate Contaminant Concentrations
(cfu/mL)
F. tularensis
unspiked and
IxlO4
unspiked and
IxlO4
unspiked and
IxlO4
unspiked and
IxlO4
Y. pestis
unspiked
and
IxlO4
unspiked
and
IxlO4
unspiked
and
IxlO4
unspiked
and
IxlO4
B. anthracis
unspiked and
IxlO4
unspiked and
IxlO4
unspiked and
IxlO4
unspiked and
IxlO4
Brucella suis
unspiked and
IxlO4
unspiked and
IxlO4
unspiked and
IxlO4
unspiked and
IxlO4
E. coli
unspiked
and
IxlO4
unspiked
and
IxlO4
unspiked
and
IxlO4
unspiked
and
IxlO4
3.2.3 Quality Control Samples
QC samples included method blank (MB) samples consisting of ASTM Type n DI water and
positive and negative controls, as provided by the vendor. All of the MB QC samples were
exposed to sample preparation and analysis procedures identical to the test samples. External
positive and negative controls were prepared and used according to the protocol provided by the
vendor. At least one positive and negative control were prepared with each batch of samples
placed on the R.A.P.I.D.® 7200 instrument. The MB samples were used to confirm negative
responses in the absence of any contaminant and to ensure that no sources of contamination
8
-------
were introduced into handling and analysis procedures. At least 10% of the test samples (eight
replicates) for each bacteria and target reagent set were MB samples. The vendor-provided
control samples indicated to the technician whether the R. A.P.I.D.® System was functioning
properly. If the controls failed for any reason, that batch of samples would be discarded and the
extracts reanalyzed. To the extent practicable, the test samples were analyzed blindly by having
the technician label the vials with only a sample number prior to the DNA purification step, so
that the samples were tracked through the purification, PCR, and detection steps by only a
sample number. Due to special facility use, the identity of the target bacteria was always known
by the technician.
3.3 Reference Methods
3.3.1 Plate Enumeration
For all contaminants, plate enumeration was used to quantify bacteria to confirm the
concentration of the stock solutions of these contaminants. The Battelle standard operating
procedure (SOP) followed was SOP No. MREF X-054, Standard Operating Procedure (SOP)
for the Enumeration ofBSL-2 and BSL-3 Bacteria Samples Via the Spread Plate Technique.
Prior to testing, the F. tularensis, Y. pestis, Brucella suis, and E. coli were grown and then
suspended in phosphate buffered saline (PBS). Twenty-five or more individual 1 mL aliquots of
stock solution were prepared from each original PBS stock solution. Three ImL aliquots were
randomly taken for enumeration, while the others were frozen for later use in sample
preparation. Each bacteria was enumerated on each of the three selected 1 mL aliquot to confirm
the determined concentration.
The B. anthracis came from a lot of spores prepared by Battelle and stored in a 1% stock
solution of phenol in water. Prior to testing, an aliquot of the B. anthracis solution described
above was centrifuged, the supernatant consisting of the phenol/water solution was decanted
from the spores, and the spores were reconstituted with DI water. This process was repeated two
times to ensure that the spores were suspended only in DI water. This DI water suspension of
spores was then aliquoted into 1 mL portions as with the F. tularensis, Y. pestis, Brucella suis,
and E. coli. Because of the known stability of B. anthracis spores and based on general facility
protocol, the aliquots were refrigerated instead of frozen. An aliquot was enumerated in triplicate
prior to testing to confirm the concentration. Another aliquot was enumerated during the
verification test to further verify the concentration of B. anthracis in the stock solution vials.
3.3.2 Drinking W
Because most of the contaminants tested can occur naturally in water, and because rapid PCR
technologies cannot distinguish between live and dead organisms, each unspiked concentrated
DW sample was plate enumerated to verify, to the extent practicable, the presence or absence of
the contaminant of interest. The samples were plated onto tryptic soy agar plates with 5% sheep
blood and incubated at 30 to 35°C. After 20 hours of incubation, the unspiked OH, CA, and NY
-------
DW samples produced lawns of bacteria with a level of contamination estimated to be greater
than lxl03cfu/mL. The unspiked FL DW sample showed only 10 to 100 cfu/mL estimated
concentration levels after 20 hours. After further incubation, the FL DW sample produced
bacteria at a concentration estimated to be greater than IxlO3 cfu/mL. Each DW sample had at
least three distinct types of bacteria growing. Gram stains were performed on any distinct colony
types visible in each sample to gain further insight into the colony morphology. For OH and CA
DW, three Gram negative bacteria colonies were identified. For NY, four Gram negative
colonies were identified; and, for FL, both Gram negative and positive colonies were present.
The CA DW was further evaluated for the presence of F. tularensis based on the potential
positive results for unspiked CA DW samples during the verification test. An aliquot of the
water was plated onto cystine heart agar (F. tularensis selective media) and incubated at 30 to
35°C. A single colony type (Gram negative rods) grew on the plates and was subjected to
biochemical tests (catalase, oxidase, p-lactamase, and urease) for the presumptive identification
of F. tularensis. The biochemical test results came back oxidase positive, indicating that the
bacteria were not F. tularensis. Further identification tests were not conducted on other DW
samples because no confirmed positive responses were detected in the remaining unspiked DW
samples.
3.4 Test Procedure
3.4.1 Sample Handling
All testing for this verification test was conducted within Battelle laboratories staffed with
technicians trained to safely handle F. tularensis, Y. pestis, B. anthracis, Brucella suis, and
E. coli bacteria. The technician operating the R.A.P.I.D.® System had prior PCR experience.
F. tularensis and E. coli samples were tested in a Biosafety Level 2 (BSL-2) laboratory; while
Y. pestis, B. anthracis, and Brucella suis samples were tested in a BSL-3 laboratory. Appropriate
safety guidelines associated with each laboratory were followed throughout the verification test.
Each day, fresh samples were prepared from a thawed vial of frozen or refrigerated stock
solution in either DI water, an interferent matrix, or a DW matrix. Concentration levels for
spiked samples at various multiples of the R.A.P.I.D.® System's LOD (2, 5, 10, and 50 times the
system LOD for PT samples, and 10 times the system LOD for interferent and DW samples)
were calculated from the system LOD provided by the vendor. Sample solutions were prepared
to these concentrations based on the concentration of the bacteria stock solution, which was
determined through triplicate plate enumeration prior to testing. Each sample was prepared in its
own container and labeled only with a sample identification (ID) number that also was recorded
in a laboratory record book along with details of the sample preparation. Samples were diluted to
the appropriate concentration using volumetric pipettes and glassware. Each sample was
prepared in 5 mL quantities.
Despite rigorous sample preparation efforts, solutions consisting of low bacterial concentrations,
such as the Brucella suis or Y. pestis infective/lethal dose, may have no DNA present in a given
sample or aliquot.(3>4) The rationale for this is based on the Poisson statistical distribution, where
there is some probability that a sample taken will contain no particles (i.e., bacteria or target
10
-------
DNA) and thus yield a negative result.(3>4) As a practical example, assume that 1 mL contains
exactly five particles (i.e., bacteria or target DNA) of interest. If one takes ten 0.1 mL samples
and analyzes them, the maximum number of positives will be five out of the ten samples. From
this it follows that there will be at least five negatives. Random variation in the sampling will
cause this ratio to change. This verification test was not designed to differentiate between the
stochastic nature of the low concentration samples and the capabilities of the assays, but this
phenomenon should be noted.
3.4.2 Sample Preparation and Analysis
Three steps were carried out to test a liquid sample for the presence of F. tularensis, Y. pestis,
B. anthracis, Brucella suis, and E. coli bacteria: (1) DNA extraction and purification using the
ITI1-2-3 Flow Kit, (2) PCR setup using the freeze-dried reagents, and (3) PCR and analysis
using the R.A.P.I.D.® 7200 instrument and software. To perform these steps, the laboratory
work area was separated into three distinct areas: DNA extraction and purification was
performed in one area; the PCR setup, involving reconstituting the freeze-dried reagents and
loading the capillaries, was performed in a separate area; and loading and running the
instrument (the R.A.P.I.D.®) was done in another area. These steps are described below.
First, the DNA was extracted and purified from the sample using the ITI 1-2-3 Flow Kit. The
entire 5 mL sample was taken through this purification procedure. The kit instructions specific
to processing a water sample were followed. After the purification step was complete, the PCR
samples were prepared in another area using the freeze-dried reagents. This process involved
reconstituting the freeze-dried reagents. The reagents for each sample and control placed on the
R.A.P.I.D.® 7200 instrument come in individually sealed vials labeled as "positive," "negative,"
and "unknown." First, the negative controls were reconstituted by combining 40 |iL of provided
sterile PCR grade water with the freeze-dried reagent in the "negative" vial. The reconstituted
negative control was then split into two capillaries, which were capped and briefly centrifuged to
pull all of the liquid into the bottom of the capillaries. The test samples were prepared next. For
the sample preparation, the "unknown" freeze-dried reagent vial specific to the bacteria being
tested was used. A volume of 20 |iL of purified sample DNA and 20 |iL of sterile PCR grade
water were combined with the freeze-dried reagents in the vial. As with the negative controls,
the sample "unknown" reagent mix was split into two capillaries, capped, and then briefly
centrifuged. Each test sample replicate was prepared using an individual pathogen-specific
"unknown" freeze-dried reagent vial and then split into two capillaries per the R. A.P.I.D.®
protocol. After all of the samples were prepared, the positive control was prepared in the same
manner as the negative control, only using the "positive" freeze-dried reagent vial. The controls
and "unknown" freeze-dried reagent vials came packaged together for each target.
Once all of the capillaries were filled and capped, they were loaded onto the R.A.P.I.D.® 7200
instrument carousel and the PCR run was started per the instructions provided with the
instrument. The carousel can hold up to 32 capillaries, or two split controls (positive and
negative) and 14 split samples. For this verification test, to ensure that an entire set of replicates
for a sample was run at the same time, three sets of four replicates along with positive and
negative controls were generally placed on the carousel for one PCR run. Three sets of four
sample replicates and positive and negative controls are considered a batch. The PCR program
11
-------
was loaded using the Advanced Options "Run" feature (version 1.2.14). After the R.A.P.I.D.®
7200 instrument had completed its PCR program run, which consisted of 45 cycles on the
thermal cycler, the results were analyzed using the R. A.P.I.D.® software. For the purposes of this
test, the Advanced Options LCDA (LightCycler Data Analysis) (version 3.5.28) quantification
function was used to interpret the results. In this data analysis option, the amplification data for
each capillary are plotted as fluorescence versus cycle number (from the thermal cycler
program). The Second Derivative Maximum method was used to determine a crossing point, or
the fractional cycle number where the sample fluorescence is differentiated from the background
fluorescence. The baseline adjustment was made by the software using the "Arithmetic" setting,
and the noise band (discrimination between noise and actual amplification in the fluorescence
curves) was set automatically by the software. The crossing point was generated by the software
based on the second derivative maximum value of each amplification curve. The resulting
amplification plots (plots of fluorescence versus the thermal cycler cycle number) and crossing
point values were used to determine the results for each sample. A sample was considered
positive (bacteria detected in the sample) if a crossing point was assigned to that sample and the
amplification curve for that sample was above the baseline. A sample was considered negative
(no bacteria detected in the sample) when no crossing point value was assigned by the software
and the amplification curve was along the baseline with no sign of exponential increase. A
sample was also considered negative if a crossing point value was assigned to the sample but no
amplification above the baseline was apparent for that sample's amplification curve. Samples
were considered positive or negative only if both capillaries from the split analysis of the sample
showed positive or negative results. Results for a sample were deemed inconclusive when the
two capillaries for the sample did not agree with each other (one was positive and one was
negative). The negative controls were considered successful if no amplification was present (and
thus no crossing point was assigned to the sample). The positive controls were considered
successful if amplification above the baseline was noted on the plot and a crossing point was
assigned to the samples. The technician recorded the sample ID number on a sample data sheet
along with the qualitative results (positive or negative) for each sample.
For F. tularensis, Y. pestis, and B. anthracis, more than one assay (freeze-dried reagent target)
was used for the analysis of each bacteria. In other words, multiple sets of "unknown" and
control assays were used to determine the presence or absence of the same bacteria in the PT and
DW samples. Each set of assays for a particular bacteria were specific to different gene targets
for that bacteria. Three assays were tested for B. anthracis (Target 1, Target 2, and Target 3),
while two assays were tested for F. tularensis and Y. pestis (Target 1 and Target 2 for each). In
each case where multiple freeze-dried reagent targets were tested for a given bacteria, the
replicates for each sample tested for each target came from the same purified sample DNA. For
example, when the infective/lethal dose sample for B. anthracis was tested, the same batch of
purified sample DNA was used to analyze four replicates for Target 1, four replicates for
Target 2, and four replicates for Target 3 for the B. anthracis reagents.
3.4.3 Drinking Water Characterization
An aliquot of each DW sample, collected as described in Section 3.2.2, was sent to ATEL prior
to concentration to determine the following water quality parameters: turbidity; concentration of
dissolved and total organic carbon; conductivity; alkalinity; pH; concentration of Ca and Mg;
12
-------
hardness; and concentration of total organic halides, trihalomethanes, and haloacetic acids.
Table 3-4 lists the methods used to characterize the DW samples, as well as the characterization
data from the four water samples used in this verification test. Water samples were collected and
water quality parameters were measured by ATEL in January 2004. Some of the water quality
parameters may have changed slightly prior to verification testing.
Table 3-4. ATEL Water Quality Characterization of Drinking Water Samples
Sources of Drinking Water Samples
Parameter
Turbidity
Dissolved organic
carbon
Total organic
carbon
Specific
conductivity
Alkalinity
PH
Ca
Mg
Hardness
Total organic
halides
Trihalomethanes
Haloacetic acids
Unit
NTU
mg/L
mg/L
micro-Siemens
mg/L
mg/L
mg/L
mg/L
Hg/L
|ig/L/analyte
|ig/L/analyte
Method
EPA180.1(5)
SM5310(6)
SM5310(6)
SM2510(6)
SM 2320(6)
EPA150.1(7)
EPA 200.8(8)
EPA 200.8(8)
EPA 130.2(7)
SM 5320(6)
EPA 524.2(9)
EPA 552.2(10)
Columbus,
Ohio
(OH DW)
0.2
1.9
1.6
357
55
7.33
42
5.9
125
360
26.9
23.2
MWD,
California
(CA DW)
0.1
2.3
2.1
740
90
7.91
35
1.5
161
370
79.7
17.6
Orlando,
Florida
(FL DW)
0.5
1.7
1.8
325
124
7.93
41
8.4
137
370
80.9
41.1
New York City,
New York
(NY DW)
1.3
1.5
2.1
85
4
6.80
5.7
19
28
310
38.4
40.3
NTU = nephelometric turbidity unit
[ig = microgram
13
-------
Chapter 4
Quality Assurance/Quality Control
Quality assurance/quality control procedures were performed in accordance with the quality
management plan (QMP) for the AMS Center(11) and the test/QA plan for this verification test.(1)
4.1 Sample Chain-of Custody Procedures
Sample custody was documented throughout collection, shipping, and analysis of the samples.
Sample chain-of-custody procedures were generally those provided in the guidelines in
ASAT.n-007, Standard Operating Procedure for Chain of Custody for Dioxin/Furan Analysis.
The chain-of-custody forms summarized the samples collected and analyses requested and were
signed by the person relinquishing samples once that person had verified that the custody forms
were accurate. The original sample custody forms accompanied the samples; the shipper kept a
copy. Upon receipt at the sample destination, sample custody forms were signed by the person
receiving the samples once that person had verified that all samples identified on the custody
forms were present in the shipping container.
4.2 Equipment Calibration
The R. A.P.I.D.® System and all associated reagents and supplies specific for the detection of
F. tularensis, Y. pestis, B. anthracis, Brucella suis, and E. coll were provided to Battelle by the
vendor. This system required no calibration. The performance of the system was monitored
through positive and negative controls. For DW characterization and confirmation of the
possible interferent, analytical equipment was calibrated by ATEL according to the procedures
specified in the appropriate standard methods. Pipettes used during the verification test were
calibrated according to Battelle SOP VI-025, Operation, Calibration, and Maintaining Fixed
and Adjustable Volume Pipettes.
After completion of the verification test, a Rainin pipette Lite 200 used during most of the
testing in the BSL-3 laboratory (for Y. pestis, B. anthracis, and Brucella suis) was found to be
out of calibration. The pipette was found to deliver liquids in excess of the volume specified on
the pipette. It is not known if the pipette went out of calibration before, during, or after the
verification test. The pipette was used in two steps: sample solution preparation and to deliver
liquids to reconstitute the freeze-dried reagents. There is no definitive evidence to indicate that
the pipette did or did not affect the results for the BSL-3 bacteria.
14
-------
4.3 Characterization of Contaminant Stock Solutions
F. tularensis, Y. pestis, B. anthracis, Brucella suis, and E. coli were grown and prepared by
Battelle. All bacteria were plate enumerated in triplicate for confirmation of the concentration of
the 1 mL aliquot stock solutions. Prior to enumeration, the B. anthracis, originally stored as a
1 % stock solution of phenol in water, was aliquoted and washed twice with DI water and
resuspended in only DI water for analysis.
The lot of B. anthracis Ames Strain spores used for this verification test were characterized in
September 2003 by Battelle and the Centers for Disease Control and Prevention. This
characterization involved evaluation of 11 criteria, including the percent of vegetative cells
present, the viable spore count, the guinea pig 10 day LD50, as well as DNA fingerprinting and
gene sequencing. This lot of spores met all 11 acceptance criteria, proving that the spores were
viable and of the specified strain (Ames). The vegetative cell count indicated that the stock
solution of spores was 99.94% pure spores, with only 0.06% of the solution containing
vegetative cells.
The Battelle SOP No. MREF X-054, Standard Operating Procedure (SOP) for the Enumeration
ofBSL-2 and BSL-3 Bacteria Samples Via the Spread Plate Technique, was followed for the
plate enumeration of F. tularensis, Y. pestis, B. anthracis, Brucella suis, and E. coli. The results
of the plate enumerations for each bacteria are presented in Table 4-1. For all bacteria, the plate
enumeration was conducted prior to testing. Because the B. anthracis stock solutions were
stored at 2 to 8 °C, another 1 mL aliquot stock solution vial was enumerated during testing
activities to further confirm the concentration of the stock solutions. The average of triplicate
enumerations for each bacteria was used to calculate and prepare all spiked sample solutions.
The percent difference between the concentration of the initial preparation of B. anthracis spores
and the second analysis of these spore during testing was 23%. Because this difference falls
within the bounds of expected plate enumeration error and is close to the standard deviations
found for the plate enumerations of other bacteria used in this verification test, the concentration
determined from the initial set of plate enumerations on the B. anthracis spores was used in
calculating solution concentrations.
4.4 Quality Control Samples
MB samples consisting of ASTM Type n DI water, and positive and negative control samples,
as provided in the R. A.P.I.D.® System, were analyzed to help identify potential cross-
contamination issues as well as verify that the PCR process was functioning properly. Positive
and negative control samples were run with each set of samples placed on the R.A.P.I.D.® 7200
instrument. Eight MB replicates were analyzed over the course of the verification test for each
bacteria (or bacteria assay).
Each set of eight MB sample replicates for all Y. pestis, B. anthracis, Brucella suis, and E. coli
targets returned negative results. One set of MB replicates for the F. tularensis Target 1 returned
one positive and two inconclusive results, indicating possible contamination. Other samples on
15
-------
Table 4-1. Contaminant Triplicate Plate Enumeration Data
Plate 1 Plate 2 Plate 3 . Relative
Averase
Bacteria Concentration Concentration Concentration * Standard
(cfu/mL) (cfu/mL) (cfu/mL) Deviation
£ tularensis
Y. pestis
B. anthracis
(initial prep)
B. anthracis
(second
analysis)
Brucella suis
E. coli
l.OxlO9
5.8xl07
8.7xl07
5.7xl07
1.5xl010
7.3xl08
l.lxlO9
6.5xl07
8.1xl07
5.7xl07
1.6xl010
5.0xl08
1.2xl09
5.0xl07
7.8xl07
7.6xl07
1.4xl010
9.0xl08
l.lxlO9
5.8xl07
8.2xl07
6.3xl07
1.5xl010
7.1xl08
9%
13%
6%
17%
7%
28%
that day's testing did not show signs of contamination, and the second F. tularensis target ran
from the same purified DNA returned all negative results for the four replicates.
No positive controls failed during the verification test. Three negative controls failed over the
entire course of testing. In these instances, the batch of samples was loaded again and rerun
using new reagents. For two batches of samples, the carousel was misaligned, producing
incomprehensible results. These batches of samples were reran. The results from all reruns were
used in the data analysis.
4.5 Audits
4.5.1 Technical Systems Audit
The Battelle Quality Manager conducted a technical systems audit (TSA) on June 11, 2004, to
ensure that the verification test was performed in accordance with the test/QA plan(1) and the
AMS Center QMP.(11) As part of the audit, the Battelle Quality Manager reviewed the standards
and methods used, compared actual test procedures to those specified in the test/QA plan, and
reviewed data acquisition and handling procedures. Observations and findings from this audit
were documented and submitted to the Verification Test Coordinator for response. No findings
were documented that required any significant action. The records concerning the TSA are
stored for at least seven years with the Battelle Quality Manager.
4.5.2 Audit of Data Quality
At least 10% of the data acquired during the verification test was audited. Battelle's Quality
Manager traced the data from the initial acquisition, through reduction and statistical analysis,
16
-------
to final reporting, to ensure the integrity of the reported results. All calculations performed on
the data undergoing the audit were checked.
4.6 QA/QC Reporting
Each assessment and audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the
QMP for the ETV AMS Center.(11) Once the assessment report was prepared, the Verification
Test Coordinator ensured that a response was provided for each adverse finding or potential
problem and implemented any necessary follow-up corrective action. The Battelle Quality
Manager ensured that follow-up corrective action was taken. The results of the TS A were sent to
the EPA.
4.7 Data Review
Records generated in the verification test were reviewed before these records were used to
calculate, evaluate, or report verification results. Table 4-2 summarizes the types of data
recorded. The review was performed by a Battelle technical staff member involved in the
verification test, but not the staff member that originally generated the record. The person
performing the review added his/her initials and the date to a hard copy of the record being
reviewed.
Table 4-2. Data Recording Process
Data to Be
Recorded
Where Recorded How Often Recorded
Disposition of Data(a)
Dates and times of
test events
ETV data sheets
Start/end of test and at
each change of a test
parameter
Used to organize/check test
results; manually incorporated
in data spreadsheets as
necessary
Sample collection
and preparation
information,
including chain-of-
custody
ETV data sheets
and chain-of-
custody forms
At time of sample
collection and
preparation
Used to organize/check test
results; manually incorporated
in data spreadsheets as
necessary
R.A.P.I.D.®
System procedures
and sample results
ETV data sheets
and data
acquisition system
Throughout test
duration
Manually incorporated in data
spreadsheets
Enumeration data
Enumeration data
forms and ETV
data sheets
With every
enumeration
Used to organize/check test
results
Reference method
procedures and
sample results
Data acquisition
system, as
appropriate
Throughout sample
analysis process
Transferred to spreadsheets
) All activities subsequent to data recording were carried out by Battelle, except for the reference method analyses
(DW characterization), which were carried out by ATEL.
17
-------
Chapter 5
Data Analysis
The R.A.P.I.D.® System was evaluated for qualitative results (i.e., positive/negative responses to
samples) based on the expected application of rapid PCR technologies as rapid screening tools.
All data analyses were based on these qualitative results. QC and MB samples were not included
in any of the analyses.
5.1 Accuracy
Accuracy was assessed by evaluating how often the R.A.P.I.D.® System results were positive in
the presence of a concentration of contaminant above the system LOD. Contaminant-only PT
samples were used for this analysis. An overall percent agreement was determined by dividing
the number of positive responses by the overall number of analyses of contaminant-only PT
samples above the system LOD.
5.2 Specificity
The ability of the R. A.P.I.D.® System to provide a negative response when the contaminant was
absent was assessed. The specificity rate was determined by dividing the number of negative
responses by the total number of unspiked samples.
5.3 False Positive/Negative Responses
A false positive response was defined as a detectable or positive R. A.P.I.D.® System response
when the ASTM Type n DI water (including interferent samples) or DW samples were not
spiked. A false positive rate was reported as the frequency of false positive results out of the
total number of unspiked samples.
A false negative response was defined as a non-detectable response or negative response when
the sample was spiked with a contaminant at a concentration greater than the system LOD.
Spiked PT (contaminant and interferent) samples and spiked DW samples were included in the
analysis. A false negative rate was evaluated as the frequency of false negative results out of the
total number of spiked samples for a particular contaminant.
18
-------
5.4 Precision
The precision of the four replicates of each sample set were assessed. Responses were
considered consistent if all four replicates gave the same result. The precision of the R.A.P.I.D.®
System was assessed by calculating the overall number of consistent responses for all the sample
sets.
5.5 Interferences
The potential effect of the DW matrix on the R. A.P.I.D.® System performance was evaluated
qualitatively by comparing the results for the spiked and unspiked DW samples to those for the
PT samples. Similarly, the potential effect of interferent PT samples containing humic and fulvic
acids at two levels, both spiked and not spiked with bacteria, were evaluated.
5.6 Other Performance Factors
Aspects of the R. A.P.I.D.® System performance such as ease of use and sample throughput are
discussed in Section 6. Also addressed are qualitative observations of the verification staff
pertaining to the performance of the R. A.P.I.D.® System.
19
-------
Chapter 6
Test Results
The results for the R. A.P.I.D.® System were evaluated based on the responses provided by the
R.A.P.I.D.® software Advanced Options LCDA quantification analysis Second Derivative
Maximum output. The R.A.P.I.D.® software offers both an Advanced Options/LCDA as well as
a simplified Detector window to view the PCR results. The Detector view is an auto-analysis
feature that provides a "Present'VNot Detected" response for the sample in each capillary in a
single table. The Advanced Options interface provides more information about the amplification
of each sample, including fluorescence curves and crossing points. In a testing scenario, if the
Detector option was used and a "Present" response was given, Idaho Technology would train the
user to proceed to the LCDA view to further evaluate the data. In the interest of maintaining
consistency in data interpretation, the LCDA interface was used for the interpretation of all
sample results.
An example amplification plot from the LCDA analysis for B. anthracis is presented in
Figure 6-1. The plot displays the fluorescence versus the thermal cycler cycle number. In this
example, all of the samples are displayed at once and assigned a different color. Using the
R. A.P.I.D.® software, the number of samples displayed on the amplification plot can be
controlled by the operator. Only qualitative (positive/negative) responses were recorded for each
sample. To determine the results of each sample, the crossing point values as well as the
amplification curves of each sample were monitored. The crossing point value, a fractional cycle
number where the sample fluorescence is differentiated from the background fluorescence, is
determined by the software and based on the second derivative maximum of the amplification
curve, or the point on the curve where the rate of fluorescence changes the fastest. A sample was
considered positive (bacteria detected in the sample) if a crossing point was assigned to that
sample and the amplification curve for that sample was above the baseline. A sample was
considered negative (no bacteria detected in the sample) when no crossing point value was
assigned by the software and the amplification curve was along the baseline with no sign of
exponential increase. A sample was also considered negative if a crossing point value was
assigned to the sample but no amplification above the baseline was apparent for that sample's
amplification curve. Samples were considered positive or negative only if both capillaries from
the split analysis of the sample showed positive or negative results. For the purposes of this test,
amplification curves that were above the baseline and assigned a crossing point by the software
were considered positive, regardless of the cycle number at which they crossed. In some cases,
many crossing point values were high (i.e., close to 40 cycles). Often, such high values,
20
-------
Baseline
•2-,
I
10
15
20 25
Cycle Number
I
30
35
40
I
45
Figure 6-1. LCDA Amplification Plot Samples Analyzed Using B. anthracis Target 2. The
amplification for each sample is represented by a different color.
-------
particularly when the duplicate (or split) samples do not agree and amplification is only slightly
above the baseline, might lead to suspect amplification and might not be considered by some to
be a conclusive positive sample without further analyses. Results for a sample were deemed
inconclusive when the two capillaries for the sample did not agree with each other (one was
positive and one was negative). For the purposes of this test, such samples were not considered
positive or negative. In a real-world scenario, Idaho Technology recommends that samples with
inconclusive results be rerun to confirm the presence or absence of the bacteria of interest.
Positive and negative controls were monitored with each set of samples placed on the
R.A.P.I.D.® 7200 instrument. No positive controls failed. Three negative controls were
unsuccessful throughout the entire verification test (for all bacteria). When the negative controls
failed, the samples were prepared again (using the freeze-dried reagents), loaded onto the
carousel, and run again.
6.1 Accuracy
The results for the R. A.P.I.D.® System using the contaminant-only PT samples containing
F. tularensis, Y. pestis, B. anthracis, Brucella suis, and E. coll are discussed in this section. The
infective/lethal dose samples for each bacteria are included in the contaminant-only PT samples.
In the case of Y. pestis, B. anthracis, Brucella suis, and E. coli, the infective/lethal doses (see
Table 3-1) were below the vendor-stated system LOD. The results for each bacteria at the
infective/lethal dose are presented in the following tables, but those for Y. pestis, B. anthracis,
Brucella suis, and E. coli were not included in the overall accuracy calculations for those
bacteria.
6.1.1 F. tularensis
Two targets were tested for samples containing F. tularensis, Target 1 and Target 2. The results
obtained for the PT samples containing F. tularensis for both targets are given in Table 6-la. All
concentration levels analyzed for both targets generated 4 out of 4 positive responses for each set
of replicate samples. An overall percent agreement was determined by dividing the number of
positive responses by the overall number of analyses of contaminant-only PT samples. This
resulted in 100% agreement for the overall accuracy of the R.A.P.I.D.® System for the detection
of F. tularensis using both Target 1 and Target 2 reagents.
6.1.2 Y. pestis
The R. A.P.I.D.® System has two assays for Y. pestis, Target 1 and Target 2. The results obtained
for the PT samples containing Y. pestis in both targets are given in Table 6-lb. All samples with
concentration levels above the vendor-stated system LOD generated positive responses using
both Target 1 and Target 2. The infective/lethal dose of Y. pestis (0.28 cfu/mL) was below the
R. A.P.I.D.® System LOD for this bacteria and produced no positive responses in four replicates
in either target. An overall percent agreement was determined by dividing the number of positive
22
-------
Table 6-la. F. tularensis Contaminant-Only PT Sample Results
Sample Type
PT Samples
Overall Accuracy
Concentration*3'
(cfu/mL)
4xl05(b)
2xl03
5xl03
IxlO4
5xl04
Positive Results Out of Total Replicates
Target 1
4/4
4/4
4/4
4/4
4/4
100% (20/20)
Target 2
4/4
4/4
4/4
4/4
4/4
100% (20/20)
-------
resulted in positive responses, while one replicate at this concentration level resulted in a
positive response for one capillary and a negative response for the other capillary of the split
replicate sample. This indicated inconclusive results for this replicate (i.e., the sample could be
declared neither positive nor negative). Idaho Technology indicated that pipetting error when
splitting the sample into two capillaries can cause mixed results for the split sample, as can
Poisson distribution effects at low bacterial concentration levels. In a screening scenario,
inconclusive results would lead to further testing of the sample, but this was beyond the scope of
this test. The infective/lethal dose of B. anthracis for Target 1 reagents resulted in three
inconclusive and one negative response. The overall accuracy of the R. A.P.I.D.® System using
Target 1 in detecting B. anthracis above the R.A.P.I.D.® System LOD was 94% (15/16).
As with Target 1, PT samples at 5, 10, and 50 times the vendor-provided system LOD generated
4 out of 4 positive responses for Target 2. Three of the four replicates at 2xl03 cfu/mL resulted in
positive responses, while one replicate at this concentration level resulted in a negative response.
At the infective/lethal dose for Target 2, two replicates produced inconclusive results, while the
remaining two replicates resulted in negative responses. The overall accuracy of the R. A.P.I.D.®
System using Target 2 in detecting B. anthracis above the R.A.P.I.D.® System LOD was 94%
(15/16).
Table 6-lc. B. anthracis Contaminant-Only PT Sample Results
Inconclusive Results(c)
Sample Type
PT
Samples
Concentration^
(cfu/mL)
200(b)
2xl03
5xl03
IxlO4
5xl04
Target
1
3
1
0
0
0
Target
2
2
0
0
0
0
Target
3
2
0
0
0
0
Overall
Accuracy(d)
Positive Results Out of
Total Replicates
Target
1
0/4
3/4
4/4
4/4
4/4
94%
(15/16)
Target
2
0/4
3/4
4/4
4/4
4/4
94%
(15/16)
Target
3
0/4
4/4(e)
4/4
4/4
4/4
100%
(16/16)
(a) Sample solutions were prepared at 2, 5, 10, and 50 times the vendor-stated system LOD from stock solutions based
on the enumeration data (see Table 4-1).
(b) Infective/lethal dose—below the R.A.P.I.D.® System LOD for B. anthracis.
(c) Inconclusive results are one positive and one negative result for each capillary of the split sample for a given
replicate. They were considered neither positive nor negative.
(d) Excludes infective/lethal dose concentration, which was below the system LOD.
(e) Samples at 2xl03 cfu/mL had to be rerun for Target 3 because of suspected sample preparation problems.Only the
rerun results are presented here.
24
-------
Using Target 3, all samples containing B. anthracis at concentration levels above the vendor-
stated system LOD generated 4 out of 4 positive responses. The infective/lethal dose replicates
for Target 3 reagents resulted in two inconclusive and two negative responses. The overall
accuracy of the R. A.P.I.D.® System using Target 3 in detecting B. anthracis above the
R.A.P.I.D.® System LOD was 100% (16/16).
6.1.4 Brucella suis
One R. A.P.I.D.® System target was tested for Brucella suis. The results obtained for the PT
samples containing Brucella suis are given in Table 6-Id. Contaminant-only PT samples at 5,
10, and 50 times the vendor-provided system LOD generated all positive responses. Two of the
four replicates at 2xl03 cfu/mL resulted in positive responses, while the remaining two replicates
at this concentration level resulted in inconclusive responses (one positive and one negative for
each split sample). The infective/lethal dose of Brucella suis was below the R. A.P.I.D.® System
LOD for this bacteria. The infective/lethal dose replicates generated three negative and one
inconclusive response.
The overall accuracy in detecting Brucella suis above the R.A.P.I.D.® System LOD was 88%
(14/16). The infective/lethal dose of Brucella suis was not included in this calculation because it
was below the R. A.P.I.D.® System LOD for this bacteria.
Table 6-ld. Brucella suis Contaminant-Only PT Sample Results
Sample Concentration*3'
Type (cfu/mL)
PT
Samples
40(b)
2xl03
5xl03
IxlO4
5xl04
Inconclusive Results'*0
1
2
0
0
0
Overall Accuracy
Positive Results Out of
Total Replicates
0/4
2/4
4/4
4/4
4/4
88% (14/16)(d)
-------
(0.2 cfu/mL) was below the R. A.P.I.D.® System LOD for this bacteria and produced no positive
responses in four replicates. All responses at the infective/lethal dose were negative. There was
100% agreement for the overall accuracy in detecting E. coli above the R.A.P.I.D.® System
LOD.
Table 6-le. E. coli Contaminant-Only PT Sample Results
Concentration'3' Positive Results Out of
Sample Type (cfu/mL) Total Replicates
0.2°° 0/4
2xl03 4/4
PT Samples 5xl03 4/4
IxlO4 4/4
5xl04 4/4
Overall Accuracy 100% (16/16)(c)
(a) Sample solutions were prepared at 2, 5, 10, and 50 times the vendor-stated system LOD from stock solutions based
on the enumeration data (see Table 4-1).
(b) Infective/lethal dose—below the R.A.P.I.D.® System LOD for E. coli.
(c) Excludes infective/lethal dose concentration, which was below the system LOD.
6.2 Specificity
Specificity assesses the R.A.P.I.D.® System's ability to provide a negative response when the
contaminant was absent. The results from all unspiked interferent PT samples and unspiked DW
samples are presented in this section. Negative results out of total replicates are presented in
each table.
6.2.1 F. tularensis
The results obtained for F. tularensis Target 1 and Target 2 for the unspiked interferent and DW
samples are given in Table 6-2a. All unspiked DW samples for Target 1 showed negative
responses. For the interferent samples, all four replicates of unspiked 0.5 mg/L humic and fulvic
acids generated negative results. Only the unspiked 2.5 mg/L humic and fulvic acid replicates
for Target 1 showed inconsistent negative responses. Three of the replicates for unspiked 2.5
mg/L humic and fulvic acids had one positive and one negative response for each capillary of
the split sample. This indicated inconclusive results for these replicates (i.e., the sample could be
declared neither positive nor negative). In general, the positive results of the split samples (for
the inconclusive results) had late-cycle crossing points (-38 cycles) with the fluorescence barely
above the baseline for many of the results. The fourth replicate was negative for this set.
All unspiked interferent PT samples showed negative responses for Target 2. Similarly, OH, FL,
and NY unspiked DW samples showed negative responses, indicating that the bacteria was not
26
-------
present in these samples, as would be expected. For the unspiked CA DW samples, one replicate
resulted in one positive and one negative response for the split sample. This indicated
inconclusive results for this replicate. The remaining three replicates were negative. The CA DW
was further analyzed to determine the presence (or absence) of F. tularensis naturally in the
water (see section 3.3.2). F. tularensis could not be identified in the sample.
An overall specificity rate was determined by dividing the number of negative responses by the
overall number of analyses of unspiked samples. This resulted in 88% agreement for the overall
specificity of the R.A.P.I.D.® System for F. tularensis Target 1, and 96% agreement for the
overall specificity for Target 2.
Table 6-2a. F. tularensis Specificity Results
Negative Results Out of
Total Replicates
Sample Type
Interferent PT Samples
DW Samples
Overall Specificity
Sample
0.5 mg/L humic acid and
0.5 mg/L fulvic acid, unspiked
2.5 mg/L humic acid and
2.5 mg/L fulvic acid, unspiked
OH DW, unspiked
CA DW, unspiked
FL DW, unspiked
NY DW, unspiked
Target 1
4/4
4/4
4/4
4/4
4/4
88%(21/24)(a)
Target 2
4/4
4/4
4/4
3/4(b)
4/4
4/4
96% (23/24)(b)
(a) Three 2.5 mg/L humic and fulvic acid replicates had one positive and one negative result for each capillary of the
split sample. These were inconclusive results.
(b) One CA DW replicate had one positive and one negative results for each capillary of the split sample. This
indicated an inconclusive result.
6.2.2 Y.pestis
The results obtained for Y. pestis Target 1 and Target 2 for the analysis of unspiked interferent
PT and DW samples are given in Table 6-2b. For both targets, all unspiked interferent PT
samples and unspiked DW samples generated negative responses for all of the replicates. This
resulted in 100% agreement for the overall specificity of the R.A.P.I.D.® System for Y. pestis
Target 1 and Target 2.
27
-------
Table 6-2b. Y. pestis Specificity Results
Negative Results Out of
Total Replicates
Sample Type
Sample
Target 1
Target 2
Interferent PT Samples
0.5 mg/L humic acid and
0.5 mg/L fulvic acid, unspiked
2.5 mg/L humic acid and
2.5 mg/L fulvic acid, unspiked
4/4(a)
4/4
(a)
4/4
4/4
DW Samples
Overall Specificity
OH DW, unspiked
CA DW, unspiked
FL DW, unspiked
NY DW, unspiked
4/4
4/4
4/4
4/4
100% (24/24)
4/4
4/4
4/4
4/4
100% (24/24)
(a)
These samples were rerun because of suspected sample preparation problems. Only the rerun results are presented
here.
6.2.3 B. anthracis
The results obtained using B. anthracis Targets 1, 2, and 3 for the analysis of unspiked
interferent and DW samples are given in Table 6-2c. All unspiked interferent PT samples and
unspiked DW samples showed negative responses for all of the replicates for all three targets. An
overall specificity rate was determined by dividing the number of negative responses by the
overall number of analyses of unspiked samples. This resulted in 100% agreement for the overall
specificity of the R.A.P.I.D.® System for each target (Targets 1, 2, and 3) for B. anthracis.
6.2.4 Brucella suis
The results obtained for the Brucella suis assay for the analysis of unspiked interferent and DW
samples are given in Table 6-2d. All unspiked interferent PT samples and unspiked DW samples
showed negative responses for all of the replicates. This resulted in 100% agreement for the
overall specificity of the R. A.P.I.D.® System for Brucella suis.
28
-------
Table 6-2c. B. anthracis Specificity Results
Sample Type
0,
Interferent PT Samples
2,
DW Samples
Overall Specificity
Table 6-2d. Brucella suis
Sample Type
Interferent PT Samples
DW Samples
Overall Specificity
Sample
.5 mg/L humic acid and 0.5 mg/L
fulvic acid, unspiked
.5 mg/L humic acid and 2.5 mg/L
fulvic acid, unspiked
OH DW, unspiked
CA DW, unspiked
FL DW, unspiked
NY DW, unspiked
Specificity Results
Sample
0.5 mg/L humic acid and
0.5 mg/L fulvic acid, unspiked
2.5 mg/L humic acid and
2.5 mg/L fulvic acid, unspiked
OH DW, unspiked
CA DW, unspiked
FL DW, unspiked
NY DW, unspiked
Negative Results Out of
Total Replicates
Target 1 Target 2 Target 3
4/4 4/4 4/4
4/4 4/4 4/4
4/4 4/4 4/4
4/4 4/4 4/4
4/4 4/4 4/4
4/4 4/4 4/4
100% 100% 100%
(24/24) (24/24) (24/24)
Negative Results Out of
Total Replicates
4/4
4/4
4/4
4/4
4/4
4/4
100% (24/24)
6.2.5 E. coli
The results obtained for the E. coli assay for unspiked interferent and DW samples are given in
Table 6-2e. All unspiked interferent PT samples and unspiked DW samples showed negative
responses for all of the replicates. An overall specificity rate was determined by dividing the
29
-------
number of negative responses by the overall number of analyses of unspiked samples. This
resulted in 100% agreement for the overall specificity of the R.A.P.I.D.® System for E. coli.
Table 6-2e. E. coli Specificity Results
Negative Results Out of
Sample Type Sample Total Replicates
0.5 mg/L humic acid and ...
0.5 mg/L fulvic acid, unspiked
Interferent PT Samples
2.5 mg/L humic acid and ...
2.5 mg/L fulvic acid, unspiked
OH DW, unspiked 4/4
CA DW, unspiked 4/4
DW Samples
FL DW, unspiked 4/4
NY DW, unspiked 4/4
Overall Specificity 100% (24/24)
6.3 False Positive/Negative Responses
Contaminant-only PT samples, interferent PT samples, and DW samples were evaluated to
determine false positive and false negative results for the R. A.P.I.D.® System. Included in the
calculations were the 16 additional interferent sample replicates (0.5 mg/L humic and fulvic
acids) per bacteria tested to determine the effects of the DNA extraction and purification on the
results. A false positive response was defined as a positive result when bacteria were not spiked
into the sample. A false negative response was defined as a negative result when the sample was
spiked with a contaminant at a concentration greater than the R. A.P.I.D.® System LOD for that
bacteria.
It should be noted that false positive responses cannot be absolutely confirmed as false because
there is a possibility of cross-contamination. All appropriate steps were taken throughout the
verification test to avoid this issue by using three separate work areas, by following daily
cleanup procedures, and by loading controls and samples into the capillaries in the appropriate
sequence (negative controls, samples, and then positive controls). However, cross-contamination
is always a possibility in any PCR process.(12) No appropriate reference method was available to
cross-check the amplified PCR product to confirm the R. A.P.I.D.® System responses. When
sample preparation error was suspected (e.g., sample appeared to be unspiked when it should
have been spiked, or spiked when it should have been blank), the sample was re-evaluated. If
sample preparation errors or cross-contamination were suspected after re-analysis, only the
results of the reruns were presented.
30
-------
6.3.1 F. tularensis
Tables 6-3a and 6-3b present F. tularensis Target 1 and Target 2 (respectively) false negative
and false positive results. The number of positive samples out of the total replicates analyzed is
presented in each table. Total negative and inconclusive results are also presented for each set of
replicates. No false positive samples were found in any of the sample matrices for either target.
No false negative results were found for Target 1. Four false negatives were found for Target 2:
one in a set of replicates for spiked 0.5 mg/L humic and fulvic acids, and three in a set of
replicates for spiked CA DW. Three replicates for unspiked 2.5 mg/L humic and fulvic acids
showed one negative and one positive result for each of the split sample for Target 1. Because
both results for a split sample did not agree, the results for the replicates were determined to be
inconclusive. Inconclusive results were also found for one replicate in spiked CA DW and one
replicate in unspiked CA DW using Target 2.
6.3.2 Y.pestis
Table 6-3c presents the false negative and false positive results for Y. pestis Target 1 and
Target 2. The number of positive samples out of the total replicates analyzed is presented in the
table. No false positive or false negative samples were found in any of the sample matrices for
either target. No inconclusive results were found in any of the sample matrices for either target.
6.3.3 B. anthracis
Tables 6-3d, 6-3e, and 6-3f present the false positive/negative results for B. anthracis Targets 1,
2, and 3 (respectively). The number of positive samples out of the total replicates analyzed is
presented in each table. The number of negative and inconclusive responses for each set of
replicates is also presented. No false positives were found in any of the sample matrices for
Targets 1, 2, and 3. For Target 1, two false negatives were found: one negative response for a
spiked NY DW replicate and one negative response for a spiked CA DW replicate. One replicate
each for spiked FL DW, spiked CA DW, and spiked DI water at 2xl03 cfu/mL showed one
positive and one negative response for the split sample, indicating a neither fully positive nor
fully negative response. The results for these replicates were determined to be inconclusive.
Target 2 also had two false negatives: one replicate of 2xl03 cfu/mL B. anthracis in DI water and
one replicate of spiked CA DW. Two inconclusive results were also found for spiked CA DW
replicates using Target 2 reagents. Target 3 reported no false negative results, although all four
spiked CA DW replicates generated inconclusive results.
6.3.4 Brucella suis
Table 6-3g presents the false negative/false positive results for Brucella suis. The number of
positive samples out of the total replicates analyzed is presented in the table. No false positive or
false negative samples were found in any of the sample matrices. Two replicates for DI water
spiked at 2xl03 cfu/mL showed one negative and one positive result for each of the split
samples. Because both results for a split sample did not agree, the results for these two replicates
were determined to be inconclusive.
31
-------
Table 6-3a. F. tularensis Target 1 False Positive/Negative Results
Sample Type
Contaminant-
Only PT
Samples
Interferent PT
Samples
DW Samples
Sample
DI water
DI water
DI water
DI water
DI water
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
OHDW
OHDW
CADW
CADW
FLOW
FLOW
NYDW
NYDW
Cone. (a)
(cfu/mL)
4xl05(c)
2xl03(d)
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Negative
Results
0
0
0
0
0
4
0
1
0
4
0
4
0
4
0
4
0
Inconclusive
Results (b)
0
0
0
0
0
0
0
3
0
0
0
0
0
0
0
0
0
False Positive Rate
False Negative
Rate
Positive Results
Out of
Total Replicates
4/4
4/4
4/4
4/4
4/4
0/4
20/20
0/4
4/4
0/4
4/4
0/4
4/4(d)
0/4
4/4
0/4
4/4
0/24
0/60
a) Sample solutions were prepared at 2, 5, 10, and 50 times the vendor-stated system LOD from stock solutions based
on the enumeration data (see Table 4-1).
(b) Inconclusive results are one positive and one negative result for each capillary of the split sample for a given
replicate. They were considered neither positive nor negative.
(c) Infective/lethal dose.
(d) Spiked CA DW samples were rerun because of suspected cross-contamination problems. Only the rerun results are
presented here.
32
-------
Table 6-3b. F. tularensis Target 2 False Positive/Negative Results
Sample Type
Contaminant-
Only PT
Samples
0.5
0
0.5
Interferent PT °
Samples 2 5
2
2.5
2
DW Samples
False Positive Rate
False Negative Rate
Sample
DI water
DI water
DI water
DI water
DI water
mg/L humic acid and
.5 mg/L fulvic acid
mg/L humic acid and
.5 mg/L fulvic acid
mg/L humic acid and
.5 mg/L fulvic acid
mg/L humic acid and
.5 mg/L fulvic acid
OHDW
OHDW
CADW
CADW
FLOW
FLOW
NYDW
NYDW
Conc.(a)
(cfu/mL)
4xl05(d)
2xl03
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Negative
Results(b)
0
0
0
0
0
4
1
4
0
4
0
3
3
4
0
4
0
Inconclusive
Results(c)
0
0
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
Positive Results
Out of
Total Replicates
4/4
4/4
4/4
4/4
4/4
0/4
19/20
0/4
4/4
0/4
4/4
0/4
0/4(e)
0/4
4/4
0/4
4/4
0/24
4/60
-------
Table 6-3c. Y. pestis False Positive/Negative Results
Positive Results Out of
Sample Type
Contaminant-
Only PT
Samples
0,
0,
Interferent PT
Samples 2
2,
DW Samples
False Positive Rate
False Negative Rate
Sample
DI water
DI water
DI water
DI water
.5 mg/L humic acid and
0.5 mg/L fulvic acid
.5 mg/L humic acid and
0.5 mg/L fulvic acid
.5 mg/L humic acid and
2.5 mg/L fulvic acid
.5 mg/L humic acid and
2.5 mg/L fulvic acid
OHDW
OHDW
CADW
CADW
FLOW
FLOW
NYDW
NYDW
Concentration^
(cfu/mL)
2xl03
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Total Replicates
Target 1
4/4
4/4
4/4
4/4
0/400
20/20(c)
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/24
0/56(c)
Target 2
4/4
4/4
4/4
4/4
0/400
20/20(c)
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/24
0/56(d)
-------
Table 6-3d. B. anthracis Target 1 False Positive/Negative Results
Sample Type
Contaminant-
Only PT
Samples
0.5
andO
0.5
Interferent PT and °
Samples 9 5
and 2
2.5
and 2
DW Samples
False Positive Rate
False Negative Rate
Sample
DI water
DI water
DI water
DI water
mg/L humic acid
.5 mg/L fulvic acid
mg/L humic acid
.5 mg/L fulvic acid
mg/L humic acid
.5 mg/L fulvic acid
mg/L humic acid
.5 mg/L fulvic acid
OHDW
OHDW 0
CADW
CADW 1
FLOW
FLDW 0
NYDW
NYDW 1
Conc.(a)
(cfu/mL)
2xl03
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Negative
Results(b)
0
0
0
0
4
0
4
0
4
4
4
4
Inconclusive
Results(c)
1
0
0
0
0
0
0
0
0
0
0
1
0
1
0
0
Positive Results
Out of
Total Replicates
3/4
4/4
4/4
4/4
0/4
20/20
0/4
4/4
0/4
4/4
0/4
2/4(d)
0/4
3/4
0/4
3/4
0/24
2/56(e)
-------
Table 6-3e. B. anthracis Target 2 False Positive/Negative Results
Sample Type
Contaminant-
Only PT
Samples
0.
and
0.
Interferent PT and
Samples 9
and
2.
and
DW Samples
False Positive Rate
False Negative Rate
Sample
DI water
DI water
DI water 0
DI water 0
5 mg/L humic acid
0.5 mg/L fulvic acid
5 mg/L humic acid
0.5 mg/L fulvic acid
5 mg/L humic acid
2.5 mg/L fulvic acid
5 mg/L humic acid
2.5 mg/L fulvic acid
OHDW
OHDW
CADW
CADW 1
FLOW
FLDW
NYDW
NYDW
Conc.(a)
(cfu/mL)
2xl03(d)
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Negative
Results(b)
1
0
4
0
4
0
4
0
4
4
0
4
0
Inconclusive
Results(c)
0
0
0
0
0
0
0
0
0
0
0
2
0
0
0
0
Positive Results
Out of
Total Replicates
3/4
4/4
4/4
4/4
0/4
20/20
0/4
4/4
0/4
4/4
0/4
1/4(d)
0/4
4/4
0/4
4/4
0/24
2/56(e)
-------
Table 6-3f. B. anthracis Target 3 False Positive/Negative Results
Sample Type
Contaminant-
Only PT
Samples
0.
and
0.
Interferent PT and
Samples 9
and
2.
and
DW Samples
False Positive Rate
False Negative Rate
Sample
DI water
DI water
DI water
DI water
5 mg/L humic acid
0.5 mg/L fulvic acid
5 mg/L humic acid
0.5 mg/L fulvic acid
5 mg/L humic acid
2.5 mg/L fulvic acid
5 mg/L humic acid
2.5 mg/L fulvic acid
OHDW
OHDWO
CADW
CADW
FLOW
FLDWO
NYDW
NYDWO
Conc.(a)
(cfu/mL)
2xl03(c)
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
0 IxlO4
Blank
IxlO4
Blank
IxlO4
Negative
Results
0
0
0
0
4
0
4
0
4
4
4
4
Inconclusive
Results(b)
0
0
0
0
0
0
0
0
0
0
0
4
0
0
0
0
Positive Results
Out of
Total Replicates
4/4
4/4
4/4
4/4
0/4
20/20
0/4
4/4
0/4
4/4
0/4
0/4(c)
0/4
4/4
0/4
4/4
0/24
0/56(d)
-------
Table 6-3g. Brucella suis False Positive/Negative Results
Sample Type
Contaminant-Only
PT Samples
Interferent PT
Samples
DW Samples
False Positive Rate
False Negative Rate
Sample
DI water
DI water
DI water
DI water
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
OHDW
OHDW
CADW
CADW
FLOW
FLOW
NYDW
NYDW
Concentration^
(cfu/mL)
2xl03
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Positive Results Out of
Total Replicates
2/4*)
4/4
4/4
4/4
0/4
20/20
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/24
0/56(b'c)
-------
6.3.5 E. coli
Table 6-3h presents the false negative and false positive results for E. coli. The number of
positive samples out of the total replicates analyzed is presented in the table. No false positive or
false negative samples were found in any of the sample matrices.
Table 6-3h. E. coli False Positive/Negative Results
Sample Type
Contaminant-Only PT
Samples
Interferent PT
Samples
DW Samples
False Positive Rate
False Negative Rate
Sample
DI water
DI water
DI water
DI water
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
0.5 mg/L humic acid and
0.5 mg/L fulvic acid
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
2.5 mg/L humic acid and
2.5 mg/L fulvic acid
OHDW
OHDW
CADW
CADW
FLOW
FLDW
NYDW
NYDW
Concentration'*0
(cfu/mL)
2xl03
5xl03
IxlO4
5xl04
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Blank
IxlO4
Positive Results Out of
Total Replicates
4/4
4/4
4/4
4/4
0/4
16/1600
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/4
4/4
0/24
0/52(b'c)
(a) Sample solutions were prepared at 2, 5, 10, and 50 times the vendor-stated system LOD from stock solutions based
on the enumeration data (see Table 4-1).
(b) One set of spiked 0.5 mg/L humic and fulvic acid replicates were suspected of having cross-contamination problems.
The samples were not rerun, so the results are not reported here for consistency with other rerun reporting.
(c) The infective/lethal dose for E. coli was below the system LOD and thus not included in this calculation.
39
-------
6.4 Precision
The performance of the F. tularensis Target 1 assay within sample sets of four replicates was
consistent. Only one set of replicates, that for interferent samples spiked with 2.5 mg/L humic
and fulvic acids, was inconsistent, with inconclusive results for three replicates and a negative
result for one replicate. All other samples showed the same results within a set of replicates.
Thus, for F. tularensis Target 1, one of the 21 sets of replicates that were analyzed was
determined to be inconsistent, indicating that 95% of the sample sets showed consistent results
among the replicates.
The precision of the F. tularensis Target 2 for the R.A.P.I.D.® System was not as consistent as
that for Target 1. Results were inconsistent within three sets of replicate samples for Target 2. In
the unspiked CA DW sample, the results for one replicate were inconclusive while the other three
were negative. The same was true for spiked CA DW. In one set of spiked 0.5 mg/L each humic
and fulvic acids, one negative result was found amongst three positive responses. All other
samples showed the same results within a set of four replicates. Thus, for F. tularensis Target 2,
three of the 21 sets of replicates that were analyzed were determined to be inconsistent, indicating
that 86% of the sample sets showed consistent results among the replicates.
The performance of the R. A.P.I.D.® System within sample sets of four replicates for Y. pestis
Targets 1 and 2 was very consistent. For all 21 sets of replicate samples for both targets, all
replicates showed the same results within the sample set. Thus, for Y. pestis Targets 1 and 2,
100% of the sample sets showed consistent results among the replicates.
As with F. tularensis, the consistency of the sample sets for B. anthracis were mixed for the three
targets that were evaluated. For B. anthracis Target 1, five sample sets had inconsistent results.
These samples included two PT samples (2 x 103 cfu/mL and the infective/lethal dose), the spiked
CA DW, the spiked FL DW, and the spiked NY DW samples. For the infective/lethal dose of
B. anthracis, three of the four samples had inconclusive results, while one sample was negative.
The infective dose of B. anthracis was below the LOD for the R. A.P.I.D.® System for this
bacteria, so the discrepancy between replicate samples likely has more to do with this fact than
the actual precision of the system. At 2 x 103 cfu/mL, three positive results and one inconclusive
result were observed. The responses for spiked CA DW were mixed, with two positive replicates,
one negative, and one inconclusive replicate response. Spiked FL DW had three positive results
and one inconclusive result, while spiked NY DW had three positive and one negative response
for B. anthracis. These five inconsistent replicate sets indicate that 76% (16/21) of the sample
sets analyzed for B. anthracis Target 1 showed consistent results among the replicates.
The number of consistent replicate sample sets for B. anthracis Target 2 was higher than that for
Target 1. Three of the 21 replicate sets of samples showed inconsistent results for Target 2. As
with Target 1, the infective/lethal dose of B. anthracis showed varied results, with two incon-
clusive and two negative responses. Again, the infective dose of B. anthracis was below the LOD
for the R. A.P.I.D.® System for this bacteria, so the discrepancy between replicate samples likely
has more to do with this fact than the actual precision of the system. Replicates at 2 x 103 cfu/mL
again showed inconsistent results, with three positive and one negative response. The remaining
inconsistent sample set was spiked CA DW. For these replicates, one was positive, one was
40
-------
negative, and the remaining two were inconclusive. Overall, 86% (18/21) of the sample sets
analyzed for B. anthracis Target 2 showed consistent results among the replicates.
The consistency within sample sets for the B. anthracis assays was the highest for Target 3. As
with Targets 1 and 2, the results for the infective/lethal dose of anthrax were inconsistent, with
two negative and two inconclusive results. The remaining 20 of the 21 sets of replicate samples
showed consistent results among the replicates. All four spiked CA DW samples were
inconclusive. Thus, for B. anthracis Target 3, one of the 21 sets of replicates that were analyzed
was determined to be inconsistent, indicating that 95% (20/21) of the sample sets showed
consistent results among the replicates.
For Brucella suis, two out of 21 sample sets were inconsistent. The replicates at 2 x 103 cfu/mL
showed two positive and two inconclusive responses. Three of the replicates at the infective/
lethal dose were negative, while one was inconclusive, resulting in an inconsistent set of
replicates. As with B. anthracis, the infective/lethal dose of Brucella suis is below the vendor-
provided system LOD for this bacteria. All other sample sets showed consistent results within a
set of replicates. Thus, 90% (19/21) of the sample sets showed consistent results among the
replicates.
Only 20 sets of replicate samples were considered for the precision calculations for E. coli (see
Table 6-3h). For all 20 sets of replicate samples, all replicates showed the same results within the
sample set. Thus, for E. coli, 100% of the sample sets showed consistent results among the
replicates.
6.5 Interferences
6.5.1 Interferent PT Samples
In both the 0.5 mg/L and 2.5 mg/L humic and fulvic acid solutions, both spiked with the bacteria
of interest and unspiked, the R. A.P.I.D.® System provided expected results for Y. pestis Targets 1
and 2; B. anthracis Targets 1, 2, and 3; Brucella suis; and E. coli. In the absence of the bacteria,
all samples for these bacteria (and all associated targets) tested negative; in the presence of the
bacteria, all samples tested positive. In the case of F. tularensis Target 1, all unspiked 0.5 mg/L
humic and fulvic acid samples returned negative results, while all spiked 0.5 and 2.5 mg/L humic
and fulvic acid samples returned positive results. The unspiked 2.5 mg/L humic and fulvic acid
replicates returned one negative response and three inconclusive results, where the split samples
of each of these three replicates had a positive and a negative response. All positive samples had
low amplification and late crossing points. The same sample DNA, when analyzed on the
R.A.P.I.D.® 7200 instrument using F. tularensis Target 2, returned all negative results. The
discrepancy between the targets could be related to the different sensitivities of the two targets
and/or potential cross-contamination. The initial spiked and unspiked 0.5 mg/L and 2.5 mg/L
humic and fulvic acid samples tested using F. tularensis Target 2 returned all positive results
when the sample was spiked and all negative results when the sample was unspiked.
Discrepancies in the results for the additional 0.5 mg/L spiked samples that were tested are
discussed below.
41
-------
As discussed in Section 3.2.1, four solutions of humic and fulvic acids at 0.5 mg/L each spiked
with each contaminant at lx!04cfu/mL were prepared in addition to the initial 0.5 mg/L and
2.5 mg/L humic and fulvic acid solutions. Each solution was put through the DNA extraction and
purification procedure, and then four replicates from each of the four purified DNA solutions
were analyzed using the R. A.P.I.D.® System. These samples were included in the verification test
in an effort to evaluate the efficacy of the DNA extraction and purification procedure in the
presence of inhibitory substances. These samples also contribute to the precision evaluations of
the R.A.P.I.D.® System. For F. tularensis Target 1; Y. pestis Targets 1 and 2; B. anthracis
Targets 1, 2, and 3; Brucella suis; and E. coli, all of the samples tested resulted in positive
responses. Thus, 20 out of the 20 (16/16 for E. coli) spiked 0.5 mg/L humic and fulvic acid
samples tested resulted in positive responses for each bacteria. For F. tularensis Target 2, one of
the 20 replicates tested produced a negative instead of positive reponse.
6.5.2 Drinking Water Samples
The R. A.P.I.D.® System DW sample results for F. tularensis, Y. pestis, B. anthracis,
Brucella suis, and E. coli are presented in Tables 6-3a through 6-3h. In general, the R.A.P.I.D.®
System showed positive results for each set of replicates for the spiked DW samples and negative
results for each set of replicates for the unspiked samples, with a few exceptions. B. anthracis
Targets 1, 2, and 3 as well as F. tularensis Targets 1 and 2 generated mixed results in spiked CA
DW. B. anthracis Target 1 also showed inconclusive and negative results in spiked FL and NY
DW, respectively. For the detection of F. tularensis Target 2 in unspiked CA DW, one of the four
replicates had an inconclusive result. Analysis of the CA DW did not indicate the presence of
F. tularensis. The possibility of cross-contamination causing the inconclusive results for
unspiked CA DW sample cannot be ruled out.
The contaminant-only PT samples at IxlO4 cfu/mL, the level at which the DW samples were
spiked, showed consistent positive responses across all bacteria. The interferent PT samples at
both 0.5 mg/L and 2.5 mg/L humic and fulvic acids also spiked at IxlO4 cfu/mL showed
consistent positive responses for all replicates across all bacteria, except for four replicates total
for F. tularensis Targets 1 and 2. The consistency of responses in these PT samples, as well as
other contaminant-only PT samples above the system LOD, would seem to indicate that some of
the DW matrices used in this test, in particular CA DW, may have inhibitory or other
confounding effects on the PCR process for the R. A.P.I.D.® System.
6.6 Other Performance Factors
The R. A.P.I.D.® System was operated by the same Battelle technician throughout the verification
test. This technician had prior PCR experience and was trained by Idaho Technology over the
course of one day in the operation of the R. A.P.I.D.® System before testing began. This training
included the use of the ITI1-2-3 Flow Kit, the R.A.P.I.D.®7200 instrument, and the freeze-dried
reagents. The Battelle technician was familiar with general DNA extraction and purification
techniques, PCR reagent preparation techniques, and general thermal cycler operation, as well as
general PCR theory, prior to training. The overall operation of the R. A.P.I.D.® System was
straightforward, and the experienced technician found the system easy to use and had no major
42
-------
difficulties operating the R. A.P.I.D.® System. The R. A.P.I.D.® System is designed to be used by
operators without prior experience who have only gone through training conducted by Idaho
Technology. Some features of the R. A.P.I.D.® System software are designed to complement such
an inexperienced user, though the Advanced Options were used in this test.
The freeze-dried reagents provided for easy PCR setup because all of the components needed to
fill two capillaries were contained within one vial. The "unknown" reagents and positive and
negative control reagent vials were packaged together and color coded. It was important to
throughly mix the reconstituted reagents before pipetting them into the capillaries. Prior
experience using pipettes would likely make the PCR setup easier and reduce potential pipetting
errors. The DNA extraction procedure using the ITI1-2-3 Flow Kit was straightforward and easy
to follow, though there were a fair number steps involved in the process. The transfer pipettes
provided with the ITI 1-2-3 Flow Kit were more rudimentary than typical laboratory pipettes, but
they worked well because exact amounts of buffer were not needed for the extraction procedure.
The glass capillaries used to hold the sample were easy to work with but posed some problems.
At one point, it became difficult to put the capillaries into the slots (rotors) in the carousel, and
the technician broke multiple capillaries trying to load the instrument. Capillaries also broke
during two PCR runs. After instruction from Idaho Technology, the rotor tool (provided with the
system) was used to clean out the capillary slots in the carousel. After the cleaning, the capillaries
posed no further problems. The R.A.P.I.D.® System software was user friendly and easy to use
with clearly labeled buttons. The technician easily learned to use the Advanced Options feature,
which was used to evaluate the results instead of the user interface intended for inexperienced
operators. In Advanced Options, the software returned crossing points for all samples that had
crossed the background fluorescence, but it was up to the technician to verify that the sample was
positive. An understanding of PCR theory and amplification would make such interpretations
easier.
All testing was performed in a laboratory setting because the R.A.P.I.D.® System as used in this
verification test is not field portable. The R. A.P.I.D.® System is intended to be field portable, and
in fact the instrument itself as used for this test could be easily transported to the field. However,
to perform the DNA extraction and purification of 5 mL of water, a 50 mL capacity centrifuge
had to be used. For this reason, the R. A.P.I.D.® System as used in this test was not considered
field portable. Three different testing areas were required in each laboratory to operate the
R. A.P.I.D.® System: one for the DNA purification, one for the reagent preparation, and one for
the instrument operation. All of the reagents, including those for the ITI 1-2-3 Flow Kit and the
freeze-dried reagents, were stable at room temperature. The reagents came packaged in
individual, vacuum-sealed foil pouches containing 20 vials (four positive controls, two negative
controls, and 14 unknown vials) along with sterile PCR grade water. The R.A.P.I.D.® 7200
instrument was approximately 20 inches x 14 inches x 10 inches its own hardened case, with a
backpack for carrying. It came with its own laptop, which can be placed in the backpack.
R tularensis and E. coli samples were tested in a BSL-2 laboratory; while Y. pestis, B. anthracis,
and Brucella suis were tested in a BSL-3 laboratory. Because live bacteria were being handled,
special safety requirements and protocols had to be implemented in both the BSL-2 and BSL-3
laboratories. Some of these requirements impacted the analysis time for the R. A.P.I.D.® System
and are inherently present in any throughput estimations for this verification test. Thus, such
43
-------
performance factors mentioned here also incorporate the safety and facility requirements
necessary for this test.
A total of 92 or more samples (including method blanks) were tested for each bacteria using the
R. A.P.I.D.® System. On average, the DNA extraction and isolation step for between three and
nine solutions took approximately 2 hours. The PCR setup steps, including reconstituting the
freeze-dried reagents and loading the capillaries, took approximately 30 minutes for each batch of
samples. The thermal cycler run on the R.A.P.I.D.® 7200 instrument took approximately
30 minutes for each batch of samples. The verification staff analyzed on average three to four
batches of samples on the R. A.P.I.D.® 7200 instrument per day, in some instances analyzing up
to six batches a day. This equated to approximately 36 to 48 duplicate samples being analyzed
per day (72 to 96 actual capillaries loaded) for three to four batches of samples and 72 duplicate
samples a day (144 actual capillaries loaded) for six batches, including two controls (one split
positive and one split negative) for every batch of samples analyzed.
For the purposes of this test, the R. A.P.I.D.® output was monitored using the LCDA
quantification view to determine the results for each sample. The Second Derivative Maximum
function was used to determine the crossing points. Within the LCDA quantification view of the
data, the operator has other options with which he or she can manipulate and inspect the data. A
Fit Points analysis can be used to determine the crossing points for each positive sample. This
option allows for more user-defined criteria for determining the crossing points. For the Fit Points
analysis, the user picks the appropriate baseline adjustment method (as with the Second
Derivative Maximum analysis) and then sets the noise band threshold. The software then
generates the crossing points for each positive sample based on the user-defined criteria. The
R.A.P.I.D.® System also offers a melt cycle and melting curve analysis to aid in product
identification.
Another more simplified screen that was not used in this verification test is also available from
the R. A.P.I.D.® software for interpreting the results. The software also offers a Detector view of
the results at the completion of a PCR run. In this view, all of the samples from a single run on
the R.A.P.I.D.®7200 instrument are shown in one table, including the positive and negative
controls. In this table, the software generates a "Present" response beside a sample when the
bacteria of interest have been detected, and a "Not Detected" beside the sample when the bacteria
have not been detected in the sample. The software also monitors the control samples and will
indicate to the user if a batch of samples needs to be rerun because the controls failed. An
example of the Detector output is shown below in Figure 6-2. An option also exists within the
R. A.P.I.D.® 7200 instrument to test for multiple assays on one carousel batch. Though this option
was not fully verified in this test, for one batch of samples, in an effort to conserve time, two
different assays for the same bacteria (Y. pestis) were analyzed in the same R.A.P.I.D.® 7200
instrument run. No problems were encountered, and the setup was easy.
44
-------
Irucella - Organism Present
*
1
2
3
4
b
6
7
8
9
10
11
12
13
14
15
16
I/
18
19
20
21
22
23
24
2b
l?fi
__ u
27
28
Sample Name
Positive Control
Positive Control
Negative Control
Neqative Control
Sample 7 al
Samp e / a 2
Samp e / bl
Sample / b2
Sample 7 cl
Samp e / c2
Sample 7 dl
Sample 7 c!2
Sample Sal
Samp-C 8 a 2
Samp c 8 bl
Samp e 8 b2
Samp e 8 cl
Sample 8 c2
Sample 8 dl
Sample 8 d2
Sample 9 al
Sample 9 a2
Sample 9 bl
Samp c 9 b2
Samp e 9 cl
Sample 9 c2
Sarnp.e 9 ell
Sample 9 d2
Control
Positive
Positive
Neqatve
Neqat ve
Unknown
Unknown
Unknown
Unknown
Unknown
Ur known
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Unknown
Score
234
195
-8
-8
183
249
127
123
159
132
1/6
212
0
-8
-•:;
-12
— Jj
-8
-5
~ /
-13
-4
-6
-6
-4
.5
_5
-10
Result
Present
Present
Nci Detected
Nci Detected
Present
PreserK
Present
Present
Present
Present
Present
Present
Not Detected
No; Detected
f\c Detected
Nc Detected
l\c Detected
Nc: Detected
Not Detected
No: Detected
Not Detected
Not Detected
Not Detected
Ken Detected
Nci Detected
Nc: Detected
Not Detected
Not Detected
SSN
Enc. 1C
Figure 6-2. R.A.P.LD.® Detector View Output for Spiked (Sample 7) and Unspiked
(Samples 8 and 9) Brucella suis Samples.
45
-------
Chapter 7
Performance Summary
The R.A.P.I.D.® System results for this verification test for samples containing F. tularensis,
Y. pestis, B. anthracis, Brucella suis, and E. coli are presented in Tables 7-1 through 7-9. The
results for each bacteria assay are presented in a separate table. Qualitative responses for each set
of sample replicates as well as accuracy, specificity, false negatives and positives, and precision
are presented in each table. A summary of the other performance factors associated with the
R. A.P.I.D.® System is presented at the end of this chapter. These performance factors apply to the
entire system, across all bacteria.
46
-------
Table 7-1. F. tularensis Target 1 Summary Table
Parameter
Qualitative
results
Contaminant-only
PT samples
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Number Detected/
Sample Information Concentration Number of Samples
4xl05 cfu/mL(a) 4/4
2xl03 cfu/mL 4/4
DI water 5xl03 cfu/mL 4/4
IxlO4 cfu/mL 4/4
5xl04 cfu/mL 4/4
Humic and 1 1A4 , o/i/o/i
,. , . ., 1x10 cfu/mL 24/24
fulvic acids
Concentrated DW 1 x 1 04 cfu/mL 16/16
100% (20 out of 20) of the contaminant-only PT samples above the
system LOD were positive.
88% (21 out of 24) of the unspiked interferent and DW samples were
negative. Three unspiked 2.5 mg/L each humic and fulvic acid
replicates returned inconclusive results.*'
No false positives resulted from the analysis of the unspiked interferent
or DW samples. Three unspiked 2.5 mg/L each humic and fulvic acid
replicates returned inconclusive results.*'
No false negative results were obtained from the analysis of samples
spiked with levels of F. tularensis above the system LOD.
95% (20 out of 21) of the sample sets showed consistent results among
the individual replicates within that set.*'
(a) Infective/lethal dose.
(b) Three unspiked 2.5 mg/L each humic and fulvic acid replicates had one positive and one negative result in the split
samples. These were inconclusive results and would require re-analysis in a real-world scenario. The remaining
replicate was negative.
47
-------
Table 7-2. F. tularensis Target 2 Summary Table
Parameter
Contaminant-
only PT
Qualitative samPles
results
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Sample Information Concentration
4xl05cfu/mL(a)
2xl03 cfu/mL
DI water 5 x 1 03 cfu/mL
IxlO4 cfu/mL
5x1 04 cfu/mL
Humic and 1 1 n4 - , T
,, , . . , lxl04cfu/mL
fulvic acids
Concentrated DW 1 x 1 04 cfu/mL
Number Detected/
Number of Samples
4/4
4/4
4/4
4/4
4/4
23/24(b)
12/16(c)
100% (20 out of 20) of the contaminant-only PT samples above the
system LOD were positive.
96% (23 out of 24) of the unspiked interferent
negative.
and DW samples were
No false positives resulted from the analysis of the unspiked
interferent or DW samples. One unspiked CA DW replicate returned
an inconclusive result. (d)
Four false negative results were obtained from the analysis of samples
spiked with levels of F. tularensis above the system LOD. Three
spiked CA DW and one spiked 0.5 mg/L each humic and fulvic acid
replicates returned negative results.
86% (18 out of 21) of the sample sets showed
among the individual replicates within that set
consistent results
(b, c, d)
(a) Infective/lethal dose.
(b) One spiked 0.5 mg/L each humic and fulvic acid replicate had one negative and three positive results.
(c) One spiked CA DW replicate had one positive and one negative results in the split samples. This indicated an
inconclusive result and would require re-analysis in a real-world scenario. The remaining replicates were negative.
-------
Table 7-3. Y. pestis Target 1 Summary Table
Parameter
Contaminant-
only PT
Qualitative samPles
results
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Sample Information Concentration
0.28 cfu/mL(a)
2xl03 cfu/mL
DI water 5 x 1 03 cfu/mL
IxlO4 cfu/mL
5x1 04 cfu/mL
Humicand 1 1A4 - , T
,, , . . , IxlCrcfu/mL
fulvic acids
Concentrated DW 1 x 1 04 cfu/mL
Number Detected/
Number of Samples
0/4(a)
4/4
4/4
4/4
4/4
24/24
16/16
100% (16 out of 16) of the contaminant-only PT samples above the
system LOD were positive.
100% (24 out of 24) of the unspiked interferent
were negative.
and DW samples
No false positives resulted from the analysis of the unspiked
interferent or DW samples.
No false negative results were obtained from the analysis of the
interferent and DW samples spiked with levels of Y. pestis above the
system LOD.
100% (21 out of 21) of the sample sets showed
among the individual replicates within that set.
consistent results
(a)
Infective/lethal dose—below the R.A.P.I.D.® System LOD for Y. pestis.
49
-------
Table 7-4. Y. pestis Target 2 Summary Table
Parameter
Contaminant-
only PT
Qualitative samPles
results
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Sample Information Concentration
0.28 cfu/mL(a)
2xl03 cfu/mL
DI water 5 x 1 03 cfu/mL
IxlO4 cfu/mL
5x1 04 cfu/mL
Humic and 1 1 n4 , T
,, , . . , IxlCrcfu/mL
fulvic acids
Concentrated DW 1 x 1 04 cfu/mL
Number Detected/
Number of Samples
0/4 (a)
4/4
4/4
4/4
4/4
24/24
16/16
100% (16 out of 16) of the contaminant-only PT samples above the
system LOD were positive.
100% (24 out of 24) of the unspiked interferent and DW samples
were negative.
No false positives resulted from the analysis of the unspiked
interferent or DW samples.
No false negative results were obtained from the analysis of the
interferent and DW samples spiked with levels of Y. pestis above the
system LOD.
100% (21 out of 21) of the sample sets showed
among the individual replicates within that set.
consistent results
(a)
Infective/lethal dose—below the R.A.P.I.D.® System LOD for Y. pestis.
50
-------
Table 7-5. B. anthracis Target 1 Summary Table
Parameter
Qualitative
results
Contaminant-
only PT
samples
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Number Detected/
Sample Information Concentration Number of Samples
200 cfu/mL(a) 0/400
2xl03 cfu/mL 3/4(c)
DI water 5xl03cfu/mL 4/4
IxlO4 cfu/mL 4/4
5x1 04 cfu/mL 4/4
Humic and 1 1A4 , o/i/o/i
,. , . ., 1x10 cfu/mL 24/24
fulvic acids
Concentrated DW lxl04cfu/mL 12/16(4e)
94% (15 out of 16) of the contaminant-only PT samples above the
system LOD were positive.
100% (24 out of 24) of the unspiked interferent and DW samples
were negative.
No false positives resulted from the analysis of the unspiked
interferent or DW samples.
Two false negative results were obtained from the analysis of samples
spiked with levels of B. anthracis above the system LOD, one for
spiked NY DW and the other for spiked CA DW. Inconclusive results
were found for one replicate each for spiked CA DW, spiked FL DW,
and DI water at 2xl03 cfu/mL.(c' ^ e)
76% (16 out of 21) of the sample sets showed consistent results
among the individual replicates within that set.*' c> ^ e)
(a) Infective/lethal dose—below the R.A.P.I.D.® System LOD for B. anthracis.
(b) Three samples in the infective/lethal dose PT sample replicates had one positive and one negative result in the split
samples. This indicated an inconclusive result and would require re-analysis in a real-world scenario. The remaining
replicate was negative.
(c) One PT sample replicate at 2xl03 cfu/mL had one positive and one negative result in the split sample. This indicated
an inconclusive result and would require re-analysis in a real-world scenario. The remaining replicates were positive.
(d) One spiked CA DW replicate had one positive and one negative result in the split sample. This indicated an
inconclusive result and would require re-analysis in a real-world scenario. Two of the remaining replicates were
positive, the other negative.
(e) One spiked FL DW replicate had one positive and one negative result in the split sample. This indicated an
inconclusive result and would require re-analysis in a real-world scenario. Two of the remaining replicates were
positive.
51
-------
Table 7-6. B. anthracis Target 2 Summary Table
Parameter
Qualitative
results
Contaminant-
only PT
samples
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Number Detected/
Sample Information Concentration Number of Samples
200 cfu/mL(a) 0/400
2xl03 cfu/mL 3/4
DI water 5xl03cfu/mL 4/4
IxlO4 cfu/mL 4/4
5x1 04 cfu/mL 4/4
Humic and 1 1A4 , o/i/o/i
,. , . ., 1x10 cfu/mL 24/24
fulvic acids
Concentrated DW lxl04cfu/mL 13/16(c)
94% (15 out of 16) of the contaminant-only PT samples above the
system LOD were positive.
100% (24 out of 24) of the unspiked interferent and DW samples
were negative.
No false positives resulted from the analysis of the unspiked
interferent or DW samples.
Two false negative results were obtained from the analysis of samples
spiked with levels of B. anthracis above the system LOD, one for
spiked CA DW and the other for DI water at 2xl03 cfu/mL.
Inconclusive results were found for two spiked CA DW replicates. (c)
86% (18 out of 21) of the sample sets showed consistent results
among the individual replicates within that set.(b'c)
(a) Infective/lethal dose—below the R.A.P.I.D.® System LOD for B. anthracis.
-------
Table 7-7. B. anthracis Target 3 Summary Table
Parameter
Qualitative
results
Contaminant-
only PT
samples
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Number Detected/
Sample Information Concentration Number of Samples
200 cfu/mL(a) 0/400
2xl03 cfu/mL 4/4
DI water 5xl03cfu/mL 4/4
IxlO4 cfu/mL 4/4
5x1 04 cfu/mL 4/4
Humic and 1 1A4 , o/i/o/i
,. , . ., 1x10 cfu/mL 24/24
fulvic acids
Concentrated DW lxl04cfu/mL 12/16(c)
100% (16 out of 16) of the contaminant-only PT samples above the
system LOD were positive.
100% (24 out of 24) of the unspiked interferent and DW samples
were negative.
No false positives resulted from the analysis of the unspiked
interferent or DW samples.
No false negative results were obtained from the analysis of samples
spiked with levels of B. anthracis above the system LOD. All four
replicates for spiked CA DW returned inconclusive results. (c)
95% (20 out of 21) of the sample sets showed consistent results
among the individual replicates within that set.*'
(a) Infective/lethal dose—below the R.A.P.I.D.® System LOD for B. anthracis.
-------
Table 7-8. Brucella suis Summary Table
Parameter
Contaminant-
only PT
Qualitative samPles
results
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Sample Information
DI water
Humic and
fulvic acids
Concentrated DW
88% (14 out of 16) of the
LOD were positive.
Number Detected/
Concentration Number of Samples
40 cfu/mL(a) 0/4 (b)
2xl03 cfu/mL 2/4 (c)
5xl03 cfu/mL 4/4
IxlO4 cfu/mL 4/4
5 xlO4 cfu/mL 4/4
IxlO4 cfu/mL 24/24
IxlO4 cfu/mL 16/16
contaminant-only PT samples above the system
100% (24 out of 24) of the unspiked interferent and DW samples were
negative.
No false positives resulted from the analysis of the unspiked interferent
or DW samples.
No false negative results were obtained from the analysis of samples
spiked with levels of Brucella suis above the system LOD.
90% (19 out of 21) of the sample sets showed consistent results among
the individual replicates within that set. (b> c)
(a) Infective/lethal dose—below the R.A.P.I.D.® System LOD for Brucella suis.
(b) One sample in the infective/lethal dose PT sample replicates had one positive and one negative result in the split
samples. This indicated an inconclusive result and would require re-analysis in a real-world scenario. The remaining
replicates were negative.
(c) Two PT sample replicate at 2xl03 cfu/mL had one positive and one negative result in the split sample. This indicated
an inconclusive result and would require re-analysis in a real-world scenario. The remaining replicates were positive.
54
-------
Table 7-9. E. coli Summary Table
Parameter
Qualitative
results
Contaminant-
only PT
samples
Interferent
PT samples
DW samples
Accuracy
Specificity
False positives
False negatives
Precision
Sample Information
DI water
Humic and
fulvic acids
Concentrated DW
100% (16 out of 16) of the
system LOD were positive
100% (24 out of 24) of the
negative.
No false positives resulted
or DW samples.
Concentration
0.2 cfu/mL(a)
2xl03 cfu/mL
5xl03 cfu/mL
IxlO4 cfu/mL
5 xlO4 cfu/mL
IxlO4 cfu/mL
IxlO4 cfu/mL
contaminant-only PT
Number Detected/
Number of Samples
0/4
4/4
4/4
4/4
4/4
20/20(b)
16/16
samples above the
unspiked interferent and DW samples were
from the analysis of the unspiked interferent
No false negative results were obtained from the analysis of samples
spiked with levels of E. coli above the system LOD.
100% (20 out of 20) of the sample sets showed consistent results among
the individual replicates within that set.
(a) Infective/lethal dose— below the R.A.P.I.D.® System LOD for E. coli
(b) One set of spiked 0.5 mg/L each humic and fulvic acid replicates were suspected of having cross-contamination
problems. The samples were not rerun, so the results are not reported here for consistency with other rerun reporting.
Other performance factors: A technician with prior PCR experience operated the R.A.P.I.D.®
System at all times. All three components of the R.A.P.I.D.® System (the m 1-2-3 Flow Kit, the
freeze-dried reagents, and the R. A.P.I.D.® 7200 instrument) were straightforward and easy to use.
Three separate work areas were needed for testing to minimize cross-contamination. The freeze-
dried reagents were color coded, contained all of the necessary components for PCR in one vial,
and were reconstituted in the same vial, making PCR setup easy. Reagents for the DNA purifica-
tion and PCR setup had room temperature storage requirements. The glass capillaries used on the
R.A.P.I.D.® 7200 instrument were problematic when the rotors on the carousel were dirty, but
posed no problems once the rotors were properly cleaned. The sample throughput for this
verification test was 36 to 72 samples per day. Approximate operational times were 2 hours for
DNA extraction/purification, 30 minutes for reconstituting the reagents and loading the
capillaries for one carousel batch, and 30 minutes/carousel batch for PCR. The R.A.P.I.D.®
software was easy to use, and additional software analysis tools other than those used in this test
are available. The cost for the R.A.P.I.D.® System is around $8 per sample for the FTI1-2-3 Flow
Kit DNA purification step, $17 to test each split sample using the freeze-dried reagents, and
approximately $55,000 for the R.A.P.I.D.® 7200 instrument itself (20 inches x!4 inches xlO
inches, 50 pounds). The R.A.P.I.D.®7200 instrument can hold up to 32 capillaries (14 split
samples, plus controls).
55
-------
Chapter 8
References
1. Test/QA Plan for Verification of Rapid PCR Technologies, Battelle, Columbus, Ohio,
May 2004.
2. Burrows, W. D.; Renner, S. E. "Biological Warfare Agents as Threats to Potable Water,"
Environmental Health Perspectives, 107,975-984, 1999.
3. Stenman, L; Orpana, A. "Accuracy in amplification," Nature Biotechnology, 19, 1011-
1012,2001.
4. Hughes, L; Totten, P. "Estimating the accuracy of polymerase chain reaction-based tests
using endpoint dilution," University of Washington Biostatistics Working Paper Series,
Working Paper 196, 2003.
5. U.S. EPA Method 180.1, "Turbidity (Nephelometric)" in Methods for the Determination
of Inorganic Substances in Environmental Samples, EPA/600/R-93-100, August 1993.
6. American Public Health Association, et al. Standard Methods for Examination of Water
and Waste\vater. 19th Edition. 1997. Washington D.C.
7. U.S. EPA, Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79-020,
March 1983.
8. U.S. EPA Method 200.8, "Determination of Trace Elements in Waters and Wastes by
Inductively-Coupled Plasma Mass Spectrometry" in Methods for the Determination of
Organic Compounds in Drinking Water, Supplement I, EPA/600/R-94/111, October
1994.
9. U.S. EPA Method 524.2, "Permeable Organic Compounds by Capillary Column
GC/Mass Spectrometry" in Methods for the Determination of Organic Compounds in
Drinking Water, Supplement m, EPA/600/R-95-131. August 1995.
10. U.S. EPA Method 552.2, "Haloacetic Acids and Dalapon by Liquid-Liquid Extraction,
Derivatization and GC with Electron Capture Detector" in Methods for the Determination
of Organic Compounds in Drinking Water, Supplement ffl, EPA/600/R-95-131, August
1995.
56
-------
11. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center,
Version 5.0. EPA Environmental Technology Verification Program, prepared by Battelle,
Columbus, Ohio, March, 2004.
12. Shapiro, D. S. "Quality Control in Nucleic Acid Amplification Methods: Use of
Elementary Probability Theory," Journal of Clinical Microbiology, 37, 848-851, 1999.
57
------- |