September 2004
Environmental Technology
Verification Report
RESPONSE BIOMEDICAL CORP.
RAMP® ANTHRAX, BOTULINUM TOXIN,
AND RlCIN iMMUNOASSAYlEST
CARTRIDGES
Prepared by
Baiteiie
The Business of Innovation
Under a cooperative agreement with
U.S. Environmental Protection Agency
-------
September 2004
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Response Biomedical Corp.
RAMP®
Anthrax, Botulinum Toxin, and Ricin
Immunoassay Test Cartridges
by
Ryan James
Amy Dindal
Zachary Willenberg
Karen Riggs
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six verification centers. Information about
each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received funding to plan, coordinate,
and conduct such verification tests for "Advanced Monitoring Systems for Air, Water, and Soil"
and report the results to the community at large. Information concerning this specific
environmental technology area can be found on the Internet at
http ://www. epa.gov/etv/centers/center 1 .html.
in
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We sincerely appreciate the
contribution of drinking water samples from the New York City Department of Environmental
Protection (Paul Bennett), the City of Orlando (Terri Slifko), and the Metropolitan Water
District of Southern California (Paul Rochelle). Also, thank you to the Metropolitan Water
District of Southern California for concentrating each drinking water sample. We would also
like to thank Karen Bradham, U.S. EPA National Exposure Research Laboratory (NERL); Steve
Allgeier, U.S. EPA Office of Water; Ricardo DeLeon, Metropolitan Water District of Southern
California; and Stanley States, Pittsburgh Water and Sewer Authority, for their careful review of
the test/QA plan and this verification report. Thanks go to Linda Sheldon, U.S. EPA NERL, for
her review of the verification reports and statements.
IV
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations vii
1 Background 1
2 Technology Description 2
3 Test Design and Procedures 4
3.1 Introduction 4
3.2 Test Samples 5
3.2.1 Performance Test Samples 7
3.2.2 Drinking Water Samples 8
3.2.3 Quality Control Samples 9
3.3 Test Procedure 9
3.3.1 Laboratory Testing 9
3.3.2 Non-Laboratory Testing 9
3.3.3 Drinking Water Characterization 10
4 Quality Assurance/Quality Control 12
4.1 Sample Chain-of-Custody Procedures 12
4.2 Equipment/Calibration 12
4.3 Characterization of Contaminant Stock Solutions 12
4.3.1 Characterization of Botulinum Toxin and Ricin 12
4.3.2 Characterization of Anthrax Spores 13
4.3.3 Anthrax Enumeration Data 14
4.4 Technical Systems Audit 16
4.5 Audit of Data Quality 18
4.6 QA/QC Reporting 18
4.7 Data Review 18
5 Statistical Methods and Reported Parameters 20
5.1 Qualitative Contaminant Presence/Absence 20
5.2 False Positive/Negative Responses 20
5.3 Consistency 20
5.4 Lowest Detectable Concentration 20
5.5 Other Performance Factors 21
-------
6 Test Results 22
6.1 Qualitative Contaminant Presence/Absence 22
6.1.1 Anthrax 22
6.1.2 Botulinum Toxin 25
6.1.3 Ricin 26
6.2 False Positive/Negative Responses 27
6.2.1 Interfered PT Samples 27
6.2.2 DW Samples 28
6.2.3 Cross-Reactivity PT Samples 29
6.3 Consistency 30
6.4 Lowest Detectable Concentration 30
6.5 Other Performance Factors 31
7 Performance Summary 32
8 References 37
Appendix Other Evaluations
Figures
Figure 2-1. Response Biomedical Corp. RAMP® Immunoassay Test Cartridge and Reader .. 2
Tables
Table 3-1. Lethal Dose and Source of Contaminants 5
Table 3-2. Performance Test Samples 6
Table 3-3. Drinking Water Samples 7
Table 3-4. ATEL Water Quality Characterization of the Drinking Water Samples 11
Table 4-1. Characterization Information for Battelle Preparation of Anthrax Spores 13
Table 4-2. Anthrax Enumeration Data for PT Samples 15
Table 4-3. Anthrax Enumeration Results for Fortified Interferent and
Drinking Water Sample 17
Table 4-4. Summary of Data Recording Process 19
Table 6-la. Anthrax Contaminant-Only PT Sample Results 23
Table 6-lb. Botulinum Toxin Contaminant-Only PT Sample Results 26
Table 6-lc. Ricin Contaminant-Only PT Sample Results 26
Table 6-2. Interferent PT Sample Results 27
Table 6-3. DW Sample Results 29
Table 6-4. Potentially Cross-Reactive PT Sample Results 30
Table 7-1. Anthrax Summary Table 32
Table 7-2. Botulinum Toxin Summary Table 34
Table 7-3. Ricin Summary Table 36
VI
-------
List of Abbreviations
AMS
ATEL
Ca
CDC
cfu
COA
DI
DW
EPA
ETV
L
LOD
MB
Mg
mg/L
HL
mL
PT
QA
QC
QMP
RAMP®
RPD
SOP
ISA
Advanced Monitoring Systems
Aqua Tech Environmental Laboratories, Inc.
calcium
Centers for Disease Control and Prevention
colony-forming units
certificate of analysis
deionized
drinking water
U.S. Environmental Protection Agency
Environmental Technology Verification
liter
limit of detection
method blank
magnesium
milligram per liter
microliter
milliliter
performance test
quality assurance
quality control
quality management plan
Rapid Analyte Measurement Platform
relative percent difference
standard operating procedure
technical systems audit
vn
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative tech-
nologies by developing test plans that are responsive to the needs of stakeholders, conducting
field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance
(QA) protocols to ensure that data of known and adequate quality are generated and that the
results are defensible.
The EPA's National Exposure Research Laboratory and its verification organization partner,
Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center
recently evaluated the performance of the Response Biomedical Corp. Rapid Analyte
Measurement Platform (RAMP®) anthrax, botulinum toxin, and ricin immunoassay test kits.
Immunoassay test kits were identified as a priority technology category for verification through
the AMS Center stakeholder process.
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of environ-
mental monitoring technologies for air, water, and soil. This verification report provides results
for the verification testing of the RAMP® test cartridges (Figure 2-1). The following is a
description of the RAMP® system based on information provided by the vendor. The
information provided below was not subjected to verification in this test.
RAMP® is a rapid immunochromatographic system for screening environmental samples. The
RAMP® system comprises a portable fluorescence reader and RAMP® test cartridges specific for
detecting anthrax, botulinum toxin, and ricin. Test cartridges specific for detecting smallpox are
also available, but were not tested.
The RAMP® reader is a scanning fluorometer and
data analysis system used to measure fluorescence
from RAMP® test cartridges. The reader can be
operated on built-in battery power or using an
alternating current adapter. RAMP® uses an
immunochromatographic strip, housed in the
disposable test cartridges. Each test cartridge is
single-use, disposable, and analyte-specific and is
used to detect whether an analyte (e.g., anthrax
spores) is present in an aqueous sample.
Twenty-five individually packaged RAMP® test
cartridges are provided in a small box. In addition
to the test cartridges, the box contains 25 small
plastic screw-top vials containing approximately
250 microliters (|J,L) of buffer, a box of sample collection swabs, a 70-|_iL micropipette, a lot
card for insertion into the reader, a marking pen, and step-by-step instructions. To perform a test
on a liquid sample, a small amount (10 jiL) of sample is added to the provided buffer, and that
solution is mixed and 70 |^L of sample is pipetted onto the RAMP® test cartridge. The cartridge
is then read using the reader, and a positive or negative result is generated on the reader's
display. Each result, along with the time, date, and sample identification is printed using a
printer provided by Response Biomedical Corp. The reader is also capable of downloading the
results to a computer.
Figure 2-1. Response Biomedical Corp.
RAMP® Immunoassay Test Cartridge
and Reader
-------
The dimensions of the RAMP® are 10.5 inches wide by 10 inches deep by 6 inches high
(27 centimeters wide by 25 centimeters deep by 5 centimeters high), and it weighs 4.6 pounds
(2.1 kilograms). A RAMP® system including 25 test cartridges, a reader, a printer, and a
carrying case costs approximately $10,000. Regardless of whether the test strips are specific to
anthrax, botulinum toxin, ricin, or smallpox, each additional box of 25 test cartridges costs
approximately $500.
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
The objective of this verification test of immunoassay test kits was to evaluate their ability to
detect specific biological toxins and agents in water samples and to determine their
susceptibility to specific interferents added to pure water and to interferents inherently present in
several drinking water (DW) samples. The detection devices are based on immunological
interactions, where specific antibodies are used to detect contaminants of interest. For the
RAMP® test cartridges, the contaminants, or antigens, react with a selective antibody to produce
a result that is indicated by fluorescence from the test cartridge in less than 15 minutes. The
presence of contaminants is indicated by positive or negative reading from the fluorescence
reader. The single-use test cartridges detect only one contaminant at a time.
During this verification test, the RAMP® test cartridges were subjected to various concentrations
of anthrax spores, botulinum toxin, and ricin in American Society for Testing and Materials
Type n deionized (DI) water. Table 3-1 shows the contaminants and information about their
detection, including the vendor-stated limit of detection (LOD), the lethal dose concentrations,
and the source. The RAMP® test cartridges also were used to analyze contaminant-fortified DW
samples that were collected from four water utilities that use a variety of treatment methods. The
effect of interferents was evaluated by analyzing individual solutions of organic acids (humic
and fulvic) and magnesium (Mg) and calcium (Ca) in DI water both with and without the
addition of the contaminants using the RAMP® test cartridges. In addition, specificity was
evaluated by exposing the RAMP® test cartridges to a potentially cross-reactive compound or
spore for each target contaminant.
-------
Table 3-1. Lethal Dose and Source of Contaminants
Contaminant
Bacillus anthracis
Ames Strain (anthrax)
Botulinum toxin
Types A and B
Ricinus communis
Agglutinin n (ricin)
Vendor-Stated
LOD
4 x 105
spores/mL
0.5 mg/L
1 mg/L
Lethal Dose
Concentration^
200 spores/mL(1)
0.3 mg/L(2)
15 mg/L(3)
Source of Contaminant
BattelleandU.S. Army
Dugway Proving Ground
Metabiologics, Inc.
(Madison, Wisconsin)
Vector Laboratories, Inc.
(Burlingame, California)
^ The lethal dose of each contaminant was determined by calculating the concentration at which 250 mL of water
would probably cause the death of a 154-pound person based on human mortality data.
mL = milliliter
mg/L = milligrams per liter
The verification test for the RAMP® test kits was conducted January 14 through April 23, 2004,
according to procedures specified in the Test/QA Plan for Verification of Immunoassay Test
Kits.m This test was conducted at Battelle laboratories in Columbus and West Jefferson, Ohio.
Aqua Tech Environmental Laboratories, Inc. (ATEL) of Marion, Ohio, performed physico-
chemical characterization for each DW sample to determine the following parameters: turbidity;
concentration of dissolved and total organic carbon; specific conductivity; alkalinity; concentra-
tion of Mg and Ca; pH; hardness; and concentration of total organic halides, trihalomethanes,
and haloacetic acids. Battelle confirmed the presence of anthrax spores using plate enumeration.
The RAMP® test cartridges were evaluated for the following parameters:
• Qualitative contaminant presence/absence
• False positive/false negative response
- Interferents
- DW matrix effects
- Cross-reactivity
• Consistency
• Lowest detectable concentration
• Other performance factors
- Field portability
- Ease of use
- Sample throughput.
3.2 Test Samples
Tables 3-2 and 3-3 summarize the samples analyzed for each contaminant. The ability of the
RAMP® test cartridges to individually detect various concentrations of anthrax spores,
botulinum toxin, and ricin was evaluated by analyzing performance test (PT) and DW samples.
PT samples
-------
Table 3-2. Performance Test Samples
Type of PT
Sample
Contaminant-only
Interferent
Potentially
Cross-reactive
Sample Characteristics
Anthrax spores
Botulinum toxin Type A
Botulinum toxin Type B
Ricin
Contaminants in 46 mg/L Ca
and 1 8 mg/L Mg
Contaminants in 230 mg/L Ca
and 90 mg/L Mg
Contaminants in 0.5 mg/Lhumic
acid and 0.5 mg/L fill vie acid
Contaminants in 2.5 mg/L humic
acid and 2.5 mg/L fill vie acids
Bacillus thuringiensis (anthrax
analogue)
Lipopolysaccharide
(botulinum toxin analogue)
Lectin from soybean
(ricin analogue)
Approximate Concentrations
200tol010spores/mL(a)
0.5 to 25 mg/L
0.3 to 1,000 mg/L
1 to 50 mg/L
Anthrax - 107 spore s/mL
Botulinum toxin (Type B) - 5 mg/L
Ricin - 1 0 mg/L
Anthrax - 107 and 10s spores/mL
Botulinum toxin (Type A) - 5 mg/L
Botulinum toxin (Type B) - 5 mg/L
Ricin - 1 0 mg/L
Anthrax - 107 spore s/mL
Botulinum toxin (Type B) - 5 mg/L
Ricin - 1 0 mg/L
Anthrax - 1 07 and 1 08 spores/mL
Botulinum toxin (Type A) - 5 mg/L
Botulinum toxin (Type B) - 5 mg/L
Ricin - 1 0 mg/L
106 spore s/mL
5 mg/L
10 mg/L
This concentration range includes all samples analyzed, including spores preserved with and without phenol,
spores prepared at Battelle and at Dugway Proving Ground, and vegetative anthrax cells.
(a)
included DI water fortified with either the target contaminant, an interferent, both, or only a
cross-reactive species. DW samples were analyzed using the RAMP® test cartridges with and
without the addition of each target contaminant. All the samples listed in the test/QA plan were
initially analyzed. As discussed below, additional concentration levels and sample types were
analyzed to more thoroughly evaluate the performance of the RAMP® test cartridges.
-------
Table 3-3. Drinking Water Samples
Drinking Water Sample Description
Water Utility
Metropolitan
Water District of
California (CA)
New York City,
New York (NY)
Metropolitan
Water District of
California (CA)
New York City,
New York (NY)
Columbus, Ohio
(OH)
Orlando, Florida
(FL)
Water
Treatment
filtered
chloraminated
unfiltered
chlorinated
filtered
chloraminated
unfiltered
chlorinated
filtered
chlorinated
filtered
chlorinated
Source
Type
surface
surface
surface
surface
surface
ground
Cone. /
Unconc.
cone.
cone.
unconc.
unconc.
both
both
Approximate Contaminant
Concentrations
Anthrax
(spores/mL)
unspiked
4xl06
109
unspiked
4xl06
109
unspiked
4xl06
unspiked
4xl06
unspiked
4xl06
unspiked
4xl06
Botulinum
Toxin
(mg/L)
unspiked
5 (Type B)
5 (Type A)
unspiked
5 (Type B)
5 (Type A)
unspiked
5
unspiked
5
unspiked
5
unspiked
5
Ricin
(mg/L)
unspiked
10
unspiked
10
unspiked
10
unspiked
10
unspiked
10
unspiked
10
3.2.1 Performance Test Samples
The contaminant-only PT samples were prepared in DI water using certified standards of ricin
and botulinum toxin. Reference methods were not available for quantitative confirmation of the
botulinum toxin and ricin test solutions so certificates of analysis (COA) and QA oversight of
solution preparation were used to determine their concentrations. Anthrax PT samples also were
prepared in DI water using anthrax spores prepared and characterized by Battelle using standard
methods. All test samples were prepared from the standards or stock solutions on the day of
analysis. Spores obtained from Dugway Proving Ground were prepared there and then
enumerated by Battelle during this verification test.
Initially, the test/QA plan called for the analysis of PT samples with concentrations including the
lethal dose; the vendor-stated LOD; and approximately 5, 10, and 50 times the LOD. These
samples were analyzed using the RAMP® test cartridges. Preliminary results indicated that
anthrax and botulinum toxin were not detectable; therefore, the original test/QA plan was
amended to include preparing and analyzing higher concentration samples of anthrax and
botulinum toxin Type A and testing a preparation of anthrax spores that were never preserved in
phenol, a second source of anthrax spores, a sample of anthrax vaccine, and vegetative anthrax
cells. This testing and the subsequent results are fully described in Section 6.1.
-------
The interferent PT samples consisted of samples of humic and fulvic acids isolated from the
Elliott River (obtained from the International Humic Substances Society) and Ca and Mg
(prepared from their chlorides), each spiked into DI water at two concentration levels. These
solutions were analyzed both with the addition of each target contaminant at one concentration
level and without the addition of any target contaminant. To be able to evaluate the
susceptibility of the RAMP® test cartridges to false negative results due to interferents, the
test/QA plan was amended to include the fortification of detectable types and concentrations of
contaminants into interferent solutions.
The last type of PT sample was a cross-reactivity check sample to determine whether the test
cartridges produce false positive results in response to similar analytes. Bacillus thuringiemis
(for anthrax), lectin from soybean (for ricin), and lipopolysaccharide (for botulinum toxin) are
chemically or biologically similar to the specified targets. Solutions of these were prepared in DI
water at concentrations similar to the vendor-stated LOD of the test kits for the specified targets
and analyzed using the appropriate RAMP® test cartridge.
In most cases, three replicates of each PT sample were analyzed. In some instances, the anthrax
test samples were analyzed less than three times, depending on the number of test cartridges
available for the analysis. A total of 192 PT samples was analyzed by the RAMP® test
cartridges for this test. The results provided information about how well the RAMP® test
cartridges detected the presence of each contaminant at several concentration levels, the
consistency of the responses, and the susceptibility of the RAMP® test cartridges to some
selected interferents and possibly cross-reactive species.
3.2.2 Drinking Water Samples
Table 3-3 lists the DW samples collected from four geographically distributed municipal sources
to evaluate the performance of the RAMP® test cartridges with various sample matrices. These
samples were unique in terms of their source and treatment and disinfection process. All
collected samples were finished DW either ready for the distribution system or from within the
distribution system.
Approximately 120 liters (L) of each of the above DW samples were collected in pre-cleaned
high-density polyethylene containers. All but 20 L of the DW samples were shipped to the
Metropolitan Water District of Southern California, dechlorinated with sodium thiosulfate, and
then concentrated through ultra-filtration techniques to a final volume of 250 mL. This
concentration factor was selected because it is the goal of an EPA onsite ultra-filtration method
which is currently being developed. The remaining 20 L of each water sample was shipped to
ATEL for water quality analysis. Each DW sample (non-concentrated and concentrated) was
analyzed without adding any contaminant, as well as after fortification with individual contam-
inants at a single concentration level. A total of 156 DW samples was analyzed by the RAMP®
test cartridges for this test.
-------
3.2.3 Quality Control Samples
In addition to the 348 PT and DW samples analyzed, 43 method blank (MB) samples consisting
of DI water also were analyzed to confirm negative responses in the absence of any contaminant
and to ensure that no sources of contamination were introduced during the analysis procedures.
Each RAMP® test cartridge runs an internal fluorescent control. The RAMP® reader produces an
error message if the internal fluorescent control fails. If the RAMP® reader ever produced an
error message instead of a result, that test kit would have been discarded and a new test cartridge
used. There were no instances of this during ETV testing. Because of this internal fluorescent
control, other positive control samples were not analyzed.
3.3 Test Procedure
3.3.1 Laboratory Testing
The scope of this verification test required that most of the test samples be analyzed within
Battelle laboratories staffed with technicians trained to safely handle anthrax, botulinum toxin,
and ricin. Each day, fresh samples were prepared from standards or stock solutions in either DI
water, an interferent matrix, or a DW matrix. Each sample was prepared in its own container and
labeled only with a sample identification number that also was recorded in a laboratory record
book along with details of the sample preparation. The following procedure was performed to
test a liquid sample for the presence of anthrax spores, botulinum toxin, and ricin: (1) 10 i^L1 of
test sample were added to the sample buffer vial and mixed; (2) the reader was turned on and a
lot card was inserted, its information uploaded, and then it was removed (one time only per
manufacturer's lot of test cartridges); (3) user and sample identification were entered into the
reader: (4) the test cartridge was removed from its package along with the 70-|_iL pipette tip (the
presence of a pink dot on the inside of the tip was noted); (5) the tip was firmly pressed on the
micropipette; (6) the sample was mixed by filling and dispensing the micropipette 10 times
(after which it was confirmed that the pink dot in the tip had disappeared); (7) the micropipette
was filled and dispensed into the sample well on the test cartridge; and (8) the test cartridge was
immediately pressed into the reader until resistance was felt. The reader then automatically
performed all time and measurement processes and generated the results in approximately
15 minutes. The verification staff recorded the sample identification number on a sample data
sheet; then, after the analysis was complete, the result was recorded on the sample data sheet.
3.3.2 Non-Laboratory Testing
Because the toxic nature of the contaminants did not permit their use outside special laboratory
facilities, MB samples were analyzed at a non-laboratory location to evaluate the RAMP® per-
formance and ease of use outside of the laboratory. Both a trained technician and a
non-technical, untrained first-time user performed analyses at the non-laboratory location. The
The instructions provided within the box of test cartridges stated that the sample should be collected by
placing a swab in the test sample for 3 seconds, transferring the swab into the sample buffer, and repeating once. At
the suggestion of Response Biomedical Corp., a 10-^L pipettor was used to increase the efficiency of our analyses.
-------
purpose of these analyses was to test the performance of the RAMP® in a non-laboratory setting,
not to evaluate thoroughly the effect of changing conditions such as temperature and humidity
on the RAMP®. Initially, the non-technical, untrained first-time user was guided only by the
vendor instructions. If the operators were about to complete the test incorrectly, the Verification
Test Coordinator prompted them to re-evaluate the instructions. The operators for the rest of the
verification test had undergraduate degrees in the sciences or equivalent work experience and
either participated in a training session provided by the vendor prior to the verification test or
were trained by a vendor-trained operator.
3.3.3 Drinking Water Characterization
An aliquot of each DW sample, collected as described in Section 3.2.2, was sent to ATEL prior
to concentration to determine the following water quality parameters: turbidity; concentration of
dissolved and total organic carbon; conductivity; alkalinity; pH; concentration of Ca and Mg;
hardness; and concentration of total organic halides, trihalomethanes, and haloacetic acids.
Table 3-4 lists the methods used to characterize the DW samples, as well as the characterization
data from the four water samples collected as part of this verification test. Water samples were
collected and water quality parameters were measured by ATEL in January. Samples were then
transported and test cartridges were analyzed from January through March. Because of this,
some of the water quality parameters may have changed from the time of analysis by ATEL until
testing with the RAMP® test cartridges.
10
-------
Table 3-4. ATEL Water Quality Characterization of Drinking
Water Samples
Sources of Drinking Water Samples
Parameter
Turbidity
Dissolved
organic carbon
Total organic
carbon
Specific
conductivity
Alkalinity
pH
Calcium
Magnesium
Hardness
Total organic
halides
Trihalomethanes
Haloacetic acids
Unit
NTU
mg/L
mg/L
|iS/cm2
mg/L
mg/L
mg/L
mg/L
Hg/L
Hg/L/
analyte
Hg/L/
analyte
Method
EPA180.1(5)
SM5310(6)
SM5310(6)
SM2510(6)
SM 2320(6)
EPA150.1(7)
EPA200.8(8)
EPA200.8(8)
EPA 130.2(7)
SM 5320(6)
EPA 524.2(9)
EPA 552.2(10)
Columbus,
Ohio
(OH DW)
0.2
2
2
357
55
7.33
42
5.9
125
360
26.9
23.2
Orlando,
Florida
(FL DW)
0.5
2
2
325
124
7.93
41
8.4
137
370
80.9
41.1
New York City,
New York
(NY DW)
1.3
2
2
85
4
6.80
5.7
19
28
310
38.4
40.3
MWD,
California
(CA DW)
0.1
2
2
740
90
7.91
35
1.5
161
370
79.7
17.6
NTU = nephelometric turbidity unit
MWD = Metropolitan Water District
I^S/cm2 = micro Siemen per square centimeter
11
-------
Chapter 4
Quality Assurance/Quality Control
Quality assurance/quality control (QC) procedures were performed in accordance with the
quality management plan (QMP) for the AMS Center(11) and the test/QA plan(4) for this
verification test.
4.1 Sample Chain-of-Custody Procedures
Sample custody was documented throughout collection, shipping, and analysis of the samples.
Sample chain-of-custody procedures were in accordance with ASAT n-007, Standard Operating
Procedure for Chain of Custody for Dioxin/Fur an Analysis. The chain-of-custody forms
summarized the samples collected and analyses requested and were signed by the person
relinquishing samples once that person had verified that the custody forms were accurate. The
original sample custody forms accompanied the samples; the shipper kept a copy. Upon receipt
at the sample destination, sample custody forms were signed by the person receiving the samples
once that person had verified that all samples identified on the custody forms were present in the
shipping container.
4.2 Equipment/Calibration
The RAMP® test cartridges and all appropriate reagents and supplies specific for the detection of
anthrax, botulinum toxin, and ricin were provided to Battelle by the vendor. These test kits, each
containing an internal fluorescent control, required no calibration. For DW characterization and
confirmation of the possible interferent, analytical equipment was calibrated by ATEL according
to the procedures specified in the appropriate standard methods. Pipettes used during the
verification test were calibrated according to Battelle Standard Operating Procedure (SOP) VI-
025, Operation, Calibration, and Maintaining Fixed and Adjustable Volume Pipettes.
4.3 Characterization of Contaminant Stock Solutions
4.3.1 Characterization of Botulinum Toxin and Ricin
Certificates of analysis for botulinum toxin and ricin were provided by the supplier of those
contaminants. Because standard reference methods do not exist, the concentration of botulinum
12
-------
toxin and ricin were not independently confirmed. The COAs stated that the ricin standard
(Vector Laboratories, Inc., Burlingame, California) had a concentration of 1,000 mg/L and the
botulinum toxin standards (Metabiologics, Inc., Madison, Wisconsin) had concentrations of
2,000 mg/L for Type B and 1,000 mg/L for Type A. Test samples containing these contaminants
were prepared by diluting aliquots of these stock solutions with DI water.
4.3.2 Characterization of A nthrax Spores
Multiple sources and forms of the Ames strain of Bacillus anthracis (anthrax) were evaluated
during this verification test. The primary source was a lot of spores prepared by Battelle and
stored in a 1% stock solution of phenol in water as a means to prevent vegetative cell growth.
This lot of spores is referred to in this report as Battelle-prepared, phenol-preserved. Prior to
testing, an aliquot of the stock solution described above was centrifuged, the supernatant
consisting of the phenol/water solution was decanted from the spores, and the spores were
reconstituted with DI water. This process was repeated two times to ensure that the spores were
suspended only in DI water. This lot of spores was characterized with an 11-step
characterization process prior to use in the verification test. For confidentiality reasons, Table 4-
1 gives the outcome of only five of the characterization parameters, as well as the location at
which each step was performed. These characterization steps were performed when this lot of
spores was prepared in September 2003. It should be noted that, once a stock solution of spores
is characterized, less concentrated solutions of spores can be prepared from the stock solution
without questioning the integrity of the spores. This lot of spores met all 11 acceptance criteria.
Two parts of the characterization process—DNA sequencing and gene identification—were
performed by Dr. Alex Hoffmaster at the Epidemiologic Investigations Laboratory, Meningitis
and Special Pathogens Branch of the Centers for Disease Control and Prevention (CDC). The
CDC analyses confirmed that the spores were Ames strain anthrax spores, and the guinea pig
LD50 study confirmed their virulence. The stock solution of spores was enumerated after
preparation to determine its original concentration. In addition, a vegetative cell analysis showed
that the stock solution was 99.94% anthrax spores. Because at least one spore is needed to spur
the growth of a colony during an enumeration, the concentrations determined represented a
minimum concentration of spores. Care was taken to spread the samples to avoid clumping; but,
if clumping occurred, the spore concentrations would only be higher than shown in the data
tables.
Table 4-1. Characterization Information for Battelle Preparation of Anthrax Spores
Characterization Outcome Analysis Performed By
% vegetative cells 0.06% Battelle
Viable spore count 5.26xio9 Battelle
Guinea pig 10 day LD50 10 spores Battelle
DNA fingerprinting MLVA Genotype 62 CDC
PA gene sequencing Protective Antigen Type I CDC
13
-------
Another lot of anthrax spores prepared by Battelle was used during the verification test. This lot
had been prepared in the same way as the other, but it had never been stored in phenol or any
other preservative and had not been characterized like the previously described lot. The second
lot had been subjected only to enumeration in order to determine the concentration. Test
solutions were made from this stock solution to investigate whether the phenol preservation was
affecting the sensitivity of the test cartridges.
Similarly, a lot of anthrax spores from Dugway Proving Ground was obtained and used to
investigate the sensitivity of RAMP® test cartridges to a different spore preparation (referred to
as Dugway-prepared in this report). Again, enumeration was the only characterization step
performed on this lot of spores.
A stock solution of vegetative anthrax cells also was prepared and used during this verification
test. Vegetative cells from an enumeration of the Battelle-prepared, phenol-preserved spores
were collected and placed in a solution and then enumerated to determine the concentration of
vegetative Bacillus anthracis in the solution. No further characterization was performed on these
vegetative anthrax cells. Solutions of these cells were used to determine the sensitivity of the
RAMP® test cartridges to vegetative cells. At the request of Response Biomedical Corp., Battelle
obtained an aliquot of anthrax vaccine from Colorado Serum Company, Denver, Colorado,
which is Stern Bacillus anthracis, to confirm the functionality of their technology. No
characterization or enumeration was performed on the vaccine.
Regardless of the source and type of anthrax stock solution used to make test samples, its
concentration was confirmed by a plate enumeration method. This was done within 24 hours of
any stock solution being used for test sample preparation and is described in Battelle SOP
MREF X-054, Enumeration ofBL-2 andBL-3 Bacteria Samples Via the Spread Plate
Technique. In addition, four times during the verification test the serial dilution method was
validated by enumerating the PT samples. For example, for a 109 spores/mL sample to be
enumerated, the method requires that it be diluted to at least 103 spores/mL so 100 |^L of sample
will provide a countable number of spores on a culture plate. Therefore, if 100 |^L of the 103
spores/mL solution provided the correct number of spores to the plate, the concentration of every
serial dilution made to obtain that concentration was confirmed.
4.3.3 Anthrax Enumeration Data
Table 4-2 gives the results of all plate enumerations performed throughout the verification test
on anthrax solutions prepared in DI water. The data from enumerations to validate the serial
dilution method are also given in Table 4-2. The expected concentration, as determined from a
previous enumeration (if available), the actual concentration, and the relative percent difference
between the two are given in the table. Relative percent difference (RPD) is determined using
the following equation, where E is the expected concentration and A is the actual concentration
as determined by the enumeration.
E-A\
RPD= x 100%
14
-------
Table 4-2. Anthrax Enumeration Data for PT Samples
Spore Solution
Description (units)
Date
Expected Actual
Concentration Concentration^
RPD
Battelle-prepared,
phenol -preserved
stock solution
(108 spores/mL)
Battelle-prepared,
phenol-preserved serial
dilution validations
(104 spores/mL)
Battelle-prepared, non-
phenol-preserved
(108 spores/mL)
Vegetative anthrax
(104 cfu/mL)
Dugway-prepared
(106 spores/mL)
January 28
January 28
January 30
February 2
February 10
February 26
March 1
March 23
January 28
January 30
March 2
March 23
February 5
February 12
March 23
March 24
March 22
March 23
March 24
53
58
53
61
61
82
63
67
10
40
10
1,000
Unknown
14
Unknown
260
Unknown
0.010
10
58
53
61
53
82
63
67
57
7.8
32
7.7
992
14
106
26
350
666
0.0081
8.0
9
9
15
14
55
23
5
14
22
20
24
1
NA
657
NA
35
NA
19
20
(a) Each enumeration involved the development of three to five plates. The average, standard deviation, and relative
standard deviation for each set of Battelle-prepared, phenol-preserved enumeration data were determined, and the
average relative standard deviation of all enumerations was calculated to estimate the variability in the enumeration
process, which was 15%.
NA = not applicable.
For the Battelle-prepared, phenol-preserved spores, only one enumeration resulted in a concen-
tration that was more than 25% different from the expected concentration. The average
concentration of the Battelle stock solution was 6 x 109 spores/mL (ranging from 5.3 x 109 to
8.2 x 109 spores). Over the two-month period that the stocks were used and the enumerations
performed, the relative standard deviation of the eight results was 15%. The accuracy and
precision of these enumerations indicate that the concentration of the spore stock solution was
consistent over several months and was usually close to the expected concentration. The serial
dilution validation data confirm that the PT samples containing the Battelle-prepared, phenol-
preserved spores were prepared accurately at various concentration levels. Also shown in Table
4-2 are the enumerations performed to determine the concentration of the alternate Battelle
preparation of spores (not preserved with phenol), vegetative anthrax cells, and a stock solution
of spores obtained from Dugway Proving Ground. Notable among these results was the
15
-------
significant increase in concentration of the alternative Battelle-prepared stock solution from
February 5 to February 12, 2004. Because this lot of spores was used only to determine the
effect of phenol preservation on the sensitivity of the RAMP® test cartridges, this observation
was not fully investigated. For enumerations with unknown expected concentrations, the
concentration of that particular solution or the stock from which it had been prepared had not
previously been determined.
Table 4-3 gives the enumeration data for all of the interferent PT (shaded) and DW samples that
were spiked with anthrax spores. For possible interferent samples and samples prepared in DW,
the addition of spores was confirmed by enumeration for at least one sample representing each
matrix. The results of the DW samples enumerated in late January and early February indicated
that the relative difference between the expected concentration and the actual concentration
ranged from 17 to 96%. The larger percent differences for the DW samples as compared with the
PT samples were not a surprise, considering that DW is presumably an interferent-prone matrix.
These data suggest that spore health is dependent on whether the solution is in DI water or DW.
However, the effect of DW on spore health seemed to be less significant when the concentration
of spores was higher. For example, in March, when the DW and interferent samples were spiked
with higher concentrations of anthrax spores, the difference between the expected concentration
and the actual concentration for the interferent samples was between 0 and 21% and for the DW
samples between 7 and 55%. Enumerations were performed to characterize the concentration of
spores in each sample matrix. For each test matrix, spores were enumerated within a day of
testing. In the Chapter 6 tables, the actual concentrations of the test samples have been corrected
for the result of the appropriate enumeration for that sample. Because not every test sample was
enumerated and some of the test samples were the result of dilutions of enumerated samples, not
every actual concentration will be represented directly in Table 4-2 or Table 4-3.
The concentrations of the possible cross-reactive interferents of soybean lectin (analogue of
ricin) and lipopolysaccharide (analogue of botulinum toxin) were not confirmed independent of
the COA received from the supplier because of the lack of available analytical methodologies for
these analytes. Samples containing Bacillus thuringiensis (analogue of anthrax) were confirmed
by the same enumeration method used for anthrax and were approximately an order of
magnitude less than expected because some spores were lost during washing with water.
Because the lowest detectable concentration of anthrax was much higher than Response
Biomedical Corp. had claimed, additional samples containing higher concentration levels of
anthrax were prepared and analyzed. Additional resources were not expended to determine the
cross-reactivity of Bacillus thuringiensis at comparable concentration levels.
4.4 Technical Systems Audit
The Battelle Quality Manager conducted a technical systems audit (TSA) to ensure that the
verification test was performed in accordance with the test/QA plan(4) and the AMS Center
QMP(11) As part of the audit, the Battelle Quality Manager reviewed the standards and methods
used, compared actual test procedures with those specified in the test/QA plan,(4) and reviewed
data acquisition and handling procedures. Observations and findings from this audit were
16
-------
Table 4-3. Anthrax Enumeration Results for Fortified
Sample
Sample
Description
Cone. CA DW
Cone. CA DW
Unconc. CA DW
0.5mg/LOC
2.5mg/LOC
230 mg/L Ca
90 mg/L Mg
46 mg/L Ca
Cone. CA DW
Unconc. CA DW
Cone. OH DW
Unconc. OH DW
Cone. NY DW
Unconc. NY DW
Cone. FL DW
Unconc. FL DW
Cone. NY DW
Cone. CA DW
2. 5 mg/L OC
230 mg/L Ca
90 mg/L Mg
2. 5 mg/L OC
Cone. CA DW
230 mg/L Ca
Cone. NY DW
Date
(2004)
January 28
January 30
January 30
February 2
February 3
February 3
February 3
February 3
February 3
February 3
February 3
February 3
February 3
February 3
February 3
March 3
March 3
March 3
March 3
March 23
March 23
March 24
March 24
Expected
Concentration
(10s spores/mL)
10
100
40
15
15
15
15
15
15
15
15
15
15
15
15
1,000
1,000
1,000
1,000
1,000
1,000
1,000
1,000
Interferent and Drinking Water
Actual
Concentration'"'
(10s spores/mL)
0.38
8.7
8
16
16
5.6
8.3
6.9
6.5
5.7
6.9
13
12
9.1
7.5
933
1,100
993
1,000
962
448
788
486
RPD
96
91
80
9
9
63
45
54
57
62
54
17
21
39
50
7
10
1
0
4
55
21
51
OC = Organic carbon (humic and fulvic acids)
Shading on table distinguishes the interferent and cross-reactivity PT samples from the DW samples.
The uncertainty of the enumeration technique is approximately 15%.
17
-------
documented and submitted to the Battelle Verification Test Coordinator for response. No
findings were documented that required any significant action. The records concerning the TSA
are permanently stored with the Battelle Quality Manager.
4.5 Audit of Data Quality
At least 10% of the data acquired during the verification test was audited. Battelle's Quality
Manager or designee traced the data from the initial acquisition, through reduction and
statistical analysis, to final reporting, to ensure the integrity of the reported results. All
calculations performed on the data undergoing the audit were checked.
4.6 QA/QC Reporting
Each internal assessment and audit was documented in accordance with Sections 3.3.4 and 3.3.5
of the QMP for the ETV AMS Center.(11) Once the assessment report was prepared, the Battelle
Verification Test Coordinator responded to each potential problem and implemented any
necessary follow-up corrective action. The Battelle Quality Manager ensured that follow-up
corrective action was taken. The results of the TSA were sent to the EPA.
4.7 Data Review
Records generated in the verification test were reviewed before they were used to calculate,
evaluate, or report verification results. Table 4-4 summarizes the types of data recorded. The
review was performed by a technical staff member involved in the verification test, but not the
staff member who originally generated the record. The person performing the review added
his/her initials and the date to a hard copy of the record being reviewed.
18
-------
Table 4-4. Summary of Data Recording Process
Data to Be Recorded
Dates and times of test
events
Responsible
Party
Battelle
How Often
Where Recorded Recorded
ETV data sheets Start/end of test,
and at each
change of a test
parameter
Disposition
of Data
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary
Sample collection and
preparation information,
including chain-of-
custody
Battelle
ETV data sheets
and chain-of-
custody forms
At time of sample
collection and
preparation
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary
Detection device Battelle
procedures and sample
results
ETV data sheets
Throughout test
duration
Manually incorporated
in data spreadsheets
Anthrax enumeration
data
Battelle
Enumeration data
forms
With every
enumeration
Manually incorporated
in data spreadsheets
Reference method ATEL
procedures and sample
results
Data acquisition
system, as
appropriate
Throughout
sample analysis
process
Transferred to
spreadsheets and
reported to Battelle
19
-------
Chapter 5
Statistical Methods and Reported Parameters
The methods presented in this chapter were used to verify the performance parameters listed in
Section 3.1. The RAMP® test cartridges produce qualitative results; i.e., they indicate only the
presence or absence of a contaminant, not a measure of the concentration present. Therefore, the
data evaluation methods were used in that context.
5.1 Qualitative Contaminant Presence/Absence
Accuracy was assessed by reporting the number of positive results out of the total number of
samples tested for the RAMP® test cartridges at each concentration level of contaminant-only PT
sample tested for anthrax spores, botulinum toxin, and ricin.
5.2 False Positive/Negative Responses
A false positive response was defined as a positive response when the DI water or DW sample
was spiked with a potential interferent, a cross-reactive compound, or not spiked at all. A false
negative response was defined as a negative response when any sample was spiked with a
contaminant at a concentration greater than the lowest detectable concentration of the test
cartridge for each analyte in DI water. Interferent PT samples, cross-reactivity PT samples, and
DW samples were included in the analysis. The number of false positive and negative results is
reported.
5.3 Consistency
The reproducibility of the results was assessed by calculating the percentage of individual test
samples that produced positive or negative results without variation within replicates.
5.4 Lowest Detectable Concentration
The lowest detectable concentration for each contaminant was determined to be the
concentration level at which at least two out of the three replicates generated positive responses.
These concentration levels are determined for each target contaminant in solutions of DI water.
20
-------
5.5 Other Performance Factors
Aspects of the RAMP® test cartridge performance such as ease of use, field portability, and
sample throughput are discussed in Section 6. Also addressed are qualitative observations of the
verification staff pertaining to the performance of the RAMP®.
21
-------
Chapter 6
Test Results
6.1 Qualitative Contaminant Presence/Absence
The responses for the RAMP® test cartridges using the contaminant-only PT samples containing
anthrax, botulinum toxin, and ricin are discussed in the following sections. The RAMP® test
cartridges provide indication of only a positive or negative response based on whether or not the
level of fluorescence is above the detectable threshold. Upon sample application, the test
cartridge was inserted into the RAMP® reader, which determined conclusively whether the test
result was positive or negative. Note that the appendix to this report contains a vendor-supplied
reference on the performance of the test cartridges.
6.1.1 Anthrax
The results obtained for the performance test samples containing anthrax spores are given in
Table 6-1 a. The first five concentration levels listed were initially analyzed, and the results
indicated that none of these samples (up to 50 times the vendor-stated LOD) produced
detectable results. The Battelle-prepared, phenol-preserved serial dilution validation
enumeration on February 2 confirmed the concentration of the stock solution used to prepare the
solutions analyzed by the RAMP® test cartridges. In addition, the serial dilution validation
enumeration on January 30 (4 x 10s spores/mL expected) confirmed that the anthrax solutions
can be accurately diluted using standard techniques. After discussions with Response
Biomedical Corp., the following speculative explanations for these results were considered:
1. The target proteins on the spore's surface may have been stripped off or chemically altered
by phenol in the storage solution. (The absence or alteration of these proteins would
probably decrease the sensitivity of the RAMP® test cartridges to the affected spores.)
2. The sensitivity of the RAMP® test cartridges to anthrax spores is dependent on the method
used to prepare the spores; therefore, the spores prepared at Battelle may result in decreased
responsiveness compared with spores prepared elsewhere.
3. The RAMP® test cartridges are more sensitive to vegetative anthrax cells than spores. (This
hypothesis stemmed from the analysis of one sample that was prepared by collecting a single
vegetative anthrax colony from an enumeration plate and placing it into DI water and mixing
well. This sample produced one out of two positive results using the RAMP® test cartridges;
however, the solution was not enumerated so the concentration was not known.)
22
-------
Table 6-1 a. Anthrax Contaminant-Only
Purpose of
Analysis
Original test/QA
plan PT samples
Investigation of
phenol storage
of spores
Sensitivity
determination
Alternate spore
preparation
Vegetative cell
sensitivity
Actual Fortified
Concentration^
200 spores/mL(b)
3 x 105spores/mL(c)
2 x 106spores/mL
4 x 106 spores/mL
2 x 107spores/mL
2 x 107spores/mL
1 x 109spores/mL
8 x 108spores/mL
1 x 1010spores/mL
8 x 109spores/mL
8 x 108spores/mL
8 x 107spores/mL
8 x 106spores/mL
7 x 108 spores/mL
8 x 1 Q7 spores/mL
Unknown
concentration
Unknown Cone.
3 x 105cfu/mL
3 x 104cfu/mL
PT Sample Results
Anthrax
Description
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Spores
Stern
vaccine
Vegetative
Vegetative
Vegetative
Prep Phenol-
Location Preserved
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
Dugway
Dugway
Colorado
Serum
Battelle
Battelle
Battelle
Yes
Yes
Yes
Yes
Yes
No
No
Yes
No
Yes
Yes
Yes
Yes
No
No
No
NA
NA
NA
Positive
Results
Out of
Total
Replicates
0/3
0/3
0/3
0/3
0/3
0/3
2/2(d)
3/3
low signal
low signal
3/3
0/3
0/3
3/3
0/1
4/4
2/2
2/3
0/1
NA = not applicable. Vegetative cells were not prepared from any stock solution; they were grown and placed in
solution.
® Actual concentrations were corrected for the enumeration of the stock solution from which each sample was
prepared. The uncertainly of the enumeration technique is approximately 15%.
(b:i Lethal dose concentration.
^ This concentration is very close to the vendor-stated LOD.
' ' The result for one additional replicate was "low signal" error.
23
-------
Additional testing beyond that described in the test/QA plan was performed to explore these
possible explanations and to gain more information about the performance of the RAMP® test
cartridges. It included evaluating whether Battelle's storage of the stock solution of anthrax
spores in a 1% solution of phenol had any impact on the performance of the RAMP® test
cartridges; increasing the concentration of spores beyond what was required by the test/QA plan;
subjecting the test cartridges to Ames strain anthrax spores prepared by Dugway Proving
Ground using a preparation method that is different from the one Battelle uses; subjecting the
test cartridges to the anthrax vaccine (Stern strain anthrax spores); and testing the response of
the test cartridges to vegetative anthrax cells at various concentrations.
To address the possibility that storing spores in phenol affected the sensitivity of the RAMP®
test cartridges, a series of samples was prepared and analyzed using one anthrax spore stock
solution that had been stored in a phenol solution and one that had not. The data are given in
Table 6-1 a under "Purpose of Analysis, Investigation of phenol storage of spores." Both
solutions had been prepared at Battelle using the same preparation method. The 2 x io7
spores/mL sample made with spores not stored in phenol produced negative results in all three
samples, as did the spores stored in phenol. In addition, samples containing concentrations of
approximately IO10 and IO9 spores/mL of spores from both phenol and non-phenol stock
solutions were analyzed. The IO10 spores/mL solutions produced a "low signal" error message
for each replicate, a message that indicates that the rate of sample moving across the adsorbent
cartridge was not adequate. Response Biomedical Corp. informed Battelle that this error can
occur when analyzing highly concentrated solutions of spores because of the increased viscosity
of the sample. The approximately IO9 spores/mL solution made with the Battelle-prepared non-
phenol-preserved spores generated two out of three positive results and the Battelle-prepared
phenol-preserved spores generated three out of three positive results. These results suggested
that the effect of phenol storage was probably inconsequential to the sensitivity of the RAMP®
test cartridges to anthrax spores.
The second explanation of the results at the first five concentration levels was investigated by
preparing and analyzing samples containing approximately IO9, IO8, and IO7 spores/mL from the
original stock solution that had been stored in phenol, but washed with water prior to testing.
Since phenol storage apparently did not affect the sensitivity of the technologies to spores, this
series of samples was analyzed to determine the approximate sensitivity of the RAMP® test
cartridges to the Battelle-prepared spores. Only the highest concentration level was detectable;
therefore, the lowest detectable concentration was approximately IO9 spores/mL. Solutions of
spores that were prepared at Dugway Proving Ground and received at Battelle in 2001 were then
analyzed. Since 2001, the Dugway stock solution had been refrigerated as a solution of spores in
spent media. The solution was washed in DI water as described for the phenol storage solution
above and diluted by tenfold factors several times. Both the stock solution concentration and the
dilution methodology were confirmed by plate enumeration as shown in Table 4-2. These
samples were analyzed one concentration level at a time by decreasing concentration to
determine the approximate sensitivity to these spores. Three replicate analyses were performed
on the lowest detectable individual replicate. When determined in this manner, the lowest
detectable concentration of Dugway spores was 7* IO8 spores/mL, a level similar to that
determined for the Battelle-prepared spores. In addition, upon discussions with Response
Biomedical Corp., Battelle obtained Stern anthrax vaccine and analyzed it using two separate
24
-------
lots of the RAMP® test cartridges. The vaccine generated two out of two positive results for two
lots of test cartridges (4 out of 4 positive results overall). This indicated that Battelle was
operating the test cartridges properly and that the test cartridges were functioning.
The third explanation of the results was investigated by preparing a solution of vegetative cells
as described above. This solution was diluted by a factor of 10 four times, and then the stock
and two diluted samples were enumerated to determine the concentration of vegetative cells in
each sample. These samples were analyzed one concentration level at a time by decreasing
concentration to determine the approximate sensitivity to these vegetative cells. The lowest
detectable concentration of vegetative cells was 3x 10s colony-forming units (cfu)/mL, an order
of magnitude lower than the vendor-stated LOD for anthrax spores. While Response Biomedical
Corp. has not provided information with regard to the RAMP® test kit's vegetative cell
sensitivity, these results suggest that RAMP® test cartridges are much more sensitive to
vegetative cells than to spores.
6.1.2 Botulinum Toxin
The results obtained for the PT samples containing botulinum toxin are given in Table 6-lb.
Upon analyzing the first five concentration levels listed in the test/QA plan, there were no
positive responses. In response to these results, Response Biomedical Corp. informed Battelle
that their test cartridges were only sensitive to botulinum toxin Type A. Since botulinum toxin
Type B was described for use in the test/QA plan, samples were not initially analyzed using
botulinum toxin Type A. To more completely verify the RAMP® test cartridges, a limited
amount of expanded testing was conducted by analyzing four PT samples containing a range of
concentration levels (0.5, 2, 5, and 25 mg/L) of botulinum toxin Type A and two higher
concentration levels (200 and 1,000 mg/L) of botulinum toxin Type B. The results showed that
the RAMP® test cartridges were sensitive to botulinum toxin Type A at approximately 2 mg/L,
but were not able to detect botulinum toxin Type B at concentration levels up to 1,000 mg/L.
25
-------
Table 6-1 b. Botulinum Toxin Contaminant-Only PT Sample Results
Purpose
of Analysis
Original test/QA plan
PT samples
Expanded testing
Concentration
(mg/L)
0.3(a)
0.5®
2.5
5
25
0.5
2
5
25
200
1,000
Type of Botulinum
Toxin
B
B
B
B
B
A
A
A
A
B
B
Positive Results Out of Total
Replicates
0/3
0/3
0/3
0/3
0/3
0/3
2/3
2/3
3/3
0/3
0/3
(a) Lethal dose concentration.
(b) Vendor-stated LOD.
6.1.3 Ricin
The results obtained for the PT samples containing ricin are given in Table 6-1 c. With the
exception of the 1 mg/L sample, the vendor-stated LOD, each concentration level analyzed
generated 3 out of 3 positive responses.
Table 6-1 c. Ricin Contaminant-Only PT Sample Results
Purpose
of Analysis
Original test/QA plan
PT samples
Concentration
(mg/L)
J(a)
5
15<">
20
50
Positive Results Out of
Total Replicates
0/3
3/3
3/3
O /O
3/3
o /o
3/3
^Vendor-stated LOD.
(b:i Lethal dose concentration.
26
-------
6.2 False Positive/Negative Responses
Three types of samples were analyzed to evaluate the susceptibility of RAMP® test cartridges to
false positive and negative results. These included interferent PT samples, made up of DI water
fortified with Ca and Mg and samples fortified with humic and fulvic acids with and without the
addition of target contaminants; cross-reactivity PT samples, made up of DI water fortified with
a contaminant similar biologically or chemically with each specific target contaminant; and DW
samples both concentrated and unconcentrated and both with and without the addition of target
contaminants. A false positive result was defined as a positive result in the absence of the target
contaminant and a false negative result was defined as a negative result from a sample
containing detectable levels of each target contaminant.
6.2.1 Interferent PT Samples
The results from the interferent PT samples are given in Table 6-2. For test cartridges specific to
each contaminant, the number of positive results out of the number of replicates is given for PT
samples containing only the possible interferents and those possible interferents in the presence
of the listed concentration of target contaminant. For anthrax and botulinum toxin, expanded
testing included additional interferent PT samples (a higher concentration in the case of anthrax
and a different type in the case of botulinum toxin). No expanded testing involving interferent
PT samples was done for the ricin test cartridges.
Table 6-2. Interferent PT Sample Results
Interferent
Sample
46 mg/L Ca
18mg/LMg
230 mg/L Ca
90 mg/L Mg
0.5 mg/L
humic and
fulvic acid
2.5 mg/L
humic and
fulvic acid
Positive Results Out
Anthrax (spores/mL)
Blank 4xl06(a)
nn 0/3
U/J 2xl06^
0/3 2xl06^
0/3 0/3
4xio6(b)
0/3 °/3
4xio6(b)
lxl09(a)
NA
3/3
IxlQ9^
NA
3/3
IxlQ9^
of Total Replicates
Botulinum Toxin
Blank
0/3
0/3
0/3
0/3
TypeB
5
0/3
0/3
0/3
0/3
(mg/L)
Type A
5
NA
3/3
NA
2/3
Ricin
Blank
0/3
0/3
0/3
0/3
(mg/L)
10
3/3
3/3
3/3
3/3
NA = not applicable. Sample not analyzed during expanded testing.
(^ Expected concentration.
(b:i Actual concentration.
27
-------
When interferent solutions not fortified with target contaminants were analyzed, no false
positive results occurred for the test cartridges specific for any of the three target contaminants.
The lack of detectable results at 4 x 106 spores/mL for anthrax and 5 mg/L for botulinum toxin
Type B indicated false negative responses with respect to the vendor-stated LOD; however,
because those tested concentration levels for anthrax were not detectable when analyzed in DI
water, and the test cartridges were not sensitive to botulinum toxin Type B (see Section 6.1.1),
the lack of sensitivity within this testing scenario cannot be attributed to the presence of the
possible interferents. Expanded testing was performed by analyzing samples prepared using
concentration levels of anthrax detectable when prepared in DI water only and botulinum toxin
Type A. For anthrax spores, there were no false negative responses for the expanded testing. For
botulinum toxin Type A, there was one false negative response in the 2.5 mg/L humic and fulvic
acid sample. The lower concentration interferent matrix was not analyzed during the expanded
testing of anthrax and botulinum toxin samples.
6.2.2 DW Samples
The results from the DW samples are given in Table 6-3. For test cartridges specific to each
contaminant, the number of positive results out of the number of replicates is given for the DW
samples containing no target contaminants and also the DW samples in the presence of the listed
concentration of each target contaminant. For anthrax and botulinum toxin, expanded testing
included additional DW samples (a higher concentration in the case of anthrax and a different
type in the case of botulinum toxin) fortified with those two target contaminants. No expanded
testing involving DW samples was performed for the ricin test cartridges.
Table 6-3 shows that there were no false positive results for the test cartridges specific for any of
the three target contaminants when the unspiked DW samples were analyzed. The second
column of results under anthrax and botulinum toxin and the only results under ricin show false
negative responses with respect to the vendor-stated LOD (not as defined in Section 5.2). But for
the reasons detailed in the previous section, many of the negative results in Table 6-3 cannot be
attributed to the presence of the DW matrix. No further testing was completed for ricin, but for
anthrax spores and botulinum toxin, expanded testing for the spiked DW samples was done as
for the spiked interferent testing (Section 6.2.1). Only two DW samples, concentrated CA and
concentrated NY DW, were analyzed during the expanded testing of anthrax and botulinum
toxin samples. No false negative responses resulted from this expanded testing; in both cases the
analyses resulted in three out of three positive results.
28
-------
Table 6-3. DW Sample Results
DW Sample
Unconcentrated
CADW
Concentrated
CADW
Unconcentrated
FLOW
Concentrated
FLOW
Unconcentrated
NYDW
Concentrated
NYDW
Unconcentrated
OHDW
Concentrated
OHDW
Positive Results Out
Anthrax (spores/mL)
Blank 4xl06(a) IxlO9
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
2xl06(b)
0/3
2xl06(b)
0/3
2xl06(b)
0/3
2xl06(b)
0/3
3xl06(b)
0/3
3xl06(b)
0/3
2xl06(b)
0/3
2xl06(b)
NA
3/3
5xl08(b)
NA
NA
NA
3/3
5xl08(b)
NA
NA
of Total
Replicates
Botulinum Toxin (mg/L)
Type B Type A
Blank 5 5
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
NA
3/3
NA
NA
NA
3/3
NA
NA
Ricin
Blank
0/3
0/3
0/3
0/3
0/3
0/3
0/3
0/3
(mg/L)
10
3/3
3/3
3/3
3/3
3/3
3/3
3/3
3/3
NA = not applicable. Sample not analyzed during expanded testing.
(^ Expected concentration.
(b:i Actual concentration.
6.2.3 Cross-Reactivity PT Samples
The results from the cross-reactivity PT samples are given in Table 6-4. For test cartridges
specific to each target contaminant, a PT sample fortified with a spore or chemical similar to
each target contaminant was analyzed in the absence of any of the target contaminant. The
number of positive results out of the number of replicates is given for each sample. There were
not any false positive results for this evaluation. Each sample generated 3 out of 3 negative
responses. For Bacillus thuringiensis, the concentration analyzed was significantly less than the
lowest detectable concentration of anthrax; therefore, the results may not indicate a lack of
cross-reactivity.
29
-------
Table 6-4. Potentially Cross-Reactive PT Sample Results
Positive Results Out of Total Replicates
Botulinum
Anthrax Toxin Ricin
Bacillus thuringiensis (5 xio5 spores/mL)(a)
Lipopolysaccharide (5 mg/L)
Lectin from soybean (10 mg/L)
(^ Concentration was determined after the fact to be below the lowest detectable concentration. Therefore, the non-
detectable results may not indicate a lack of cross-reactivity.
6.3 Consistency
For the anthrax testing, at times the number of replicate analyses was reduced to conserve time
or available supplies. However, the available replicate data for anthrax suggests that
performance of the test cartridges was very consistent. Including all the sample sets analyzed for
anthrax, only one sample set generated results that were not either all negative or all positive, for
a consistency of 96%.
For botulinum toxin, the 2- and 5-mg/L samples of botulinum toxin Type A produced positive
results in two out of three replicates, while the 25 mg/L botulinum toxin Type A samples
produced three out of three positive results and the 0.5 mg/L Type A botulinum toxin generated
three out of three negative results. These results may suggest a correlation between consistency
and concentration. The rest of the botulinum toxin sample sets, with the exception of the
2.5 mg/L humic and fulvic acid interferent sample spiked with 5 mg/L botulinum toxin Type A
(two out of three positive), produced either all positive or all negative results. For botulinum
toxin, the consistency was 95%. For ricin, the results were consistent 100% of the time for all
the sample matrices. Either all replicates within a sample were positive or all were negative.
Overall, 95% of all the results were obtained in sets of two or three replicates in which all the
individual replicates had the same result, whether positive or negative.
6.4 Lowest Detectable Concentration
The lowest detectable concentration of each target contaminant was defined as the lowest
concentration of contaminant-only PT sample to have at least two out of three positive results.
For anthrax, that concentration was determined to be 8 x 108 spores/mL (Battelle-prepared),
7 x 108 spores/mL (Dugway-prepared), and 3 x 10s cfu/mL (vegetative cells); for botulinum
toxin Type A, 2 mg/L; and for ricin, 5 mg/L. The RAMP® test cartridges could not detect
botulinum toxin Type B.
30
-------
6.5 Other Performance Factors
Battelle technicians, who had been trained by Response Biomedical Corp. to perform testing
using the RAMP® test cartridges, performed all of the testing in a laboratory setting. The
technicians had no problem performing the tests as they were trained. The RAMP® test
cartridges require the use of a reader contained in a rugged carrying case that weighs
approximately 20 pounds and is about the size of a medium-sized suitcase. To test the ability of
the RAMP® test cartridges to be used outside a laboratory environment and by a non-trained
user, both a trained operator and an operator without any training in the sciences or in the
operation of the RAMP® test cartridges were given a liquid sample (DI water) and told to
analyze the sample three times. For the samples analyzed in a non-laboratory setting, the reader
was powered by batteries. Initially, the non-technical operator was guided only by the
instructions provided with each box of test cartridges. However, if the operator was about to
complete the test incorrectly, the Verification Test Coordinator prompted the operator to
reevaluate the instructions. Initially, the non-technical operator was confused by what the
micropipette was and how to use it, but figured it out without any assistance. During the first
sample analysis, the Verification Test Coordinator stopped the non-technical operator before
attempting to perform the mixing step after applying the sample to the sample well. The
instructions seemed to be clear, but the operator apparently misunderstood. When directed back
to the instructions, the operator then understood the proper technique and was able to perform
each step, including the operation of the reader, successfully. The non-technical operator
repeated the sample analysis two additional times. On the second attempt, the reader prompted
the operator to repeat the test because it took too long to insert the cartridge into the reader after
sample application. The operator completed the final two samples successfully with no
assistance. The experienced operator analyzed this sample in the correct way on the first attempt.
Over 400 RAMP® test cartridges were tested during the verification test. In nine instances, the
result instead of positive or negative was "low signal." This error message occurred when
insufficient sample was transferred to the test cartridge. However, eight out of nine instances
occurred when analyzing highly concentrated anthrax solutions when we were very confident of
the testing procedure. Upon discussions with Response Biomedical Corp., we were informed
that "low signal" result can also be generated from highly concentrated spore solutions because
of the increased viscosity of the sample. According to Response Biomedical Corp., this
precaution is in place to stop the over-sampling of powdered samples. Sample throughput was 4
samples per hour.
31
-------
Chapter 7
Performance Summary
Table 7-1. Anthrax Summary Table
Parameter
Qualitative
contaminant
results
Contaminant-
only PT samples
Interferent
PT samples
DW samples
Cross-reactivity
False positives
False negatives
Consistency
Positive Results
Actual Fortified Anthrax Out of Total
Sample Information Concentration'3' Replicates
8 x 108 spores/mL 3/3
Battelle-prepared, phenol- 8 x 107 spores/mL 0/3
preserved spores 8 x 106 spores/mL 0/3
3 x 105 spores/mL 0/3
3 x 105cfu/mL 2/3
Vegetative cells
3xl04cfu/mL 0/1
7 x 108spores/mL 3/3
Dugway-prepared spores
8 x 107spores/mL 0/1
230 mg/L Ca , , _9 , T CM *>n
„„„-., 1 x 10 spores/mLw 3/3
90 mg/L Mg r
2.5 mg/L humic acid , ,,,9 / T rw in
„ ._ B, , . . ., 1 x 109 spores/mL(b) 3/3
2.5 mg/L fulvic acid
Concentrated CA 5 x 108 spores/mL® 3/3
Concentrated NY 5 x 108 spores/mL00 3/3
Unconcentrated DW 4 x 106 spores/mL00 0/24
5 x 105 spores/mL
_ „ , unspiked 0/3
Bacillus thurmgiensis
No false positives resulted from the analysis of the interferent, DW, or cross-
reactivity samples. Bacillus thurmgiensis was prepared at concentrations much
lower than the lowest detectable concentration of Bacillus anthracis. Therefore,
negative results with these samples do not necessarily indicate a lack of cross-
reactivity.
No false negative results were generated from the analysis of the interferent and
DW samples spiked with detectable levels of anthrax spores; the RAMP® test
cartridges were not able to detect anthrax spores at the vendor-stated limit of
detection (LOD), but they were able to detect much higher concentration levels.
All of the unconcentrated DW samples were spiked at concentrations less than
detectable by the test strips and, therefore, were, as expected, negative.
96% of the results were obtained in replicate sets in which all the individual
replicates had the same result, whether positive or negative.
32
-------
Table 7-1. Anthrax Summary Table (continued)
Parameter
Sample Information
Lowest detectable concentration
8 x 108 spores/mL - Battelle prep ; 7x 108 spores/mL - Dugway prep (vendor-
stated LOD: 4 x 105 spores/mL); 3 x 105 cfu/mL - vegetative anthrax (no
vendor-stated LOD)
Other performance factors
All components for testing were provided in a box of 25 test cartridges; the
required cartridge reader was operated using electricity or batteries, was easy to
operate, and was contained in a rugged carrying case; test cartridges used easily
inside and outside a laboratory with trained operator; non-technical operator
needed minor direction from a trained operator; "low signal" resulted from
highly concentrated anthrax solutions; and sample throughput was 4 samples per
hour.
(a) The uncertainty of the enumeration technique was approximately 15%.
^ -* Battelle-prepared, phenol-preserved spores.
33
-------
Table 7-2. Botulinum Toxin Summary Table
Parameter
Contaminant-only PT
samples
Qualitative
contaminant
positive results
Interferent
PT samples
DW samples
Cross-reactivity
False positives
False negatives
Consistency
Botulinum Toxin
Sample Information Concentration (mg/L)
0.5
2
Type A
25
0.3
0.5
2.5
TypeB
5
200
1,000
230 mg/L Ca (a)
90 mg/L Mg
2.5 mg/L humic acid _(a)
2.5 mg/L fulvic acid
Concentrated CA 5(a)
Concentrated NY 5(a)
Unconcentrated DW 500
5 mg/L .. ,
, . . . . , unspiked
Lipopolysaccnande
Positive Results
Out of Total
Replicates
0/3
2/3
2/3
3/3
0/3
0/3
0/3
0/3
0/3
0/3
3/3
2/3
3/3
3/3
0/24
0/3
No false positives resulted from the analysis of the interferent, DW, or
cross-reactivity samples.
One false negative replicate resulted from the analysis of the 2.5 mg/L
humic and fulvic acid interferent samples spiked with a detectable level
of Type A botulinum toxin; in addition, the RAMP® test cartridges were
not able to detect Type B botulinum toxin spiked into DW at 5 mg/L or
in DI water at concentrations up to 1,000 mg/L.
95% of the results were obtained in replicate sets in which all the
individual replicates had the same result, whether positive or negative.
34
-------
Table 7-2. Botulinum Toxin Summary Table (continued)
Parameter
Sample Information
Lowest detectable concentration
2 mg/L (Type A), Type B was not detectable up to concentrations of
1,000 mg/L. (vendor-stated LOD for botulinum toxin [non-specific]: 0.5
mg/L)
Other performance factors
All components for testing were provided in a box of 25 test cartridges;
the required cartridge reader was operated using electricity or batteries,
was easy to operate, and was contained in a rugged carrying case; test
cartridges used easily inside and outside a laboratory with trained
operator; non-technical operator needed minor direction from a trained
operator; and sample throughput was 4 samples per hour.
(a) Type A botulinum toxin.
(b:i Type B botulinum toxin.
35
-------
Table 7-3. Ricin Summary Table
Parameter
Contaminant-
only PT
Samples
Qualitative
positive Interferent PT
results Samples
DW Samples
Cross-reactivity
False positives
False negatives
Consistency
Lowest detectable concentration
Other performance factors
Ricin Concentration
Sample Information (mg/L)
1
5
Ricin in DI water 1 5
20
50
CaandMg 10
Fulvic and humic acid 1 0
Concentrated DW 10
Unconcentrated DW 10
10 mg/L .. ,
, . ,, . unspiked
Lectm from soybean
Positive Results Out
of Total Replicates
0/3
3/3
3/3
3/3
3/3
6/6
6/6
12/12
12/12
0/3
No false positives resulted from the analysis of the interferent, DW, or cross-
reactivity samples.
No false negative results were generated by analyzing DW and interferent
samples spiked with detectable levels of ricin.
1 00% of the results were obtained in replicate sets in which all the individual
replicates had the same result, whether positive or negative.
5 mg/L (vendor-stated LOD: 1 mg/L)
All components for testing were provided in a box of 25 test cartridges; the
required cartridge reader was operated using electricity or batteries, was easy to
operate, and was contained in a rugged carrying case; test cartridges used easily
inside and outside a laboratory with trained operator; non-technical operator
needed minor direction from a trained operator; and sample throughput was 4
samples per hour.
36
-------
Chapter 8
References
1. Personal communication with Dick Burrows, U.S. Army Center for Health Promotion and
Preventive Medicine.
2. U.S. EPA threat prioritization study provided by Steve Allgeier, U.S. EPA Office of Water.
3. Center for Defense Information Fact Sheet: Ricin, www.cdi.org/terrorism;ricin-pr.cfm.
4. Test/QA Plan for Verification of Immunoassay Test Kits, Battelle, Columbus, Ohio,
January 2004.
5. U.S. EPA Method 180.1, "Turbidity (Nephelometric)," Methods for the Determination of
Inorganic Substances in Environmental Samples, EPA/600/R-93/100, August 1993.
6. American Public Health Association, et al. Standard Methods for the Examination of Water
and Wastewater. 19th Edition, Washington, D.C., 1997.
7. U.S. EPA, Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020,
March 1983.
8. U.S. EPA Method 200.8, "Determination of Trace Elements in Waters and Wastes by
Inductively-Coupled Plasma Mass Spectrometry," in Methods for the Determination of
Organic Compounds in Drinking Water, Supplement I, EPA/600/R-94/111, October 1994.
9. U.S. EPA Method 524.2, "Permeable Organic Compounds by Capillary Column GC/Mass
Spectrometry," Methods for the Determination of Organic Compounds in Drinking
Water—Supplement III, EPA/600/R-95/131, August 1995.
10. U.S. EPA Method 552.2, "Haloacetic Acids and Dalapon by Liquid-Liquid Extraction,
Derivatization and GC with Electron Capture Detector," Methods for the Determination of
Organic Compounds in Drinking Water—Supplement III, EPA/600/R-95/131, August 1995.
11. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center,
Version 5.0, U.S. EPA Environmental Technology Verification Program, Battelle,
Columbus, Ohio, March 2004.
37
-------
Appendix
Other Evaluations
It is the belief of Response Biomedical Corp. that the anthrax data generated during this
verification test is discrepant from that of data generated by other groups. For more information,
please review the following publication, which can be obtained from Response Biomedical Corp.
"Evaluation of a Rapid Immunoassay System for the Detection of Bacillus anthracis Spores,"
K. Heroux and P. Anderson; U.S. Army Edgewood Chemical Biological Center, Aberdeen
Proving Ground, Maryland.
------- |