November 2004
Environmental Technology
Verification Report
MICROSENSOR SYSTEMS INC.
HAZMATCAD™ PLUS
SURFACE ACOUSTIC WAVE/
ELECTROCHEMICAL DETECTOR
Prepared by
Battelle
Balteiie
I fie liusiiiess i>/ Innovation
Under a contract with
vxEPA
U.S. Environmental Protection Agency
ElV ET1/ ET1/
-------
November 2004
Environmental Technology Verification
Report
ETV Safe Buildings Monitoring and Detection
Technology Verification Program
Microsensor Systems Inc.
HAZMATCAD™ Plus
Surface Acoustic Wave/
Electrochemical Detector
by
Kent Hofacre
Tricia Derringer
Dale Folsom
Thomas Kelly
Loraine Sinnott
Cody Hamilton
Zachary Willenberg
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development, has financially supported and collaborated in the extramural program described
here. This document has been peer reviewed by the Agency. Mention of trade names or
commercial products does not constitute endorsement or recommendation by the EPA for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development provides data and science support that
can be used to solve environmental problems and to build the scientific knowledge base needed
to manage our ecological resources wisely, to understand how pollutants affect our health, and to
prevent or reduce environmental risks. EPA also addresses responsibilities in homeland security
through the National Homeland Security Research Center, by means of research programs in
Drinking Water Security, Safe Buildings, and Rapid Risk Assessment.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six verification centers. Information about
each of these centers can be found on the Internet at http://www.epa.gov/etv.
The ETV approach has also been applied to verification of homeland security technologies. The
verification reported herein was conducted by Battelle as part of the Safe Buildings Monitoring
and Detection Technology Verification Program, which is funded by EPA. Information
concerning this specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/centers/centerl 1 .html.
111
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We also would like to thank Tom
Taylor of the Centers for Disease Control and Prevention, Donald Stedman of the University of
Denver, and Mark Durno of the U.S. Environmental Protection Agency Region 5 for their
reviews of the test/quality assurance plan and this report.
IV
-------
Contents
Page
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations viii
1 Background 1
2 Technology Description 2
3 Test Design and Procedures 3
3.1 Introduction 3
3.2 Test Design 4
3.2.1 Chemical Test Compounds 4
3.2.2 Test Matrix 5
3.2.3 Test Locations 5
3.2.4 Test Sequence and Schedule 6
3.2.5 Reference Methods 8
3.2.6 Interferents 10
3.2.7 Materials and Equipment 12
3.3 Test Procedure 15
3.3.1 Response Time 16
3.3.2 Recovery Time 17
3.3.3 Accuracy 17
3.3.4 Repeatability 17
3.3.5 Response Threshold 17
3.3.6 Temperature and Humidity Effects 18
3.3.7 Interference Effects 18
3.3.8 Cold-/Hot-Start Behavior 19
3.3.9 Battery Life 19
3.3.10 Operational Characteristics 20
4 Quality Assurance/Quality Control 21
4.1 Equipment Calibration 21
4.1.1 Reference Methods 21
4.1.2 Instrument Checks 22
-------
4.2 Audits 23
4.2.1 Performance Evaluation Audit 23
4.2.2 Technical Systems Audit 23
4.2.3 Data Quality Audit 24
4.3 Quality Assurance/Quality Control Reporting 24
5 Statistical Methods 25
5.1 Statistical Analyses 25
5.1.1 Analysis of Response, Response Time, and Recovery Time 26
5.1.2 Analysis of Accuracy 27
5.1.3 Analysis of Repeatability 28
5.1.4 False Negatives and Positives Analysis 28
5.1.5 Analysis of Response to Alternating Concentrations 29
5.2 Other Analyses 29
6 Test Results 30
6.1 Response Time 30
6.2 Recovery Time 32
6.3 Accuracy 32
6.4 Repeatability 33
6.5 Response Threshold 34
6.6 Temperature and Humidity Effects 35
6.7 Interference Effects 36
6.8 Cold-/Hot-Start Behavior 38
6.9 Battery Life 40
6.10 Operational Characteristics 40
7 Performance Summary 43
8 References 46
Appendix A. Detailed Statistical Analysis Results
VI
-------
Figures
Figure 2-1. Microsensor Systems Inc. HAZMATCAD Plus 2
Figure 3-1. Sequence of TIC Verification Tests 7
Figure 3-2. Logic Diagram for Determining TIC Test Sequence 8
Figure 3-3. Test System Schematic 13
Tables
Table 3-1. Target TIC and CW Agent Challenge Concentrations 5
Table 3-2. Evaluations 6
Table 3-3. Test Schedule 8
Table 3-4. Primary Reference Methods 9
Table 3-5. Interferent Test Concentrations 11
Table 3-6. Temperature and Relative Humidity Conditions 14
Table 4-1. Performance Evaluation Audit Results 23
Table 6-1. Illustrative Data 31
Table 6-2. Response Threshold Data 35
Table 6-3. Interference Effects Data 37
Table 6-4. Cold-/Hot-Start Effects Data 39
Table 6-5. Responses Recorded in Battery Life Testing 41
vn
-------
List of Abbreviations
AC
AEGL
ANOVA
C12
CG
CK
CW
DEAE
EC
EPA
ETV
FID
FPD
GB
GC
HD
HMRC
IDLH
L
L/min
mg/m3
mL
mL/min
MSD
PE
ppb
ppm
ppmC
psig
QA
hydrogen cyanide
acute exposure guideline level
analysis of variance
chlorine
phosgene
cyanogen chloride
chemical warfare
N,N-diethylaminoethanol
electrochemical
U.S. Environmental Protection Agency
Environmental Technology Verification
flame ionization detection
flame photometric detection
sarin
gas chromatography
sulfur mustard
Hazardous Materials Research Center
immediately dangerous to life and health
liter
liter per minute
microgram
microgram per cubic meter
microliter
milligram per cubic meter
milliliter
milliliter per minute
mass selective detection
performance evaluation
part per billion
part per million
part per million of carbon
pound per square inch gauge
quality assurance
Vlll
-------
QMP quality management plan
RH relative humidity
SA arsine
SAW surface acoustic wave
THC total hydrocarbon
TIC toxic industrial chemical
TSA technical systems audit
VOC volatile organic compound
IX
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental tech-
nologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative tech-
nologies by developing test plans that are responsive to the needs of stakeholders, conducting
field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-
reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance
(QA) protocols to ensure that data of known and adequate quality are generated and that the
results are defensible.
Subsequent to the terrorist attacks of September 11, 2001, this ETV approach has been applied
to verify the performance of homeland security technologies. Monitoring and detection
technologies for the protection of public buildings and other public spaces fall within the Safe
Buildings Monitoring and Detection Technology Verification Program, which is funded by EPA
and conducted by Battelle. In this program, Battelle recently evaluated the performance of the
Microsensor Systems Inc. HAZMATCAD Plus portable detector, which uses surface acoustic
wave (SAW) and electrochemical (EC) technologies for detecting chemical warfare (CW) agents
and toxic industrial chemicals (TICs), respectively.
-------
Chapter 2
Technology Description
The objective of the ETV Safe Buildings Monitoring and Detection Technology Verification
Program is to verify the performance characteristics of monitoring technologies for chemical
and/or biological contaminants that might be introduced into the building environment. This
verification report provides results for the verification testing of the HAZMATCAD Plus
portable SAW/EC detector made by Microsensor Systems Inc. Following is a description of the
HAZMATCAD Plus, based on information provided by the vendor. The information provided
below was not subjected to verification in this test.
The HAZMATCAD Plus is a hybrid system using SAW sensors and ECs to detect CW agents
and TICs, respectively. SAW sensors are small, solid-state devices that are extremely sensitive to
minute changes in mass. These devices are coated with specific
polymers that selectively absorb contaminants. The polymer
surface responds rapidly and reversibly to nerve and blister
agents. Using an array of three coated SAW sensors provides a
response pattern that is unique to CW agents. The architecture
of the SAW system allows for a high level of specificity in a
complex sample. A preconcentrator is used to prepare sample
for delivery to the SAW array. In this test, the HAZMATCAD
Plus was operated in its "Fast" response mode, which employs a
20-second preconcentration cycle. The ECs are semi-selective
and provide a rapid response to contaminants. Amperometric
ECs use an electrolyte that is sealed behind a gas-permeable
membrane. Gases and vapors diffuse through the membrane and
dissolve in the electrolyte. Subsequent oxidation/reduction
processes release electrons that are collected at an electrode. The
resulting current signal is proportional to the amount of gas or
vapor sampled.
Figure 2-1. Microsensor
Systems Inc.
HAZMATCAD Plus
(3.4 pounds) including batteries.
HAZMATCAD Plus software logs and date stamps all alarms or
systems faults. The HAZMATCAD Plus weighs 1.5 kilograms
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
When first responders arrive at a potentially contaminated site, they need to immediately and
accurately identify chemicals that may be present. Chemicals and chemical agents that may pose
a threat in a building could include both TICs and CW agents.
The objective of this verification test of the HAZMATCAD Plus, a commercially available,
portable SAW/EC detector, was to evaluate its ability to detect toxic chemicals and chemical
agents in indoor air. This verification focused on the scenario of a portable detector used by first
responders to identify contaminants and guide emergency response activities after chemical
contamination of a building. This verification was conducted according to a peer-reviewed
test/QA plan(1) that was developed according to the requirements of the Quality Management
Plan (QMP) for the ETV program.(2) The following performance characteristics of the
HAZMATCAD Plus were evaluated:
Response time
Recovery time
Accuracy
Repeatability
Response threshold
Temperature and humidity effects
Interference effects
Cold-/hot-start behavior
Battery life
Operational characteristics.
Response time, recovery time, accuracy, and repeatability were evaluated by challenging the
HAZMATCAD Plus with known vapor concentrations of target TICs and CW agents.
HAZMATCAD Plus performance at low target analyte concentrations was evaluated to assess
the response threshold. Similar tests conducted over a range of temperatures and relative
humidities (RH) were used to establish the effects of these factors on detection capabilities. The
effects of potential interferences in an emergency situation were assessed by sampling those
interferences both with and without the target TICs and CW agents present. The HAZMATCAD
Plus was tested after a cold start (i.e., without the usual warm-up period) and after hot storage to
evaluate the delay time before readings could be obtained and the response speed and accuracy
-------
of the HAZMATCAD Plus once readings were obtained. All readings of a target chemical were
obtained with the HAZMATCAD Plus operated on battery power. Battery life was determined as
the time until HAZMATCAD Plus performance degraded as battery power was exhausted in
continuous operation. Operational factors such as ease of use, data output, and cost were
assessed by observations of the test personnel and through inquiries to the vendor.
It was intended that all tests would be conducted with two HAZMATCAD Plus units (Units 22
and 27). However, as a result of occasional problems, testing continued in a few cases with only
one HAZMATCAD Plus unit.
Testing was limited to detecting chemicals in the vapor phase because that mode of application
is most relevant to use by first responders. Testing was conducted in two stages: detection of
TICs (conducted in a non-surety laboratory at Battelle) and detection of CW agents (conducted
in a certified surety laboratory at Battelle's Hazardous Materials Research Center [HMRC]).
3.2 Test Design
3.2.1 Chemical Test Compounds
The TICs used in testing were
Hydrogen cyanide (HCN, North Atlantic Treaty Organization designation AC)
Cyanogen chloride (C1CN, designated CK)
Phosgene (COC12, designated CG)
Chlorine (C12, no military designation)
Arsine (AsH3, designated SA).
The CW agents were sarin (GB) and sulfur mustard (HD).
It should be noted that the HAZMATCAD Plus was programmed to detect all of these
compounds expect CK; the library in the HAZMATCAD Plus at the time of testing did not
include an alarm for the presence of CK. However, the instruments were challenged briefly with
CK to confirm absence of response. Also, it should be noted that the chemical identification that
the HAZMATCAD Plus units were programmed to display upon detecting a chemical varied
among the target compounds. For AC, the HAZMATCAD Plus identifiers were "BLOD" at
higher concentration levels and "CHOK" at lower concentration levels, indicating the presence
of a blood or choking agent. For CG, the HAZMATCAD Plus identifier was "CHOK,"
indicating the presence of a choking agent. For SA, the identifier was "HYDR," indicating the
presence of a hydride. For C12, the HAZMATCAD Plus identifier was "HALO," indicating the
presence of a halogen. For the CW agents, GB was identified as "G," and HD was identified as
"H." HAZMATCAD Plus Unit 22 did not alarm when challenged with GB, so the GB test
results are based on Unit 27 responses only.
Table 3-1 summarizes the concentrations of each TIC and CW agent used in this verification
test. For the TICS AC, CK, CG, SA, and C12, tests were conducted at the immediately
-------
dangerous to life and health (IDLH) levels. For the CW agents GB and HD, testing was
conducted at a concentration level that produced a mid-scale to full-scale reading on the
HAZMATCAD Plus under normal temperature and humidity conditions. The concentration used
for GB was 0.39 part per million (ppm) (2.2 milligrams per cubic meter (mg/m3), which is 11
times the IDLH concentration of 0.035 ppm (0.2 mg/m3). No IDLH level has been set for HD, so
the concentration used was based on an alternative toxic effects guideline, as noted in the
footnote of Table 3-1.
Table 3-1. Target TIC and CW Agent Challenge Concentrations
Chemical
AC
CK
CG
SA
C12
GB
HD
Challenge Concentrations
50 ppm (50 mg/m3)
20 ppm (50 mg/m3)
2 ppm (8 mg/m3)
3 ppm (10 mg/m3)
10 ppm (30 mg/m3)
0.39 ppm (2.2 mg/m3)
0.6 ppm (4 mg/m3)
Type of Level
IDLH(a)
IDLH
IDLH
IDLH
IDLH
11*IDLH
7*AEGL(b)
(a) IDLH value for CK estimated from value for AC.
(b) AEGL = Acute Exposure Guideline Level; AEGL-2 levels are those expected to produce a serious hindrance to
efforts to escape in the general population. The AEGL-2 value of 0.09 ppm (0.6 mg/m3) for HD is based on a 10-
minute exposure.
3.2.2 Test Matrix
Table 3-2 summarizes the evaluations that were conducted in the verification test. As Table 3-2
indicates, except for cold-/hot-start behavior, battery life, and assessment of false positive inter-
ference effects (i.e., the interferent alone), all performance parameters were evaluated during
both the TIC and CW agent testing. Cold-/hot-start behavior and battery life were tested only
with AC.
3.2.3 Test Locations
Two laboratories were used to conduct the verification test. Testing with the non-chemical
surety materiel—TICs and interferents—was conducted in a new laboratory at Battelle's
Columbus, Ohio, campus, which has the needed vapor generation, collection, and analysis
equipment. This laboratory has been used previously to conduct detector and filter tests using
CG, AC, CK, SA, and C12 under controlled environmental conditions. Testing with CW agents
was conducted at the HMRC at Battelle's West Jefferson, Ohio, campus. Battelle's HMRC is an
ISO 9001-certified facility that provides a broad range of materials testing, system and
component evaluation, research and development, and analytical chemistry services requiring
the safe use and storage of highly toxic substances. Battelle operates the HMRC in compliance
with all applicable federal, state, and local laws and regulations, including Army regulations.
-------
Table 3-2. Evaluations
Performance
Parameter
Objective
Comparison Based On
Response Time
Recovery Time
Accuracy
Repeatability
Response
Threshold
Temperature and
RH Effects
Interference
Effects
Cold Start
Hot Start
Battery Life
Determine rise time of
HAZMATCAD Plus response
Determine fall time of
HAZMATCAD Plus response
Characterize reliability of
identification of TICs or CW agents
Characterize consistency of
HAZMATCAD Plus readings with
constant analyte concentration
Estimate minimum concentration that
produces HAZMATCAD Plus
response
Evaluate effect of temperature and
RH on HAZMATCAD Plus
performance
Evaluate effect of building
contaminants that may
interfere with
HAZMATCAD Plus performance
Characterize start-up performance
after cold storage
Characterize start-up performance
after hot storage
Characterize battery life and
performance
HAZMATCAD Plus readings with step rise
in analyte concentration
HAZMATCAD Plus readings with step
decrease in analyte concentration
HAZMATCAD Plus identifier display in
testing each TIC or CW agent
HAZMATCAD Plus readings with constant
input
Reference method results
Repeat above evaluations with different
temperature and RH
Sample interferents and TICs/CW agents
together (and interferents alone(a))
Repeat tests with no warm-up(a)
Repeat tests with no warm-up(a)
HAZMATCAD Plus readings with constant
input,(a) while depleting batteries in
continuous operation
(a) Indicates this part of the test performed only during TIC testing.
3.2.4 Test Sequence and Schedule
The sequence of tests performed with the TICs in this verification test is outlined in Figure 3-1.
Since HAZMATCAD Plus performance was not known a priori, the concentrations used in
testing depended on the results of the first few tests performed. The decision logic used to
determine the actual TIC concentrations and the test sequence is shown in Figure 3-2. After
completion of TIC testing, a similar, but slightly reduced, set of tests was performed with CW
agents. Table 3-3 summarizes the actual schedule of testing for the TICs and CW agents. As
Table 3-3 indicates, testing with CK took only one day because this TIC is not detected by the
HAZMATCAD Plus.
-------
Testl: Vapor challenge with TIC
Alternating clean air with immediately dangerous to life and health (IDLH) level concentration of TIC five times with
HAZMATCAD Plus operating on battery power, fully warmed up per manufacturer's instructions prior to testing,
and room temperature (22 ± 3°C) and 50 ± 5% RH.
Test 2: Vapor challenge with TIC at reduced concentration
Test 1 is repeated at a lower concentration giving mid-range on-scale readings (only if off-scale response at IDLH).
The concentration that gives a mid-range on-scale reading is then referred to as the target concentration for all
subsequent tests.
Test 3: Vapor challenge with TIC at increased concentration
Test 1 is repeated at roughly 10 times the IDLH concentration (only if no response at IDLH).
Test 4: Response threshold of TIC
Test 1 is repeated at a concentration below IDLH. If a response is recorded, the concentration is cut in half until no
response is recorded. If no initial response is recorded, the concentration is increased by a factor of 2 until a response
is recorded.
Test 5: Target/low/clean air challenge
Test 1 is repeated by alternating target concentrations, a low concentration (either 0.1 IDLH or response threshold
concentration) and clean air six times and alternating order of low concentration and target concentration.
Test 6: Vapor challenge with TIC at room temperature, low humidity
Test 1 is repeated at room temperature (22 ± 3°C) and less than 20% RH. The test is performed at the
concentration(s) determined via the logic in Figure 3-2.
Test 7: Vapor challenge with TIC at room temperature, high humidity
Test 1 is repeated at room temperature (22 ± 3°C) and 80% RH. The test is performed at the concentration(s)
determined via the logic in Figure 3-2.
Test 8: Vapor challenge with TIC at high temperature, medium humidity
Test 1 is repeated at high temperature (35 ± 3°C) and 50% RH. The test is performed at the concentration(s)
determined via the logic in Figure 3-2.
Test 9: Vapor challenge with TIC at high temperature, high humidity
Test 1 is repeated at high temperature (35 ± 3°C) and 80% RH. The test is performed at the concentration(s)
determined via the logic in Figure 3-2.
Test 10: Vapor challenge with TIC at low temperature, medium humidity
Test 1 is repeated at low temperature (5 ± 3°C) and 50% RH. The test is performed at the concentration(s)
determined via the logic in Figure 3-2.
Test 11: Interferent false positive tests
Test 1 is repeated alternating interferent only with clean air. The test is repeated for all interferents in both libraries.
Test 12: Interferent false negative tests
Test 1 is repeated alternating TIC and interferent with clean air. The test is repeated for all interferents.
Test 13: Room temperature, cold start behavior
Repeat Test 1 with the HAZMATCAD Plus at room temperature for a minimum of 12 hours and no warm-up.
Test 14: Cold-/cold-start behavior
Repeat Test 1 after the HAZMATCAD Plus has been kept refrigerated (5-8°C) overnight for a minimum of 12 hours,
with no warm-up.
Test 15: Hot-/cold-start behavior
Repeat Test 1 after the HAZMATCAD Plus has been kept heated (40°C) overnight for a minimum of 12 hours, with
no cool-down or warm-up.
Test 16: Battery test
Repeat Test 1 with the HAZMATCAD Plus operating on battery power. The TIC at target concentration is alternated
with clean air once every half hour until the unit stops responding or shuts down due to loss of power.
Figure 3-1. Sequence of TIC Verification Tests
-------
Step 1: Perform Test 1. Depending on the results of this test, go to Step 2a, 2b, or 2c as
appropriate.
Step 2a: If there is no response in Test 1, perform Test 3, then go to Step 4.
Step 2b: If the response in Test 1 is on scale, skip to Step 3 and perform all subsequent
tests at the IDLH concentration.
Step 2c: If the response in Test 1 is off-scale, perform Test 2. Establish the concentration
that gives a mid-range on-scale response and proceed with Step 3, using that established
concentration in all subsequent tests.
Step 3: Perform Test 4 (if not already done), Tests 5 through 10, and Test 12 at the
concentration(s) determined above. For the first TIC, also perform Test 11 and Tests 13
through 16.
Step 4: Return to Step 1 and repeat Test 1 through 10 and 12 for all other TICs.
Step 5: Repeat Tests 1 through 10 and 12 for all CW agents
Figure 3-2. Logic Diagram for Determining TIC Test Sequence
Table 3-3. Test Schedule
Chemical Test Dates
AC May 4-14, 2004
CK May 17, 2004
CG May 25-June 1,2004
SA May 18-24, 2004
C12 June 2-14, 2004
GB July 16-August 12, 2004
HP August 16-27, 2004
3.2.5 Reference Methods
Table 3-4 summarizes the primary reference methods used to determine the challenge
concentrations of the target TICs and CW agents. Listed in the table are the target TICs and CW
agents, the sampling and analysis methods used for each compound, and the applicable
concentration range of each method. For AC and CK, samples were injected directly for
determination by gas chromatography (GC) with flame ionization detection (FID). A
-------
Table 3-4. Primary Reference Methods
Analyte
Concentration
Range (ppm)
Sampling Method
Analysis Method
AC
CK
CG
C12
SA
GB
HD
0.05 to 100 Air sample injected directly
2 to 100 Air sample injected directly
0.05 to 10 Collection in impingers with nitrobenzyl
pyridine
0.1 to 100 Continuous EC detector with chlorine-
specific sensor
0.05 to 100 Capillary GC with syringe injection from
bag sample
0.01 to 100 Air sample collected in gas sample bag
0.01 to 100 Air sample collected in gas sample bag
GC/FID
GC/FID
Visible absorption at
475 nanometers
Continuous
detection
MSD
GC/FPD
GC/FPD
colorimetric method, using a liquid reagent solution in a small impinger train was used for CG(3)
C12 was determined by a continuous EC analyzer with a Cl2-specific sensor to allow rapid
determination of C12 levels delivered to the HAZMATCAD Plus during testing. SA was
determined by a GC with a capillary column and mass selective detection (MSD), using samples
collected by syringe from the test apparatus. The CW agents GB and HD were collected in a gas
sample bags and determined by GC with flame photometric detection (FPD), according to
existing HMRC test procedures.
Summaries of these primary reference methods, and of supplemental methods also used, are as
follows.
The analytical instrument used for AC and CK reference measurements was an Agilent 6890 GC
with a capillary column and FID. This GC was positioned next to the laboratory hood containing
the test system during the TIC testing and sampled automatically from the flow line delivering
the challenge gases to the HAZMATCAD Plus.
To quantify CG, a sample of air was drawn at a known, constant flow rate through a midget
impinger containing 10 milliliter (mL) of an indicating solution, consisting of a mixture of 4-
(4'-nitrobenzyl)pyridine (0.25% w/w) and n-phenylaniline (0.5% w/w) in diethyl phthalate.(3) In
this solution, CG reacts to produce a product having a brilliant red color. The amount of CG
reacted was determined by measuring the absorbance of the indicating solution at 475
nanometers using a spectrophotometer.
C12 was monitored with a Draeger MiniWarn Multi-Gas Monitor, which was calibrated by
preparing known concentrations of C12 in gas sample bags.
SA was determined using an Agilent 6890 GC coupled to a 5970 mass selective detector. Air
samples [100 microliter (|_iL)] were injected onto a GS-Q capillary column held at -30°C. SA
was separated from xenon at a constant flow of 3 niL/minute (niL/min) helium using the
-------
following temperature program: hold at -30°C for 1 minute; ramp at 50°C/minute to 150°C; hold
at 150°C for 5 minutes (total time 9.6 minutes). The injector temperature was maintained at
100°C, operated in splitless mode, and was purged after 0.5 minute at 5 mL/min. Single ion
monitoring at m/z =129 and 76 was used to record signals for xenon and SA, respectively. SA
was quantified using its relative response compared with xenon as the internal standard.
The reference method for GB and HD involved collecting the agents by flowing air from the test
apparatus into gas sample bags. The agent concentrations were then determined using a capillary
GC with FPD. Concentrations were determined based on a linear regression of peak area with
the amount of agent.
A continuous FID was used for determining the total hydrocarbon (THC) content of interferent
mixtures provided to the HAZMATCAD Plus during testing. The THC concentrations character-
istic of realistic interferent levels in buildings were determined, either by direct measurement or
by interpretation of published data. The interferent delivery systems were then adjusted to
achieve the desired THC indication in parts per million of carbon (ppmC) for each interferent
during testing.
3.2.6 Interferents
Interferents were selected for testing based upon their prevalence in a building. The interferents
selected were the volatile chemicals in latex paint, air freshener, and ammonia-based floor
cleaner, as well as gasoline engine exhaust and N,N-diethylaminoethanol (DEAE). DEAE is a
common additive to boiler systems and is released into the heating, ventilating, and air
conditioning system when boiler steam is used to humidify the air. These selected interferents
were tested for false positives by exposing the HAZMATCAD Plus to selected levels of the
interferents in clean air, to see whether the interferents generated a positive response from the
HAZMATCAD Plus when no TICs or CW agents were present. Each interferent also was
introduced to the HAZMATCAD Plus along with each TIC and CW agent to determine false
negatives, i.e., whether the interferent prevents the HAZMATCAD Plus from indicating that the
TIC or CW agent is present. The following sections describe the materials and concentrations
used for testing.
The interferents are mixtures of chemicals, and determining the interferent concentration
requires the quantification of all the chemicals present. However, monitoring each component
would be time and cost prohibitive. For this reason, interferent concentrations were monitored
using a THC analyzer. THC analysis is appropriate because all the interferents consist of a
significant amount of carbon-containing compounds. Because quantification is based on carbon
content, the test concentrations are reported on a per carbon basis such as ppmC. The use of the
hydrocarbon analyzer also provided real-time continuous monitoring of the interferent
concentration during testing.
Test concentrations for the interferents were based on direct measurements or published data.
Concentrations found in published data were converted to a per carbon basis as described below.
Table 3-5 is a summary of the test concentrations. The following sections contain a detailed
description of how the test concentrations were determined.
10
-------
Table 3-5. Interferent Test Concentrations
Interferent Test Concentration (ppmC)
Latex Paint Fumes 5-10
Floor Cleaner Vapors 10
Air Freshener Vapors 1
Gasoline Exhaust Hydrocarbons 2.5
DEAF 0.02
3.2.6.1 Latex Paint Fumes
The appropriate concentrations of latex paint fumes were established directly by measurements
in and around a freshly painted office. Samples were obtained using a 25-liter (L) Teflon bag
and analyzed for THC content. Each wall in the office was painted, and the room dimensions are
11 feet by 11 feet with an alcove that is 4 feet by 10 feet and ceiling that is 12 feet high.
Immediately after painting, the hydrocarbon concentration was 170 ppmC. After 2.5 hours, the
hydrocarbon concentration in the office fell to 38 ppmC. At this time, the hydrocarbon content
was determined just outside the entrance to the office and in the hallway 80 feet away from the
office. Hydrocarbon content just outside the office was 20 ppmC; in the hallway 80 feet away
from the office, it was 3 ppmC. Based on these measurements, the test concentration was
maintained at 5 to 10 ppmC.
3.2.6.2 Floor Cleaner Vapors
The test concentration for the ammonia-based floor cleaner was inferred from the information
cited in Section 3.2.6.1 on latex paint fumes. Similar to paint, floor cleaner is applied to a
surface and allowed to dry. Floor cleaner vapors containing both ammonia and fragrances will
disperse into the hallway. Because of the similarity, a test concentration of 10 ppmC was used
for the floor cleaner.
3.2.6.3 Air Freshener Vapors
Concentration levels of air freshener for interferent testing were based upon values reported at an
indoor air quality conference. Volatile organic compound (VOC) emission for a plug-in air
freshener was reported to be 30 to 80 milligrams per hour, resulting in a concentration of 300 to
500 micrograms per cubic meter (|ag/m3) for the average room. Assuming the VOC emitted
consists of hydrocarbons similar to limonene, a common fragrance component, the
concentration on a per carbon basis can be calculated. Limonene contains 10 carbons and has a
molecular weight of 136. A concentration of 5.56 |_ig/m3 of limonene is the same as 1 part per
billion (ppb). With a room concentration of 500 |-ig/m3 and limonene as a representative
molecule, the fragrance concentration on a per carbon basis is estimated to be 1 ppmC. This
THC level was maintained for all tests with the air freshener.
11
-------
3.2.6.4 Gasoline Engine Exhaust
Of the constituents in gasoline engine exhaust fumes, the aromatic components were considered
most likely to interfere with the performance of the HAZMATCAD Plus. A recent study
reported that urban areas have benzene concentrations of 7 to 9 ppb with comparable
concentrations of other aromatics.(4) The test mixture used to simulate exhaust contains 61
compounds ranging in size from 2 to 10 carbons, with an average concentration of 200 ppb for
each component. To obtain a concentration of 7 to 9 ppb for the aromatic compounds, the test
mixture was diluted 30:1. Assuming an average size of six carbons, the THC of the mixture was
approximated to be 73 ppmC. After dilution, the THC content was 2.5 ppmC, and this target
concentration was maintained for all the experiments.
3.2.6.5 DEAE
DEAE is a common additive to boiler systems to prevent corrosion. When boiler steam is used
to humidify the air in a building, DEAE is released into the building as well. Generally, the
DEAE concentration is kept below 40 ppb, the threshold for odor detection. One study has
shown DEAE concentrations of 1 ppb in a building that uses direct steam injection for
humidification.(5) For testing purposes, the concentration was set at 20 ppbC, which correlates to
3.3 ppb DEAE given that DEAE contains six carbons. This concentration was not detectable by
THC analysis, so the interferent concentration was set by dilution of a concentrated standard.
3.2.7 Materials and Equipment
3.2.7.1 TICs and CWAgents
The commercial gas standards used as sources of the TICs for testing included standards of
10,020 ppm AC (Cylinder B0005506, Scott Specialty Gases); 991 ppm CG (Cylinder
RR37345), 997 ppm SA (Cylinder KE-50368), and 5,811 ppm C12 (Cylinder RA64239), all
from Matheson Gas products. The source of CK was a 10,000 ppm compressed gas standard
made at Battelle starting with neat CK gas. The neat CK gas was from Atomergic Chemetals
Corp. (Lot No. L6196). To prepare the CK standard, 210 mL of the neat gas was diluted by
pressurizing to 1,000 pounds per square inch gauge (psig) with hydrocarbon-free air in a 5.9-L
cylinder. The CW agents GB and HD were obtained as neat materials from the U.S. Army under
Bailment Agreement No. DAAD13-03-H-0003.
3.2.7.2 Vapor Delivery Equipment
The compressed gas mixtures noted in Section 3.2.7.1 were diluted as the vapor sources for AC,
CK, CG, C12, and SA. For the CW agents GB and HD, a diffusion cell containing the pure agent
was used. A temperature-controlled water bath was installed to control the temperature of the
diffusion cell to maintain a stable and controllable vapor generation rate. A two-way valve was
included in the flow path downstream of the vapor generation source, so that the dilution and
test equipment could be totally isolated from the source. A schematic of the entire vapor
generation, dilution, and delivery system is shown in Figure 3-3.
12
-------
Figure 3-3. Test System Schematic
13
-------
3.2.7.3 Temperature/Humidity Control
The HAZMATCAD Plus was evaluated at the temperature and humidity conditions indicated by
an "X" in Table 3-6. Both the delivered air temperature and the HAZMATCAD Plus units were
maintained within the specified temperature range. For testing at 35°C, the vapor delivery
system was warmed with a heat-traced line, using an electronic temperature controller. For
testing at 5°C, the dilution and delivery system was enclosed in a cooled chamber to provide
approximate temperature control. For all tests, thermocouples were installed in both the clean air
plenum and the challenge plenum to provide real-time temperature monitoring.
A commercial Nafion® humidifier (Perma Pure, Inc.) was used to generate controlled high-
humidity air (50 to 100% RH), which was then mixed with dry dilution air and the target vapor
stream to obtain the target RH (< 20% to 80%) in the challenge air.
Table 3-6. Temperature and Relative Humidity Conditions
Temperature (°C)
RH (%) 5±3 22 ±3 35 ± 3
< 20 X
50 ±5 X X X
80 ±5 X X
3.2.7.4 InterferentSources
Interference test concentrations were obtained by diluting a concentrated feed with air. For latex
paint, floor cleaner, and air freshener, the concentrated feeds were made by purging the head
space of a large boiling flask containing about 100 mL of the bulk liquid of each interferent
using approximately 0.1 liter per minute (L/min) flow of clean air. THC analysis of the head
space samples found that the concentrated feeds contained 394, 886, and 233 ppmC for latex
paint, floor cleaner, and air freshener, respectively. Gasoline engine exhaust was simulated using
a mixture of 61 organic compounds ranging from 2 to 10 carbon atoms (C2 to C10). This mixture
was prepared by adding 1 |^L of 51 neat liquid components and 250 |_iL of 10 gaseous
components into a 15.7-L cylinder and diluting to a final pressure of 1,200 psig with nitrogen. A
concentrated standard of 1 ppm for DEAF was made by adding zero nitrogen to 6 |_iL of liquid
neat DEAF to a final pressure of 1,200 psig. In all cases these cylinder gases or concentrated
vapor streams were diluted to the appropriate level by addition to the large flows of clean air
passing through the test apparatus (Figure 3-3).
3.2.7.5 Performance Evaluation Audit Materials
As part of the QA effort in this verification test, a performance evaluation (PE) audit was
performed on reference methods used to confirm the TIC concentrations provided to the
HAZMATCAD Plus. This audit involved conducting analyses on independent standards,
obtained from different sources than those used for the calibration standards. The results from
14
-------
the independent standards were then compared with those from the calibration standards, to
assess the degree of agreement. The target agreement in the TIC PE audit was within 20%.
For the TICs, the PE audit standards were 10,000 ppm AC (Cylinder LL320), 1,043 ppm CG
(Cylinder ASS 1117), 1,090 ppm SA (Cylinder AS73486), and 6,015 ppm C12 (Cylinder
LL23078), all from Linde Gas EEC.
A comparable PE audit could not be done for the CW agents because of the lack of independent
standards. In lieu of a PE audit for the CW agents, check samples were prepared at the HMRC
by an analyst other than the staff who conducted routine calibration of the reference method.
These samples were analyzed by the same approach used for analysis of calibration samples
from GB and HD testing, and the results were compared.
3.3 Test Procedure
The test system (Figure 3-3) consisted of a vapor generation system, a Nafion® humidifier, two
challenge plenums, a clean air plenum, an RH sensor, thermocouples, and mass flow meters. The
challenge vapor or gas was generated by the vapor generation system. The challenge vapor was
then mixed with the humid dilution air and flowed into the challenge plenum. Interference
vapors were added to the challenge mixtures as needed for testing.
The RH and target concentration of the challenge vapor were obtained by adjusting the mixing
ratio of the humid air (from the Nafion® humidifier) to the dry dilution air, and the mixing ratio
of the vapor generation stream to the humid dilution air, respectively. To avoid potential
corrosion or malfunction of the RH sensor from exposure to the challenge vapor, the RH meter
was installed upstream of the inlet of the vapor stream. The RH of the challenge vapor stream
was calculated based on the measured RH of the humid dilution air and the mixing ratio of the
vapor generation stream to the humid dilution air.
To establish the background readings of the two HAZMATCAD Plus units being tested, a clean
air plenum was installed. Part of the humid dilution air was introduced directly into the clean air
plenum. When establishing the HAZMATCAD Plus units' background, the four-way valves
connected to the two HAZMATCAD Plus units were switched to the clean air plenum to collect
baseline data.
After the background measurement, the four-way valves connected to the two HAZMATCAD
Plus units were switched to one of the challenge plenums to allow the HAZMATCAD Plus units
to sample the challenge mixture. Switching between the challenge and clean air plenums was
rapid, and the residence time of gas in the test system was short to allow determination of the
response and recovery times of the HAZMATCAD Plus units. The reference methods described
in Section 3.2.5 were used to confirm that the TIC concentrations in the challenge plenums were
within ± 20% of the target level (or within 35% of the target level for the CW agents).
Concentrations outside those tolerance ranges triggered a repeat of any test procedures
conducted since the last analysis.
15
-------
3.3.1 Response Time
To evaluate HAZMATCAD Plus response time, the environmental conditions were established
at the target conditions of 22 ± 3°C and 50 ± 5% RH. Initially 10 IVmin of the clean humidified
air passed through the clean air plenum. The HAZMATCAD Plus sampled the clean air for a
minimum of 30 seconds or until a stable reading was indicated, but not exceeding 10 minutes, to
obtain a baseline for the HAZMATCAD Plus. The clean air plenum also was sampled with the
appropriate reference method. This sampling took place after the HAZMATCAD Plus readings
had stabilized.
Concurrent with the background measurements the target challenge concentration in the high
challenge plenum was established. The high challenge concentration was generated at the target
environmental conditions. For the TICs, the generator operating conditions and the dilution flow
were adjusted as needed to establish a challenge concentration within ± 20% of the IDLH or
other target. For the CW agents, a delivered concentration within 35% of the target level was
acceptable. Reference samples were collected and analyzed immediately to establish the
challenge concentration and demonstrate stability. A challenge concentration was considered
stable if it could be maintained within the target challenge bounds on three consecutive
reference sample measurements for AC and CK over a minimum of 5 minutes of continuous
operation prior to the test. A challenge concentration was considered stable for CG and SA if
one sample taken prior to testing was consistent with the method calibration curve. A challenge
concentration was considered stable for C12 when the continuously monitoring reference method
reached a stable reading.
After a stable reading was obtained from the HAZMATCAD Plus on background air, and the
challenge mixture was stable and at the target concentration, the four-way valve at the
HAZMATCAD Plus inlet was switched to sample from the challenge plenum. The response of
the HAZMATCAD Plus was recorded and the time to produce an alarm was considered the
response time. When feasible, based on the time response of the reference method, the challenge
vapor concentration also was determined by reference method sampling periodically during the
procedure. The HAZMATCAD Plus sampled from the challenge plenum for a minimum of
30 seconds, up to a maximum of 10 minutes. The high challenge vapor concentration was
determined by the reference method as frequently as possible during the procedure. For AC, CK,
and C12, a reference sample was taken prior to, during, and after every set of tests with the
HAZMATCAD Plus. For CG and SA, a reference sample was taken prior to and at the
conclusion of every set of tests.
After the challenge sampling, the sample inlet four-way valve was switched to again sample
from the clean air plenum. The time required for the HAZMATCAD Plus to clear, i.e., the time
to return to starting baseline or non-alarm readings, was recorded as the recovery time. After a
maximum of 10 minutes, regardless of whether the HAZMATCAD Plus had returned to
baseline, subsequent cycles of alternating challenge/clean air sampling were carried out,
controlled by the four-way valve. A total of five such challenge/clean air cycles were completed.
The same sampling procedure was carried out at different temperature and RH conditions or
challenge concentration to evaluate temperature and RH effects and response thresholds. For
16
-------
each TIC or CW agent, the initial test was conducted at the levels shown in Table 3-1. If the
instrument gave an over-scale reading when challenged at the initial level at the normal
temperature and RH conditions (22°C and 50% RH), a lower challenge concentration was
chosen that provided an on-scale reading. All subsequent tests for that TIC or CW agent used
that lower challenge concentration. If the instrument did not respond to the IDLH or other initial
concentration selected, then the response threshold procedure in Section 3.3.5 was conducted;
but, all subsequent tests planned for that TIC or CW agent were eliminated. Otherwise, testing
proceeded as described.
Following the five challenge/clean air cycles, six cycles were conducted in which the
HAZMATCAD Plus sampled sequentially from the high, low, and clean air challenge plenums.
The high challenge plenum provided the IDLH or other primary target concentration, and the
low challenge plenum provided a concentration of approximately 0.1 times that level, or the
response threshold (see Section 3.3.5), whichever was greater. Clean air was sampled alternately
with sampling from the challenge plenums, and the order of sampling from the high (H) and low
(L) challenge plenums was reversed, i.e., the order of sampling was clean air/H/L/clean
air/L/H/clean air/H/L/ . .,. for a total of six such cycles. This procedure simulated use of the
HAZMATCAD Plus in locations having different degrees of contamination.
3.3.2 Recovery Time
The time for the HAZMATCAD Plus to return to its baseline reading or non-alarm state after
removing a challenge concentration was measured as described in Section 3.3.1.
3.3.3 Accuracy
In all of the response threshold and response time tests, the challenge concentration was
measured using a reference method or monitor. Those measurements confirmed that the target
TIC or CW agent was present at the appropriate challenge concentration. The degree to which
the HAZMATCAD Plus correctly identified the challenge TIC or CW agent was evaluated as the
measure of accuracy.
3.3.4 Repeatability
Repeatability was assessed using data obtained from the five repeated challenge/clean air cycles
or the high challenge/low challenge cycles. The repeated test results at the same environmental
and concentration conditions were used to quantify the repeatability of the measurements and
the effects of test conditions on repeatability.
3.3.5 Response Threshold
The response threshold of each HAZMATCAD Plus unit was evaluated by repeating the
procedure in Section 3.3.1 at successively lower (or if necessary, higher) concentrations. The
response threshold was determined at the baseline environmental condition of 22 ± 3°C and
50 ± 5% RH, in the absence of any interfering chemicals. The manufacturer's reported detection
limit (± 50%) was used as the starting concentration. If the manufacturer did not provide a
17
-------
detection limit, a concentration at least 10 times lower that the IDLH or target concentration was
chosen. If there was no response at the starting test concentration, then the concentration of the
challenge was increased by a factor of two. Similarly, if the HAZMATCAD Plus units
responded to the starting concentration, then the challenge concentration was decreased by a
factor of two. The increase or decrease in concentration was continued accordingly, until the
response threshold had been bracketed. The minimum concentration producing a
HAZMATCAD Plus unit response was denoted as the response threshold. If the duplicate
HAZMATCAD Plus unit tested simultaneously produced different instrument responses, the
concentrations were varied as needed to assess the response thresholds of each HAZMATCAD
Plus unit.
3.3.6 Temperature and Humidity Effects
The tests described in Section 3.3.1 were repeated at the target concentrations shown in
Table 3-1, over the range of environmental conditions shown in Table 3-6. Five repeat runs were
performed at each set of test conditions for each target TIC or CW agent. The data at different
temperature and RH conditions were used to infer whether these conditions affected the
detection (i.e., accuracy, repeatability, response threshold) of the HAZMATCAD Plus for the
target chemical. The effect on response time and recovery time also was assessed.
3.3.7 Interference Effects
To evaluate the effects of the interferents described in Section 3.2.6, the test system shown in
Figure 3-3 was modified by adding an interferent vapor generator. The output from this source
was directed as needed to mix with the humidified air flowing to the challenge plenum. The test
chemical generation was independently controlled to generate interferent in the absence or
presence of the test chemical. This allowed interference effects to be evaluated with the
interferent alone and with each interferent and TIC or CW agent together. Testing with the
interferent alone allowed evaluation of false positive responses, and testing with the interferent
and chemical together allowed evaluation of false negatives. The test procedures also allowed
observation of interferent effects on the response time and recovery time of the HAZMATCAD
Plus. The target concentrations of the planned interferents are shown in Table 3-5. Those
concentrations are shown in terms of the equivalent THC concentration in ppmC. These target
concentrations are based on actual indoor measurements by Battelle or on published data, as
described in Section 3.2.6.
Interferent testing involved only one interferent at a time. Testing was done by alternately
sampling clean air and the interferent mixture, for a total of up to five times each, in a procedure
analogous to that described in Section 3.3.1. However, if no interferent effect was observed
after three such test cycles, the test was truncated. Testing with interferents alone involved
alternately sampling from the clean air plenum and then from the challenge plenum, to which
only the interferent in clean air was delivered. The same process was used for testing with
interferents and TICs or agents together, with the two compounds diluted together in humidified
air delivered to the challenge plenum. The same TIC and CW agent concentrations used in the
initial testing under Section 3.3.1 were used in this test, i.e., the levels shown in Table 3-1.
18
-------
A response from the HAZMATCAD Plus with the interferent alone was recorded as a false
positive; and the absence of a response, or a reduced response, to the TIC or CW agent in the
presence of the interferent was recorded as a false negative.
The replicate test runs conducted with the interferent plus TIC or agent also allowed the
response time and recovery time of the HAZMATCAD Plus to be assessed with interferents
present. Differences in response and recovery times, relative to those in previous tests with only
the TIC or agent present, were attributed to the effect of the interferent vapor.
3.3.8 Cold-/Hot-Start Behavior
The cold-/hot-start tests were conducted only with the TIC AC in a manner similar to the
response time test in Section 3.3.1. Prior to these tests, however, the HAZMATCAD Plus units
were not allowed to warm up per the manufacturer's recommendation.
The cold-start test was conducted twice, once with the HAZMATCAD Plus units at room
temperature and, subsequently, at reduced temperature, prior to start-up. In the former test, the
HAZMATCAD Plus units were stored with the power off at 22 ± 3°C for at least 12 hours prior
to testing. The cold-start effect was assessed at the baseline conditions of 22 ± 3°C and 50 ± 5 %
RH. The time from powering up the HAZMATCAD Plus units to their first readiness to provide
readings was considered start-up delay time. The response time (as defined in Section 3.3.1) was
measured, followed by the recovery time. Repeatability and accuracy in five replicate clean
air/challenge cycles with AC were also noted.
For the reduced temperature cold start, HAZMATCAD Plus units were placed in a refrigerated
enclosure (5 to 8°C) with the power off for at least 12 hours overnight. At the start of the next
test day, the cold-start test was repeated, again using AC and using the same baseline conditions
(22°C and 50% RH) and again recording the start-up delay time and other performance
parameters.
For the hot-start test, the HAZMATCAD Plus units were placed in a heated enclosure at 40 ±
3°C for at least 12 hours overnight. At the start of the next test day, the hot-start test was
conducted in the same fashion as in the cold-start test, with AC at the baseline test conditions
(22°C and 50% RH). Only one cold-/hot-start test was performed per day.
3.3.9 Battery Life
Battery life was evaluated by assessing the degradation of performance with extended
continuous operation. Fully charged batteries were installed, and the HAZMATCAD Plus units
were turned on and allowed to warm up, and an initial response time test was performed (see
Section 3.3.1). A single TIC (AC) was used in this evaluation. The indicated concentration
signal from the HAZMATCAD Plus units was recorded. At each sampling of the AC mixture,
the battery level of the HAZMATCAD Plus units was recorded. The HAZMATCAD Plus units
sampled clean air for 30 minutes, and then the AC mixture was sampled again. This procedure
was repeated with each HAZMATCAD Plus unit operating continuously until it no longer
19
-------
responded to the AC challenge. The total time of operation was recorded as the measure of
battery life.
3.3.10 Operational Characteristics
Key operational characteristics of the HAZMATCAD Plus were evaluated by means of the
observations of test operators and inquiries to the HAZMATCAD Plus vendor. Ease of use was
assessed by operator observations, with particular attention to the conditions of use by
emergency first responders.
Signal or data output capabilities were assessed by observations of the personnel who operated
the HAZMATCAD Plus units during testing. The type of data that was output was noted on the
data sheets (e.g., audio and/or visual alarm, bar graph, low/med/high indication, and/or
quantitative measure of concentration). In addition, the clarity and readability of the output were
noted, especially in low light conditions or when holding the HAZMATCAD Plus while
walking, as in use by a first responder. The availability of multiple forms of data output or
display also was assessed (e.g., the availability of both a visual display and an analog voltage
output for recording purposes).
The vendor was asked for the purchase and operational costs of the HAZMATCAD Plus as
tested. Estimates for key maintenance items also were requested from the vendor.
20
-------
Chapter 4
Quality Assurance/Quality Control
QA/quality control procedures were performed in accordance with the ETV QMP(2) and the
test/QA plan for this verification test.(1)
4.1 Equipment Calibration
4.1.1 Reference Methods
The reference methods used for determining TICs and CW agents are summarized in
Section 3.2.5. The analytical equipment needed for these methods was calibrated, maintained,
and operated according to the quality requirements of the reference methods and Battelle's
normal documentation. Procedures for blank sampling during testing and for calibration of
reference methods are described below.
For AC, CK, and C12 testing, blank reference samples were run before, during, and after each set
of tests. The sequence of reference sampling thus included running a blank on clean air and then
switching to challenge gas and taking a reference sample immediately prior to challenging the
HAZMATCAD Plus units with the challenge gas, and again running a blank when the
HAZMATCAD Plus units were once more sampling clean air. Blank reference samples were
taken before and after, but not during, SA and CG testing, because the methods used were not
conducive to running the large number of blank reference samples analyzed during the other TIC
tests. In testing with GB and HD, blank gas sample bags were run at the start of each test day.
Calibration procedures for the reference and other analyses were as follows:
The GC reference method for AC and CK was calibrated by preparing gas mixtures in 1-L gas
sampling bags. For AC, calibration standards were prepared by diluting 0.5 to 4 mL of
commercial concentrated AC gas standards (e.g., 10,000 ppm AC in N2) in 800 mL of clean air
in a bag. Three samples from each bag were injected by syringe into the GC, and the peak area
was recorded. Several such calibration standards ranging from 12.5 to 50 ppm AC were
prepared and analyzed over a three-day period. The regression of peak area versus AC standard
concentration had the form Peak Area = 0.7192 x (AC, ppm), with an r2 value of 0.9961. For
CK, 800-mL bag standards were prepared in a similar manner; but, since CK was available as a
neat gas, a two-stage dilution was needed. An initial bag dilution of the neat material was made,
and then a small volume of that mixture was diluted further in a second bag, producing CK
21
-------
standards of about 12.5 to 50 ppm. Again, three samples from each final bag were analyzed, and
the peak areas were used to generate a regression line, which had the form Peak Area = 1.4292 x
(CK, ppm), with an r2 value of 0.9931.
For calibrating the impinger-based method for CG, 25-L Tedlar bags were filled with a known
amount (-24 L) of zero air spiked with known amounts of CG from a high concentration
commercial standard. Final concentrations in the bags ranged from 0 (method blank) to 3 ppm.
Approximately 23 L of the CG/air mixture in each bag was then drawn through a midget
impinger containing the indicating solution. A multi-point calibration at the beginning of the
study showed linear response and an r2 value of 0.98. Before proceeding with the analysis of
samples on each test day, a one-point calibration check was conducted, typically at a CG
concentration of 2 ppm, by the procedures outlined above. Inclusion of each day's calibration
check data in the calibration curve showed correlation coefficients of 0.96 to 0.98, which were
deemed acceptable. Due to the complexity of the impinger method, a Draeger Pac HI Single Gas
Monitor also was used to provide a real-time indication of CG This device was new, and its
factory calibration was used in all monitoring.
C12 was monitored with a Draeger MiniWarn Multi-Gas Monitor, which was calibrated by
preparing standard known concentrations in sample bags. The C12 calibration was done by
injecting a known amount of high concentration gas into a known volume of clean air in a
Tedlar gas sampling bag, to give a concentration of a few ppm. The MiniWarn readings in this
calibration were within 5% of the expected standard concentrations.
SA was quantified by GC using its relative response compared with xenon as the internal
standard. Xenon is a ubiquitous component of ambient air with a concentration of 90 ppb.
Triplicate injections of SA standards at 0, 0.03, 0.3, 5.0, and 50 ppm and one method blank
yielded a linear calibration plot having an r2 value of 1.000.
Calibration for GB and HD was conducted by diluting stock agent to |_ig/mL concentrations, and
then injecting a l-|iL volume of each standard into the GC-FPD. Regression of peak area versus
amount of agent was prepared.
The THC analyzer used to document the interferent levels provided in testing was calibrated by
filling a 25-L Tedlar bag with 33 ppm of propane in air from a commercial gas standard. Since
propane is a three-carbon molecule, this standard constitutes a THC concentration of 99 ppmC.
This standard was used for calibrating the THC analyzer throughout the verification. Clean air
from the room was used for zeroing.
4.1.2 Instrument Checks
The HAZMATCAD Plus units were operated and maintained according to the vendor's
instructions throughout the verification test. Maintenance was performed according to a preset
schedule or in response to predefined HAZMATCAD Plus instrument diagnostics. Daily
operational check procedures for the HAZMATCAD Plus units were performed with a vendor-
supplied simulant tube. Proper response of the HAZMATCAD Plus units to the simulant was
required before testing could proceed. Both units were operated in the "Fast" mode at all times.
22
-------
4.2 Audits
4.2.1 Performance Evaluation Audit
As described in Section 3.2.7.5, a PE audit was conducted to assess the quality of reference
measurements made in the verification test. For TICs, the PE audit was performed once during
the verification test by analyzing a standard that was independent of the standards used during
the testing. The acceptable tolerance for each target TIC was ±20%. Table 4-1 shows that the
results of the PE audit were all within the target tolerances. For the CW agents, standards of GB
and HD were prepared in the same manner as for the normal calibration standards, but by a
different analyst. The reference data obtained for these standards were compared. For GB, four
standards were prepared at concentrations of 5, 2.5, 1.0, and 0.5 |ig/mL. For the average
responses, all results were within 7% for the standards made by the two different individuals.
For HD, five standards were prepared at concentrations of 10, 5, 2.5, 1.0, and 0.5 |ig/mL. For
the average responses, all results were within 4% for the standards made by the two different
individuals.
Table 4-1. Performance Evaluation Audit Results
(a)
TIC
AC
CG
SA
C12
Sample
Standard (Cylinder B0005506)
PE Audit Std (Cyl. LL320)
Standard (Cylinder RR37345)
PE Audit Std (Cyl. AS51117)
Standard (Cylinder KE50368)
PE Audit Std (Cyl. AS73486)
Standard (Cylinder RA64239)
PE Audit Std (Cyl. LL23078)
Date of Audit
7/12/04
7/13/04
8/5/04
7/12/04
Concentration
10,020 ppm
10,000 ppm
991 ppm
1,043 ppm
997 ppm
1,090 ppm
5,8 11 ppm
6,015 ppm
Result
43.2 ppm
47.8 ppm
2. 18 ppm
2.63 ppm
267332 PA(b)
277556 PA
9.85 ppm
9.60 ppm
Agreement
9.8
12.7
5.3
6.2
(a) As noted in Section 3.2.7.5, PE audits were not performed for CK (no independent standard available).
(b) PA = Peak area; based on comparison of neat (undiluted) standards.
4.2.2 Technical Systems Audit
The Battelle Quality Manager conducted a technical systems audit (TSA) to ensure that the
verification test was performed in accordance with the test/QA plan(1) and the ETV QMP.(2) As
part of the audit, the Battelle Quality Manager reviewed the reference sampling and analysis
methods used, compared actual test procedures with those specified in the test/QA plan,(1) and
reviewed data acquisition and handling procedures. Observations and findings from this audit
were documented and submitted to the Battelle Verification Test Coordinator for response. The
records concerning the TSA are permanently stored with the Battelle Quality Manager. The
single deviation from the test/QA plan was the inability to complete the PE audit (Table 4-1)
within the period of TIC testing as a result of delays in receiving some PE audit standards.
However, as shown in Table 4-1, all PE audit results were within the acceptance criteria, so no
impact on the test resulted from this deviation.
23
-------
A separate TSA was also conducted by EPA QA staff on June 7, 2004. That TSA produced two
findings: (1) the need to ensure traceability of electronic records from the test, and (2) the need
to complete the PE audit of the TIC challenge gases, as was noted above.
The first finding was addressed by improving the documentation of electronic files placed in the
electronic archive for this test. The second finding was motivated by delays in obtaining some of
the PE audit gases. That finding was addressed by completing the PE audit immediately once
the final audit gases were obtained. The actions taken to address these findings were
documented to the EPA Quality Manager, and records related to this TSA are stored with the
Battelle Quality Manager.
4.2.3 Data Quality Audit
At least 10% of the data acquired during the verification test was audited. Battelle's Quality
Manager traced the data from the initial acquisition, through reduction and statistical analysis,
to final reporting, to ensure the integrity of the reported results. All calculations performed on
the data undergoing the audit were checked.
4.3 Quality Assurance/Quality Control Reporting
Each assessment and audit was documented in accordance with the test/QA plan.(1) Once the
assessment report was prepared, the Battelle Verification Test Coordinator ensured that a
response was provided for each adverse finding or potential problem and implemented any
necessary follow-up corrective action. The Battelle Quality Manager ensured that follow-up
corrective action was taken. The results of the TSA were sent to the EPA.
24
-------
Chapter 5
Statistical Methods
To extract the most information about the HAZMATCAD Plus performance from the test
procedures, a statistical analysis of the test results was performed whenever appropriate. Such an
analysis used all available data to explore the impact of test parameters on the HAZMATCAD
Plus performance. Section 5.1 summarizes the statistical approaches and the parameters tested.
The performance parameters of response threshold and battery life were assessed with simple
comparisons that did not require statistical analysis. Section 5.2 describes the analyses used for
these performance parameters.
5.1 Statistical Analyses
Several successive readings of the HAZMATCAD Plus units' responses, alternating with the
HAZMATCAD Plus units' baseline readings, were recorded in each step of the test procedure.
That is, for each HAZMATCAD Plus unit and each TIC or CW agent, such readings were
recorded with each concentration, at a range of temperature and RH conditions (Table 3-6), with
the exception that data at high temperature and high humidity (35°C/>80% RH) were used in
evaluating performance, but not incorporated in statistical analyses. Response to each TIC and
CW agent was also determined at medium temperature and humidity, both with and without
each interferent. In addition, readings were obtained at the normal temperature and RH
conditions at different HAZMATCAD Plus unit starting conditions. These data were the basis
for the statistical analysis of HAZMATCAD Plus unit performance.
The statistical analyses focused on the following performance parameters:
Response time
Recovery time
Accuracy
Repeatability
False positives/false negatives
by considering the following explanatory variables:
• Identity of the target TIC or CW agent
• Concentration of the target TIC or CW agent
• Temperature
25
-------
• Humidity
• Start state (i.e., warmed up, cold start, etc.)
• Identity and presence/absence of interferent.
The following sections detail the statistical methods used for the TIC analysis. Methods used for
the CW analysis were similar. However, it must be noted that only one of the HAZMATCAD
Plus units responded to GB during testing with that CW agent. Consequently, data for testing
with that agent were limited, as compared with data from HD or TIC testing. Because of
concentration differences between chemicals (Table 3-1), the effects of concentration and
chemical on the behavior of the HAZMATCAD Plus units cannot be separated. For this reason, the
statistical analyses were performed on a per chemical basis. In descriptive comments about
figures that summarize results in Appendix A, differences that appear across chemicals are
highlighted. It should be kept in mind that the differences noted may be the result of a difference
in TIC/agent identity, a difference in concentration, or a mixture of both effects.
5.1.1 Analysis of Response, Response Time, and Recovery Time
The effects of temperature and humidity on the actual response were investigated using the
Jonckheere-Terpstra test.(6) This non-parametric method tests the hypothesis of no association
between response and temperature or humidity versus the hypothesis that the response increases
or decreases as temperature or humidity increases. The test accommodates the categorical nature
of the dependent (response) and independent (temperature or humidity) variable and is
appropriate when both the dependent and the independent variables have a natural ordering
(low, medium, and high in this case). Furthermore, the Jonckheere-Terpstra test is appropriate
when the sample size is small or the data are sparse.
Unlike temperature and humidity, start state has no natural ordering. The Kruskall-Wallis test(6)
was used to determine whether start state has an effect on machine response. This test is
equivalent to an analysis of variance (ANOVA) performed on the ranked data. Unlike the
Jonckheere-Terpstra test, the Kruskall-Wallis test simply tests for differences in response among
the start state alternatives (i.e., the alternative hypothesis is not ordered). Like the Jonckheere-
Terpstra test, it accommodates small sample sizes and sparse data.
For the analysis of response time, a standard ANOVA was used. This allowed testing for the
effect of temperature, humidity, and start state on the response time. To investigate the effect of
temperature, for example, the following model was fit:
Yy = [i + ttj + ey (1)
Here Yy denotes the log of the 1th response time for a given TIC under temperature j. The term [i
represents the mean overall treatment levels, the term «j denotes the effect of temperature j, and
the term e^ accounts for all variation not explained by the model.
The log response time was modeled because time-to-event measurements are typically skewed to
the right. The log transformation is a standard technique used to achieve normality of error(7)
effects when the data are skewed in such a manner. This model provided the average log
26
-------
response time under a given set of conditions. This average was transformed back into the
original scale (as opposed to log scale) by exponentiating it. Thus, the modeled geometric mean
of the response times was reported under the given set of conditions. The significance of effects
of interest was tested by evaluating the corresponding coefficients in the model. Thus, to test
whether temperature had an effect on log response time, a standard F test was used to test
whether «j is equal to zero for all j. For more information on the ANOVA approach, see Kirk.(7)
The analysis of recovery time was similar to that of response time unless there were recovery
times that were "censored." When the HAZMATCAD Plus did not recover within the maximum
allotted time of 600 seconds, that particular recovery time was considered censored. In a
censored model, instead of assuming that the log recovery times, Yy, have a joint normal density
function, the likelihood for the vector of recovery times, Y, is assumed to be:
where C is the collection of censored observations and C* is the collection of uncensored
observations. Here g is a normal density function and S is the "survival" function:
where O is the standard normal distribution function and a is the standard deviation for the
recovery times. The parameter [i represents the common mean; the parameter «j represents the
effect of treatment j. Once again, effects were investigated by testing the parameters of the
model. Because the model addressed the log recovery times, the geometric mean of the recovery
times was reported.®
5.7.2 Analysis of Accuracy
The HAZMATCAD Plus response was defined as "accurate" under a given set of conditions if
the HAZMATCAD Plus:
1. Alarmed in the presence of a TIC or CW agent challenge
2. Correctly identified the TIC or CW agent.
The HAZMATCAD Plus accuracy was modeled under a given set of conditions via a binomial
logit model.(9) The significance of an effect can be determined by investigating the
corresponding coefficient(s) in the model. For example, to investigate accuracy under different
temperatures, the following model applies:
log(pi/(l-pi)) = n + ai (3)
where p; is the proportion of accurate responses under temperature i. Here a4 denotes the effect
of temperature i and |_i is the common mean. By testing the significance of the a/s using a
likelihood ratio test, the effect of each factor was tested.
27
-------
As noted in Section 3.2.1, the HAZMATCAD Plus displayed specific identifiers when a
chemical was detected. For AC, the correct identifiers were "BLOD" or "CHOK." For CG, the
correct identifier was "CHOK," while the correct identifiers for C12 and SA were "HALO" and
"HYDR," respectively. The correct identifers for GB and HD were "G" and "H," respectively.
Any response with an incorrect identifier was counted as inaccurate.
5.1.3 Analysis of Repeatability
For testing the repeatability of response and recovery times for the HAZMATCAD Plus, a test of
equal variances was used. Where there is a difference between the variability in response or
recovery times for the different levels of temperature or humidity, there is evidence that
temperature or humidity has an effect on the repeatability of the response or recovery time. The
specific test used to test for equal variances was the Brown-Forsythe test.(7) This test is
essentially an ANOVA run on the absolute deviation from the treatment (level of temperature or
humidity) medians.
For testing repeatability of the HAZMATCAD Plus response, an approach was used that took
into account the categorical nature of the response data. For all responses observed under a given
set of conditions, the mode (the most common response) was computed. The number of
observed responses that equaled that mode was then determined. Thus, the proportion of
responses equaling the most common response was the measure for the HAZMATCAD Plus
response repeatability. This proportion was modeled using a binomial logit model.
5.1.4 False Negatives and Positives Analysis
To test whether interferents caused false negatives in the HAZMATCAD Plus response, Dunn's
non-parametric multiple comparisons procedure was used.(10) To employ this procedure, the
responses for all interferent tests are ranked (ties receive average ranks). The test statistic, which
is asymptotically normal, is then:
R - R
(4)
where Rj is the average rank for interferent i, Rc is the average rank for no interferent, n; is the
number of tests for interferent i, nc is the number of tests for no interferent, and N = n4 + nc. The
smaller this test statistic is, the greater the evidence that the given interferent is creating a false
negative response.
To investigate the proportion of false positives, a Clopper-Pearson approach(11) was used. To
estimate the rate of false positives, the sample proportion was used (i.e., the number of false
positives divided by the number of trials). Along with this point estimate, a measure of its
uncertainty was calculated in the form of a 95% confidence interval. Simply because the process
did not register a false positive for a particular interferent does not guarantee that it would never
register a false positive for that interferent. This methodology makes an effort to quantify such a
28
-------
possibility by determining bounds for the false positive rate estimate based on its value and the
number of trials. By assuming that the response obtained was representative of HAZMATCAD
Plus performance, the individual tests may be modeled as a binomial distribution, and standard
methods of confidence interval estimation may be employed. The Clopper-Pearson "exact"
interval is commonly used in such instances. Its endpoints are directly calculated from the
binomial distribution without approximation.
5.7.5 Analysis of Response to Alternating Concentrations
This analysis addressed the HAZMATCAD Plus response to varying concentrations of the target
TIC or CW agent. As described in Section 3.3.1, the test procedure involved sequentially
sampling clean air or a high (H) or low (L) target concentration in varying order (i.e., clean
air/H/L/clean air/L/H/clean air/H/L.. .). The data analysis involved two separate analyses. In the
first analysis, the association between the order in which the challenge levels was presented and
the HAZMATCAD Plus response was investigated after adjusting for the level of challenge.
This was accomplished using a Cochran-Mantel-Hansel statistic.(9) Empty cells for this
contingency table analysis were filled with counts of 0.01 to allow for convergence of the test
statistic. In the second analysis, the difference between the response for the two challenge levels
was investigated. More specifically, when challenged by a high concentration after being
challenged by a lower concentration, the machine response should increase. Similarly, when
challenged by a low concentration after being challenged by a higher concentration, the machine
response should decrease. The proportion of tests exhibiting this behavior for each TIC was
recorded. Clopper-Pearson bounds were placed on the probability that the machine response
would increase or decrease as appropriate. Results of this analysis are presented in Section 6.3.
5.2 Other Analyses
The data used to evaluate the response threshold were the replicate HAZMATCAD Plus
readings obtained at each succeeding TIC or CW agent concentration. These data were
tabulated, along with the corresponding reference method data that established the challenge
concentration. The response threshold was determined by inspection as the lowest reference
method concentration that produced a positive HAZMATCAD Plus response in all replicate
runs. In this evaluation, any positive HAZMATCAD Plus response was taken as detection of the
target TIC or CW agent, i.e., HAZMATCAD Plus response of L (low) was sufficient in terms of
the response threshold evaluation.
Battery life was assessed. Battery life is reported as the total time from start-up to battery
exhaustion when a HAZMATCAD Plus is warmed up and operated continuously solely on
battery power at room temperature and 50% RH. This time was measured from initial start-up to
the point when the HAZMATCAD Plus no longer responded to a challenge mixture of AC in air.
29
-------
Chapter 6
Test Results
As discussed in Chapter 5, statistical approaches were used to test for the effects of different
conditions on the HAZMATCAD Plus performance. The following sections summarize the
statistical results from this verification. A more extensive presentation of the modeled statistical
results is included as Appendix A of this report. Note that the statistical evaluations reported in
this chapter were not applied to CK since the HAZMATCAD Plus did not respond to that TIC.
Two HAZMATCAD Plus units were used in all testing except for GB (as a result of the failure
of Unit 22 to alarm for GB during testing).
6.1 Response Time
Results of the response time analysis are summarized here and detailed in Appendix A,
Section A.l. Table 6-1 summarizes data used for the analysis of response time and other
performance parameters. This table is for illustration purposes, and the TIC results shown are
drawn from data obtained at the target concentrations (see Table 3-1).
The HAZMATCAD Plus produced an alarm and a visual display of response typically within a
few seconds after the initiation of sampling of one of the TICs. Over the range of temperature
settings, modeled geometric mean response times varied from a low of about 8 seconds for AC
to a high of about 12 seconds for CG Temperature had no effect on HAZMATCAD Plus
response time. Over the humidity range settings, modeled geometric mean response times varied
from a low of about 6 seconds for AC to a high of about 14 seconds for CG A statistically
significant difference was found among the response times for the different levels of humidity
for CG; however, the difference (12 to 14 seconds) does not seem practically significant. For GB
and HD, over the range of temperature and humidity settings, temperature and humidity had no
effect on the HAZMATCAD Plus response time. The data collected for HD at the low
temperature setting were not used in the statistical analysis because the target concentration for
that test was 1 mg/m3, while the target concentration for all other HD tests was 4 mg/m3. For
GB, HAZMATCAD Plus Unit 22 did not alarm in any tests, while Unit 27 provided no response
for one of the five challenges at room temperature and high humidity and for four of the five
challenges at high temperature and high humidity. (The Microsensor Systems vendor attributed
this instability to a possible decrease in the collection efficiency of the concentrator used in the
SAW part of the HAZMATCAD Plus caused by exposure to the TICs during testing.)
30
-------
Table 6-1. Illustrative Data
HAZMATCAD Response
TIC/CW
Agent(a) Environmental Conditions
AC Control (22°C - 50% RH)
22°C - <20% RH
22°C - 80% RH
35°C - 50% RH
35°C - 80% RH
5°C - 50% RH
CG Control (22°C - 50% RH)
22°C - <20% RH
22°C - 80% RH
35°C - 50% RH
35°C - 80% RH
5°C - 50% RH
SA Control (22°C - 50% RH)
22°C - <20% RH
22°C - 80% RH
35°C - 50% RH
35°C - 80% RH
5°C - 50% RH
CL Control (22°C - 50% RH)
22°C - <20% RH
22°C - 80% RH
35°C - 50% RH
35°C - 80% RH
5°C - 50% RH
GB Control (22°C - 50% RH)
22°C - <20% RH
22°C - 80% RH
35°C - 50% RH
35°C - 80% RH
5°C - 50% RH
Control (22°C - 50% RH)
HD 22°C - <20% RH
22°C - 80% RH
35°C - 50% RH
35°C - 80% RH
5°C - 50% RH(d)
Plus Response
Level
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
M
M
L (8)-M (2)
M(9)
L
M
M
M
M(4)
M(4)-H(1)
H(l)
H
H
H
H
H
H
M
Time
Range (s)
3-13
5-10
3-10
5-14
4-10
4-10
9-16
10-16
12-18
8-13
11-20
9-18
6-12
6-12
5-11
5-11
5-11
6-11
4-12
7-13
6-12
6-14
8-69
5-11
22-39
34-40
21-42
21-33
21
23-36
40-94
21-69
15-80
31-125
22-177
82-114
(a) TIC/CW agent results shown are from data at target concentration level (see
units of the HAZMATCAD Plus, except for GB (Unit 22 did
-------
6.2 Recovery Time
Results of the recovery time analysis are summarized here and detailed in Appendix A,
Section A.2. Recovery time results also are illustrated in Table 6-1.
In general, recovery times differed considerably from one target chemical to another. For AC,
modeled geometric mean recovery times ranged from 76 to 361 seconds; for CG, from 36 to 57
seconds; for C12, from 49 to 73 seconds; and for SA, from 23 to 25 seconds (see Figures A-2 and
A-3). Modeled geometric mean recovery times for GB ranged from 20 to 80 seconds and for HD
ranged from 143 to 434 seconds.
Temperature has a significant effect on recovery time for every TIC except SA. While
differences in modeled geometric mean recovery time among the levels of temperature are
evident for AC, CG, and C12, the greatest differences are apparent for AC. In this case, recovery
time increases as temperature decreases. The modeled geometric mean recovery time for AC
under low temperature at medium humidity was 361 seconds, which is nearly four times as long
as that for AC at higher temperatures.
For GB and HD, temperature has a statistically significant effect on the HAZMATCAD Plus
recovery time. The longest modeled geometric mean recovery time for GB (80 seconds) was at
the low temperature setting. For HD, only the medium and high temperature settings were
analyzed. The medium temperature setting resulted in a much longer modeled geometric mean
recovery time (357 seconds) than the high temperature setting (143 seconds). In general, for the
agents, recovery time appears to be greater for lower temperatures.
Humidity also had statistically significant effects on the recovery time for AC, CG, and C12, but
these small effects were of minimal importance in a practical sense. The recovery time for AC
was longest for medium humidity, with a modeled geometric mean recovery time of 131
seconds. With that exception, all TICs (including AC) had modeled geometric mean recovery
times below about 90 seconds for all humidity conditions.
For both GB and HD, humidity had a statistically significant effect on the HAZMATCAD Plus
recovery time. The modeled geometric mean of recovery time for GB was minimally longer at
50% RH than at 20% RH (31 vs. 20 seconds). No recovery time information for GB at high
humidity is available because of unstable responses or no response to the presence of GB. For
HD, the opposite trend was apparent for the modeled geometric mean of recovery time, with
longer recovery times (mean 434 seconds) at 20% RH compared to 281 seconds at 80% RH.
6.3 Accuracy
Results of the accuracy analysis are summarized here and described in Appendix A, Section A.3.
Results of tests that involved alternating different challenge concentrations, as opposed to
alternating clean air and a single challenge concentration, are summarized below and detailed in
Appendix A, Section A.8. Accuracy results also are illustrated in Table 6.1. The HAZMATCAD
32
-------
Plus was nearly 100% accurate in identifying the target TICs under all levels of temperature and
humidity, the only erroneous reading among nearly 250 data points (Table 6-1) being one failure
indication during testing with C12.
For HD, the HAZMATCAD Plus identified the agents with 100% accuracy under all levels of
temperature. For HD, the HAZMATCAD Plus also performed with 100% accuracy under all
humidity conditions. For GB, Unit 22 was inaccurate in all challenges (i.e., did not alarm for
GB), whereas Unit 27 of the HAZMATCAD Plus performed with 100% accuracy under low and
medium humidity, but at high humidity was inaccurate as a result of no response or the lack of a
stable response. It should be noted that at room temperature/high humidity the HAZMATCAD
Plus Unit 27 did alarm as "G" in four out of five challenges. However, the alarm cleared before
the challenge was completed, resulting in an unstable response. (As noted in Section 6.1, the
Microsensor Systems vendor attributes this high humidity behavior to a possible effect of the
TICs on the concentrator used in the SAW part of the HAZMATCAD Plus.)
The HAZMATCAD Plus identified the target TIC or CW agent as effectively when sampling
alternating concentrations (Section 5.1.5; Appendix A, Section A.8) as when sampling a single
concentration. The HAZMATCAD Plus responded as expected to the sequential sampling of
clean air, low, and high challenge concentrations.
6.4 Repeatability
Results of the repeatability analysis are summarized below and detailed in Appendix A,
Section A.4.
Repeatability addressed the consistency of the Low, Medium, and High readings of the
HAZMATCAD Plus units. Repeatability of response was always perfect under all levels of
temperature and humidity for AC, CQ and SA. C12 exhibited more variability in response.
Typically the HAZMATCAD Plus units registered a Medium response with C12. For one of the
10 runs at high temperature and medium humidity, HAZMATCAD Plus Unit 27 responded with
a Low reading before showing an instrument failure alarm, and for eight of the 10 runs at high
humidity and room temperature it responded with a Low reading. However, neither of the
deviations was statistically significant. Given the mix of concentrations and TICs used in the
repeatability analysis, there appears to be no evidence for a temperature or humidity effect on
HAZMATCAD Plus response repeatability.
Repeatability of response was perfectly consistent for both GB and HD at each temperature
level. Repeatability of response was also perfectly consistent for HD over all humidity levels.
For GB, the repeatability of response from Unit 27 was perfectly consistent at the low and
medium humidity levels. However, for GB at the high humidity level, there was either no
response or an unstable response from Unit 27, as discussed in Sections 6.1 and 6.3. Unit 22 did
not alarm for GB during testing.
33
-------
The modeled repeatability of response times for both TICs and CW agents showed that there
were no significant differences among the variabilities in response time for the different levels of
temperature or humidity.
The modeled repeatability of recovery times for both TICs and CW agents showed that there
were no significant differences among the variabilities in recovery time for the different levels of
temperature for AC, CG, SA, GB, and HD. The effect of temperature on the repeatability of
recovery time was significant for C12. Recovery time for C12 was most repeatable under medium
temperature. The modeled repeatability of recovery times showed that there were no significant
differences among the variabilities in recovery time for the different levels of humidity for CG,
SA, C12, GB, and HD. The effect of humidity on the repeatability of recovery time was signifi-
cant for AC. The greatest variability in recovery time occurred under medium humidity for AC.
6.5 Response Threshold
Response threshold was determined by challenging each HAZMATCAD Plus unit with
successively lower concentrations of each TIC and CW agent until it no longer responded.
Table 6-2 provides the results for each TIC and CW agent for the response threshold test.
The concentrations used in each of these tests are given in the table and, for the TICs, are well
below the IDLH concentrations used in the other tests. The responses listed in the table give the
results for three successive challenge/clean air cycles. For the CW agents, the concentrations
used are below the target concentrations used in the other tests; however, the concentrations are
above the IDLH for GB and above the AEGL-2 level for HD.
For AC, the response threshold was between 0.6 and 1.25 ppm on both HAZMATCAD Plus
units. At lower concentrations, the HAZMATCAD Plus units alarmed as CHOK for AC,
whereas, at higher concentrations, the units alarmed as BLOD. According to the HAZMATCAD
Plus manual, both alarms are correct when AC is present. For CG, the response threshold was
between 0.3 and 0.6 ppm. For SA, the response threshold was between 0.2 and 0.4 ppm, and for
C12, the response threshold was between 0.5 and 1 ppm. According to the HAZMATCAD Plus
manual, C12 should alarm as HALO. At low C12 concentrations, the alarm response changed to
CHOK. This was not a correct alarm for the presence of C12; however, the HAZMATCAD Plus
units still identified the presence of a threat, so the CHOK response was used to determine the
response threshold for C12. In all cases, the two HAZMATCAD Plus units showed the same TIC
thresholds.
For GB, the response threshold was between 0.6 and 1.1 mg/m3 (0.11 and 0.2 ppm) for Unit 27.
A response threshold for GB was not determined for Unit 22 because that unit failed to respond
to GB. For HD, the response threshold was between 0.6 and 1.6 mg/m3 (0.09 and 0.24 ppm) for
Unit 22 and between 1.6 and 4 mg/m3 (0.24 and 0.6 ppm) for Unit 27. Unit 22 responded to
only two of the five challenges with HD at 1.6 mg/m3, suggesting its response threshold is close
to that level.
34
-------
Table 6-2. Response Threshold Data
TIC/CW Agent (Concentration)
AC (5 ppm)
AC (2.5 ppm)
AC (1.25 ppm)
AC (0.6 ppm)
CG (0.6 ppm)
CG (0.3 ppm)
SA (0.4 ppm)
SA (0.2 ppm)
C12 (1 ppm)
C12 (0.5 ppm)
GB(l.lmg/m3)(0.2ppm)
GB (0.6 mg/m3) (0.11 ppm)
HD (4 mg/m3) (0.6 ppm)
HD (1.6 mg/m3) (0.24 ppm)
HD (0.6 me/m3) (0.09 ppm)
HAZMATCAD
Unit 22
L (BLOD)
H (CHOK)
M (CHOK)
No Response
L (CHOK)
No Response
L (HYDR)
No Response
L-M (CHOK)
No Response
NA
NA
H(H)
L(H)
No Response
Plus Response Level (ID)
Unit 27
L (BLOD)
H (CHOK)
L (CHOK)
No Response
L (CHOK)
No Response
L (HYDR)
No Response
L-M (CHOK)
No Response
L-M (G)
No Response
H(H)
No Response
No Response
NA = Not applicable; no response to GB from this unit during testing.
6.6 Temperature and Humidity Effects
The results of investigating temperature and RH effects on the HAZMATCAD Plus response are
summarized here and are detailed in Appendix A, Section A.5.
Table 6-1 also illustrates temperature and RH effect data. The effects of temperature and RH on
the HAZMATCAD Plus response were small. The HAZMATCAD Plus response to each TIC
was tested under medium humidity at low, medium, and high temperature. A High response was
recorded for all tests under all temperatures for every TIC except C12. Typically the
HAZMATCAD Plus units registered a Medium response for C12. For one of the ten C12 runs at
high temperature and medium humidity, a Low response was observed before HAZMATCAD
Plus Unit 27 showed an instrument failure alarm. However, there is no statistically significant
evidence that temperature had a consistent effect on the HAZMATCAD Plus units' response for
the TICs.
For GB at low temperature, the response from Unit 27 was consistently High, while at medium
and high temperature, the response was consistently at the Medium level. These results provide
sufficient statistical evidence to conclude that temperature has an effect on the HAZMATCAD
Plus response to GB. For HD, the response was consistently High for each temperature level.
Thus, for HD, there was no evidence of an effect of temperature on the HAZMATCAD Plus
response.
The HAZMATCAD Plus units' response to each TIC also was tested under medium temperature
at low, medium, and high humidity. As in the temperature tests, a High response was recorded
for all tests under all temperatures for every TIC except C12. Typically the HAZMATCAD Plus
35
-------
units registered a Medium response for C12, but for eight of the ten runs at high humidity and
room temperature, the two HAZMATCAD Plus units registered a Low response. Statistically,
humidity had a significant effect on the HAZMATCAD Plus units' response for C12, i.e., the
typical HAZMATCAD Plus unit response was lower for the highest level of humidity.
For GB at high humidity, there was either no response from Unit 27 or the response was not
stable (Table 6-1). At low and medium humidity, the Unit 27 response was consistently at the
Medium level. These results provide sufficient statistical evidence to conclude that, under the
conditions of this test, humidity has an effect on the HAZMATCAD Plus response to GB. For
HD, the response was consistently High for all runs within each humidity level. Thus, for HD
there was no evidence of an effect of humidity on the HAZMATCAD Plus units' response.
6.7 Interference Effects
The results of investigating interference effects on HAZMATCAD Plus response are
summarized here and are detailed in Appendix A, Section A.6. Table 6-3 summarizes data used
for the analysis of interference effects. This table is for illustration purposes.
A false positive response would occur if the HAZMATCAD Plus responded and provided an
alarm in the presence of an interferent and in the absence of a TIC or CW agent. A false positive
was defined as any alarm under those conditions. For the five interferents tested, no false
positive responses occurred. The HAZMATCAD units provided no response to the presence of
any of the interferents when no TIC or CW agent challenge was present.
False negative responses would occur if the presence of an interferent masked the presence of a
TIC or CW agent and the HAZMATCAD Plus provided a lower response or did not respond to
the TIC or CW agent. The HAZMATCAD Plus responded to all TIC challenges when inter-
ferents were present; however, the HAZMATCAD Plus did not respond to all CW agent
challenges when interferents were present. Changes in response, response time, and recovery
time due to interferences are discussed in the following paragraphs.
The interferents did not show an effect on the response for AC, CG, and SA. For C12, the
interferents had a very small effect on the HAZMATCAD Plus response, in that air freshener
slightly reduced the level of the response to C12, while floor cleaner slightly increased the
response level. However, neither of the effects for these two interferents is statistically
significantly different from the control (i.e., the response to C12 in the absence of interferents).
The HAZMATCAD Plus Unit 27 did not respond to GB in the presence of air freshener and
paint and in two of the exhaust hydrocarbon interferent tests the response was not stable
(Unit 22 did not alarm in any test with GB). The HAZMATCAD Plus also did not respond to
HD in the presence of ammonia cleaner and paint. However, after the HD/interferent challenge
ended (after they were switched back to sample from the clean air plenum), both HAZMATCAD
Plus units alarmed for HD.
36
-------
Table 6-3. Interference Effects Data
TIC/CW Agent(a)
AC
CG
SA
C12
GB
HD
Interferent
Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
Response
Range
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
M
M
M(2)-H(8)
L(1)-M(9)
M
H(1)-M(4)
M
NR(b)
H
NR
M
H(1)-M(4)
H
H(1)-NR(11)
NR
H
H
H
Response
Time Range
(s)
3-13
4-9
4-10
6-11
5-11
4-10
9-16
9-17
11-18
10-16
8-17
9-16
6-12
5-9
5-11
6-11
7-11
5-11
4-12
6-11
8-16
5-10
5-12
6-12
22-39
NA(C)
29-41
NA
32-35
28-38
40-94
22
NA
57-112
22-129
23-65
Recovery Time
Range
(s)
78-227
96-452
96-424
84-423
91-365
108-477
36-46
43-52
39-53
42-54
39-54
34-47
18-29
17-30
22-31
23-35
18-29
18-33
54-75
60-83
44-136
62-86
55-70
55-75
28-41
NA
30-43
NA
0(d)-22
21-36
267-517
0(d)
NA
137-600(e)
274-459
238-447
Alarms
(Indicated
Chemical)
10/10 (BLOD)
10/10 (BLOD)
10/10 (BLOD)
10/10 (BLOD)
10/10 (BLOD)
10/10 (BLOD)
10/10 (CHOK)
10/10 (CHOK)
10/10 (CHOK)
10/10 (CHOK)
10/10 (CHOK)
10/10 (CHOK)
10/10 (HYDR)
10/10 (HYDR)
10/10 (HYDR)
10/10 (HYDR)
10/10 (HYDR)
10/10 (HYDR)
10/10 (HALO)
10/10 (HALO)
10/10 (HALO)
10/10 (HALO)
10/10 (HALO)
10/10 (HALO)
5/5 (G)
5/5 (G)
5/5 (G)
5/5 (G)
10/10 (H)
1/12 (H)
10/10 (H)
10/10 (H)
10/10 (H)
a) Data shown are for illustration, TIC/CWA results shown are from data at target concentration level only. All results
are from two units of the HAZMATCAD Plus, except for GB (Unit 22 did not respond to GB during testing).
(b) NR = No response.
(c) NA = Not applicable.
(d) Cleared during challenge resulting in zero recovery time.
(e) HAZMATCAD Plus did not return to a cleared response within 600 seconds.
The interferents also showed no effect on the response time for AC, CG, and SA. The presence
of floor cleaner vapors, however, had a significant effect on the response time for C12. When
floor cleaner vapors were present, the modeled geometric mean response time for C12 increased
from about 7 seconds to about 12 seconds.
37
-------
The interferents showed no effect on the response time for GB. However, there was a statistically
significant effect on response time for HD in the presence of interferents. Both DEAE and
exhaust hydrocarbons decreased the response time for HD (to 40 and 48 seconds) compared to
the control (61 seconds), and air freshener vapors increased the response time for HD (to
74 seconds).
The interferents showed an effect on the recovery time primarily for AC. The modeled geometric
mean recovery time for the AC control was about 131 seconds. In the presence of interferent, the
modeled geometric mean recovery time for AC ranged from about 299 seconds when DEAE was
present to about 207 seconds when exhaust hydrocarbons were present. Upon review of the
interferent recovery times, a trend was apparent for AC, in that recovery times tended to increase
with each successive challenge within an individual test for all five of the interferents tested.
Statistically significant effects of interferents on recovery time were also found for CG and C12,
but these effects were too small to be of any practical significance in actual use of the
HAZMATCAD Plus.
In addition to the false negatives observed with GB in the presence of paint and air freshener
vapors, the data showed a statistically significant effect on recovery time in the presence of one
interferent. Recovery time in the presence of exhaust hydrocarbons was shorter than that for GB
without an interferent. For HD, there was no evidence of an interferent effect on the
HAZMATCAD Plus units' recovery time.
6.8 Cold-/Hot-Start Behavior
Analysis of the effects of insufficient warm-up time, under start-up conditions ranging from cold
(5 to 8°C) to hot (40°C), are summarized here and described in Appendix A, Sections A. 1.3,
A.2.3, and A.5.3. Table 6-4 illustrates the data obtained in testing for cold-/hot-start effects,
showing the HAZMATCAD Plus unit used, the start condition, sequential experiment number,
delay time, response reading, response and recovery times, and indicated chemical. Such testing
was conducted only with AC, at the IDLH concentration. Results from the two HAZMATCAD
Plus units were consistent, as shown in Table 6-4, except for recovery times exceeding
10 minutes observed with Unit 22 in the room temperature cold start.
The effect of start state of the HAZMATCAD Plus was tested with AC at the IDLH level and
was found to have no effect on response time, but a strong effect on recovery times (data shown
in Table 6-4). Under fully warmed-up conditions, the modeled geometric mean recovery time for
AC was about 131 seconds. The longest mean recovery time resulted from a cold-start condition
after maintaining the HAZMATCAD Plus at room temperature overnight with a mean recovery
time of 315 seconds. Under cold-start conditions, after an overnight cold soak (5°C), the
modeled
38
-------
Table 6-4. Cold/Hot-Start Effects Data
(a)
HAZMATCAD
Plus Unit
22
27
Start
Condition
Control
(Fully
Warmed Up)
Room
Temperature
(Cold Start)
Cold
Temperature
(Cold Start)
Hot
Temperature
(Cold Start)
Control
(Fully
Warmed Up)
Room
Temperature
(Cold Start)
Cold
Temperature
(Cold Start)
Hot
Temperature
(Cold Start)
Experiment
Number
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
Delay(b)
Time
(mm:ss)
0:18
0:40
0:20
0:16
0:40
0:20
Response
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
Response
Time
(seconds)
3
4
7
6
8
12
10
6
8
7
6
8
9
5
7
9
10
10
9
9
13
8
5
7
8
12
4
6
8
7
6
5
8
7
6
9
5
5
6
4
Recovery
Time
(seconds)
171
227
220
143
178
156
219
600(c)
600(c)
600(c)
166
209
283
223
277
95
174
318
311
379
78
97
109
82
101
90
198
373
287
317
158
164
213
182
215
73
120
336
405
568
Alarm
(Indicated
Chemical)
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
BLOD
w All cold-hot-start tests conducted with AC.
-------
geometric mean recovery time was about 205 seconds. Under cold-start conditions, after an
overnight hot soak (40°C), the recovery time was also long, with a modeled geometric mean
recovery time of 230 seconds. Comparing the responses by start state (cold, room temperature,
and hot soak) with the responses from the control runs, no effect of start state on response could
be detected for AC at IDLH.
6.9 Battery Life
The battery life test was conducted by placing fully charged batteries in the HAZMATCAD Plus
units. The HAZMATCAD Plus units were then powered on and allowed to warm up fully
according to the manufacturer's directions.
An initial response time test was conducted with AC at the IDLH concentration level. The
HAZMATCAD Plus units then sampled clean air for approximately 30 minutes, and the AC
mixture was then sampled again. This procedure was repeated with the HAZMATCAD Plus
units operating continuously until the batteries were depleted, and the HAZMATCAD Plus units
no longer responded to the presence of the AC.
Table 6-5 provides the results for the battery life test. The time of each AC challenge, and the
response (and response time), are provided in the table for both HAZMATCAD Plus Unit 22
and Unit 27. Both HAZMATCAD Plus units gave a response of H BLOD and showed response
times of a few seconds every time the AC challenge was sampled. HAZMATCAD Plus Unit 22
responded in a normal fashion until the low-battery light began flashing at 15:42. Several
minutes later the low-battery light became a solid light and, within seven minutes, the all clear
light went out. During the next AC challenge (at 16:00), Unit 22 did not respond to the AC
challenge. The unit powered off at 16:12. The battery life for this unit was 9 hours and 49
minutes. HAZMATCAD Plus Unit 27 responded in a normal fashion until the low-battery light
began flashing at 16:52. During the next AC challenge, Unit 27 responded as H BLOD within
5 seconds. However, during this challenge, the low-battery light became a solid light. Six
minutes later the all clear light went out, and the unit powered off at 17:16. The battery life for
this unit was 10 hours and 53 minutes. There was no degradation of response or response time
with either HAZMATCAD Plus unit as the batteries approached depletion. Even when the low-
battery light was flashing and when it was solid, Unit 27 responded normally for the IDLH level
AC challenge. However, when the all clear light was extinguished, Unit 22 failed to respond.
6.10 Operational Characteristics
General performance observations noted during verification testing:
• Instrument Operation—The HAZMATCAD Plus has a button that is pushed to turn the
detector on. It also has a separate button to turn the detector off. Other buttons include an
ALARM SILENCE button that can be used to turn off the audible alarm and a MODE
SELECT button that can be used to switch between fast and sensitive modes.
40
-------
Table 6-5. Responses Recorded in Battery Life Testing
.(a)
Test
Start-up
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Time
0623
0630
0700
0730
0800
0830
0900
0930
1000
1030
1100
1130
1200
1230
1300
1330
1400
1430
1500
1530
1542
1550
1557
1600
1612
1630
1652
1700
1706
1716
HAZMATCAD Plus Unit
Unit 22
Response
(Response Time
[seconds])
HBLOD(IO)
H BLOD (5)
H BLOD (9)
H BLOD (7)
H BLOD (5)
H BLOD (8)
H BLOD (10)
H BLOD (5)
H BLOD (7)
H BLOD (5)
H BLOD (7)
H BLOD (5)
H BLOD (9)
H BLOD (8)
H BLOD (7)
H BLOD (6)
H BLOD (7)
H BLOD (6)
H BLOD (8)
No Response
Battery Indicator
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Low battery (flashing)
Low battery (solid)
All clear light out
Power Off
(9 hours, 49 minutes)
Unit 27
Response
(Response Time
[seconds])
H BLOD (2)
H BLOD (6)
H BLOD (7)
H BLOD (9)
H BLOD (8)
H BLOD (5)
H BLOD (5)
H BLOD (9)
H BLOD (10)
H BLOD (10)
H BLOD (7)
H BLOD (5)
H BLOD (5)
H BLOD (5)
H BLOD (5)
H BLOD (9)
H BLOD (6)
H BLOD (5)
H BLOD (7)
H BLOD (6)
H BLOD (11)
H BLOD (5)
Battery Indicator
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Low battery (flashing)
Low battery (solid)
All clear light out
Power off
(10 hours, 53
minutes)
) All battery life tests were conducted with AC as the challenge TIC at the IDLH concentration of 50 ppm
(50 mg/m3).
Instrument Indicators—The HAZMATCAD Plus has several lighted indicators to show the
status of the detector. These indicators include ALARM, ALL CLEAR, LO BAT, and SYS
OK. The HAZMATCAD Plus also has an LED display with large letters that indicate the
type of chemical detected and the level of alarm (high, medium, low) detected. The display
flashes the type of chemical and the level of alarm alternately until the detector clears.
41
-------
Warm-Up—The HAZMATCAD Plus units generally reach a ready state within one minute
after turning the detectors on.
Batteries—The HAZMATCAD Plus unit uses two batteries that are easy to obtain and easy
to install. The batteries fit only one way into the detector. Also, low batteries can be changed
during operation by changing one battery at a time so that the detector does not need to be
shut down to change the batteries.
Errors—One of the HAZMATCAD Plus units occasionally showed an XECM failure on the
LED display. The HAZMATCAD Plus unit would remain in this state until it was rebooted.
After rebooting, the HAZMATCAD Plus unit operated correctly. One of the HAZMATCAD
Plus units also would not alarm for GB.
Vendor Support—Before the verification, vendor representatives trained three Battelle
employees to operate the HAZMATCAD Plus units. Testing proceeded according to the
vendor's recommendations on how to operate the HAZMATCAD Plus units for testing. The
vendor also responded promptly when information was needed.
Cost—The list price of the HAZMATCAD Plus units, as used in this verification test, is
approximately $8,349 each.
42
-------
Chapter 7
Performance Summary
This chapter summarizes the overall performance results found in testing of the HAZMATCAD
Plus portable SAW and EC detector with both TICs and CW agents. This summary focuses on
aspects of the performance that are most important in field use of the HAZMATCAD Plus by
emergency first responders. Consistent with that use, most test procedures were conducted with
challenge levels that were at or near IDLH or comparable concentrations. Response thresholds
were tested by stepping downward in concentration from these challenge levels. The
HAZMATCAD Plus was not programmed to respond to CK, so no results are reported for that
TIC. Results summarized here are from two units of the HAZMATCAD Plus in all cases, except
GB testing (as a result of the failure of one unit to alarm for GB).
The HAZMATCAD Plus response for AC, CG, and SA at IDLH concentration was a full-scale
(high alarm) reading. The response for C12 at IDLH concentration was a mid-scale (medium
alarm) reading. The HAZMATCAD Plus response for GB was a mid-scale reading at 2.2 mg/m3
(11 times the IDLH concentration). The response for HD was a full-scale reading at 4 mg/m3
(7 times the AEGL-2 level for this agent).
In nearly all cases, the HAZMATCAD Plus provided an audible and visual alarm within 3 to
20 seconds after exposure to AC, CG, SA, or C12 over the range of temperature and humidity
tested (5 to 35°C; <20 to >80% RH). Over the different temperatures and humidities, response
time ranges for GB and HD were from 21 to 42 seconds and 15 to 177 seconds, respectively.
Over the ranges of 5 to 35 °C and <20 to >80% RH, temperature and RH had no practically
significant effect on response time for any TIC or CW agent. Response times for AC were
unaffected by operating the HAZMATCAD Plus from a cold start (i.e., with insufficient warm-
up time).
HAZMATCAD Plus recovery times (i.e., the time needed for the HAZMATCAD Plus to return
to baseline after the end of exposure to a TIC or CW agent) varied widely, depending on the TIC
or CW agent sampled and also on the sampling conditions. Recovery times differed considerably
from one TIC to another. For AC, modeled recovery times ranged from 76 to 361 seconds; for
CG, from 36 to 57 seconds; for SA, from 23 to 25 seconds; and for C12, 49 to 73 seconds. The
effect of temperature on recovery time was small, except for AC, for which recovery times
increased by about a factor of four as temperature decreased from 35°C to 5°C. Temperature had
an effect on recovery time for GB and HD, with recovery times at 35°C less than half of those at
5°C or 22°C. All TICs had recovery times less than about 90 seconds under all RH conditions,
with the exception of a modeled mean recovery time for AC of 131 seconds at medium (50%)
43
-------
RH. Recovery time for GB was slightly longer at higher humidity, whereas HD showed the
opposite trend, with recovery time increased by about 50% at low (<20%) RH relative to high
(>80%) RH. In operation from a cold start at normal temperature and humidity, the recovery
time for AC was lengthened to over 600 seconds.
The HAZMATCAD Plus was nearly 100% accurate in identifying the TIC being sampled under
all temperature and humidity conditions, with only one erroneous reading among nearly 250
data points. For HD, the response was 100% accurate under all temperature and humidity
settings. For GB, one unit of the HAZMATCAD Plus did not alarm during testing. With the
other unit, 100% accuracy was achieved for all conditions except the high humidity tests, where
unstable responses or no responses were recorded. (This behavior at high humidity is attributed
by the Microsensor Systems vendor to a possible decrease in the collection efficiency of the
concentrator in the SAW part of the HAZMATCAD Plus as a result of exposure to the TICs
during testing.)
The accuracy of the HAZMATCAD Plus in identifying the TIC or CW agent was the same when
alternating between different challenge concentrations as that when alternating between clean air
and a challenge concentration.
The repeatability, or consistency, of HAZMATCAD Plus response, response times, and recovery
times also was evaluated. Repeatability of response was always perfect under all levels of
temperature and humidity for AC, CG, and SA. C12 exhibited more variability in response.
Repeatability of response for GB and HD was unaffected by temperature level. At the different
humidity levels, HD had a consistent response. For GB, the response from the one
HAZMATCAD Plus unit was consistent at low and medium humidities, but either no response
or an unstable response was reported for GB at the high humidity, as noted above. The modeled
repeatability of response times for both TICs and CW agents showed no effect from the different
levels of temperature or humidity. The modeled repeatability of recovery times for AC, CG, SA,
GB, and HD showed no effect from the different temperature levels. The effect of temperature
on the repeatability of recovery time was significant for C12, with the recovery time most
repeatable under medium temperature. The modeled repeatability of recovery times for CG, S A,
C12, GB, and HD showed no effect from the different humidity levels. The effect of humidity on
the repeatability of recovery time was significant for AC, with the greatest variability for
recovery time under medium humidity.
The response thresholds of the HAZMATCAD Plus were 0.6 to 1.25 ppm for AC; 0.3 to 0.6
ppm for CG; 0.2 to 0.4 ppm for SA; 0.5 to 1 ppm for C12, 0.6 to 1.1 mg/m3 for GB; and 0.6 to
1.6 mg/m3 and 1.6 to 4 mg/m3 for HD on Units 22 and 27, respectively.
The effects of temperature and RH on the HAZMATCAD Plus TIC response were small, with
the largest effect that, at high temperature or high humidity, C12 produced some Low rather than
Medium responses. For GB at low temperature, the response was consistently High, while at
medium and high temperature, the response was consistently Medium. For HD, the response was
consistently High for each temperature level. For GB at high humidity, the response was either
44
-------
an unstable response or no response, as noted above. For HD, the response was consistently
High for each humidity level.
Interferents likely to be present indoors had large effects with individual TICs and CW agents.
No false positive responses occurred. In terms of false negatives, however, neither
HAZMATCAD Plus responded to GB in the presence of air freshener and latex paint fumes or
to HD in the presence of ammonia cleaner and latex paint fumes. Interferents had almost no
effect on response times for the TICs. The interferents showed an effect on the recovery time
primarily for C12, with all interferents increasing the C12 recovery time substantially.
Operating the HAZMATCAD Plus with insufficient warm-up time showed no effect on the
instrument response or response times for AC, regardless of whether the cold start occurred after
storage at 5°C, at room temperature, or at 40°C. There was a strong effect on recovery time
depending on start state, with the longest recovery time occurring at the room temperature cold-
start condition. The delay time (time for the HAZMATCAD Plus unit to be ready for a first
reading after start-up) ranged from 16 to 40 seconds.
The useful operating lives for fully charged batteries in two HAZMATCAD Plus units in
continuous operation were found to be 9 hours and 49 minutes and 10 hours and 53 minutes,
respectively. Both units showed rapid response and consistent readings throughout this test until
the last few minutes of useful battery life.
In general, the HAZMATCAD Plus was easy to use, gave clear alarms and a readable and
informative display, and provided error messages. Batteries are easy to obtain and install, and
new batteries can be installed without interrupting continuous operation. The one operational
limitation was the failure of one unit to respond during GB testing.
45
-------
Chapter 8
References
1. Test/QA Plan for Verification of Surface Acoustic Wave (SAW) and combined
SAW/Electrochemical Detectors for Detection of Chemicals and Chemical Agents in
Buildings, Battelle, Columbus, Ohio, April 2004
2. Environmental Technology Verification Program Quality Management Plan, Version 2.0,
December 2002.
3. Lodge, J. P. (ed.) "Determination of Phosgene in Air," in Methods of Air Sampling and
Analysis, Third Edition, Lewis Publishers, Chelsea, Michigan, 1989.
4. Spicer, C. W., Gordon, S. M., Holdren, M. W., Kelly, T. J., and Mukund, R., Hazardous
Air Pollutant Handbook: Measurements, Properties, and Fate in Ambient Air, ISBN 1-
56670-571-1, CRC Press, Boca Raton, Florida, 2002.
5. Turpin, J. "Direct Steam Injection Humidification: Is It Safe for Building Occupants?"
Engineered Systems, www.esmagazine.com/CDA/ArticleInformation/features/BNP
Features_Item/0,2503,23246,OO.html.
6. Sprent, P. and Smeeton, M. C. Applied Non-Parametric Statistical Methods, Chapman and
Hall, New York, 2001.
7. Kirk, R. E. Experimental Design: Procedures for Behavioral Sciences, Third Edition,
Brooks/Cole Publishing Co., 1995.
8. Klein, J. P. and Moeschberger, M. L., Survival Analysis: Techniques for Censored and
Truncated Data, Springer, New York, 1997.
9. Agresti, A. Categorical Data Analysis, John Wiley and Sons, New York, 1990.
10. Dunn, O. J., "Multiple comparisons using rank sums." Technometrics 6, 241-252, 1964.
11. Clopper, C. J. and Pearson, E. S.,"The use of confidence or fiducial limits illustrated in the
case of the binomial," Biometrika 26, 404-13, 1934.
46
------- |