&EPA
United States
Environmental Protection
Agency
REPORT
Science Applications
International Corporation
S-CAD Chemical Agent Detection System
Office of Research and Development
National Homeland Security
Research Center
-------
EPA600/R-06/140
June 2006
Technology Evaluation Report
Science Applications International
Corporation
S-CAD Chemical Agent Detection
System
By
Tricia Derringer, Thomas Kelly, Peter Bujnak,
Robert Krile, and Zachary Willenberg
Battelle
505 King Avenue
Columbus, OH 43201
Eric Koglin
Task Order Project Officer
National Homeland Security Research Center
Office of Research and Development
U.S. Environmental Protection Agency
944 East Harmon Ave.
Las Vegas, NV 89119
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and
Development's National Homeland Security Research Center, funded and managed this
technology evaluation through a Blanket Purchase Agreement under General Services
Administration contract number GS23F0011L-3 with Battelle. This report has been peer and
administratively reviewed and has been approved for publication as an EPA document. Mention
of trade names or commercial products does not constitute endorsement or recommendation for
use of a specific product.
-------
Preface
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
nation's air, water, and land resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, the EPA's Office of Research and Development (ORD) provides data and science
support that can be used to solve environmental problems and to build the scientific knowledge
base needed to manage our ecological resources wisely, to understand how pollutants affect our
health, and to prevent or reduce environmental risks.
In September 2002, EPA announced the formation of the National Homeland Security Research
Center (NHSRC). The NHSRC is part of the ORD; it manages, coordinates, and supports a
variety of research and technical assistance efforts. These efforts are designed to provide
appropriate, affordable, effective, and validated technologies and methods for addressing risks
posed by chemical, biological, and radiological terrorist attacks. Research focuses on enhancing
our ability to detect, contain, and clean up in the event of such attacks.
NHSRC's team of world-renowned scientists and engineers is dedicated to understanding the
terrorist threat, communicating the risks, and mitigating the results of attacks. Guided by the
roadmap set forth in EPA's Strategic Plan for Homeland Security, NHSRC ensures rapid
production and distribution of security-related products.
The NHSRC has created the Technology Testing and Evaluation Program (TTEP) in an effort to
provide reliable information regarding the performance of homeland security-related
technologies. TTEP provides independent, quality-assured performance information that is
useful to decision makers in purchasing or applying the tested technologies. It provides potential
users with unbiased, third-party information that can supplement vendor-provided information.
Stakeholder involvement ensures that user needs and perspectives are incorporated into the test
design so that useful performance information is produced for each of the tested technologies.
The technology categories of interest include detection and monitoring, water treatment, air
purification, decontamination, and computer modeling tools for use by those responsible for
protecting buildings, drinking water supplies, and infrastructure and for decontaminating
structures and the outdoor environment.
The evaluation reported herein was conducted by Battelle as part of the TTEP program.
Information on NHSRC and TTEP can be found at http://www.epa.gov/ordnhsrc/index.htm.
in
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
evaluation, analyze the data, and prepare this report. We also would like to thank
Donald Stedman of the University of Denver and Dale Werkema of EPA/ORD for their reviews
of this report.
IV
-------
Contents
Notice ii
Preface iii
Acknowledgments iv
Abbreviations/Acronyms vii
Executive Summary viii
1.0 Introduction 1
2.0 Technology Description 4
3.0 Quality Assurance/Quality Control 5
3.1 Equipment Calibration 5
3.1.1 Reference Methods 5
3.1.2 Instrument Checks 6
3.2 Audits 6
3.2.1 Performance Evaluation Audit 6
3.2.2 Technical Systems Audit 7
3.2.3 Data Quality Audit 7
3.3 QA/QC Reporting 7
4.0 Test Results 8
4.1 Response Time 8
4.2 Recovery Time 10
4.3 Accuracy 11
4.4 Repeatability 12
4.5 Response Threshold 13
4.6 Temperature and Humidity Effects 14
4.7 Interference Effects 14
4.8 Cold-/Hot-Start Behavior 18
4.9 Battery Life 20
4.10 Operational Characteristics 21
5.0 Performance Summary 23
6.0 References 26
v
-------
Figures
Figure 2-1. SAIC S-CAD Chemical Agent Detection System 4
Tables
Table 1-1. Target TIC and CW Agent Challenge Concentrations 2
Table 3-1. Performance Evaluation Audit Results 7
Table 4-1. TIC and CW Agent Results from S-CAD Evaluation 9
Table 4-2. Response Threshold Data for the TIC and CW Agent Evaluation 14
Table 4-3. Interference Effects 15
Table 4-4. Start State Effects 19
Table 4-5. Responses Recorded from the S-CAD in Battery Life Evaluation 21
VI
-------
Abbreviations/Acronyms
AC
CW
DEAE
EPA
GB
GC/FID
GC/FPD
HD
IDLH
IMS
L
ug
uL
mg/m3
mL
mm
NHSRC
ORD
PE
ppm
ppmC
QA
QC
QMP
RH
SAIC
SAW
THC
TIC
ISA
TTEP
hydrogen cyanide
chemical warfare
N,N-diethylaminoethanol
U.S. Environmental Protection Agency
sarin
gas chromatography/flame ionization detection
gas chromatography/flame photometric detection
sulfur mustard
immediately dangerous to life and health
ion mobility spectrometer(ry)
liter
microgram
microliter
milligram per cubic meter
milliliter
millimeter
National Homeland Security Research Center
Office of Research and Development
performance evaluation
parts per million
parts per million of carbon
quality assurance
quality control
quality management plan
relative humidity
Science Applications International Corporation
surface acoustic wave
total hydrocarbon
toxic industrial chemical
technical systems audit
Technology Testing and Evaluation Program
vn
-------
Executive Summary
The U.S. Environmental Protection Agency's (EPA's) National Homeland Security Research
Center (NHSRC) Technology Testing and Evaluation Program (TTEP) is helping to protect
human health and the environment from adverse impacts as a result of acts of terror by carrying
out performance tests on homeland security technologies. Under TTEP, Battelle recently
evaluated the performance of the Science Applications International Corporation (SAIC) S-CAD
Chemical Agent Detection System in detecting a toxic industrial chemical (TIC) and chemical
warfare (CW) agents in indoor air.
The S-CAD uses electrochemical cells to detect TICs and both ion mobility spectrometry (IMS)
and surface acoustic wave (SAW) methods to detect CW agents. With its data fusion algorithm,
it is designed to provide a higher probability of detection with a reduced false alarm rate. The
S-CAD gathers and stores data for future analysis, and its modular design allows it to be
integrated with nuclear and biological agent detectors and other application-specific sensors. It is
designed to operate in 10% to 90% relative humidity (RH).
The following performance characteristics of the S-CAD were evaluated:
# Response time
# Recovery time
# Accuracy of hazard identification
# Repeatability
# Response threshold
# Temperature and humidity effects
# Interference effects
# Cold-/hot-start behavior
# Battery life
# Operational characteristics.
This evaluation addressed detection of chemicals in the vapor phase. The TIC and the challenge
concentration delivered to the S-CAD during the evaluation were hydrogen cyanide (HCN;
North Atlantic Treaty Organization military designation AC; 50 mg/m3), and the CW agents and
concentrations were sarin (GB; 0.060 mg/m3) and sulfur mustard (HD; 0.54 mg/m3). Two
S-CAD units (designated A and B) were evaluated simultaneously with the TIC; one unit
(Unit B) of the S-CAD was evaluated with the CW agents. The use of only one unit in testing
with CW agents minimized the expense to the vendor because that unit could not be returned
after contamination with agents.
The evaluation included sampling potential indoor interferents, both with and without the target
TIC and CW agents. The interferents used were latex paint fumes; air freshener vapors; ammonia
cleaner vapors; a mixture of hydrocarbons representing motor vehicle exhaust; and
Vlll
-------
diethylaminoethanol (DEAE), a boiler water additive that can enter indoor air via steam
humidification. A range of temperatures (5 to 35°C) and relative humidities (<20 to 80%) was
used to assess the effects of these conditions. The S-CAD units were challenged at the start of
every test day with a confidence check sample provided by the vendor. No test activities were
initiated until a valid response to the confidence check sample was obtained.
Summary results from testing the S-CAD are presented below for each performance parameter
evaluated. Discussion of the observed performance can be found in Chapter 4 of this report.
Results with AC are from testing two units of the S-CAD; results for GB and HD are from
testing one unit.
Response Time: When the S-CAD responded to challenges with AC, the time required was
35 seconds or less, with no consistent effect of temperature or RH. Similarly, most response
times for GB were 43 seconds or less, but response time increased at the highest humidity
conditions to about 60 to 260 seconds. Response times for HD ranged from about 30 to
60 seconds, and were not affected by the temperature and RH. These results do not include
instances in which the S-CAD failed to respond to TIC or CW agent challenges; those instances
are noted below under Accuracy.
Recovery Time: Recovery times for AC ranged widely, from 13 seconds to over 600 seconds,
with no consistent temperature or RH effects for the two units. For the most part, recovery times
for GB were less than 30 seconds, regardless of temperature or RH conditions. Recovery times
for HD ranged from 35 to 146 seconds, with faster recovery at higher temperatures and higher
RH. These results exclude those instances in which the S-CAD did not respond to a TIC or agent
challenge.
Accuracy: Of the 120 challenges with AC, GB, and HD used to assess accuracy, the S-CAD
responded accurately to 102 and did not respond to the other 18. Both S-CAD units identified
AC with 100% accuracy under almost all temperature and RH conditions. The primary exception
was that one unit correctly identified AC in only one of five challenges at room temperature and
high (80%) RH (i.e., 20% accuracy). Accuracy of identifying GB was 80 to 100% in most tests,
but was 20 to 60% in tests at high RH, indicating a dependence on RH. Accuracy of identifying
HD was 100% in most conditions, except for values of 80% at the high temperature (35 °C) and
50% RH condition, and 0% at high temperature and high RH.
[Failure to respond to AC challenges was also observed occasionally during cold-/hot-start and
battery life tests, but those observations were not used in the calculation of the accuracy results
noted above.]
Repeatability: When responding to an AC challenge, the repeatability of the S-CAD's
Low/Medium/High readings for AC was not affected by temperature, for either unit tested. One
unit tended to show higher readings at higher RH, but the other unit did not. Repeatability of
responses for GB was unaffected by temperature, but RH had an effect, with Low readings at the
highest (80%) RH and Medium readings at other conditions. Repeatability for HD was affected
IX
-------
by temperature, with readings dropping from High at 5 °C to Low at 35 °C, and by RH, with
readings changing from High at 20% RH to Medium at other conditions.
Response Threshold: For AC, the response threshold was between 1.5 and 3 parts per million
(ppm) [1.5 and 3 milligrams per cubic meter (mg/m3)] on one unit, and between 6 and 12.5 ppm
(6 and 12.5 mg/m3) on the other unit. For GB, the response threshold was less than 0.002 ppm
(0.01 mg/m3), and for HD it was less than 0.03 ppm (0.2 mg/m3).
Temperature and Humidity Effects: These effects are described in the preceding summaries
of other performance parameters.
Interference Effects: There were almost no false positive readings from the two S-CAD units
when tested with each of the five interferences one at a time in clean air. The only such false
positive response was a single reading (out of five separate challenges) from Unit B with latex
paint fumes.
[Erroneous positive responses of a different kind (i.e., alarms while the S-CAD sampled clean
air) were observed in several cases during tests addressing accuracy, interference effects, and
cold-/hot-start behavior.]
When added to challenge mixtures of AC, the interferences had relatively small effects on the
performance of the S-CAD. For Unit A, most interferences did not affect the accuracy of
identification, response time, or response level (Low/Medium/High), but all five interferences
did lengthen the recovery time after detection of AC. Engine exhaust reduced Unit A accuracy
for AC to 60%, although this result was not statistically significant. For Unit B, the interferences
had no effect on response time or response level, and most interferences had no effect on
accuracy. However, accuracy of AC identification by Unit B was reduced to 0% by latex paint
fumes. Recovery time was lengthened for Unit B by air freshener vapors and DEAE, but was
shortened by ammonia cleaner and engine exhaust hydrocarbons.
False negative responses with GB and HD reduced the accuracy of identification, but response
time, recovery time, and response levels were unaffected by the interferences when the unit did
alarm. Accuracy for GB was reduced to 0% by latex paint fumes, ammonia cleaner, and engine
exhaust hydrocarbons, and to 20% by air freshener vapors. With paint fumes, the S-CAD failed
to respond during the GB challenges, but alarmed upon sampling clean air. Accuracy for HD was
reduced to 0% by the paint fumes, and to 20% by ammonia cleaner vapors. In at least one HD
challenge with each of these two interferences, the S-CAD stopped alarming while the agent
challenge was in progress.
Cold-/Hot-Start Behavior: The response time, recovery time, response level, and identification
accuracy of the two S-CAD units were essentially the same in operation after a cold start and
during fully warmed up operation. The delay times (time before readiness to take a reading) were
160 to 240 seconds with the two S-CAD units after start-up from room temperature or hot (40
°C) storage, but the delay times after cold (5 °C) storage were 1,218 seconds and 1,440 seconds,
respectively.
x
-------
Battery Life: One unit of the S-CAD shut down after 6 hours and 30 minutes of continuous
operation on battery power. The other unit shut down after 9 hours and 18 minutes.
Operational Characteristics: The display of the S-CAD was easy to read as long as the
contrast mechanism worked properly, but when this failed the display was unreadable. Such
failure occurred at both hot and cold temperatures during testing. When that happened, test
procedures were continued by observing S-CAD responses on a laptop computer, but that option
will not typically be available to a user in the field. The red alarm light on top of the S-CAD was
not easily visible when looking directly at the face of the instrument, and the volume of the
audible alarm was weak.
Before this evaluation began, an SAIC representative trained Battelle testing personnel to operate
the S-CAD. Testing proceeded according to the vendor's recommendations, and the vendor
responded promptly when information was needed during the evaluation. The list price of the
S-CAD is approximately $25,000 to $35,000, depending on the instrument configuration and the
number of units ordered.
Conclusion: The S-CAD provided accurate detection and identification of AC, GB, and HD
under most temperature and RH conditions in air. There was little effect from temperature, but
failures to respond to challenges were seen in some tests, especially at high RH, and there was
some variation in response and recovery times. Start-up conditions had no effect other than a
lengthy delay time before readings could be obtained after storage under cold conditions.
Erroneous positive readings were seen in a few tests when the S-CAD alarmed while sampling
clean air. A key performance issue disclosed by this test is the suppression of response to GB and
HD caused by some interferences. This behavior is unexpected in that the S-CAD uses IMS and
SAW principles simultaneously for CW agent detection, specifically to minimize interferences.
Other areas for improvement include the visibility and audibility of the alarm indicators, and the
reliability of the visual display.
XI
-------
1.0 Introduction
The U.S. Environmental Protection Agency's (EPA's) National Homeland Security Research
Center (NHSRC) is helping to protect human health and the environment from adverse impacts
as a result of intentional acts of terror. With an emphasis on decontamination and consequence
management, water infrastructure protection, and threat and consequence assessment, NHRSC is
working to develop tools and information that will help detect the intentional introduction of
chemical or biological contaminants in buildings or water systems, the containment of these
contaminants, the decontamination of buildings and/or water systems, and the disposal of
material resulting from clean-ups.
NHSRC's Technology Testing and Evaluation Program (TTEP) works in partnership with
recognized testing organizations; with stakeholder groups consisting of buyers, vendor
organizations, and permitters; and with the full participation of individual technology developers
in carrying out performance evaluations on homeland security technologies. The program
evaluates the performance of innovative homeland security technologies by developing
evaluation plans that are responsive to the needs of stakeholders, conducting evaluations,
collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are
conducted in accordance with rigorous quality assurance (QA) protocols to ensure that data of
known and high quality are generated and that the results are defensible. TTEP provides high
quality information that is useful to decision makers in purchasing or applying the evaluated
technologies. It provides potential users with unbiased, third-party information that can
supplement vendor-provided information. Stakeholder involvement ensures that user needs and
perspectives are incorporated into the evaluation design so that useful performance information
is produced for each of the evaluated technologies.
Under TTEP, Battelle recently evaluated the performance of the Science Applications
International Corporation (SAIC) S-CAD Chemical Agent Detection System in detecting a toxic
industrial chemical (TIC) and chemical warfare (CW) agents in indoor air. This evaluation was
conducted according to a peer-reviewed test/QA plan(1) that was developed in accordance with
the requirements of the quality management plan (QMP) for TTEP.(2) The following
performance characteristics of the S-CAD were evaluated:
# Response time
# Recovery time
# Accuracy of hazard identification
# Repeatability
# Response threshold
-------
# Temperature and humidity effects
# Interference effects
# Cold-/hot-start behavior
# Battery life
# Operational characteristics.
In this evaluation, two units of the S-CAD (designated A and B) were evaluated simultaneously
with one TIC (hydrogen cyanide). In evaluating two CW agents (sarin and sulfur mustard), only
one unit (Unit B) of the S-CAD was used, with the other kept in reserve. This approach
minimized the expense to the vendor of the S-CAD because the unit tested with CW agents could
not be returned after testing. Results are reported for the two units separately. The S-CAD units
were challenged at the start of every test day with a confidence check sample provided by the
vendor. No test activities were initiated until a valid response to the confidence check sample
was obtained on each S-CAD unit being tested. This challenge was also repeated as needed
during testing (e.g., in the case of an unexpected response) before continuing the test procedures.
This evaluation addressed detection of chemicals in the vapor phase, because that application is
most relevant to use in a building contamination scenario. This evaluation took place between
May 2 and August 11, 2005, in two phases: detection of TICs (conducted in a non-surety
laboratory at Battelle) and detection of CW agents (conducted in a certified surety laboratory at
Battelle's Hazardous Materials Research Center). The one TIC used was hydrogen cyanide
(HCN; North Atlantic Treaty Organization military designation AC). The CW agents were sarin
(GB) and sulfur mustard (HD). Most evaluation procedures were conducted with challenge
concentrations of the TIC or CW agent that were at or near immediately dangerous to life and
health (IDLH) or similar levels, as specified in the test/QA plan.(1) Table 1 summarizes the
primary challenge concentrations used.
Table 1-1. Target TIC and CW Agent Challenge Concentrations
Chemical Concentration Type of Level
Hydrogen cyanide (AC) 50 parts per million (ppm) IDLH(a)
[50 milligrams per cubic
meter (mg/m3)]
Sarin (GB) 0.011 ppm (0.060 mg/m3) 0.3 x IDLH
Sulfur mustard (HD) 0.081 ppm (0.54 mg/m3) 0.9 x AEGL-2(b)
-1 IDLH = Immediately dangerous to life and health.
(b) AEGL = Acute exposure guideline level; AEGL-2 levels are those expected to produce a serious hindrance to
efforts to escape in the general population. The AEGL-2 value of 0.09 ppm (0.6 mg/m3) for HD is based on a
10-minute exposure.
In all evaluations, the TIC or CW agent challenge concentrations were confirmed by means of
reference analysis of the challenge air stream. The reference method for AC was a gas chroma-
tography method using flame ionization detection (GC/FID), with sample collection from the
challenge air stream into gas sampling bags. The reference method for GB and HD was gas
-------
chromatography with flame photometric detection (GC/FPD), again using bags for sample
collection.
As described in the test/QA plan,(1) response time, recovery time, accuracy, and repeatability
were evaluated by alternately challenging the S-CAD units with clean air and known vapor
concentrations of the target TIC or CW agent. Response thresholds were evaluated by challenges
with concentrations well below the target values shown in Table 1-1. Evaluations conducted over
the range of 5 to 35°C and 20 to 80% relative humidity (RH) were used to establish the effects of
temperature and humidity on detection capabilities. The test apparatus allowed RH to be changed
rapidly; a few minutes of continuous operation were allowed to thoroughly flush all flow paths
after a change in the RH (with no change in temperature). On the other hand, typically two to
three hours of stabilization time were allowed after a change in the test temperature. In all cases,
testing resumed only after the temperature and RH sensors in the test apparatus showed readings
stabilized within the required ranges. Throughout the stabilization period after any change, the
S-CAD units remained enclosed in the test apparatus, sampling clean air of the target RH.
The effects of potential indoor interferences were assessed by sampling selected interferences
both with and without the target TIC or CW agent present. The interferences used were latex
paint fumes, ammonia floor cleaner vapors, air freshener vapors, a mixture of gasoline exhaust
hydrocarbons, and diethylaminoethanol (DEAE), a boiler water additive potentially released to
indoor air by humidification systems. The concentrations of the interferents were checked during
the evaluation by means of a total hydrocarbon (THC) analyzer, calibrated with known
concentrations of propane. The S-CAD units were also evaluated with AC after a cold start (i.e.,
without the usual warm-up period) from room temperature, from cold storage conditions (5 °C),
and from hot storage conditions (40 °C) to evaluate the delay time before readings could be
obtained and the response speed and accuracy once readings were obtained. Battery life was
determined as the time until S-CAD performance degraded as battery power was exhausted in
continuous operation. Operational factors such as ease of use, data output, and cost were
assessed through observations made by evaluation personnel and through inquiries to the vendor.
The evaluation data were subjected to multivariate and other statistical analyses, as described in
the test/QA plan,(1)to characterize the performance of the S-CAD.
QA oversight of this evaluation was provided by Battelle and EPA. Battelle QA staff conducted
a technical systems audit (TSA) and a data quality audit of all the evaluation data. A
performance evaluation (PE) audit of the reference method for AC was also conducted.
-------
2.0 Technology Description
This report provides results for the evaluation of the S-CAD hand-held chemical agent detector.
Following is a description of the S-CAD, based on information provided by the vendor. [Contact:
Scott Smith, Science Applications International Corporation, 16701 W. Bernardo Drive, San
Diego, California, 92127, 858-826-9775, smithsl@saic.com] The information provided below
was not verified in this evaluation.
The S-CAD is designed to be used by military,
security, first responder, and medical personnel
to detect, identify, and determine the concentra-
tion of CW agents and TICs. The S-CAD uses
electrochemical cells to detect TICs and uses
both ion mobility spectrometry (IMS) and
surface acoustic wave (SAW) methods simul-
taneously to detect CW agents. The S-CAD
provides an indication of the chemical class of a
detected hazard (e.g., NERVE, BLOOD,
BLISTER), along with a relative indication
(Low, Medium, High) of the intensity of
response. With its data fusion algorithm, it is
designed to provide a higher probability of
detection with a reduced false alarm rate. The
S-CAD gathers and stores data for future
analysis, and its modular design allows it to be
integrated with nuclear and biological agent
detectors and other application-specific sensors.
It is designed to operate in 10 to 90% RH.
Figure 2-1. SAIC S-CAD Chemical
Agent Detection System The S-CAD can be used with or without
batteries. Without batteries, it weighs
1.8 kilograms (4 pounds) and with batteries, 2.1 kilograms (4.6 pounds). The S-CAD can be
operated with rechargeable batteries or using 12-volt, direct current power. It has an audible
alarm and a visual display screen that indicates agent/TIC type and concentration. The price
range for the S-CAD, depending on the configuration and the number of units ordered, is
$25,000 to $35,000.
-------
3.0 Quality Assurance/Quality Control
QA/quality control (QC) procedures were performed in accordance with the program QMP(2) and
the test/QA plan(1) for this evaluation.
3.1 Equipment Calibration
3.1.1 Reference Methods
AC reference sampling was performed on 12 days between May 2 and May 17, 2005, by
collecting the challenge mixtures into 1-liter (L) Tedlar gas sampling bags with GC septum
fittings. A 20 milliliter (mL) glass syringe was then used to withdraw an aliquot through the
sealed septum for injection into a 100 microliter (uL) loop for on-column manual injection.
A new AC calibration curve was prepared on each day of testing. Initially, standards at levels of
4, 10, and 60 ppm were prepared by diluting a certified 967-ppm AC standard (Scott Specialty
Gases) into high purity air in 1-L Tedlar gas sampling bags. From May 9 onward, the same
standard concentrations were prepared by dilution of a certified 10,000-ppm AC standard (Scott
Specialty Gases). Linear regression of each day's calibration data provided the calibration
equation that was applied to samples from that day. Over the 12 days of testing, the average
regression result was Peak Area = 155.8 (± 27.3) (AC, ppm) + 94.5 (± 174.0) area units, with r2
= 0.9999, where the error bars indicate ± one standard deviation. These results show a 17.5%
relative standard deviation of the daily slopes, with an average intercept that did not differ
significantly from zero, and r2 values close to 1.0.
Each sample was injected twice, and the average response was used in calculating AC
concentration. Also, a known propane standard was injected each day to track any drift of the
FID signal. The propane standard used was a Scott Specialty Gases 33-ppm compliance class
standard. On May 9, 10, and 11, the propane was introduced into an empty Tedlar bag and then
injected from the bag to the filling loop using a 20-mL syringe. From May 12 onward, a septum
was installed on the propane cylinder, and the syringe was filled directly from the tank and
injected into the loop. The three bag injections of propane standard displayed an average peak
area of 43,857 area counts and a variability of 12.3% relative standard deviation. The 12
subsequent direct syringe injections showed an average peak area of 52,614 area units, with a
3.1% relative standard deviation. The latter degree of variability is consistent with that expected
for FID response and shows minimal drift over the course of the reference analyses. The greater
-------
variability of the first three propane analyses is attributed to the transfer of the propane standard
to gas sampling bags.
Calibration standards for the CW agents GB and HD were prepared by diluting stock agent to
micrograms (Og) per mL concentrations and then injecting a 1-microliter (OL) volume of each
standard into the GC/FPD. Calibration was based on a regression of peak area versus amount of
agent injected.
For GB and HD testing, new calibration plots were prepared at least once a week during detector
evaluation for a total of six GB calibrations and four HD calibrations. The concentrations of the
standards used were 0.0075, 0.1, 0.25, 0.5, and 0.75 ^g/mL for GB and 0.25, 0.5, 1.0, 2.5, 5.0,
and 10 [ig/mL for HD. Low range calibrations were used to determine agent concentrations for
the response threshold and high/low tests. In all cases, agent concentrations were determined by
using the most recent calibration plot. All calibration plots for both agents were linear, with r2
values of greater than 0.99.
The THC analyzer used to document the interferent levels provided in the evaluation was
calibrated by filling a 25-L Tedlar bag with the same 33-ppm propane standard noted above.
Since propane is a three-carbon molecule, this standard constitutes a THC concentration of
99 ppm of carbon (ppmC). This standard was used for single-point calibration of the THC
analyzer on each test day. Clean air from the analytical laboratory was used for zeroing the
analyzer.
3.1.2 Instrument Checks
The S-CAD was operated and maintained according to the vendor's instructions throughout the
evaluation. Maintenance was performed according to predefined diagnostics. Daily operational
check procedures were performed with vendor-supplied simulant tubes. Proper response of the
S-CAD to the simulant was required before testing could proceed.
3.2 Audits
3.2.1 Performance Evaluation Audit
A PE audit was conducted to assess the quality of reference measurements made in the
evaluation. For AC, the PE audit was performed once prior to the start of testing by diluting and
analyzing a standard that was independent of the standard used for testing. The acceptable
tolerance for this PE audit was ±20%. Table 3-1 shows that the results of the PE audit were well
within the target tolerance.
Independent PE audit samples do not exist for GB and HD. Instead, for the CW agents, check
standards of GB and HD were prepared by individuals other than the staff conducting the
reference analyses. The check standards were prepared in the same way as the reference
calibration standards, i.e., by dilution of military grade agent. The results obtained for these two
-------
sets of standards were then compared. For GB, standards were prepared at concentrations of
0.75, 0.50, 0.25, and 0.1 Og/mL. All results were within 9% for the separate standards made by
two individuals. For FID, standards were prepared at concentrations of 5, 2.5, 1.0, and
0.5 Og/mL. All results were within 15% for the separate standards made by two individuals.
Table 3-1. Performance Evaluation Audit Results
TIC
AC
Standard
PE Audit
Date of
Sample Audit
(Cylinder C74059)(a)
Std (Cylinder LL320) (b) 3/3/05
Standard
Concentration
10,
10,
000
000
ppm
ppm
Diluted Agreement
Result (%)
45.
51.
.8 ppm
.5 ppm
11.1
Obtained from Scott Specialty Gases.
(a) Obtained from Linde Gas.
3.2.2 Technical Systems Audit
The Battelle Quality Manager conducted a TSA to ensure that the evaluation was performed in
accordance with the test/QA plan(1) and the TTEP QMP.(2) As part of the audit, the Battelle
Quality Manager reviewed the reference sampling and analysis methods used, compared actual
test procedures with those specified in the test/QA plan,(1) and reviewed data acquisition and
handling procedures. No significant adverse findings were noted in this audit. The records
concerning the TSA are permanently stored with the Battelle Quality Assurance Manager.
3.2.3 Data Quality Audit
At least 10% of the data acquired during the evaluation were audited. The Battelle Quality
Assurance Manager traced the data from the initial acquisition, through reduction and statistical
analysis, to final reporting, to ensure the integrity of the reported results. All calculations
performed on the data undergoing the audit were checked.
3.3 QA/QC Reporting
Each assessment and audit was documented in accordance with the test/QA plan(1) and the
QMP(2) Once the assessment report was prepared by the Battelle Quality Manager, it was routed
to the Test Coordinator and Battelle TTEP Program Manager for review and approval. The
Battelle Quality Manager then distributed the final assessment report to the EPA Quality
Manager and Battelle staff.
-------
4.0 Test Results
The S-CAD was evaluated with the TIC AC and the CW agents HD and GB. Test procedures
were based on sets of five challenges with a TIC or CW agent, alternating those challenges with
intervals of sampling clean air.(1) Statistical approaches were used to assess the performance
parameters listed in Chapter 1 for the S-CAD for these compounds, as specified in the test/QA
plan.(1) Two S-CAD units (Units A and B) were used during TIC evaluation, and one S-CAD
unit (Unit B) was used during CW agent evaluation. The following sections summarize the
findings of this evaluation; results for both TIC and CW agents are included for each
performance parameter.
Table 4-1 summarizes the results used in the analysis of performance parameters for the TIC and
CW agent evaluations. This table shows data from all evaluations for both S-CAD units for
illustration purposes, and the TIC and CW agent results shown are drawn from data obtained at
the target concentrations (see Table 1-1).
4.1 Response Time
Results of the response time analysis are summarized here, including temperature and humidity
effects. Note that only challenges to which the S-CAD actually gave a response are included in
the analysis of response time. As Table 4-1 shows, in 120 total challenges with AC, GB, and HD,
the S-CAD failed to respond in 18 cases. Failure to respond is addressed under Accuracy in
Section 4.3.
Unit A for AC—Across the three temperatures [low temperature (5 °C), room temperature
(22 °C), and high temperature (35 °C)] evaluated at medium humidity (50% RH), the geometric
mean times to first response were 15.3, 22.3, and 14.3 seconds, respectively. The low
temperature average time was not statistically significantly different from the room temperature
average. However, the high temperature average was significantly less than the room
temperature average. Across the three humidity levels [low (<20% RH), medium, and high (80%
RH)] evaluated at room temperature, the geometric mean times to first response were 14.1, 22.3,
and 10.0 seconds, respectively. Both the low and high humidity means were statistically
significantly less than the medium humidity mean. However, the high humidity estimate should
be viewed cautiously as it is based on only one trial. Overall, this unit showed statistically
significant effects for both temperature and humidity on time to first response.
-------
Table 4-1. TIC and CW Agent Results from S-CAD Evaluation
__,, . Environmental Conditions
CW Agent
AC Control (22
22°C-
22°C-
35°C-
35°C-
5°C-
GB Control (22
22°C-
22°C-
35°C-
35°C-
5°C-
HD Control (22
22°C-
22°C-
35°C-
35°C-
5°C-
°C - 50% RH)
<20% RH
- 80% RH
- 50% RH
- 80% RH
50% RH
°C - 50% RH)
<20% RH
- 80% RH
- 50% RH
- 80% RH
50% RH
°C - 50% RH)
<20% RH
- 80% RH
- 50% RH
- 80% RH
50% RH
S-CAD
Response
M(5)/H(5)
L(2)/M(3)/H
(5)
H
H
H
H
M
M
L
M(1)/H(4)
L
H
M
H
M
L
-
H
Alarms
(Indicated Chemical)
10/10 (BLOOD)
10/10 (BLOOD)
6/10 (BLOOD)
4/10 (NR)(C)
10/10 (BLOOD)
10/10 (BLOOD)
9/10 (BLOOD)
1/10 (NR)
5/5 (NERVE)
5/5 (NERVE)
3/5 (NERVE)
2/5 (NR)
5/5 (NERVE)
1/5 (NERVE)
4/5 (NR)
4/5 (NERVE)
1/5 (NR)
5/5 (BLISTER)
5/5 (BLISTER)
5/5 (BLISTER)
4/5 (BLISTER)
1/5 (NR)
5/5 (NR)
5/5 (BLISTER)
Response
Time Range
(Seconds)00
9-35
9-25
8-17
10-19
13-22
9-24
11-25
14-15
59-257
15-30
186
29-43
31-54
32-41
31-66
42-51
-
41-49
Recovery
Time Range
(Seconds)00
17-98
13-60000
62-113
36-600
45-600
84-180
18-20
15-22
13-14
14-29
6
21-600
47-68
73-101
45-81
35-49
-
106-146
•' Response time and recovery time evaluated only when the S-CAD showed response to the challenge.
*' 600 seconds = Maximum time monitored for detector recovery time.
^ NR = No response.
Unit B for AC—Across the three temperatures (low, room, and high) evaluated at medium
humidity, the geometric mean times to first response were 12.3, 13.5, and 14.7 seconds,
respectively. The observed response times for the low and high temperatures were not
statistically significant in comparison to room temperature. Across the three humidity levels
(low, medium, and high) evaluated at room temperature, the geometric mean times to first
response were 13.2, 13.5, and 11.5 seconds, respectively. Neither the low nor the high humidity
average times were statistically significant in comparison to the medium humidity average.
Therefore, neither temperature nor humidity had a statistically significant effect on the time to
first response for this unit.
Unit B for GB—Across the three temperatures (low, room, and high) evaluated at medium
humidity, the geometric mean times to first response were 37.4, 14.7, and 21.2 seconds,
respectively. The low temperature average response time was statistically significantly greater
than the room temperature condition. Across the three humidity levels (low, medium, and high)
evaluated at room temperature, the geometric mean times to first response were 14.4, 14.7, and
97.4 seconds, respectively. The high humidity average response time was statistically
significantly greater than the medium humidity condition.
-------
Unit B for HD—Across the three temperatures (low, room, and high) evaluated at medium
humidity, the geometric mean times to first response were 44.3, 39.9, and 46.8 seconds,
respectively. Across the three humidity levels (low, medium, and high) evaluated at room
temperature, the geometric mean times to first response were 37.3, 39.9, and 39.8 seconds,
respectively. There was no statistically significant effect on time to first response of temperature
at medium humidity or of humidity at room temperature.
4.2 Recovery Time
Results of the recovery time analysis are summarized below and presented in Table 4-1, which
summarizes the data recorded in the tests conducted on the S-CAD. As with response time,
recovery time was only evaluated for those cases in which the S-CAD responded to a challenge
mixture.
Unit A for AC—Of the observations across the three temperatures (low, room, and high)
evaluated at medium humidity, the geometric mean recovery times were 94.0, 20.5, and
43.6 seconds, respectively. From these data, the low and high temperature average times to clear
were statistically significantly greater than the room temperature time to clear for this unit.
Across the three humidity levels (low, medium, and high) evaluated at room temperature, the
geometric mean recovery times were 15.2, 20.5, and 62.0 seconds, respectively. The low
humidity average recovery time was statistically significantly faster to clear than the medium
humidity time to clear. (One trial at low humidity did not clear within 600 seconds and was
removed from the data before modeling.) The high humidity average recovery time was much
longer than the medium humidity time, but should be interpreted cautiously since this S-CAD
unit gave a response in only one trial at high humidity and room temperature with AC. Overall, it
appears there were both temperature and humidity effects on time to clear for this unit.
Unit B for AC—Of the observations across the three temperatures (low, room, and high)
evaluated at medium humidity, the geometric mean recovery times were 145, 88.3, and
63.5 seconds, respectively. From these data, the low temperature average time to clear was
statistically significantly greater than the room temperature time to clear. The high temperature
average time was statistically significantly shorter than the room temperature time to clear. (Note
that two of the five high temperature trials did not achieve clearance within 600 seconds and
were not included in the analysis). Across the three humidity levels (low, medium, and high)
evaluated at room temperature, the geometric mean recovery times were 83.7, 88.3, and
96.9 seconds, respectively. Neither the low humidity nor the high humidity average recovery
times were statistically significantly different from the medium humidity condition. Overall, it
appears there was a temperature effect, but no humidity effect on time to clear for this unit.
Unit B for GB—Of the observations across the three temperatures (low, room, and high)
evaluated at medium humidity, the geometric mean recovery times were 22.3, 19.4, and
17.5 seconds, respectively. This does not represent a statistically significant difference for either
the high or low temperature compared to room temperature. Across the three humidity levels
(low, medium, and high) evaluated at room temperature, the geometric mean recovery times
10
-------
were 19.0, 19.4, and 13.3 seconds, respectively. The high humidity average time to clear was
statistically significantly less than the medium humidity condition. However, temperature and
humidity effects for GB appear to be minimal.
Unit B for HD—Across the three temperatures (low, room, and high) evaluated at medium
humidity, the geometric mean times to clear were 13.1, 55.6, and 42.1 seconds, respectively. The
effects of both low and high temperature were statistically significant with low temperature
clearance time greater than room temperature and high temperature clearance time faster than
room temperature. Across the three humidity levels (low, medium, and high) evaluated at room
temperature, the geometric mean times to clear were 82.2, 55.6, and 60.1 seconds, respectively.
The recovery time at low humidity was statistically significantly greater than at medium
humidity. The high humidity recovery time was not statistically significantly different from the
medium humidity condition. Overall, higher temperature and higher humidity tended to shorten
recovery times for HD.
4.3 Accuracy
Results of the accuracy analysis are summarized below and are based on the data presented in
Table 4-1. The accuracy of a unit was defined as the proportion of trials in which the unit
registered an accurate response to the challenge. The S-CAD was considered accurate if it
alarmed in the presence of the TIC or CW agent and correctly identified the TIC or CW agent
class. For the S-CAD, any level of response (Low, Medium, or High) and "BLOOD" were
considered by the manufacturer to be an accurate response to AC. Also, any level of response
(Low, Medium, or High) and "NERVE" for GB and "BLISTER" for HD were considered by the
manufacturer to be an accurate response to the CW agents. As noted in Section 4.1, in 18 of the
120 challenges, no S-CAD response occurred; these 18 cases are, by definition, inaccurate
responses.
Unit A for AC—For AC, Unit A displayed 100% accuracy for low and medium humidity at
room temperature as well as medium and high humidity at high temperature. For the low
temperature/medium humidity testing, 80% accuracy was observed; and for the room
temperature/high humidity testing, 20% accuracy was observed. The accuracy results for high
humidity at room temperature (20%) differ sharply from those at high temperature (100%), but
do not show a consistent humidity dependence.
Unit B for AC—For AC, Unit B achieved 100% accuracy for the all six temperature and
humidity conditions tested. Hence, there was no observed effect of either temperature or
humidity on accuracy of the unit.
Unit B for GB—For GB, the evaluated unit displayed 80% accuracy for the low
temperature/medium humidity condition, 60% accuracy at the room temperature/high humidity
condition, and 100% accuracy for the room temperature and high temperature conditions at
medium humidity and the low humidity condition at room temperature. At high temperature,
11
-------
high humidity, 20% accuracy was observed. A reduction in accuracy at higher RH is indicated
by these data.
Although the low temperature/medium humidity condition exhibited 80% accuracy for GB
according to the definition, it did exhibit unusual behavior in that it alarmed as "BLISTER"
during the clean air challenge for all five trials at these conditions. In three cases, this alarm
carried over past the end of the clean air challenge and into the beginning of the next GB
challenge before the "BLISTER" alarm cleared, and the unit then responded correctly as
"NERVE" to the challenge.
Unit B for HD—For HD, the unit displayed 80% accuracy for the high temperature/medium
humidity condition and 100% accuracy for low and room temperature at medium humidity as
well as low and high humidity at room temperature. At high temperature, high humidity, 0%
accuracy was observed. No definitive conclusion can be made from these data about temperature
or RH effects on accuracy of identifying HD.
High/Low—For the high/low test, the S-CAD was challenged with either a high concentration of
chemical followed by a low concentration, or a low concentration of chemical followed by a high
concentration. In all cases, both units responded accurately. The order of the challenge did not
affect the response of the S-CAD. If the concentration went from high to low, the S-CAD units
responded by producing a higher level alarm at the high concentration and then a lower level
alarm at the low concentration. If the concentration went from low to high, the S-CAD units
responded by producing a lower level alarm at the low concentration and then a higher level
alarm at the high concentration. For AC for Unit A, the difference was a medium "BLOOD"
alarm at high concentration and a low "BLOOD" alarm at low concentration. For AC for Unit B,
the difference was a high "BLOOD" alarm at high concentration and a low "BLOOD" alarm at
low concentration. For GB, Unit B responded with a medium "NERVE" alarm at high
concentration and a low "NERVE" alarm at low concentration. For HD, Unit B responded with a
medium "BLISTER" alarm at high concentration and a low "BLISTER" alarm at low
concentration.
4.4 Repeatability
Results of the repeatability analysis are summarized below. As with response time and recovery
time (Sections 4.1 and 4.2, respectively), the evaluation of repeatability includes only those cases
in which the S-CAD responded to a TIC or CW agent challenge. Repeatability addressed the
consistency of the Low, Medium, and High readings of the S-CAD. For each trial that had a
response, the maximum observed response level from the ordered progression (Low, Medium,
High) was identified.
Unit A for AC—At medium humidity, the four low temperature trials and the five High
temperature trials all had High responses, while the five room temperature trials had Medium
responses. This difference was not large enough to constitute a statistically significant effect for
temperature. Similarly, there was no statistically significant effect of temperature on the
12
-------
repeatability of the maximum response level. At room temperature, the low humidity trials
showed a mixture of Low (2) and Medium (3) alarms. At medium humidity, the alarms were all
Medium. At high humidity, the only trial with an alarm registered at the High level. This
progression resulted in a statistically significant effect of humidity at room temperature. There
was not enough evidence to conclude that there was a humidity effect on repeatability of
maximum response level. Overall, this unit showed a statistically significant effect (with
repeatability) of higher humidity levels leading to higher maximum responses. However, no
significant effect was found for temperature.
Unit B for AC—There was no observed effect of temperature or humidity on maximum level of
response (or repeatability of such) since all trials for this unit attained the High alarm level.
Unit B for GB—With four alarms at the High level for low temperature/medium humidity, five
alarms at the Medium level for room temperature/ medium humidity, and four High and one
Medium alarm for high temperature/medium humidity, the maximum level of alarm did not
differ significantly across the three temperatures (low, room, and high) evaluated at medium
humidity. Across the three humidity conditions (low, medium, and high) evaluated at room
temperature, there was a statistically significant effect of maximum alarm response with all five
responses at the Medium level for low and medium humidity, but only Low alarm levels for the
three responses at high humidity.
Unit B for HD—Across the three temperatures (low, room, and high) at medium humidity, the
data showed a statistically significant decreasing trend in response level with increasing
temperature. All five trials at low temperature showed High maximum response levels, all five
trials at room temperature showed Medium maximum response levels, and all four accurate trials
at high temperature showed Low maximum response levels. Across the three humidity levels
(Low, medium, and high) at room temperature, the data showed a statistically significant
decreasing trend in response level with increasing humidity. All five trials at low humidity
showed High maximum response levels, while all five trials at both medium and high humidity
only achieved a Medium maximum response level.
4.5 Response Threshold
Response thresholds were determined by challenging the S-CAD with successively lower
concentrations of TIC or CW agent until it no longer responded or the response was not
maintained during a challenge. Table 4-2 provides the results for the response threshold test,
showing the concentrations used for each target compound. For all three target compounds, the
concentrations used are mostly below the target concentrations used in the other tests
(Table 1-1).
For AC, the response threshold was between 6 and 12.5 ppm for Unit A and between 1.5 and
3 ppm for Unit B. For GB, the response threshold was below 0.01 mg/m3 for Unit B. The S-CAD
responded to seven out of 10 challenges with GB at that concentration, but evaluating response at
lower concentrations was not possible due to reference method limitations. For HD, the response
13
-------
Table 4-2. Response Threshold Data for the TIC and CW Agent Evaluation
TIC/CW Agent
(Concentration)
AC (50 ppm; 50 mg/m3)
AC (12.5 ppm;12.5 mg/m3)
AC (6 ppm; 6 mg/m3)
AC (3 ppm; 3 mg/m3)
AC (1.5 ppm; 1.5mg/m3)
GB (0.002 ppm; 0.01 mg/m3)
HD (0.03 ppm; 0.2 mg/m3)
A
M BLOOD (5)
L BLOOD (5)
No Response (5)
No Response (5)
No Response (5)
NA
NA
S-CAD Unit
B
H BLOOD (5)
L BLOOD (2) / M BLOOD (3)
L BLOOD (4) / No Response (1)
L BLOOD (4) / No Response (1)
No Response (5)
L NERVE (7) / No Response (3)
L BLISTER (5) / M BLISTER (5)
threshold was below 0.2 mg/m3 for Unit B because it responded to 10 out of 10 challenges with
HD at that concentration. Once again, evaluating response at lower concentrations was not
possible due to reference method limitations.
4.6 Temperature and Humidity Effects
The effects of temperature and humidity on the S-CAD are summarized in Sections 4.1 through
4.4.
4.7 Interference Effects
Five interferents (latex paint fumes, ammonia floor cleaner vapors, air freshener vapors, gasoline
engine exhaust hydrocarbons, and DEAE) were used in the evaluation. The effect of these
interferences on the S-CAD response to AC, GB, and HD is summarized below and in Table 4-3.
False Positive—A false positive response (not shown in Table 4-3) was noted if the S-CAD
responded and provided an alarm in the presence of an interferent alone (i.e., in the absence of
AC or a CW agent). A false positive was defined as any alarm under those conditions.
Erroneous positive response of a different kind (i.e., alarms while the S-CAD sampled clean air
during testing) is noted in Sections 4.3 and 4.8. Similar responses to clean air challenges were
also noted during tests with GB and interferent together, and are noted later in this section.
Unit A (false positive)—None of the five interferents produced a false positive with this unit in
the five separate trials conducted for each interferent.
Unit B (false positive)—One of the five trials with paint fumes produced a response of
"BLISTER." This resulted in a false positive rate for paint of 20%. None of the other four
interferents produced a false positive in the five separate trials conducted for each interferent.
14
-------
Table 4-3. Interference Effects
TICorCW ¥ , ,
Inter terent
Agent
AC Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
GB Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
HD Control
Paint Fumes
Floor Cleaner
Air Freshener
Gasoline Engine Exhaust
DEAE
S-CAD
Response
M(5)/H(5)
L(2)/M(3)
H
M (4) / H (6)
H
M(5)/H(5)
M
-
-
M
-
L(1)/M(4)
M
M
L(1)/H(1)
H
H
M(4)/H(1)
Alarms
(Indicated Chemical)
10/10 (BLOOD)
5/10 (BLOOD)
5/10 (NR)(a)
10/10 (BLOOD)
10/10 (BLOOD)
8/10 (BLOOD)
2/10 (NR)
10/10 (BLOOD)
5/5 (NERVE)
5/5 (NR)
5/5 (NR)
1/5 (NERVE)
4/5 (NR)
5/5 (NR)
5/5 (NERVE)
5/5 (BLISTER)
3/5 (BLISTER)(C)
2/5 (NR)
2/5 (BLISTER)(d)
3/5 (NR)
5/5 (BLISTER)
5/5 (BLISTER)
5/5 (BLISTER)
Response Time
Range (Seconds)
9-35
16-33
8-23
10-17
9-38
10-21
11-25
-
-
17
-
17-55
31-54
43-46
22-64
27-43
28-46
32-42
Recovery Time
Range (Seconds)
17-98
20-41
55-82
34-162
12-600°°
36-600
18-20
-
-
52
-
14-18
47-68
-
39
26-53
50-66
53-64
(a) NR = No response.
°° 600 seconds = Maximum time monitored for detector recovery time.
^ During all three challenges, unit cleared while still being challenged with HD.
*• During one of these challenges, unit cleared while still being challenged with ITD.
False Negative-A false negative response was noted if the presence of an interferent masked the
presence of AC or a CW agent and the S-CAD provided a lower response or did not respond to
the AC or CW agent. Changes in response, response time, and recovery time due to interferences
are discussed in the following paragraphs.
Other erroneous negative responses occurred in the absence of interferences (i.e., failure to
respond to a TIC or CW agent challenge in clean air) and are discussed under accuracy in
Section 4.3, cold-/hot-start behavior (Section 4.8), and battery life (Section 4.9).
Unit A (false negative)—For this unit, the accuracy in detecting AC in the presence of the
interferents was very similar to the accuracy without the interferent. The accuracy of the
non-interferent trials and each of the interferent trials was 100% with the exception of the trials
with engine exhaust as an interferent. The engine exhaust interferent accuracy was only 60%.
While this was not different enough from the 100% accuracy of the non-interferent trials to be
statistically significant, these results suggest that the engine exhaust hydrocarbons may have
suppressed Unit A response to AC.
15
-------
With S-CAD Unit A, all five responses to AC for the non-interferent test reached a Medium
alarm level, as did all five of the responses for DEAE. The paint trials had two Low responses
and three Medium responses. The air freshener trials had four Medium responses and one High
response. All trials with responses for either ammonia cleaner or engine exhaust reached a High
response level. The overall result of this variability was that the presence of interferent was
found to have a statistically significant effect on the maximum level of response. However,
compared to the non-interferent trials, none of the interferents individually showed a statistically
significant effect.
The geometric mean time to first response for the non-interferent test for AC was 22.3 seconds.
At 20.5, 13.9, 14.7, 15.6, and 17.3 seconds, the response times for paint, ammonia cleaner, air
freshener, engine exhaust, and DEAE, respectively, were comparable to no interferent.
The geometric mean recovery time for the non-interferent test for AC was 20.5 seconds. The
interferents paint, ammonia cleaner, air freshener, engine exhaust, and DEAE all produced
longer mean recovery times with estimates of 28.4 seconds, 70.9 seconds, 41.1 seconds,
39.0 seconds, and 37.5 seconds, respectively, than that of the non-interferent test.
Unit B (false negative)—For this unit for AC, the accuracy of the non-interferent trials and each
of the interferent trials was 100% with the exception of the trials with paint as an interferent. The
paint interferent accuracy was 0%, and this was a statistically significant difference from the
non-interferent, indicating suppression of response by the paint vapors.
With Unit B for AC, every trial that recorded a response across all the interferents achieved a
maximum response level of High. Hence, there is no evidence that interferent affected the
maximum response level for this unit.
The geometric mean time to first response for the non-interferent test for AC was 13.5 seconds.
At 11.8, 13.2, 15.3, and 12.6 seconds, the response times for ammonia cleaner, air freshener,
engine exhaust, and DEAE, respectively, were comparable to no interferent.
The geometric mean recovery time for the non-interferent test for AC was 88.3 seconds. The
ammonia cleaner (62.4 seconds) and engine exhaust (45.2 seconds) produced shorter average
recovery times. The engine exhaust estimate was statistically significant. Both air freshener
(136 seconds) and DEAE (137 seconds) produced longer recovery times, but the differences
were not statistically significant compared to the non-interferent tests.
The accuracy of Unit B in detecting GB was not the same for all interferents tested. The detector
exhibited 100% accuracy at room temperature and medium humidity without an interferent and
showed no difference in accuracy with DEAE as an interferent. All other interferents appeared to
inhibit accurate response with the unit showing no response (i.e., 0% accuracy) in the presence of
paint, ammonia cleaner, and engine exhaust and only 20% accuracy in the presence of air
freshener. These accuracy observations were all statistically significant when compared to the
non-interferent condition. The response failures for ammonia cleaner, engine exhaust, and air
freshener were complete failures of the unit to alarm. However, for the five paint interferent
16
-------
trials, the unit failed to respond during the interferent/agent challenge, but did alarm during the
clean air portion of the evaluation after each interferent/agent challenge. These responses are
essentially a different type of erroneous positive response, as they occurred while only clean air
was being sampled by the S-CAD.
The interferents did not show a statistically significant effect on the maximum level of response
observed for GB with S-CAD Unit B. All five of the responses for the non-interferent evaluation
as well as four of the five for the DEAE interferent evaluation and the one air freshener test trial
reached a Medium alarm level. The other trial of the DEAE interferent evaluation achieved a
maximum alarm level of Low.
The geometric mean time to first response for the non-interferent evaluation for GB was
14.7 seconds. The mean time to first response for air freshener was 17 seconds (based on only
one observation) and for DEAE was 26.0 seconds. These response times were not statistically
significantly different from that in the non-interferent evaluation.
The geometric mean recovery time for the non-interferent evaluation for GB was 19.4 seconds.
Air freshener displayed a statistically significant longer average recovery time at 52.0 seconds,
but this is based on only a single observation for air freshener and should be viewed with
caution. At 16.1 seconds, the geometric mean recovery time for DEAE was statistically
significantly lower than that of the non-interferent evaluation.
The accuracy in detecting HD was not the same for all interferents tested. The detector exhibited
a range of behaviors for the interferents tested:
• Paint—Three of the trials had a response, but cleared while the agent challenge was ongoing
and were therefore considered inaccurate. The other two trials showed no response. The net
result was an estimate of 0% accuracy with paint as an interferent. This was a statistically
significant effect compared to the non-interferent test.
• Ammonia Cleaner—In one trial, the SCAD Unit B responded accurately. One trial had a
response, but cleared during the challenge and was therefore considered inaccurate. The other
three trials showed no response. Therefore, this interferent showed 20% accuracy. This was a
statistically significant difference from the non-interferent test.
• Air Freshener, Engine Exhaust, and DEAE—In each case, the S-CAD Unit B responded
accurately to HD in the presence of the interferent for 100% accuracy, the same level as
observed for the non-interferent test.
After determining that the interferents did seem to affect the accuracy of detecting the CW
agents, further analysis was performed on the maximum response level, time to first response,
and recovery time for each interferent compared to the non-interferent test. Note that these
analyses incorporated data from trials determined to be inaccurate if such data were appropriate.
For example, the time to first response analysis used data from trials that recorded an alarm, even
if the unit subsequently cleared during the challenge and was therefore counted as inaccurate.
17
-------
The interferents exhibited a statistically significant effect on the maximum level of Unit B
response to HD. All five responses for the non-interferent test reached a Medium alarm level as
did all three of the responses for the paint test and four of the five for the DEAE test. One trial
for ammonia cleaner and one for DEAE reached a High alarm level as did all five trials for both
air freshener and engine exhaust. The only occurrence of a Low maximum response was for one
trial with ammonia cleaner as an interferent. Even though the overall differences in maximum
response level were statistically significant, there were no statistically significant differences
between any of the interferents and the non-interferent test.
The geometric mean time to first response for the non-interferent test for HD was 39.9 seconds.
At 45.0, 37.5, 36.1, 39.2, and 37.6 seconds, the response times for paint, ammonia cleaner, air
freshener, engine exhaust, and DEAE, respectively, were comparable to no interferent.
The geometric mean recovery time for the non-interferent test for HD was 55.6 seconds. At 39.0,
42.6, 60.3, and 60.3 seconds, the recovery times for ammonia cleaner, air freshener, engine
exhaust, and DEAE, respectively, were comparable to no interferent.
4.8 Cold-/Hot-Start Behavior
Analysis of the effects of insufficient warm-up time, under start-up conditions ranging from cold
(5 to 8°C) to hot (40°C), are summarized below. Table 4-4 illustrates the data obtained in testing
for cold-/hot-start effects, showing the S-CAD units used, the start condition, delay time,
sequential experiment number, response reading, response and recovery times, and indicated
chemical. Such testing was conducted only with AC at the IDLH concentration.
Unit A—Delay time is the time it took the S-CAD to achieve a ready state after powering the
unit on. For the room temperature cold start, the delay time was 209 seconds. For the cold
temperature cold start, the delay time was 1,440 seconds. For the hot temperature cold start, the
delay time was 180 seconds.
Accuracy for the cold storage/cold start was 40%. At room temperature storage/cold start,
accuracy was 100% as it also was for the standard control (not cold start) condition. For the hot
storage/cold start, the accuracy was 80% (in one trial the unit failed to alarm to the AC
challenge, but then alarmed on clean air after the challenge). While these data show variability in
response accuracy between the standard control condition and the three cold-start conditions
tested, none of the individual comparisons between cold-start condition and control condition
was statistically significant.
18
-------
Table 4-4. Start State Effects
S-CAD
Unit
A
B
Start Condition
Control
Room
Temperature
(Cold Start)
Cold
Temperature
(Cold Start)
Hot
Temperature
(Cold Start)
Control
Room
Temperature
(Cold Start)
Cold
Temperature
(Cold Start)
Hot
Temperature
(Cold Start)
Delay Time
(s)
NA
209
1440
180
NA
160
1218
240
Experiment
Number
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
S-CAD
Response
M
M
M
M
M
M
M
M
M
M
H
NR(b)
NR
NR
H
M
NR(C)
M
H
M
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
NR(C)
H
H
Response
Time
(Seconds)
35
22
22
18
18
20
20
21
10
18
17
-
-
-
15
19
-
23
18
16
14
14
17
15
9
10
12
10
18
17
13
46
29
15
17
17
10
-
7
13
Recovery
Time
(Seconds)
17
22
23
21
20
20
22
78
39
37
600(a)
-
-
-
68
38
-
44
50
35
77
87
90
98
91
82
84
147
118
111
82
117
139
117
89
77
105
-
600
62
Alarm
(Indicated
Chemical)
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
-
-
-
BLOOD
BLOOD
-
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
BLOOD
-
BLOOD
BLOOD
•' 600 seconds = Maximum time monitored for detector recovery time.
^ NR = No response.
^ Unit alarmed while sampling clean air after the end of the AC challenge.
The room temperature/cold start had exactly the same maximum response level profile (five of
five trials at Medium alarm level) as the control condition. The cold storage/cold start resulted in
High alarm level responses on the two trials where a response was recorded. One of the four
trials with responses for the hot storage/cold start achieved a High alarm level, while the other
19
-------
three trials showed Medium. Overall, this variability is large enough to reject the hypothesis that
maximum response level is dependent on the type of start conditions.
There was no statistically significant impact of cold-start condition on response time. With a
geometric mean of 22.3 seconds, the time to first response for the control condition was similar
to the mean time for a cold storage/cold start (16.0 seconds), a room temperature storage/cold
start (17.2 seconds), and a hot storage/cold start (18.8 seconds).
The geometric mean recovery time for the cold start from room temperature (34.6 seconds) was
not significantly different from that for the control condition (20.5 seconds). However, the
recovery time for the cold start from hot storage, averaging 41.4 seconds was statistically
significantly longer. The estimated recovery time for the cold storage/cold start, at 68.0 seconds,
was even longer, but should be viewed with caution since it is based on only one measured
recovery time. In the only other trial at that start condition in which a response to AC was
observed, Unit A failed to clear after 600 seconds of sampling clean air.
Unit B—For the room temperature cold start, the delay time was 160 seconds. For the cold
temperature cold start, the delay time was 1,218 seconds. For the hot temperature cold start, the
delay time was 240 seconds.
All five trials with the S-CAD in the fully warmed up control condition produced a response.
The same was true for the room-temperature and cold-storage cold-start tests. The hot-
storage/cold-start test exhibited 80% accuracy. (In one trial the unit failed to alarm on challenge,
but then alarmed on clean air after the challenge. This was not the trial noted above in which
Unit A exhibited similar behavior.) None of the cold-start conditions had a statistically
significant difference in accuracy relative to the control condition.
All the trials for all the standard and cold start conditions that had a response reached a
maximum response level of High. Hence, there is no statistically significant effect of cold start
on level of maximum response.
There was no statistically significant impact of cold-start condition on response time. With a
geometric mean of 13.5 seconds, the time to first response for the control condition was similar
to the mean time for a cold storage/cold start (21.3 seconds), a room temperature storage/cold
start (13.0 seconds), and a hot storage/cold start (11.2 seconds).
The geometric mean recovery times for the cold storage/cold start (107 seconds), the cold start
from room temperature (106 seconds), and the hot storage/cold start (79.4 seconds) were not
significantly different from that for the control condition (88.3 seconds).
4.9 Battery Life
The S-CAD can be powered by rechargeable batteries. The battery life evaluation was conducted
by placing fully charged batteries provided by the vendor in the S-CAD. The S-CAD was then
20
-------
powered on and allowed to warm up fully according to the manufacturer's directions. The
battery life test was conducted by successive challenges with AC at the IDLH concentration
delivered for 5 minutes every half hour, and the results are shown in Table 4-5. During the
battery life evaluation, both erroneous positive and negative responses were observed. Erroneous
negatives are noted as "No Response" in Table 4-5. Unit A had to be rebooted once due to the
unit alarming as "BLOOD" on clean air and not clearing. Unit B was rebooted at 4 hours and
45 minutes (i.e., at 11:30) after the unit failed to alarm with three consecutive challenges. After
6 hours and 30 minutes (i.e., at 13:15), the unit was rebooted again because it was alarming on
clean air as "BLOOD" and would not clear. At this point, the unit would not power on. Therefore
the battery life for Unit B was judged to be 6 hours and 30 minutes. For Unit A, the low battery
indicator came on after 9 hours and 15 minutes (i.e., at 16:00). The unit shut down after 9 hours
and 18 minutes.
Table 4-5. Responses Recorded from the S-CAD in Battery Life Evaluation(a)
Test
Start-up
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Time
0645
0700
0730
0800
0830
0900
0930
1000
1030
1100
1130
1200
1230
1300
1315
1330
1400
1430
1500
1530
1600
1603
S-CAD Identification Number
A
Response (Response
Time in Seconds)
No Response
H BLOOD (20)
H BLOOD (17)
No Response
H BLOOD (14)
H BLOOD (14)
H BLOOD (12)
H BLOOD (16)
No Response
H BLOOD (42)
H BLOOD (16)
No Response
H BLOOD (14)
H BLOOD (16)
H BLOOD (8)
H BLOOD (14)
H BLOOD (18)
H BLOOD (11)
H BLOOD (18)
Battery Indicator
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Low (1/4) indicator
Power Off
(9 hours, 18 minutes)
B
Response (Response
Time in Seconds)
M BLOOD (9)
H BLOOD (13)
H BLOOD (6)
H BLOOD (15)
H BLOOD (14)
H BLOOD (9)
H BLOOD (16)
No Response
No Response
No Response
H BLOOD (16)
H BLOOD (9)
No Response
Battery Indicator
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Full
Power Off
(6 hours, 30 minutes)
All battery life tests were conducted with AC as the challenge TIC at the IDLH concentration of 50 ppm (50 mg/m ).
4.10 Operational Characteristics
General performance observations noted during evaluation testing:
• Instrument Operation—The S-CAD has a large display that was easy to read as long as the
contrast mechanism was working properly. During some tests, the contrast mechanism
stopped functioning, and the S-CAD display faded or went black. This effect was related to
21
-------
temperature, because the display turned entirely black at elevated test temperatures and was
dim at cold temperatures. To continue with testing, data were collected from a laptop
computer that was connected to the S-CAD. It should be noted that first responders will
likely not have this option in the field. The S-CAD also had a background light that could be
controlled from the control panel.
Instrument Indicators—The S-CAD had a flashing red light on the top of the instrument that
indicated an alarm. However, this light could not easily be seen when looking at the face of
the S-CAD. Visual alarms on the display were small, compared to the size of the display face
itself, but were easy to read. Audio alarms were hard to hear because the speaker was located
behind the outer shell. The audio alarm volume could be controlled from a menu, but even at
the highest level the alarms could barely be heard. Use of personal protective equipment by
an operator of the S-CAD could exacerbate these limitations of the alarms and indicators.
Warm-Up—The S-CAD took up to 1,440 seconds (24 minutes) to reach a ready state after
being turned on from cold (5 to 8°C) storage. Start-up from room temperature or hot (40°C)
storage conditions required 160 to 240 seconds to reach a ready state.
Batteries—The S-CAD can operate on a rechargeable battery pack.
Errors—During testing, the S-CAD occasionally remained in an alarm state and did not clear
for long periods of time, even after extensive periods of sampling clean air. The S-CAD had
to be rebooted to clear the alarm in these cases.
Vendor Support—Before the evaluation, a vendor representative trained Battelle employees
to operate the S-CAD. Testing proceeded according to the vendor's recommendations. The
vendor responded promptly when information was needed during the evaluation.
Cost—The list price of the S-CAD, as used in this evaluation, is approximately $25,000 to
$35,000, depending on the instrument configuration and the number of units ordered.
22
-------
5.0 Performance Summary
Summary results from testing the S-CAD are presented below for each performance parameter
evaluated. Discussion of the observed performance can be found in Chapter 4 of this report.
Results with AC are from testing two units of the S-CAD; results for GB and HD are from
testing one unit.
Response Time: When the S-CAD responded to challenges with AC, the time required was
35 seconds or less, with no consistent effect of temperature or RH. Similarly, most response
times for GB were 43 seconds or less, but response time increased at the highest humidity
conditions to about 60 to 260 seconds. Response times for HD ranged from about 30 to
60 seconds, and were not affected by the temperature and RH. These results do not include
instances in which the S-CAD failed to respond to TIC or CW agent challenges; those instances
are noted below under Accuracy.
Recovery Time: Recovery times for AC ranged widely, from 13 seconds to over 600 seconds,
with no consistent temperature or RH effects for the two units. For the most part, recovery times
for GB were less than 30 seconds, regardless of temperature or RH conditions. Recovery times
for HD ranged from 35 to 146 seconds, with faster recovery at higher temperatures and higher
RH. These results exclude those instances in which the S-CAD did not respond to a TIC or agent
challenge.
Accuracy: Of the 120 challenges with AC, GB, and HD used to assess accuracy, the S-CAD
responded accurately to 102 and did not respond to the other 18. Both S-CAD units identified
AC with 100% accuracy under almost all temperature and RH conditions. The primary exception
was that one unit correctly identified AC in only one of five challenges at room temperature and
high (80%) RH (i.e., 20% accuracy). Accuracy of identifying GB was 80 to 100% in most tests,
but was 20 to 60% in tests at high RH, indicating a dependence on RH. Accuracy of identifying
HD was 100% in most conditions, except for values of 80% at the high temperature (35 °C) and
50% RH condition, and 0% at high temperature and high RH.
[Failure to respond to AC challenges was also observed occasionally during cold-/hot-start and
battery life tests, but those observations were not used in the calculation of the accuracy results
noted above.]
Repeatability: When responding to an AC challenge, the repeatability of the S-CAD's
Low/Medium/High readings for AC was not affected by temperature, for either unit tested. One
unit tended to show higher readings at higher RH, but the other unit did not. Repeatability of
responses for GB was unaffected by temperature, but RH had an effect, with Low readings at the
highest (80%) RH and Medium readings at other conditions. Repeatability for HD was affected
23
-------
by temperature, with readings dropping from High at 5 °C to Low at 35 °C, and by RH, with
readings changing from High at 20% RH to Medium at other conditions.
Response Threshold: For AC, the response threshold was between 1.5 and 3 ppm (1.5 and
3 mg/m3) on one unit, and between 6 and 12.5 ppm (6 and 12.5 mg/m3) on the other unit. For
GB, the response threshold was less than 0.002 ppm (0.01 mg/m3), and for HD it was less than
0.03 ppm (0.2 mg/m3).
Temperature and Humidity Effects: These effects are described in the preceding summaries
of other performance parameters.
Interference Effects: There were almost no false positive readings from the two S-CAD units
when tested with each of the five interferences one at a time in clean air. The only such false
positive response was a single reading (out of five separate challenges) from Unit B with latex
paint fumes.
[Erroneous positive responses of a different kind (i.e., alarms while the S-CAD sampled clean
air) were observed in several cases during tests addressing accuracy, interference effects, and
cold-/hot-start behavior.]
When added to challenge mixtures of AC, the interferences had relatively small effects on the
performance of the S-CAD. For Unit A, most interferences did not affect the accuracy of
identification, response time, or response level (Low/Medium/High), but all five interferences
did lengthen the recovery time after detection of AC. Engine exhaust reduced Unit A accuracy
for AC to 60%, although this result was not statistically significant. For Unit B, the interferences
had no effect on response time or response level, and most interferences had no effect on
accuracy. However, accuracy of AC identification by Unit B was reduced to 0% by latex paint
fumes. Recovery time was lengthened for Unit B by air freshener vapors and DEAE, but was
shortened by ammonia cleaner and engine exhaust hydrocarbons.
False negative responses with GB and HD reduced the accuracy of identification, but response
time, recovery time, and response levels were unaffected by the interferences when the unit did
alarm. Accuracy for GB was reduced to 0% by latex paint fumes, ammonia cleaner, and engine
exhaust hydrocarbons, and to 20% by air freshener vapors. With paint fumes, the S-CAD failed
to respond during the GB challenges, but alarmed upon sampling clean air. Accuracy for HD was
reduced to 0% by the paint fumes and to 20% by ammonia cleaner vapors. In at least one HD
challenge with each of these two interferences, the S-CAD exhibited a different type of
erroneous negative response, in that it stopped alarming while the agent challenge was in
progress.
Cold-/Hot-Start Behavior: The response time, recovery time, response level, and identification
accuracy of the two S-CAD units were essentially the same in operation after a cold start as in
fully warmed up operation. The delay times (time before readiness to take a reading) were 160 to
240 seconds with the two S-CAD units after start-up from room temperature or hot (40 °C)
24
-------
storage, but the delay times after cold (5 °C) storage were 1,218 seconds and 1,440 seconds,
respectively.
Battery Life: One unit of the S-CAD shut down after 6 hours and 30 minutes of continuous
operation on battery power. The other unit shut down after 9 hours and 18 minutes.
Operational Characteristics: The display of the S-CAD was easy to read as long as the
contrast mechanism worked properly, but when this failed the display was unreadable. This
occurred at both hot and cold temperatures during testing. When that happened, test procedures
were continued by observing S-CAD responses on a laptop computer, but that option will not
typically be available to a user in the field. The red alarm light on top of the S-CAD was not
easily visible when looking directly at the face of the instrument, and the volume of the audible
alarm was weak.
Before this evaluation began, an SAIC representative trained Battelle testing personnel to operate
the S-CAD. Testing proceeded according to the vendor's recommendations, and the vendor
responded promptly when information was needed during the evaluation. The list price of the
S-CAD is approximately $25,000 to $35,000, depending on the instrument configuration and the
number of units ordered.
Conclusion: The S-CAD provided accurate detection and identification of AC, GB, and HD
under most temperature and RH conditions in air. There was little effect from temperature, but
failures to respond to challenges were seen in some tests, especially at high RH, and there was
some variation in response and recovery times. Start-up conditions had no effect other than a
lengthy delay time before readings could be obtained after storage under cold conditions.
Erroneous positive readings were seen in a few tests, when the S-CAD alarmed while sampling
clean air. A key performance issue disclosed by this test is the suppression of response to GB and
HD caused by some interferences. This behavior is unexpected in that the S-CAD uses IMS and
SAW principles simultaneously for CW agent detection, specifically to minimize interferences.
Other areas for improvement include the visibility and audibility of the alarm indicators, and the
reliability of the visual display.
25
-------
6.0 References
1. Technology Testing and Evaluation Program Test/QA Plan for Evaluation of Portable Ion
Mobility Spectrometers for Detection of Chemicals and Chemical Agents, Version 1, Battelle,
Columbus, Ohio, February 2005.
2. Quality Management Plan (QMP) for the Technology Testing and Evaluation Program
(TTEP), Version 1, Battelle, Columbus, Ohio, January 2005.
26
------- |