April 2011
Environmental Technology
Verification Report
EnviroScan, Inc. Ozone Detector Card
Prepared by
Battelle
The Business of Innovation
Under a cooperative agreement with
U.S. Environmental Protection Agency
-------
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
EiwiRoScAN, INC. OZONE DETECTOR CARD
by
Thomas Kelly, Brad Goodwin, and Amy Dindal, Battelle
John McKernan, U.S. EPA
-------
Notice
The U.S. Environmental Protection Agency, through its Office of Research and Development,
funded and managed, or partially funded and collaborated in, the research described herein. It
has been subjected to the Agency's peer and administrative review. Any opinions expressed in
this report are those of the author(s) and do not necessarily reflect the views of the Agency,
therefore, no official endorsement should be inferred. Any mention of trade names or
commercial products does not constitute endorsement or recommendation for use.
-------
Foreword
The EPA is charged by Congress with protecting the nation's air, water, and land resources.
Under a mandate of national environmental laws, the Agency strives to formulate and implement
actions leading to a compatible balance between human activities and the ability of natural
systems to support and nurture life. To meet this mandate, the EPA's Office of Research and
Development provides data and science support that can be used to solve environmental
problems and to build the scientific knowledge base needed to manage our ecological resources
wisely, to understand how pollutants affect our health, and to prevent or reduce environmental
risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa. gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http://www.epa.gov/nrmrl/std/etv/verifiedtechnologies.htmltfair.
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. Quality assurance (QA) oversight was
provided by Michelle Henderson, U.S. EPA and Rosanna Buhl and Zachary Willenberg,
Battelle. We thank Rudy Eden, Philip Fine, Jason Low, and their staff of the South Coast Air
Quality Management District for their contributions in field testing of the Ozone Detector Cards.
We also thank Daniel Witzling and Neal Richman of Breathe California of Los Angeles for their
financial support of this test, and their volunteers for using the Ozone Detector Cards in the field.
We also thank Mr. Eden, Mr. Richman, Mr. David Shelow (U.S. EPA), and Dr. Will Ollison
(American Petroleum Institute) for their reviews of this verification report.
in
-------
Contents
Notice i
Foreword ii
Acknowledgments iii
List of Abbreviations vi
Chapter 1 Background 1
Chapter 2 Technology Description 2
Chapters Test Design and Procedures 4
Chapter 4 Quality Assurance/Quality Control 11
Chapters Statistical Methods 15
Chapter 6 Test Results 18
Chapter 7 Performance Summary 45
Chapters References 47
Appendix A Initial SCAQMD Field Tests 48
IV
-------
LIST OF FIGURES
Figure Page
2-1 EnviroScan Ozone Detector Card folded (top) and open (bottom) 3
3-1 Laboratory Test Setup 6
LIST OF TABLES
Table Page
4-1 Results of Pre-Test Ozone FEM Calibration 12
6-1 Laboratory Ozone Detector Card Data - 0 ppbv 19
6-2 Laboratory Ozone Detector Card Data - 30 ppbv 19
6-3 Laboratory Ozone Detector Card Data - 50 ppbv 20
6-4 Laboratory Ozone Detector Card Data - 70 ppbv 20
6-5 Laboratory Ozone Detector Card Data - 90 ppbv 21
6-6 Laboratory Ozone Detector Card Data - 130 ppbv 22
6-7 Laboratory Ozone Detector Card Data - 150 to > 200 ppbv 23
6-8 Laboratory Ozone Detector Card Data -Blank Spots 24
6-9 Accuracy Results at Laboratory Ozone Concentrations 25
6-10 Summary of Variability of Ozone Detector Card Readings for Individual Ozone
Concentrations 26
6-11 Summary of Intra-Card Duplication for Laboratory Ozone Concentrations 27
6-12 Summary of Inter-Card Duplication for Laboratory Ozone Concentrations 27
6-13 Summary of User Agreement for Laboratory Ozone Concentrations 28
6-14 Data from Laboratory Testing of Effect of Light Intensity on Ozone Indicator Card
Readings 30
6-15 Laboratory Results for Light Intensity Testing on Ozone Detector Cards 30
6-16 Data from 2010 Field Testing by SCAQMD 32
6-17 Ozone Detector Card Accuracy in Fall Season 2010 SCAQMD Testing by Expected Card
Reading Category 37
6-18 Comparison of Meteorological Conditions with Accurate and Inaccurate Ozone Detector
Card Readings in Fall Season 2010 SCAQMD Field Testing 38
6-19 Comparison of Meteorological Conditions with Accurate and Inaccurate Ozone Detector
Card Readings by Expected Ozone Card Reading 39
6-20 BCLA Field Results, 12/18/09 to 5/13/10 40
7-1 Performance Summary for Ozone Indicator Cards 46
-------
List of Abbreviations
AMS Advanced Monitoring Systems
ADQ audit of data quality
BCLA Breathe California of Los Angeles
cm centimeter(s)
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
FEM Federal Equivalent Method
FRM Federal Reference Method
ft foot (feet)
in inch(es)
L Liter(s)
m meter(s)
min minute(s)
mph miles per hour
nm nanometer
OEPA Ohio Environmental Protection Agency
PE performance evaluation
ppbv parts per billion by volume
QA quality assurance
QC quality control
QMP quality management plan
RH relative humidity
sec second(s)
SCAQMD South Coast Air Quality Management District
ISA technical systems audit
UV ultraviolet
r\
|iW/cm microwatts per square centimeter
VI
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of
the ETV Program is to further environmental protection by accelerating the acceptance and
use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing
high-quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and
preparing peer-reviewed reports. All evaluations are conducted in accordance with rigorous
quality assurance and quality control (QA/QC) protocols to ensure that data of known and
adequate quality are generated and that the results are defensible.
The EPA's National Risk Management Research Laboratory and its verification organization
partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The
AMS Center recently evaluated the performance of EnviroScan, Inc.'s Ozone Detector Cards
in laboratory tests conducted at Battelle's laboratories in Columbus, OH and field tests
conducted in southern California. Ozone indicator cards were identified as a priority
technology category for verification through the AMS Center stakeholder process.
-------
Chapter 2
Technology Description
This report provides results for the verification testing of EnviroScan, Inc.'s Ozone Detector
Card. The following is a description of the Ozone Detector Card, based on information
provided by the vendor. The information provided below was not verified in this test.
The Ozone Detector Card (Figure 2-1) is approximately 8 centimeters (cm) x 11 cm (3.25
inches (in) x 4.25 in) in size. Each card has a row of five spots of solid reagent, with each
spot covered by a protective strip of foil. When a foil strip is removed and the reagent spot is
exposed to air, the reagent reacts with ozone in the air to produce a color change in the spot
proportional to the ozone concentration. An ozone measurement is conducted by removing
the foil from a spot and placing the card in the atmosphere to be tested for 10 minutes (min).
At the end of the 10 min measurement, the card is folded onto itself and the color of the
reacted spot is compared visually through a hole in the card to a reference color wheel
printed on the front of the Ozone Detector Card. The color wheel has four gradations
corresponding to four different ranges of ozone concentrations. The vendor describes those
four ranges as follows:
• Range 1 (10 to 45 parts per billion by volume (ppbv) ozone) corresponds to
conditions with no or very little ozone pollution.
• Range 2 (45 to 75 ppbv ozone) corresponds to normal ozone pollution on sunny days.
• Range 3 (75 to 105 ppbv ozone) corresponds to potentially unhealthy conditions in
which children, asthmatics, and people with other respiratory diseases should limit
prolonged outdoor exertion.
• Range 4 (>105 ppbv ozone) corresponds to unhealthy conditions in which children,
asthmatics, and people with other respiratory diseases should avoid outdoor exertion
and everyone else should limit outdoor exertion.
With a 10-minute exposure period, the degree of reagent color change increases
progressively from little to no change for Range 1 to significant darkening of the reagent spot
for Range 4.
For ambient ozone measurements, the Ozone Detector Card should be placed outdoors in a
location where it is protected from the wind. Measurements conducted in windy conditions
-------
may produce incorrect results. Additionally, the Ozone Detector Card can be used to
measure indoor ozone concentrations with a recommended exposure time of 20 min (as
opposed to 10 min for outdoor measurements). The longer indoor exposure time is intended
to produce a measurable reagent color change with the relatively lower ozone concentrations
expected indoors. (Note added by Battelle: the instructions for use of the Ozone Detector
Card do not include any adjustment to the visual readings to account for the longer exposure
time when used indoors.)
Ozone ltat«etor Card
en* ft B* itMTttno* off » MM
» n*Ce QiCn* OMceior Cart M Urn open ML for outdoor nrnaamtmrtt^,
(tee IN; c»W •rtwra R ii pratednt torn the ml
• Measurement (ME. Iw outdoor
10 mnu8«. IMoor n*lii»
Figure 2-1. EnviroScan Ozone Detector Card folded (top) and open (bottom).
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
Ozone is a widespread pollutant that is formed by photochemical reactions involving
sunlight, nitrogen oxides, and volatile organic compounds in air. The U.S. Clean Air Act and
its Amendments led to the establishment of air quality standards for ozone, and pollution
control strategies that require state and local authorities to regulate for compliance with the
standards. Ozone is regulated because of its effect on human health when air containing
elevated concentrations of ozone is inhaled.
Because of the costs associated with emission control programs and penalties for those
regions that are not in compliance, it is essential that ozone measurements that determine
compliance with standards be accurate. For that purpose EPA has established Federal
Reference and Equivalent Methods (FRM and FEM) for monitoring ozone.1 The method
currently widely used is the FEM, which makes use of the ozone molecule's strong
absorption band in the ultraviolet (UV) region of the spectrum, with a maximum coinciding
with the strong mercury vapor emission line at 254 nanometers (nm). The FEM has
completely supplanted the FRM for all compliance monitoring in the U.S., because of the
greater complexity of the FRM and its requirement for flammable ethylene gas. Commercial
FEM instruments measure the transmission of UV light through an air sample and compare
the intensity with that obtained along the same path length through air containing no ozone.
A scrubber (typically MnO2 or heated silver wool) designed to selectively remove ozone
from the air, is used to allow determination of the background absorption of UV light by non-
ozone species such as aromatic organic compounds. Potential interferences in the FEM
ozone measurement, due to removal of UV-absorbing aromatic compounds by the ozone
scrubber, have been identified and may be significant in polluted conditions that can produce
elevated ozone levels exceeding regulatory standards.
For rapid assessment of human exposure to ozone, methods less expensive and complex than
a FRM or FEM can be useful. Semi-quantitative methods that indicate a range of ozone
concentrations in air can be helpful to people with ailments that cause respiratory sensitivity
to ozone. One simple and inexpensive approach to personal ozone measurement is the use of
-------
a colorimetric indicator card which incorporates a reagent that undergoes a color change
when exposed to ozone in air. To use the card, a protective foil over a reagent spot is
removed and the spot is exposed to ambient air for a specified period of time (e.g., 10 min).
Then the intensity of the resulting color change in the reagent spot is visually compared to a
color reference printed on the card, providing an estimate of the ozone concentration range.
Such indicator cards typically indicate ozone concentrations in a few broad concentration
ranges from near zero to over 100 ppbv. The EnviroScan, Inc. Ozone Detector Card is an
example of this class of indicator cards. This verification evaluates the ease of use of the
Ozone Detector Cards and quantifies their response relative to the response of FEM
instrumentation so potential users can make informed decisions about the potential benefits
and limitations of the cards.
This verification test was conducted over a 12-month period beginning in October 2009 and
ending in October 2010, according to procedures specified in the Test/QA Plan for
Verification of Ozone Indicator Cards3 As indicated in the test/QA plan, the testing
conducted satisfied EPA QA Category III requirements. The test/QA plan and/or this
verification report were reviewed by:
• Rudy Eden, California South Coast Air Quality Management District (SCAQMD)
• Will Ollison, American Petroleum Institute
• David Shelow, U.S. EPA
• Daniel Witzling (test/QA plan only) and Neal Richman (report only), Breathe
California of Los Angeles (BCLA).
3.2 Test Procedures
The verification of the EnviroScan Ozone Detector Card included both laboratory and field
testing. Laboratory testing was conducted in Battelle's facilities in Columbus, Ohio. Field
testing was conducted at monitoring sites in the Los Angeles Basin of Southern California by
staff of the SCAQMD. Additional data were obtained in the field by volunteers coordinated
by BCLA. These testing efforts are described below.
3.2.1 Laboratory Testing
The laboratory portion of the test was conducted over a period of approximately 30 days and
involved delivering known concentrations of ozone in a continuously flowing air stream at
room temperature to an 8 L test chamber. The delivered ozone concentrations were
monitored simultaneously by both an FEM and Ozone Detector Cards. The FEM used was a
Thermo Environmental Model 49C Ozone Monitor (FEM EQOA-0880-047) with an upper
range limit of 200 ppbv. Prior to the start of testing, the Model 49C Ozone Monitor was
calibrated against a Dasibi Model 1008 UV calibration photometer which had itself been
calibrated against the Ohio Environmental Protection Agency's (OEPA) ozone calibration
standard photometer.
At the start of every day of testing, a zero and span check of the Model 49C Ozone Monitor
-------
were conducted using the Dasibi reference photometer. During testing of the Ozone Detector
Cards, the Dasibi was removed and a humidification system was connected to the test
apparatus. Humidified air from this system was mixed upstream of the test chamber with the
dry ozonated air flow from an Environics Model 6100 ozone generator to provide
approximately 20 L/min total air flow with a relative humidity (RH) of 50 (±5) % for all
tests. The resulting speed of air movement through the 8 L test chamber was approximately
1 cm/sec (0.6 meter (m)/min (2 feet (ft)/min)). Ozone Detector Cards were inserted into the
test chamber and exposed to the delivered ozone concentrations for 10-min periods. FEM
ozone data were logged continuously before, during, and after each Ozone Detector Card
exposure. The temperature and relative humidity in the chamber were also monitored and
logged continuously during each test. Figure 3-1 shows the test setup including the chamber,
Environics ozone generator, FEM, Dasibi photometer, data logger (Fluke, Hydra Series II),
humidity generator (Fuel Cell Technologies, Model LF-HBA), and temperature/humidity
probe (Onset, HOBO Model H14-002/S-THA-M002).
Figure 3-1. Laboratory Test Setup
Following each exposure to the delivered ozone concentrations, the exposed Ozone Detector
Cards were removed from the test chamber and taken within 30 sec to two separate Battelle
administrative staff members who independently recorded their visual readings of the color
of the exposed reagent spot(s) (i.e., readings from 1 to 4 based on the reference color wheel
printed on the card). Those staff members (the card readers) did not know the ozone
concentration delivered to the cards, which was known only by the Battelle employee
conducting the test. Each reader's observations were collected and entered into an electronic
spreadsheet at the end of each testing day. The two administrative staff members providing
the visual readings were non-technical personnel who were trained in use of the cards before
testing began. These staff members did not confer with each other about their observations at
any time during testing.
-------
During the laboratory testing, two reagent spots on each of four Ozone Detector Cards were
exposed simultaneously during each test (for a total of eight spots exposed simultaneously
per test). In half of the tests, the foil was also removed from one additional reagent spot on
each card after the cards were removed from the test chamber, and within 30 sec before the
cards were given to the two card readers. These additional spots were used as blanks to
determine if the readers perceived a color change on a spot that had not been exposed to
ozone. Readers were asked to record observations of the color of all reagent spots that had
been uncovered.
While conducting the initial laboratory tests, it was observed by the staff conducting the test
that insertion of the Ozone Detector Cards into the test chamber resulted in a decrease in the
ozone concentration in the chamber, as measured by the FEM ozone monitor. The
percentage decrease in ozone due to the presence of the four Ozone Detector Cards was
approximately 20%, and was the same when all reagent spots were covered with foil as when
all reagent spots were uncovered. That decrease was not transitory, but persisted as long as
the Ozone Detector Cards remained in the test chamber. To counteract this effect, the initial
ozone concentration in the chamber was set approximately 20% higher than the target test
concentration so that the target concentration was achieved once the Ozone Detector Cards
were in place.
A test matrix with six ozone concentrations (0, 30, 50, 70, 90, and 130 ppbv) was used for
this evaluation. These values were chosen to include both concentrations near the midpoint
of card ranges (i.e., 30 and 90 ppbv) and concentrations near the boundaries of a range (i.e.,
50 and 70 ppbv). Testing at each concentration was conducted twice for a total of 12
separate ozone exposure tests, with the six concentrations delivered in random order. At the
start of each test the ozone concentration in the chamber was determined by the FEM prior to
the insertion of the Ozone Detector Cards. FEM ozone monitoring then continued as the foil
covering was removed from two spots on each of four cards and the cards were placed in the
chamber for a 10 min exposure. Each Ozone Detector Card was numbered uniquely and the
spots on each card were numbered from 1 to 5. The card/spot numbers for each test were
recorded at the start of every test. The ozone concentration during the 10 min exposure was
recorded in a laboratory notebook as well as being recorded continuously by the data logging
system. At the end of the 10 min exposure, the Ozone Detector Cards were removed from
the chamber and the foil was removed from any scheduled blank spots. Within 30 sec after
the end of the ozone exposure period, the cards were then taken to the two Battelle card
readers to be read. Each card reader visually inspected each card at their desk under
overhead fluorescent lighting in a windowless office, and recorded the card and spot numbers
and their observations of the ozone range readings for each exposed spot.
During laboratory testing, the staff conducting the test observed that even with delivered
ozone concentrations exceeding 120 ppbv, neither Battelle card reader ever classified the
Ozone Detector Card color change as Range 4. Consequently, to determine the ozone
concentrations at which users would classify the color change as Range 4, additional tests
were performed at higher concentrations (up to and exceeding the 200 ppbv range limit of the
FEM).
-------
Tests were also conducted to evaluate the effect of light intensity during ozone exposure on
reagent color development in the Ozone Detector Cards. In this testing, two reagent spots on
four Ozone Detector Cards were exposed simultaneously to approximately 100 ppbv of
ozone in air at 22 °C and 50% RH, with the test conducted under normal indoor laboratory
illumination. The same test was then repeated with the Ozone Detector Cards in darkness
during Oj exposure, and was then repeated once more with the Ozone Detector Cards
illuminated by sunlamps and a UV lamp to achieve a light intensity simulating illumination
outdoors by a full overhead sun. These test conditions were designated as the Laboratory,
Dark, and Bright conditions. After each ozone exposure the Ozone Indicator Cards were
removed from the test chamber and taken within 30 sec to the two Battelle card readers for
visual reading under overhead fluorescent lighting in windowless offices. The Ozone
Detector Card results from the three trials (i.e., eight readings by each of two readers at each
of three light intensities) were compared to assess whether the light intensity during ozone
exposure affects the Ozone Detector Card readings.
For each light intensity test the total light intensity at the card surfaces was measured using a
DLM 2000 lux meter (Mannix, New York, NY) (1 lux = 1 lumen/m2 (~ 0.1 footcandle)). For
the Bright light test, the solar intensity and spectrum were simulated using the combination
of two full daylight spectrum fluorescent flood lamps (GE BE 26 PAR38/D) and a mercury
vapor UV flood lamp (PowerSun UV 100W, Zoo Med Laboratories, Inc., San Luis Obispo,
CA). These lamps were positioned at the test chamber to directly illuminate the exposed
surfaces of the Ozone Indicator Cards during the ozone exposure. In the Bright light test the
intensities at the card surfaces of UV-A (280-400 nm wavelength), UV-B (280-320 nm), and
UV-C (< 280 nm) were also measured using Solarmeter® Digital Ultraviolet Radiometers,
Model 5.7 (UV-A+B), Model 6.2 (UV-B), and Model 8.0 (UV-C) (Solartech, Inc., Harrison
Twp, MI). The measured total light intensities at the card surfaces in the Laboratory, Dark,
and Bright light conditions were 175, 3.5, and over 40,000 lux, respectively. The UV
intensities during the Bright light test were 146, 84, and zero microwatts per square
centimeter |iW/cm for UV-A, UV-B, and UV-C, respectively.
3.2.2 Field Testing
Testing conducted in the field by SCAQMD personnel consisted of exposing one or more
spots on one or more Ozone Detector Cards simultaneously, at ambient ozone monitoring
sites where an FEM monitor is deployed for continuous compliance monitoring. SCAQMD
personnel used pre-printed data collection forms to record the date, site location, user initials,
Ozone Detector Card identification number(s), reagent spot number(s), start and end times of
exposure, initial and final visual readings of the spots, and the ambient temperature, RH,
wind speed, and FEM ozone reading during each exposure period. The SCAQMD ozone
monitoring sites used in the field testing were those in Crestline, Riverside, Rubidoux, Santa
Clarita, and Upland, California. The elevations of these sites range from approximately 250
m (800 ft) above sea level at Rubidoux and Riverside to 1,400 m (4,600 ft) above sea level at
Crestline. These elevation differences result in an average difference in atmospheric pressure
of approximately 100 millimeters of mercury between the highest and lowest sites.
Consequently, the absolute concentration of ozone molecules in air at a constant ambient
ppbv concentration (i.e., mixing ratio) is approximately 13% lower at the Crestline site than
at the lowest sites. This difference in absolute ozone concentration at the same ppbv
-------
concentration was considered in evaluating the accuracy of the Ozone Detector Cards in the
SCAQMD testing.
At each SCAQMD site the Ozone Detector Cards were exposed to ambient air at a height of
approximately 3 to 6 ft above the ground, whereas the sample inlets of the FEM ozone
monitors were approximately 12 to 15 ft above the ground. Over those vertical distances,
ozone at the SCAQMD sites is not expected to exhibit vertical gradients significant enough
to affect the comparison of FEM and Ozone Detector Card readings. A total of 462 reagent
spots on 99 Ozone Detector Cards were exposed and read by SCAQMD personnel for
comparison to FEM readings in the field.
Additional testing by BCLA consisted of having volunteers expose and subsequently visually
read reagent spots on Ozone Detector Cards in indoor and outdoor locations at schools,
playgrounds, and parks. BCLA volunteers used pre-printed data collection forms to record
the date, location, user initials, Ozone Detector Card identification number(s), reagent spot
number(s), times of exposure, and the visual readings of the spots. A total of 64 reagent
spots on 14 cards were exposed by BCLA volunteers for recording of readings in the field.
3.3 Performance Parameters Tested
3.3.1 Accuracy
The accuracy of the Ozone Detector Cards was determined by comparing Ozone Detector
Card readings determined visually to simultaneous measurements made using the FEM both
in the laboratory and in the field.
3.3.2 Variability of Readings
Variability of Ozone Detector Card readings refers to the consistency, or lack thereof, in
visually determined Ozone Detector Card results with a constant ozone concentration.
Variability was assessed using the multiple readings made by each of two users at six ozone
concentrations in the laboratory testing described in Section 3.2.1.
3.3.3 Duplication
The degree of agreement of ozone measurements made simultaneously on different Ozone
Detector Cards or different reagent spots was assessed using all data in which simultaneously
exposed reagent spots were read by a single user. Similarly, the degree of agreement of
ozone measurements made by separate users was assessed using all data in which the same
exposed Ozone Detector Card reagent spot was read by more than one user. These two
measures of performance are termed Ozone Detector Card duplication and user agreement,
respectively, and are intended to address two types of variation in Ozone Detector Card
readings. Ozone Detector Card duplication addresses within-user variation (and consists of
both inter-card duplication and intra-card duplication), whereas user duplication addresses
between-user variation. Within-user variation arises from differences in the color
development of different reagent spots that have been exposed under identical conditions.
Between-user variation arises from visual perception differences in readings made by
different individual users on identically the same exposed reagent spot. These two forms of
duplication were evaluated by means of the unique numbering of Ozone Detector Cards and
reagent spots, and by separate recording of readings made by different users.
-------
Data to assess both Ozone Detector Card duplication and user agreement were generated
from the laboratory testing. Relevant data also originated in the field testing, whenever either
SCAQMD or BCLA representatives reported visual readings for simultaneously exposed
reagent spots, or visual readings from two different users for the same reagent spot. Such
duplicate readings were recorded on the field data sheets used by SCAQMD and BCLA
representatives. In all cases duplicate readings (whether of duplicate Ozone Detector Card
reagent spots by a single user or of a single Ozone Detector Card reagent spots by multiple
users) were taken in close succession and under identical lighting conditions immediately
after the Ozone Detector Card exposure period.
3.3.4 Effect of Light Intensity
The effect of light intensity on reagent spot color development was assessed based on the
readings made at a constant ozone concentration under three different light intensity
conditions.
3.3.5 Effect of Ambient Conditions
The effect of ambient temperature, RH, and wind speed was evaluated based on information
provided by SCAQMD personnel from the field sites. The Ozone Detector Card and FEM
field ozone results used to determine Ozone Detector Card accuracy were segregated into
those showing agreement between Ozone Detector Card and FEM results and those showing
disagreement. Then the temperature, RH, and wind speed conditions associated with these
two data sets were compared.
3.3.6 Operational Factors
Operational factors associated with use of the Ozone Detector Cards were evaluated based on
the comments and observations of all users (Battelle, SCAQMD, and BCLA) in the
laboratory and field testing. Such observations addressed the convenience of the Ozone
Detector Cards, their readability under differing conditions, the apparent consistency of
Ozone Detector Card readings, and other factors. Cost was evaluated based on published
price lists obtained from the vendor.
3.4 Verification Schedule
The Ozone Detector Card laboratory testing was conducted at Battelle's laboratories in
Columbus, Ohio between January 20, 2010 and February 25, 2010. Initial field testing with
the Ozone Detector Cards was conducted by SCAQMD between October 1 and November 2,
2009. Although conducted in accordance with the test/QA plan,3 this initial testing took
place before final approval of the test/QA plan. Consequently, data and results from that
portion of testing are included only as auxiliary data in Appendix A of this report. Field
testing was conducted by BCLA volunteers between December 18, 2009 and May 13, 2010.
Those initial SCAQMD and BCLA field tests used Ozone Detector Cards from the same
batch of cards used for the laboratory testing. The vendor states that the shelf life of the
Ozone Detector Cards is one year, however a new batch of cards was obtained in June 2010
for use when field testing resumed during the late summer and fall of 2010. With the new
batch of cards, SCAQMD conducted field testing between August 24 and October 5, 2010.
10
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures and all verification testing were performed in accordance with test/QA
plan for this verification test3 and the quality management plan (QMP) for the AMS Center.4
QA/QC procedures and results are described below.
4.1 Reference Methods
The following sections describe the QA/QC procedures employed in the collection and
analysis of reference samples.
The quality of the laboratory reference ozone measurements was assured by a calibration of
the FEM ozone analyzer before any testing, and a daily zero and span check of the FEM
analyzer at the start of each day of testing, using the Dasibi 1008 UV ozone transfer standard.
The pre-testing calibration data are presented in Table 4-1. The slope of the linear regression
of Thermo 49C FEM readings against the transfer standard Dasibi 1008 UV readings was
0.96, the intercept was -0.2 ppbv, and the coefficient of determination (r2) was 0.9999. This
calibration was used to correct all laboratory FEM data to the Dasibi 1008 UV readings. It
should be noted that some of the highest ozone concentrations used in laboratory testing (see
Section 3.2.1) exceeded the 150 ppbv upper end of the FEM calibration. Those data points
were used to assess the ability of the Ozone Detector Cards to exhibit a range reading of 4.
The validity of FEM readings at those concentrations is supported by the demonstrated
linearity of the FEM response on its 200 ppbv range.
The FEM reading on the one-point daily span check was within 5% of the ozone
concentration from the Dasibi 1008 UV on every day of testing. Quality of the field
reference ozone measurements was assured by QA review of the SCAQMD records of the
calibration and maintenance of FEM monitors used for ozone compliance monitoring at the
field sites.
11
-------
Table 4-1. Results of Pre-Test Ozone FEM Calibration
Nominal Concentration
Delivered by Environics
6100 (ppbv)
0
30
60
90
120
150
Measured Dasibi 1008 UV
Photometer Concentration
(ppbv)
1.8
35
65
98
127
159
Measured Thermo 49C
Ozone Monitor
Concentration (ppbv)
1.6
32
62
93
122
152
4.2 Audits
Three types of audits were performed during the verification test: a performance evaluation
(PE) audit of the Dasibi 1008 UV photometer transfer standard, a technical systems audit
(TSA) of the verification test procedures, and an audit of data quality (ADQ). Each audit
was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the ETV AMS
Center.4 Audit procedures are described further below.
4.2.1 Performance Evaluation Audit
A PE audit of the ozone measurements was performed to confirm the accuracy of the
Battelle-owned Dasibi 1008 UV photometer as the basis for the FEM calibration in this
verification. A side-by-side comparison was conducted to establish the traceability of the
Battelle Dasibi 1008 UV photometer relative to the ozone standard owned by the OEPA,
which is a Thermo Environmental Model 49, Serial Number 72903-372, and which is
traceable to the primary ozone standard reference photometer located at EPA Region 5,
Chicago, Illinois. In the side-by-side comparison, the slope of the regression of Battelle
Dasibi 1008 UV readings against OEPA readings was 0.99, the intercept was 1.3 ppbv, and
the coefficient of determination (r ) was greater than 0.9999. All of these measures fell
within the OEPA requirements for verification (slope of 1.00 ± 0.05 and intercept < 5 ppbv).
4.2.2 Technical Systems Audit
The Battelle Quality Manager performed a TSA of the laboratory testing procedures during
the first week of the laboratory testing. The purpose of the laboratory TSA was to ensure that
the verification test was being performed in accordance with the AMS Center QMP,4 the
test/QA plan for this verification test,3 and the reference methods. In this TSA, the Battelle
Quality Manager reviewed the reference method used, compared the actual test procedures
being performed to those specified or referenced in the test/QA plan, and reviewed data
acquisition and handling procedures. During the laboratory TSA, the Battelle Quality
Manager observed the reference method sampling; inspected documentation of test
procedures; and reviewed laboratory record books. Six deviations from the test/QA plan
were noted in this TSA. Some were the result of deliberate choices to improve the test
procedures; the others were readily addressed and had no significant effect on data quality.
The six deviations were:
12
-------
• Extension of the lower limit of calibration of the FEM from 10 ppbv to zero ppbv.
• Use of additional challenge ozone concentrations above the 150 ppbv upper limit of
the FEM calibration range (noted in Section 4.1).
• Use of the Dasibi 1008 UV readings, rather than the Environics 6100 ozone source
output, as the transfer standard for comparison to FEM readings in daily calibration
checks.
• Use of a 100 ppbv concentration of ozone in the light intensity testing, instead of 60
ppbv as stated in the test/QA plan, to assure production of a color change with the
Ozone Detector Cards and increase the likelihood of detecting a light intensity effect.
• Exposure of eight spots (two spots on each of four cards) rather than 10 (five spots on
each of two cards) in the light intensity testing, to provide greater consistency with
other test procedures.
• Occasional departures from recording procedures in the laboratory notebook (e.g.,
absence of units on ozone concentrations; corrections not properly initialed and
dated).
A formal description and response to these deviations was prepared, signed by Battelle's
Verification Test Coordinator, AMS Center Manager, and AMS Center Quality Manager,
provided to the EPA AMS Center Project Officer and Quality Manager, and retained in the
study files. A second such deviation form was similarly prepared, approved and distributed,
documenting that BCLA was unable to carry out field testing of the Ozone Detector Cards to
the extent originally planned.
The SCAQMD Quality Assurance Manager (QAM) also conducted a TSA of the SCAQMD
field activities. Battelle's Quality Manager prepared a TSA checklist, which was reviewed
and approved by EPA's AMS Center Quality Manager and then provided to the SCAQMD
QAM for use in the TSA. In the TSA, the SCAQMD QAM visited the monitoring sites in
Upland and Rubidoux on September 1 and 3, 2010, respectively, and observed the use of the
Ozone Detector Cards, operation of the FEM, and the recording of ozone and other data by
SCAQMD field operators. The SCAQMD QAM reported his observations in writing to the
Battelle QA Manager. No deviations from the test/QA plan were found as a result of the
field TSA of SCAQMD activities.
TSA reports were prepared and copies were distributed to EPA.
4.2.3 Audit of Data Quality
Records generated in the verification test received a one-over-one review before these
records were used to calculate, evaluate, or report verification results. Data were reviewed by
a Battelle technical staff member involved in the verification test. The person performing the
review added his/her initials and the date to a hard copy of the record being reviewed.
13
-------
100% of the verification test data was reviewed for quality by the Verification Test
Coordinator, and at least 10% of the data acquired during the verification test and 100% of
the calibration and QC data were audited. Battelle's Quality Manager, or designee, traced the
data from the initial acquisition, through reduction and statistical analysis, to final reporting,
to ensure the integrity of the reported results. All calculations performed on the data
undergoing the audit were checked.
14
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the quantitative performance factors listed in Section
3.3 are presented in this chapter. The semi-quantitative nature of the Ozone Detector Card
readings determines the types of statistical comparisons that can be done to evaluate the
performance parameters. The statistical comparisons are described in the following sections.
Qualitative observations were also used to evaluate verification test data.
5.1 Accuracy
The accuracy of the Ozone Detector Cards with respect to the FEM was assessed as a
percentage of readings in the correct Ozone Detector Cards indication range, i.e.:
Accuracy = (Number in Range/Number of Trials) x 100 (1)
Where the Number in Range is the number of total Ozone Detector Card readings reported as
being in the Ozone Detector Card indication range that encompasses the corresponding FEM
ozone reading, and the Number of Trials is the total number of such comparisons for which
FEM readings fell within that range. This calculation of accuracy was performed with the
data from each of the challenge ozone concentrations in the laboratory testing, and with the
data obtained by SCAQMD at field sites. Readings from Ozone Detector Cards exposed
simultaneously, and readings from Ozone Detector Cards read in the laboratory by more than
one user, were included in this calculation as separate trials (i.e., all Ozone Detector
Card/FEM comparisons were treated as independent data for this calculation).
5.2 Variability of Readings
Variability of the Ozone Detector Card readings was evaluated using only the data from the
repeated laboratory trials at five ozone challenge concentrations. For each of those
concentrations, variability was determined as the number of Ozone Detector Card indication
ranges into which the user readings fell. That is, at ozone concentration X (e.g.):
Variability X= (# of Ranges with Readings at Ozone Concentration X) (2)
15
-------
The ideal result for variability is a value of 1 (i.e., all readings in a single Ozone Detector
Card range). Note that the variability is independent of, and not an indication of, the
accuracy of the card readings.
5.3 Duplication
Ozone Detector Card duplication was assessed in terms of the percentage of readings in
which a single user reported a result in the same card indicator range from two reagent spots
exposed simultaneously (regardless of whether that range agreed with FEM ozone results).
Ozone Detector Card duplication was calculated as:
Duplication = (Number Same Ranged/Number Duplicatesd) x 100 (3)
where the Number Same Range^ is the number of cases in which a user reading
simultaneously exposed reagent spots reported the same indication range from each spot.
The Number Duplicatesd is the total number of cases in which one user read such duplicate
exposed reagent spots. Duplication was calculated both for those cases in which the
duplicate spots were on the same card (intra-card duplication) and for those cases in which
the duplicate spots were on different cards (inter-card duplication). These calculations were
performed for the data from each of the ozone concentrations in laboratory testing, and
separately for any cases of duplicate cards used in the field testing performed by SC AQMD
and BCLA.
User agreement was calculated in the same manner, except using data from multiple users
reading the same exposed reagent spot on an Ozone Detector Card, i.e.:
User Agreement = (Number Same Range JNumber Duplicates^ x 100 (4)
Where the Number Same Rangeu is the number of cases in which two users reading the same
exposed reagent spot reported the same Ozone Detector Card indication range, and the
Number Duplicatesu is the total number of cases in which two users read the same exposed
reagent spot. This calculation was performed for the data from the duplicate users with each
of the ozone concentrations in laboratory testing, and separately for any cases of duplicate
users in the field testing performed by SCAQMD and BCLA.
5.4 Effect of Light Intensity
The effect of light intensity on Ozone Detector Card performance was assessed using the data
from the laboratory testing (Section 3.2.2), by calculating the accuracy, variability, and
duplication of readings (Sections 5.1 through 5.3) of the test data at each of the three light
intensity conditions. Those results were compared to indicate whether light intensity has any
apparent effect on the Ozone Detector Card performance at a constant ozone concentration.
Accuracy or duplication results that differed by more than 20% accuracy or 20% duplication
were taken as evidence of a significant light intensity effect. Similarly, variability results that
differed by one or more Ozone Detector Card indication ranges were taken as evidence of a
significant light intensity effect.
16
-------
5.5 Effect of Ambient Conditions
To assess the impact of outdoor ambient conditions, the field data from the SC AQMD field
testing were separated into those results for which the Ozone Detector Card reading agreed
with that expected based on the FEM ozone reading, and those for which the card and FEM
results did not agree. The temperature, RH, and wind speed data for the two sets of results
were then compared to investigate whether there were significant differences in conditions
that may have contributed to the differences in accuracy of the Ozone Detector Cards.
Comparison of the data sets for significance of differences was based on t-test comparisons
of means, using a 95% significance level of any differences. When data sets contained small
numbers of samples (n < 6), an alternative procedure was also used in which the confidence
interval of the mean of those data was estimated based on tabulated values of Cn and the
equation:
|i = x ± CnR (5)
where |i is the population mean, x is the mean of a small set of samples, and R is the range of
the values.5
5.6 Operational Factors
Operational factors such as ease of use, readability of color ranges, etc., were evaluated based
on observations recorded by Battelle and field staff. A laboratory notebook was maintained
and data sheets were filled out by Ozone Detector Card users to record their observations.
Cost was evaluated by reviewing price lists provided by the vendor for the Ozone Indicator
Cards.
17
-------
Chapter 6
Test Results
6.1 Laboratory Results
Tables 6-1 through 6-7 show the laboratory test results obtained at ozone concentrations of
approximately 0, 30, 50, 70, 90, 130, and 150 to >200 ppbv, respectively. Each of these
tables shows the Ozone Detector Card number and spot number, the FEM ozone
concentration measured during the Ozone Detector Card exposure, the expected Ozone
Detector Card range based on the FEM reading, and the range readings recorded by the two
Battelle card readers. Table 6-8 shows the results obtained with blank spots (i.e., newly
uncovered, not exposed to ozone) during the laboratory testing. Cases in which a reader's
visual reading does not match the expected range are shaded gray in Tables 6-1 to 6-8.
Tables 6-1 through 6-7 illustrate that the Ozone Detector Card ranges reported by the two
Battelle card readers often were lower than the expected range based on the corresponding
FEM ozone reading, especially at the higher delivered ozone concentrations. At ozone
concentrations of 90 ppbv and above (Tables 6-5 to 6-7), the great majority of user range
readings were lower than the expected range. Tables 6-1 to 6-8 also show that in most cases
the readings recorded by the two card readers (who were non-technical Battelle staff
members) were the same. The readings recorded by these staff from reagent spots exposed to
zero ozone concentrations in the test chamber (Table 6-1) always matched the expected
reading of 1. The readings recorded by these staff from reagent spots not exposed at all
before the visual reading was made (Table 6-8) matched the expected reading of 1 in all but
two of 72 total readings.
18
-------
Table 6-1. Laboratory Ozone Detector Card Data - 0 ppbv.
Card
Number
21
21
22
22
23
23
24
24
25
25
26
26
27
27
28
28
s FEM Ozone
„ , Concentration
Number , , ,
(ppbv)
1
5
1
5
1
5
1
5
3
5
3
5
3
5
3
5
a: Based on FEM ozone
Table 6-2.
Card
Number
25
25
26
26
27
27
28
28
17
17
18
18
19
19
20
20
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
concentration reading;
Laboratory Ozone
FEM Ozone
„ , Concentration
Number , , ,
(ppbv)
1
2
1
2
1
2
1
2
2
3
2
3
2
3
2
3
28.1
28.1
28.1
28.1
28.1
28.1
28.1
28.1
32.7
32.7
32.7
32.7
32.7
32.7
32.7
32.7
Expected
Range"
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Userl
Range
Reading
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
User 2
Range
Reading
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
, range 1 = nominally 10 to 45 ppbv.
Detector Card Data - 30 ppbv.
Expected
Range"
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Userl
Range
Reading
1
1
1
1
1
1
1
1
1
1
2
2
2
2
1
1
User 2
Range
Reading
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
a: Based on FEM ozone reading; range 1 = nominally 10 to 45 ppbv. User readings that do not match
the expected range are shown in shaded cells.
19
-------
Table 6-3. Laboratory Ozone Detector Card Data - 50 ppbv.
„ , „ . FEM Ozone
Card Snot „ ...
„ , „ , Concentration
Number Number , , ,
(ppbv)
13
13
14
14
15
15
16
16
9
9
10
10
11
11
12
12
1
3
1
3
1
3
1
3
3
5
3
5
3
5
3
5
49.3
49.3
49.3
49.3
49.3
49.3
49.3
49.3
52.8
52.8
52.8
52.8
52.8
52.8
52.8
52.8
Expected
Range"
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Userl
Range
Reading
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
a: Based on FEM ozone reading; range 2 = nominally 45 to 75 ppbv. User readings
the expected range are shown in shaded cells.
Table 6-4. Laboratory Ozone Detector Card Data - 70 ppbv.
„ , „ . FEM Ozone
Card Spot „ , ,.
„ , „ , Concentration
Number Number (ppby)
17
17
18
18
19
19
20
20
29
29
30
30
31
31
32
32
1
4
1
4
1
4
1
4
2
4
2
4
2
4
2
4
63.8
63.8
63.8
63.8
63.8
63.8
63.8
63.8
68.6
68.6
68.6
68.6
68.6
68.6
68.6
68.6
Expected
Range3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Userl
Range
Reading
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
User 2
Range
Reading
1
1
2
2
2
3
2
2
2
2
2
3
1
1
1
1
that do not match
User 2
Range
Reading
1
1
1
1
2
2
2
1
2
2
1
1
1
1
1
1
a: Based on FEM ozone reading; range 2 = nominally 45 to 75 ppbv. User readings that do not match
the expected range are shown in shaded cells.
20
-------
Table 6-5. Laboratory Ozone Detector Card Data - 90 ppbv.
Card
Number
9
9
10
10
11
11
12
12
21
21
22
22
23
23
24
24
33
33
34
34
35
35
36
36
37
37
38
38
39
39
40
40
Spot
Number
1
2
1
2
1
2
1
2
2
4
2
4
2
4
2
4
1
5
1
5
1
5
1
5
2
3
2
3
2
3
2
3
FEM Ozone
Concentration
(ppbv)
93.9
93.9
93.9
93.9
93.9
93.9
93.9
93.9
89.9
89.9
89.9
89.9
89.9
89.9
89.9
89.9
89.8
89.8
89.8
89.8
89.8
89.8
89.8
89.8
90.2
90.2
90.2
90.2
90.2
90.2
90.2
90.2
Expected
Range"
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
Userl
Range
Reading
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
User 2
Range
Reading
2
2
2
2
3
3
3
2
2
2
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
2
2
2
a: Based on FEM ozone reading; range 3 = nominally 75 to 105 ppbv. User readings that do not match
the expected range are shown in shaded cells.
21
-------
Table 6-6. Laboratory Ozone Detector Card Data - 130 ppbv.
Card
Number
29
29
30
30
31
31
32
32
13
13
14
14
15
15
16
16
33
33
34
34
35
35
36
36
37
37
38
38
39
39
40
40
Spot
Number
3
5
3
5
3
5
3
5
4
5
4
5
4
5
4
5
3
4
3
4
3
4
3
4
4
5
4
5
4
5
4
5
FEM Ozone
Concentration
(ppbv)
132.3
132.3
132.3
132.3
132.3
132.3
132.3
132.3
121.6
121.6
121.6
121.6
121.6
121.6
121.6
121.6
131.5
131.5
131.5
131.5
131.5
131.5
131.5
131.5
132.0
132.0
132.0
132.0
132.0
132.0
132.0
132.0
Expected
Range"
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
a: Based on FEM ozone reading; range 4 = nominally > 105 ppbv.
expected range are shown in shaded cells.
User 1 User 2
Range Range
Reading Reading
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
User readings that
2
2
3
3
2
2
2
2
2
2
2
2
2
3
3
3
2
2
2
2
2
3
3
3
2
3
2
3
2
2
2
3
do not match the
22
-------
Table 6-7. Laboratory Ozone Detector Card Data - 150 to >200 ppbv.
Card
Number
41
41
42
42
43
43
44
44
41
41
42
42
43
43
44
44
45
45
46
46
47
47
48
48
45
45
46
46
47
47
48
48
Spot
Number
2
5
2
5
2
5
2
5
1
3
1
3
1
3
1
3
1
2
1
2
1
2
1
2
4
5
4
5
4
5
4
5
FEM Ozone
Concentration
(ppbv)
155.2
155.2
155.2
155.2
155.2
155.2
155.2
155.2
160.9
160.9
160.9
160.9
160.9
160.9
160.9
160.9
196.3
196.3
196.3
196.3
196.3
196.3
196.3
196.3
199.6
>200b
>200
>200
>200
>200
>200
>200
Expected
Range"
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
Userl
Range
Reading
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
User 2
Range
Reading
3
2
2
3
3
3
2
2
2
4
3
4
3
4
2
4
3
3
3
3
3
2
2
3
2
3
2
2
2
3
2
2
a: Based on FEM ozone reading; range 4 = nominally > 105 ppbv. User readings that do not match the
expected range are shown in shaded cells.
b: Exceeded 200 ppbv upper range limit of FEM.
23
-------
Table 6-8. Laboratory Ozone Detector Card Data - Blank Spots.
Card
Number
Spot
Number
Expected
Range"
User 1
Range
Reading
User 2
Range
Reading
13
14
15
16
25
26
27
28
29
30
31
32
9
10
11
12
17
18
19
20
21
22
23
24
33
34
35
36
37
38
39
40
41
42
43
44
2
2
2
2
4
4
4
4
1
1
1
1
4
4
4
4
5
5
5
5
3
3
3
3
2
2
2
2
1
1
1
1
4
4
4
4
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
3
a: Based on absence of ozone exposure; range 1 = nominally 10 to 45 ppbv. User readings that do not
match the expected range are shown in shaded cells.
24
-------
6.1.1 Accuracy
The accuracy of the Ozone Detector Cards with respect to the FEM was assessed as a
percentage of readings in the correct Ozone Detector Card indication range as described in
Section 5.1. Table 6-9 summarizes the accuracy found at each of the ozone concentrations
used in the laboratory.
Table 6-9. Accuracy Results at Laboratory Ozone Concentrations.
FEM Ozone
Concentration, ppbv
(Expected Range)
Blank (1)
0(1)
30(1)
50(2)
70(2)
90(3)
130 (4)
150to>200(4)
# of Card
Readings in
Range
70
32
28
20
21
5
0
4
#of
Trials
72
32
32
32
32
64
64
64
Accuracy
97%
100%
88%
63%
66%
7.8%
0%
6.3%
Table 6-9 shows that the accuracy of the Ozone Detector Cards was 100% or nearly so for
unexposed blanks (97%) and for cards exposed in the test chamber to 0 ppbv ozone (100%).
Accuracy decreased slightly for cards exposed to low levels of ozone (88% at 30 ppbv), and
was markedly lower when cards were exposed to moderate levels of ozone (63% at 50 ppbv
and 66% at 70 ppbv). At higher ozone concentrations, the accuracy of the Ozone Detector
Cards was very low with less than 8% accuracy at 90 ppbv and an aggregate accuracy of
3.1% (4 out of 128 readings) for ozone concentrations of 130 ppbv and greater. Overall
accuracy thus was 95.6% when a range reading of 1 was expected, 64.1% when a range
reading of 2 was expected, 7.8% when a range reading of 3 was expected, and 3.1% when a
range reading of 4 was expected. Accuracy results in Table 6-9 for ozone concentrations
near range boundaries (i.e., 50 and 70 ppbv) fall between those for concentrations near range
midpoints (i.e., 30 and 90 ppbv). However, Table 6-9 indicates that ozone concentration
rather than position within range is the dominant factor affecting Ozone Detector Card
accuracy.
The great majority of the discrepancy between the FEM ozone concentrations and the user
Ozone Detector Card readings were due to user readings that were lower than expected based
on the FEM readings. Of the 212 total card readings that did not agree with the expected
range based on the FEM reading, 1 user reading was two ranges high, 7 user readings were
one range high, 104 user readings were one range low, and 100 user readings were two
ranges low.
25
-------
6.1.2 Variability of Readings
The variability of the Ozone Detector Card readings was evaluated as described in Section
5.2. Table 6-10 shows the variability of the user readings of the Ozone Detector Cards at
each of the ozone concentrations evaluated. Cases in which a user's visual reading does not
match the expected range are shaded gray in Table 6-10.
Table 6-10. Summary of Variability of Ozone Detector Card Readings
for Individual Ozone Concentrations.
FEM Ozone
Concentration
ppbv
(Expected
Range)
Blank (1)
0(1)
30(1)
50(2)
70(2)
90(3)
130 (4)
150to>200(4)
# of Card
Readings
in Range 1
70
32
28
10
11
1
0
0
# of Card
Readings in
Range 2
1
0
4
20
21
58
53
46
# of Card
Readings in
Range 3
1
0
0
2
0
5
11
14
# of Card
Readings in
Range 4
0
0
0
0
0
0
0
4
Variability
(# of ranges)
3
1
2
3
2
3
2
3
Table 6-10 shows that blank reagent spots were classified in three ranges. It is possible that
the one blank reading classified as Range 3 was the result of a data recording error by the
reader, as the blanks showed little discoloration when viewed by the test operator. Ozone
Detector Cards exposed to 0 ppbv of ozone were always classified as Range 1 for a
variability of 1. Ozone Detector Cards exposed to 30, 70, or 130 ppbv of ozone showed a
variability of 2, and Ozone Detector Cards exposed to 50, 90, or 150 to >200 ppbv of ozone
showed a variability of 3. Overall variability thus was 3, whether a range reading of 1, 2, 3,
or 4 was expected. Table 6-10 shows that the overall variability at ozone concentrations near
range boundaries (i.e., 50 and 70 ppbv) was essentially the same as the variability at
concentrations near range midpoints (i.e., 30 and 90 ppbv). However, the fraction of
readings outside the central range is greater at 50 or 70 ppbv (i.e., 11 or 12 out of 32
readings) than at 90 ppbv (i.e., 6 out of 64 readings).
6.1.3 Duplication
Ozone Detector Card duplication was assessed as described in Section 5.3. Intra-card
duplication was determined by comparing readings from the same user for two spots on the
same Ozone Detector Card which had been exposed to the same ozone concentration at the
same time. Table 6-11 presents results for intra-card duplication in the laboratory testing.
In that testing, two spots from each of four Ozone Detector Cards were exposed
simultaneously and then read by two readers. A minimum of two replicates of each test were
26
-------
performed so there were a minimum of 16 readings of duplicate spots from the same card (8
from each reader). Table 6-11 shows that intra-card duplication ranged from 75% at 50 ppbv
and 150 to >200 ppbv ozone, to 100% with 0 ppbv ozone. Intra-card duplication was 94% at
70 and 90 ppbv ozone. Overall intra-card duplication thus was 93.8% when a range reading
of 1 was expected, 84.4% when a range reading of 2 was expected, 93.8% when a range
reading of 3 was expected, and 76.6% when a range reading of 4 was expected. Table 6-11
shows that intra-card variability at ozone concentrations near range boundaries (i.e., 50 and
70 ppbv) was not markedly different from the variability at concentrations near range
midpoints (i.e., 30 and 90 ppbv).
Table 6-11. Summary of Intra-Card Duplication for Laboratory Ozone Concentrations.
FEM Ozone
Concentration
(ppbv)
0
30
50
70
90
130
150to>200
#of
Duplicate
Spot
Readings in
Agreement
16
14
12
15
30
27
22
Total # of
Duplicate Spot
Readings
16
16
16
16
32
32
32
Intra-Card
Duplication
100%
88%
75%
94%
94%
84%
75%
Inter-card duplication was assessed by comparing readings from a single card user for
reagent spots on different Ozone Detector Cards exposed at the same time. Each set of four
cards results in 24 individual comparisons per reader for this analysis. With two readers and
two replicates the total number of comparisons is at least 96. Table 6-12 presents the results
for inter-card duplication. Table 6-12 shows that inter-card duplication ranged from 48% at
50 ppbv ozone, to 100% with 0 ppbv ozone. Inter-card duplication was 71% to 85% at other
ozone concentrations. Overall inter-card duplication thus was 91.7% when a range reading
of 1 was expected, 59.4% when a range reading of 2 was expected, 85.4% when a range
reading of 3 was expected, and 47.4% when a range reading of 4 was expected. Table 6-12
shows that inter-card variability at ozone concentrations near range boundaries (i.e., 50 and
70 ppbv) was lower than the variability at either of the concentrations near range midpoints
(i.e., 30 and 90 ppbv).
Table 6-12 shows that except for the 0 ppbv data, at all ozone concentrations the inter-card
duplication was lower than the corresponding intra-card duplication (Table 6-11). The
greatest differences occurred at 50 and 70 ppbv ozone. This comparison indicates that
reagent spots exposed simultaneously on the same cards provide greater consistency of
readings than do reagent spots exposed simultaneously on different cards.
27
-------
Table 6-12. Summary of Inter-Card Duplication for Laboratory Ozone Concentrations.
FEM Ozone
Concentration
(ppbv)
0
30
50
70
90
130
150to>200
#of
Duplicate
Spot
Readings in
Agreement
96
80
46
68
164
140
142
Total # of
Duplicate Spot
Readings
96
96
96
96
192
192
192
Inter-Card
Duplication
100%
83%
48%
71%
85%
73%
74%
User agreement in the laboratory testing was also assessed as described in Section 5.3. Table
6-13 presents the results for user agreement at each ozone concentration tested, based on
comparison of readings by multiple users of the same exposed reagent spots. As in the
calculation of intra-card duplication, there were a minimum of 16 readings of the same spots
by the two readers. Table 6-13 shows that user agreement ranged from 31% at 70 ppbv
ozone, to 100% with 0 ppbv ozone. User agreement was 44% to 81% at other ozone
concentrations, and 94% with blank (unexposed) reagent spots. Overall user agreement thus
was 91.2% when a range reading of 1 was expected, 53.1% when a range reading of 2 was
expected, 81.3% when a range reading of 3 was expected, and 54.7% when a range reading
of 4 was expected. The lowest user agreement (31%) was seen at the 70 ppbv
concentration, which is near the upper boundary of range 2 of the Ozone Detector Cards.
Table 6-13. Summary of User Agreement for Laboratory Ozone Concentrations.
FEM Ozone
Concentration
(ppbv)
Blank
0
30
50
70
90
130
150to>200
# of Spots
with
Readings in
Agreement
34
16
12
12
5
26
21
14
Total # of
Spots read
by
Multiple
Users
36
16
16
16
16
32
32
32
User
Agreement
94%
100%
75%
75%
31%
81%
66%
44%
28
-------
It might be expected that the duplication of readings would be reduced with ozone
concentrations near the boundary of an Ozone Detector Card range, and this expectation is
borne out to some extent by the results for 50 and 70 ppbv ozone in Tables 6-11 to 6-13.
However, the results at these two concentrations are often not consistent, so any such
boundary effect does not strongly determine the test results.
Table 6-13 shows varying agreement between the two Battelle staff members who read the
Ozone Detector Cards in the laboratory testing. User agreement was high for reagent spots
unexposed to ozone, but ranged from 31% agreement to 81% agreement with exposed
reagent spots. The disagreement between the two card readers was not due to random
variation in their readings. The readings of User 1 showed less variation than did those of
User 2. When the two disagreed on the reading of a reagent spot, the readings of User 1 were
generally higher than those of User 2 at ozone concentrations up to 70 ppbv, and were
generally lower than those of User 2 at ozone concentrations of 90 ppbv and greater. These
differences were not due to any color-blindness, visual impairment, or use of sunglasses by
the card readers, as these factors were avoided in selecting the staff members to read the
cards, consistent with the test/QA plan.3
6.1.4 Effect of Light Intensity
The data from the testing of light intensity effects during laboratory ozone exposures are
listed in Table 6-14, which shows the Ozone Detector Card identification (lettered A through
H), the reagent spot number, the lighting conditions, the expected Ozone Detector Card range
based on the delivered ozone concentrations of 97 to 100 ppbv, and the visual readings of
Ozone Detector Card range provided by the two Battelle card readers. Visual readings that
did not match the expected Ozone Detector Card range are shaded in Table 6-14. The results
of the light intensity testing are summarized in Table 6-15. Based on the criteria indicated in
Section 5.4, Table 6-15 shows no significant effect of light intensity on the accuracy,
variability, or user agreement of the Ozone Detector Cards. On the other hand, both intra-
and inter-card duplication showed differences greater than the 20% criterion, with the lowest
duplication in both cases occurring at the brightest illumination condition. The results for
the effect of light intensity on duplication of readings are not entirely internally consistent, as
a substantial difference between the laboratory and dark conditions was seen for intra-card
duplication, but not for inter-card duplication. Note that the ozone concentration of
approximately 100 ppbv was near the upper limit of range 3 (i.e., nominally 75 to 105 ppbv),
and this may have contributed to variability in the users' range readings.
6.2 SCAQMD Field Results
This section reports the results of the field testing conducted by SCAQMD in the fall season
of 2010. Additional data and results from initial field testing conducted by SCAQMD in the
fall of 2009 are shown in Appendix A. As noted in Section 3.4, a new batch of Ozone
Detector Cards was used for the fall 2010 field period.
Field testing in the fall of 2010 took place at SCAQMD's Crestline, Upland, and Rubidoux
monitoring sites. The elevation difference among these sites was noted in Section 3.2.2.
Note that solar UV intensity increases by approximately 5% per 1,000 m increase in
29
-------
elevation, so the maximum difference in site elevations would lead to approximately 6%
higher solar UV radiation at the Crestline site than at the lowest sites. Based on the results in
Section 6.1.4, this difference is clearly insignificant in terms of the performance of the cards.
Table 6-14. Data from Laboratory Testing of Effect of Light Intensity
on Ozone Indicator Card Readings.
Card
A
A
B
B
C
C
D
D
A
A
B
B
C
C
D
D
E
E
F
F
G
G
H
H
Spot
Number
4
5
4
5
4
5
4
5
1
2
1
2
1
2
1
2
2
5
2
5
2
5
2
5
Lighting
Conditions
Dark
Dark
Dark
Dark
Dark
Dark
Dark
Dark
Laboratory
Laboratory
Laboratory
Laboratory
Laboratory
Laboratory
Laboratory
Laboratory
Bright
Bright
Bright
Bright
Bright
Bright
Bright
Bright
Expected
Range3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
Userl
Reading
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
4
2
3
User 2
Reading
3
4
3
4
3
4
3
4
2
2
3
4
2
3
3
3
3
4
3
4
3
4
3
4
a: Based on approximately 100 ppbv ozone test concentration; range 3 = nominally 75 to 105 ppbv.
30
-------
Table 6-15. Laboratory Results for Light Intensity Testing on Ozone Detector Cards.
Lighting
Condition
Dark
Laboratory
Bright
Accuracy
75%
75%
63%
Variability
(# of ranges)
2
3
3
Intra-Card
Duplication
50%
75%
25%
Inter-Card
Duplication
75%
65%
52%
User
Agreement
50%
50%
50%
All card exposures were made during daylight hours, and almost entirely between 10:00 am
and 5:00 pm, consistent with normal schedules for the testing personnel. Average ozone
levels during the card exposure periods ranged from approximately 28 to 126 ppbv. Ambient
conditions during the testing varied widely, e.g., the average air temperature during card
exposures ranged from 44 to 107 °F. Table 6-16 shows the Ozone Detector Card number and
spot number, the average FEM ozone concentration measured during the Ozone Detector
Card exposure, the expected Ozone Detector Card range based on the average FEM reading,
the Ozone Detector Card range reported by the SC AQMD user of each card, and the ambient
temperature, RH, and wind speed during the exposure. Cards and reagent spots that were
exposed simultaneously can be identified in Table 6-16 by the identical values for FEM
ozone, temperature, RH, and wind speed. Cases in which the user's visual reading does not
match the expected range are shaded in Table 6-16.
In the fall season field period in 2010, SCAQMD staff also read every Ozone Detector Card
reagent spot immediately after the protective foil covering was removed from the spot. All
of the 249 such readings were recorded by the users as indicating a range of 1. This result
contrasts with the blank spot readings obtained by SCAQMD in the fall season of 2009 (see
Appendix A, Section Al.l) which showed readings of 2 before any ozone exposure on the 17
reagent spots checked. This difference is likely due to the use of a new set of Ozone
Detector Cards in the fall season 2010 testing, and indicates that color development can
occur in the reagent spots after extended storage as suggested in Appendix A.
6.2.1 Accuracy
Table 6-16 lists 246 cases in which the expected Ozone Detector Card reading can be
compared to the user's reading (in three other cases the user recorded an initial reading when
the foil was first removed from a reagent spot, but then did not record a final reading after the
ozone exposure). Of those 246 cases, the user's reading agreed with the expected reading in
172 cases, resulting in an overall accuracy of 69.9%.
The accuracy is further broken down in Table 6-17, which shows the number of cases,
number of cases with agreement, and resulting accuracy for each category of the expected
Ozone Detector Card range (i.e., 1, 2, 3, or 4). Table 6-16 shows that the Ozone Detector
Card accuracy was nearly 100% when the reading expected based on FEM data was a 2, but
that accuracy fell off at both lower and higher expected readings. When a reading of 3 was
expected, accuracy of about 70% was achieved. However, when a reading of 1 was
expected, the Ozone Detector Card readings frequently overestimated the ozone level,
resulting in accuracy of 16.7%. Most importantly, the Ozone Detector Cards showed
accuracy of only 10% in indicating ozone levels in the highest range of 4 (nominally above
31
-------
105 ppbv). This result is similar to that observed in the laboratory testing (see Table 6-9)
indicating low accuracy for the Ozone Detector Cards when challenged with relatively high
ozone concentrations.
Table 6-16. Data from 2010 Field Testing by SCAQMD.3
Card
Number
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
6
6
6
6
7
7
7
7
8
8
8
8
9
9
9
9
10
10
10
10
11
11
11
11
Spot
Number
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
FEM Ozone
Concentration
(ppbv)
89.0
89.0
91.8
91.8
101.2
101.2
95.9
95.9
91.2
91.2
89.8
89.8
82.4
82.4
81.6
81.6
83.7
83.7
89.1
89.1
98.5
98.5
83.8
83.8
89.4
89.4
93.7
93.7
55.2
55.2
53.7
53.7
51.8
51.8
48.4
48.4
64.8
64.8
60.5
60.5
77.9
77.9
76.0
76.0
Expected
Rangeb
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
User
Range
Reading
2
2
2
2
3
3
3
3
3
3
3
3
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Temperature
(°F)
100.8
100.8
106.5
106.5
105.8
105.8
103.2
103.2
103.8
103.8
102.2
102.2
91.2
91.2
89.0
89.0
98.6
98.6
97.0
97.0
104.1
104.1
102.7
102.7
103.3
103.3
100.1
100.1
74.8
74.8
75.9
75.9
70.3
70.3
70.7
70.7
77.3
77.3
77.6
77.6
86.2
86.2
85.6
85.6
RH
(%)
15.8
15.8
13.2
13.2
20.2
20.2
22.9
22.9
22.4
22.4
24.0
24.0
25.2
25.2
29.8
29.8
19.0
19.0
24.1
24.1
10.2
10.2
17.2
17.2
17.9
17.9
20.6
20.6
56.6
56.6
54.7
54.7
57.6
57.6
56.9
56.9
39.7
39.7
41.0
41.0
30.3
30.3
34.3
34.3
Wind
Speed
(mph)
7.4
7.4
8.3
8.3
8.7
8.7
0.4
0.4
9.6
9.6
10.7
10.7
4.9
4.9
7.2
7.2
9.4
9.4
13.0
13.0
8.3
8.3
5.5
5.5
5.4
5.4
6.0
6.0
5.9
5.9
6.5
6.5
6.2
6.2
8.1
8.1
5.8
5.8
7.8
7.8
7.3
7.3
8.6
8.6
32
-------
Card
Number
12
12
12
12
13
13
13
13
14
14
14
14
15
15
15
15
16
16
16
16
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
17
17
17
18
18
18
18
19
19
19
19
20
20
Spot
Number
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
1
2
3
4
1
2
3
4
1
2
3
4
1
2
FEM Ozone
Concentration
(ppbv)
53.7
53.7
49.7
49.7
59.5
59.5
58.5
58.5
71.7
71.7
71.2
71.2
79.5
79.5
76.5
76.5
55.1
55.1
55.0
55.0
73.2
73.2
73.7
73.7
74.8
74.8
74.8
74.8
67.5
67.5
71.2
71.2
76.6
76.6
77.3
77.3
111.0
111.0
126.0
126.0
70.5
70.5
108.0
108.0
114.0
114.0
116.5
116.5
55.3
55.3
Expected
Rangeb
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
4
4
4
4
2
2
4
4
4
4
4
4
2
2
User
Range
Reading
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
4
2
2
2
3
3
3
3
3
2
2
Temperature
(°F)
94.6
94.6
93.5
93.5
89.6
89.6
88.7
88.7
91.1
91.1
89.1
89.1
00 0
OO.O
88.8
88.1
88.1
77.5
77.5
76.5
76.5
101.6
101.6
100.1
100.1
100.7
100.7
102.3
102.3
100.5
100.5
100.4
100.4
100.5
100.5
97.1
97.1
102.0
102.0
102.0
102.0
107.0
107.0
102.0
102.0
99.1
99.1
99.0
99.0
88.0
88.0
RH
(%)
20.2
20.2
22.2
22.2
28.6
28.6
28.4
28.4
24.1
24.1
29.9
29.9
28.4
28.4
28.8
28.8
48.3
48.3
49.9
49.9
9.3
9.3
9.5
9.5
9.2
9.2
7.9
7.9
19.5
19.5
19.1
19.1
19.9
19.9
23.9
23.9
21.9
21.9
22.0
22.0
24.9
24.9
28.2
28.2
32.5
32.5
31.1
31.1
38.0
38.0
Wind
Speed
(mph)
3.7
3.7
5.9
5.9
5.1
5.1
9.1
9.1
1.6
1.6
4.9
4.9
4.4
4.4
9.7
9.7
7.4
7.4
8.6
8.6
2.3
2.3
5.1
5.1
7.3
7.3
2.5
2.5
0.5
0.5
3.2
3.2
1.5
1.5
3.4
3.4
6.0
6.0
6.0
6.0
6.0
6.0
3.0
3.0
9.0
9.0
7.0
7.0
8.0
8.0
33
-------
Card
Number
20
20
21
21
21
21
22
22
22
22
23
23
23
23
24
24
24
24
25
25
25
25
26
26
26
26
27
27
27
27
28
28
28
28
29
29
29
29
30
30
30
30
31
31
31
31
32
32
32
32
Spot
Number
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
FEM Ozone
Concentration
(ppbv)
79.4
79.4
56.3
56.3
69.5
69.5
81.5
81.5
93.0
93.0
91.0
91.0
99.5
99.5
41.0
41.0
48.0
48.0
42.0
42.0
40.5
40.5
54.5
54.5
47.5
47.5
44.5
44.5
63.5
63.5
43.5
43.5
49.5
49.5
54.0
54.0
55.5
55.5
67.0
67.0
59.0
59.0
69.0
69.0
57.0
57.0
47.5
47.5
50.5
50.5
Expected
Rangeb
3
3
2
2
2
2
3
3
3
3
3
3
3
3
1
1
2
2
1
1
1
1
2
2
2
2
1
1
2
2
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
User
Range
Reading
3
3
2
2
2
2
3
3
2
2
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Temperature
(°F)
92.0
92.0
79.0
79.0
87.0
87.0
95.0
95.0
94.0
94.0
98.0
98.0
97.0
97.0
68.0
68.0
69.0
69.0
68.0
68.0
67.0
67.0
74.0
74.0
73.0
73.0
79.0
79.0
82.0
82.0
90.0
90.0
89.0
89.0
86.0
86.0
85.0
85.0
87.0
87.0
86.0
86.0
85.0
85.0
84.0
84.0
73.0
73.0
72.0
72.0
RH
(%)
31.0
31.0
48.0
48.0
36.0
36.0
28.8
28.8
29.4
29.4
24.0
24.0
28.0
28.0
77.0
77.0
74.0
74.0
67.0
67.0
68.0
68.0
52.0
52.0
56.0
56.0
50.0
50.0
42.0
42.0
33.0
33.0
32.0
32.0
36.0
36.0
38.0
38.0
32.0
32.0
38.0
38.0
36.0
36.0
41.0
41.0
64.0
64.0
65.0
65.0
Wind
Speed
(mph)
7.0
7.0
5.0
5.0
4.0
4.0
7.0
7.0
10.0
10.0
7.0
7.0
9.0
9.0
4.0
4.0
3.0
3.0
4.0
4.0
4.0
4.0
2.0
2.0
8.0
8.0
5.0
5.0
7.0
7.0
5.0
5.0
7.0
7.0
6.0
6.0
7.0
7.0
5.0
5.0
6.0
6.0
7.0
7.0
8.0
8.0
6.0
6.0
5.0
5.0
34
-------
Card
Number
33
33
33
33
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
34
34
34
34
35
35
35
35
36
36
36
36
37
37
37
37
38
38
38
38
39
39
39
39
40
40
40
40
41
41
Spot
Number
1
2
3
4
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
FEM Ozone
Concentration
(ppbv)
36.0
36.0
38.0
38.0
65.0
65.0
75.0
75.0
78.0
78.0
71.0
71.0
60.5
60.5
68.0
68.0
51.0
51.0
63.5
63.5
101.6
101.6
89.4
89.4
69.7
69.7
70.7
70.7
80.9
80.9
80.6
80.6
75.7
75.7
97.3
97.3
81.7
81.7
77.4
77.4
70.5
70.5
76.6
76.6
72.3
72.3
75.2
75.2
86.1
86.1
Expected
Rangeb
1
1
1
1
2
2
3
3
3
3
2
2
2
2
2
2
2
2
2
2
3
3
3
3
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
2
2
3
3
2
2
3
3
3
3
User
Range
Reading
2
2
2
2
2
2
3
3
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
2
2
3
3
3
3
Temperature
(°F)
66.0
66.0
66.0
66.0
84.0
84.0
85.0
85.0
98.0
98.0
99.0
99.0
97.0
97.0
95.5
95.5
93.0
93.0
93.0
93.0
90.9
90.9
89.5
89.5
87.0
87.0
86.1
86.1
88.3
88.3
87.8
87.8
86.5
86.5
87.3
87.3
86.5
86.5
86.0
86.0
72.1
72.1
73.2
73.2
76.6
76.6
77.8
77.8
81.0
81.0
RH
(%)
71.0
71.0
71.0
71.0
39.0
39.0
37.0
37.0
18.0
18.0
17.0
17.0
28.0
28.0
30.5
30.5
39.5
39.5
37.0
37.0
19.0
19.0
20.3
20.3
29.2
29.2
32.1
32.1
30.5
30.5
34.8
34.8
32.8
32.8
31.0
31.0
32.5
32.5
33.7
33.7
37.3
37.3
37.1
37.1
27.2
27.2
25.9
25.9
23.4
23.4
Wind
Speed
(mph)
4.0
4.0
4.0
4.0
6.0
6.0
7.0
7.0
5.0
5.0
4.0
4.0
6.0
6.0
7.0
7.0
6.0
6.0
5.0
5.0
9.6
9.6
8.8
8.8
4.5
4.5
3.4
3.4
3.3
3.3
3.7
3.7
2.6
2.6
6.8
6.8
8.7
8.7
10.2
10.2
8.1
8.1
5.6
5.6
2.6
2.6
2.7
2.7
3.3
3.3
35
-------
Card
Number
41
41
42
42
42
42
43
43
43
43
44
44
44
44
45
45
45
45
46
46
46
46
47
47
47
47
48
48
48
48
49
49
49
49
50
50
50
50
34
35
36
37
38
39
40
41
42
43
44
45
Spot
Number
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
5
5
5
5
5
5
5
5
5
5
5
5
FEM Ozone
Concentration
(ppbv)
82.2
82.2
82.4
82.4
82.7
82.7
81.1
81.1
89.6
89.6
63.6
63.6
65.9
65.9
40.2
40.2
42.2
42.2
75.6
75.6
77.5
77.5
31.7
31.7
32.0
32.0
34.2
34.2
34.4
34.4
27.7
27.7
28.9
28.9
31.0
31.0
36.6
36.6
83.6
83.6
81.0
81.0
53.0
53.0
50.8
50.8
51.5
51.5
51.8
51.8
Expected
Rangeb
3
3
3
3
3
3
3
3
3
3
2
2
2
2
1
1
1
1
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
User
Range
Reading
3
3
3
2
2
2
2
2
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
2
2
2
2
2
2
2
2
Temperature
(°F)
81.2
81.2
86.8
86.8
87.1
87.1
87.4
87.4
87.2
87.2
73.4
73.4
73.6
73.6
52.0
52.0
52.7
52.7
68.9
68.9
69.9
69.9
69.7
69.7
70.3
70.3
72.2
72.2
72.8
72.8
63.3
63.3
64.0
64.0
68.7
68.7
69.0
69.0
80.2
80.2
78.4
78.4
73.5
73.5
72.2
72.2
69.9
69.9
73.8
73.8
RH
(%)
19.1
19.1
9.2
9.2
8.5
8.5
11.4
11.4
12.1
12.1
45.2
45.2
41.4
41.4
86.0
86.0
87.5
87.5
39.8
39.8
37.2
37.2
45.0
45.0
43.2
43.2
36.6
36.6
34.5
34.5
48.1
48.1
48.0
48.0
31.8
31.8
34.5
34.5
30.8
30.8
31.7
31.7
50.4
50.4
51.5
51.5
47.2
47.2
41.2
41.2
Wind
Speed
(mph)
4.6
4.6
3.8
3.8
5.1
5.1
6.5
6.5
7.6
7.6
7.0
7.0
7.8
7.8
6.2
6.2
8.5
8.5
6.8
6.8
4.6
4.6
3.4
3.4
4.0
4.0
7.8
7.8
12.4
12.4
4.5
4.5
3.7
3.7
3.6
3.6
8.8
8.8
6.5
6.5
4.1
4.1
4.8
4.8
6.3
6.3
1.7
1.7
3.8
3.8
36
-------
Card
Number
46
47
48
49
50
Spot
Number
5
5
5
5
5
FEM Ozone
Concentration
(ppbv)
39.9
39.9
35.4
35.4
35.4
Expected
Rangeb
User
Range
Reading
2
2
—
—
-
Temperature
(°F)
44.7
44.7
44.4
44.4
44.4
RH
(%)
99.5
99.5
99.5
99.5
99.5
Wind
Speed
(mph)
10.5
10.5
8.1
8.1
8.1
a: Data in this table are original measurements reported to the number of digits representative of each monitor's
accuracy, so an inconsistent number of significant figures are presented.
b: Based on FEM ozone concentration reading.
Table 6-17. Ozone Detector Card Accuracy in Fall Season 2010 SCAQMD Testing
by Expected Card Reading Category
Expected Ozone
Detector Card
Reading"
1
2
O
4
Total all ranges
Number of Casesb
36
106
94
10
246
Number of Cases with
Agreement of User
and Expected
Readings'5
6
104
61
1
172
Accuracy (%)
16.7
98.1
64.9
10.0
69.9
a: Expected range based on simultaneous average FEM reading
b: From data in Table 6-16.
6.2.2 Duplication
Most of the fall season 2010 SCAQMD field data in Table 6-16 were obtained by exposing
two reagent spots on a single Ozone Detector Card simultaneously. Those tests always used
reagent spots numbered 1 and 2, or 3 and 4, and provide data to assess intra-card duplication.
Some data were also obtained by exposing a single reagent spot on each of two different
Ozone Detector Cards simultaneously. Those tests always used reagent spots numbered 5,
and provide data to assess inter-card duplication. No data were collected in which the
exposed reagent spots were read by more than one user, so the fall 2010 SCAQMD data do
not provide any information on user agreement.
A total of 100 intra-card comparisons are shown in Table 6-16, and of that total three
comparisons show disagreement between user readings for two reagent spots on the same
card exposed simultaneously. Consequently, intra-card duplication was 97% in the fall
season 2010 SCAQMD testing. Of those three cases of disagreement, one (card number 42,
spots 1 and 2) occurred when the user recorded readings of 2 and 3 for the two spots when a
reading of 3 would have been expected based on the FEM reading. The other two cases (card
17, spots 3 and 4, and card 18, spots 3 and 4) both occurred when a reading of 4 would have
been expected based on the FEM reading. Those four spots produced readings of 2 and 3,
37
-------
and 3 and 4, respectively, illustrating that the Ozone Detector Cards did not consistently give
a reading of 4 when ambient ozone levels should have produced such a reading. Overall,
intra-card duplication was 100% when a range reading of 1 or 2 was expected, 97.3% when a
range reading of 3 was expected, and 60% (3 agreements in 5 cases) when a range reading of
4 was expected.
Table 6-16 lists 23 cases in which the reagent spots numbered 5 on two different Ozone
Detector Cards were exposed simultaneously and visual readings were recorded. Of those 23
cases (consisting of 1 case, 16 cases, and 6 cases with the expected range reading of 1, 2, and
3, respectively), there were none in which different readings were obtained from the two
reagent spots. Thus inter-card duplication was 100% in the fall 2010 SCAQMD testing.
6.2.3 Effect of Ambient Conditions
In assessing the effect of ambient conditions on the accuracy of Ozone Detector Card
readings in the SCAQMD field testing, it must be recognized that Ozone Detector Card
accuracy varies with the ozone level present (see Tables 6-9 and 6-17) and that ambient
ozone levels are clearly dependent on the ambient meteorological conditions. For example,
the observed FEM ozone readings in the fall season 2010 testing were significantly positively
correlated with ambient temperature (r2 = 0.53, by linear regression). Consequently, the
effect of ambient conditions is first evaluated using the entire data set, and then similarly
evaluated by categorizing the data by expected Ozone Detector Card reading based on the
FEM readings.
Considering the fall season 2010 SCAQMD data set as a whole, a comparison is shown in
Table 6-18 of the average meteorological conditions for the 172 cases of accurate Ozone
Detector Card readings, and for the 74 cases of inaccurate Ozone Detector Card readings.
Table 6-18. Comparison of Meteorological Conditions with Accurate and Inaccurate
Ozone Detector Card Readings in Fall Season 2010 SCAQMD Field Testing
Card Accuracy
Accurate
Inaccurate
Number of
Cases
172
74
Significant Difference13
Temperature
(°F)a
86.6 (± 11.1)
81.5 (± 15.9)
Y
Relative
Humidity (%)a
33.1 (± 13.9)
39.6 (±23.0)
Y
Wind Speed"
5. 8 (±2.4)
6.2 (± 2.4)
N
a: Mean (± standard deviation) shown.
b: Based on Mest.
Table 6-18 shows that on average the ambient temperature was significantly higher, and the
ambient RH significantly lower, during the exposure periods that resulted in accurate Ozone
Detector Card readings relative to those periods that resulted in inaccurate readings.
However, the variability of both temperature and RH was substantially greater in the dataset
of inaccurate results than in the dataset of accurate results. The ranges of conditions overlap
greatly, and it is not clear that ambient conditions strongly determined the accuracy of Ozone
Detector Card readings. There was no significant difference in ambient wind speed between
the two sets of results. These comparisons are explored further in Table 6-19, which breaks
38
-------
out the comparisons of Table 6-18 according to the category of the expected Ozone Detector
Card range (i.e., 1, 2, 3, or 4) based on the simultaneous FEM reading.
Table 6-19. Comparison of Meteorological Conditions with Accurate and Inaccurate
Ozone Detector Card Readings by Expected Ozone Card Reading
Expected
Card
Reading
1
Card
Accuracy
Accurate
Inaccurate
Number of
Cases
6b
30
Significant Difference0
2
Accurate
Inaccurate
104
2d
Significant Difference6
3
Accurate
Inaccurate
61
33
Significant Difference
4
Accurate
Inaccurate
1
9
Significant Difference6
Temperature
(°F)a
65.3 (±2.6)
67.2 (± 10.8)
N
84.7 (± 10.4)
72.1 (0)
N
91.6 (±9.1)
89.8 (± 9.8)
N
102.0
100.7 (± 1.6)
N
Relative
Humidity
(%)a
42.6 (± 8.4)
60.3 (±21.7)
Y
37.3 (± 15.0)
37.3 (0)
N
25.1 (±7.3)
24.1 (±9.4)
N
22.0
27.7 (± 4.6)
N
Wind
Speed
(mph)a
3. 9 (±0.4)
6.1 (±2.8)
Y
5. 5 (±2.0)
8.1(0)
N
6.7 (± 2.8)
6.2 (± 2.3)
N
6.0
6.2 (± 2.2)
N
a: Mean (± standard deviation) shown.
b: Confidence intervals (95%) based on treatment of ranges of data with Equation 5 are 65.3 (± 2.2) °F, 42.6 (±
6.5) %RH, and 3.9 (±0.36) mph. Significant difference also found for temperature when range of data is used.
c: Significant differences based on Mest, unless otherwise indicated.
d: Two readings taken simultaneously, therefore meteorological data are identical.
e: Differences judged significant if single value from minority result differs from mean of majority results by
more than two standard deviations of majority results.
The breakdown in Table 6-19 is limited by the small number of data points in some
categories, but does not show any consistent effects from the ambient conditions across the
reading categories. Significant effects from RH and wind speed are found for readings with
an expected level of 1, but those effects are not seen in the other categories. Overall, the
results in Tables 6-18 and 6-19 do not indicate that Ozone Detector Card readings are
affected by the ambient temperature, RH, or wind speed over the range of conditions
encountered.
6.3 BCLA Field Results
Table 6-20 summarizes the Ozone Detector Card results obtained by BCLA volunteers in
field testing in the winter and spring seasons of 2009-2010. In this test period, BCLA
39
-------
volunteers used the Ozone Detector Cards at elementary and middle schools, and at offices,
and in both outdoor and indoor air.
Table 6-20. BCLA Field Results, 12/18/09 to 5/13/10
Card
Number
BC-19
BC-19
BC-68
BC-68
BC-68
BC-69
BC-69
BC-69
BC-69
BC-69
BC-70
BC-70
BC-53
BC-12
BC-14
BC-14
BC-14
BC-14
BC-19
BC-53
BC-33
BC-33
BC-33
BC-33
BC-33
BC-32
BC-32
BC-32
BC-32
BC-32
BC-19
BC-12
BC-19
BC-53
BC-12
BC-53
BC-11
BC-11
BC-11
Spot
Number
1
2
1
2
4
1
2
3
4
5
1
2
1
1
1
2
3
4
3
2
1
2
3
4
5
1
2
3
4
5
4
2
5
3
3
4
1
2
3
User 1
Reading
2
2
3
3
4
4
3
2
3
3
3
3
4
4
2
2
2
2
3
4
3
2
2
3
4
4
4
3
2
4
3
3
2
4
3
4
3
—
3
User 2
Reading
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
—
—
—
—
—
—
—
—
—
3
-
40
-------
Card
Number
BC-11
BC-11
BC-12
BC-53
BC-12
BC-52
BC-52
BC-52
BC-52
BC-52
BC-66
BC-66
BC-66
BC-66
BC-66
BC-67
BC-67
BC-67
BC-67
BC-67
BC-54
BC-54
BC-54
BC-54
BC-54
Spot
Number
4
5
4
5
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
User 1
Reading
-
3
4
3
4
4
4
3
3
3
2
3
3
2
3
2
3
3
3
4
3
3
2
2
3
User 2
Reading
3
-
-
-
-
-
-
-
-
-
2
3
3
2
3
2
3
3
3
4
—
—
—
—
-
6.3.1 Duplication
The 2009-2010 BCLA field data summarized in Table 6-20 include no cases in which
multiple reagent spots on the same Ozone Detector Card were exposed simultaneously. As a
result, no assessment of intra-card duplication could be made with those data. Similarly, no
cases could be identified in which reagent spots on different Ozone Detector Cards were
exposed simultaneously and in the same location. Consequently, no assessment of inter-card
duplication could be made. However, Table 6-21 shows 10 cases in which the same exposed
reagent spot was read by two different users. In all 10 of those cases the readings from the
two users were in agreement, resulting in user duplication of 100%.
6.4 Operational Factors
During the laboratory portion of the testing, the following observations were made on the
operational factors of the Ozone Detector Cards.
The two Battelle staff members who served as card readers reported having a difficult time
41
-------
matching the color change in the reagent spot following exposure to the colors on the
reference color wheel printed on the Ozone Detector Cards. Common observations were that
the hue of the reference color wheel did not match the actual color change in the reagent spot.
The color change of the spot was described as more yellow in color than the reference color
wheel (at low ozone concentrations), or more red than the reference color wheel (at high
ozone concentrations). Additionally, both readers remarked that the color change was often
"in between" the colors of the reference color wheel. Ozone ranges 1, 2, and 3 are each
represented on the reference color wheel with a single color while ozone range 4 is
represented by a color gradient. Users often described the color change in a reagent spot as
being (e.g.) darker than the 1 range, but not as dark as the 2 range.
The two Battelle staff members who served as card readers rarely read reagent spots on the
Ozone Detector Cards as being in the 4 range (indicating > 105 ppbv ozone) even when the
ozone concentration in the exposure chamber was elevated to 200 ppbv or more. These high
concentration tests were the source of many comments from the two readers about the color
of the spot not matching the color of the reference color wheel. The infrequent reporting of a
range of 4, even when the delivered ozone concentrations greatly exceeded the nominal
threshold for that range, is noteworthy.
As noted in Section 3.2.1, the presence of the Ozone Detector Cards in the test chamber
during laboratory testing markedly reduced the ozone concentration in the delivered air
stream. This effect persisted as long as the cards remained in the test chamber, and was the
same whether reagent spots on the cards were covered or uncovered. This observation
suggests that the cardboard material of the cards themselves absorbed or reacted with ozone.
This characteristic of the Ozone Detector Cards could potentially introduce bias in the
measurement of ambient ozone, especially under conditions of still air. No investigation was
done in this verification of what range of wind speeds would be needed to minimize this
ozone removal effect. As noted in Section 3.2.1, air speeds through the test chamber in the
laboratory testing were low, i.e., approximately 1 cm/s (0.6 m/min (2 ft/min)).
Prior to use, all the Ozone Detector Cards were stored as received from the manufacturer,
i.e., in individual plastic sleeves inside a cardboard box. The light intensity testing was the
final laboratory test procedure, and was conducted with cards which had been stored for
approximately three months after receipt at Battelle. This time period is one-quarter of the
nominal 1 year shelf life of the cards stated by the vendor, however the length of storage
before the cards were shipped to Battelle is unknown. With a few of those cards it was
observed that reagent spots already exhibited a slight color change when the foil covering
was first removed (i.e., with no exposure to ozone), though no problem with the foil covering
was apparent. Reagent spots that had been nearest the open end of the card's plastic sleeve
during storage were more likely to be discolored upon removal of the foil than reagent spots
located farther away from the sleeve opening. This observation suggests that protection of
the reagent spots during storage in individual plastic sleeves is not absolute. A similar
observation was made by SCAQMD in initial field testing of the Ozone Detector Cards (see
Appendix A). This observation indicates that users should occasionally check the color of
unexposed reagent spots during use of the Ozone Detector Cards.
42
-------
During the field portions of the testing by SCAQMD and BCLA, the following observations
were made on the operational factors of the Ozone Detector Cards.
SCAQMD field operators noted difficulty in comparing the color change on the reagent spots
to the reference color wheel on the Ozone Detector Cards. Initially, during the fall season
2009 field period SCAQMD users reacted to this difficulty by reporting readings that were
halfway between ranges on the Ozone Detector Cards (e.g., a reading of 2.5; see discussion
in Appendix A). In the fall season 2010 field period, users noted that reagent spots exposed
outdoors at the SCAQMD sites could appear to give different readings when read under
different lighting conditions. For example, at the Upland site on September 1, 2010, the user
recorded on the data sheet: "Spots 1 and 2 borderline 2 in sunlight, 3 in shade." The
SCAQMD QAM also noted examples of this issue in conducting the ISA of SCAQMD
activities, and took photographs under differing conditions to illustrate the differing
appearance of reagent spots. The instructions printed on each Ozone Detector Card do not
call for standardization of lighting conditions when reading the reagent spot color. While
those instructions implicitly assume that the appearance of the reference color wheel and an
exposed reagent spot will change in the same way with the ambient lighting conditions, the
user comments suggest that that is not the case. It may be that the material or surface finish
of the printed color wheel cause it to absorb or reflect the ambient light differently from an
exposed reagent spot, causing a relative shift in the apparent color of the spot depending on
ambient conditions.
BCLA field operators noted two operational factors consistent with comments in other parts
of this verification. It was noted that the color change on reagent spots was hard to read, and
the visual reading could fall into either of two adjacent ranges on the card. It was also noted
that in one case a reagent spot already showed color development when its foil covering was
first removed.
After reviewing the draft of this report the vendor of the Ozone Detector Cards provided
comments on operational features of the cards. Those comments are summarized below.
Regarding user comments that the hue of the reagent spot did not match the color of the
reagent spot, the vendor noted that the exact hue of the exposed reagent spot will vary due to
the presence in air of trace reactive species other than ozone. The vendor suggested that
printing a white ring between the inner rim of the color indicator wheel and the reagent spot
would spatially separate the two, and might reduce the effect of differences in hue on the
visual reading of ozone range.
Regarding user comments that visual readings fell between two adjacent ranges on the
reference color wheel, the vendor noted that this will be the case for most readings. The
vendor stated that the Ozone Detector Card is a tool for estimating the ozone level, and not a
measuring instrument. According to the vendor, using a continuous scale of graduated color
would not be desirable as it would suggest an accuracy for the card's readings that is not real.
The vendor commented that the absorption of ozone by the material that the cards are made
of is not surprising, as ozone is known to be destroyed by reaction with many substances.
43
-------
However, the vendor indicated that coating the cards with a more inert substance such as
Teflon® is not possible. A potential solution might be to provide a Teflon mask to enclose
the card, with a hole held over the exposed spot during exposures.
Regarding color development in reagent spots during long-term storage prior to use, the
vendor indicated that gradual oxidation of the reagent spots by atmospheric oxygen causes
this color production. Although ozone is destroyed by contact with the cardboard material of
the card, oxygen is not, and may penetrate the cardboard to cause the color development.
The vendor suggested that airtight storage of the cards might minimize this effect.
Alternatively, sealing individual cards in oxygen-impermeable wrappings might minimize
this oxidation.
Regarding the effect of different lighting conditions on the visual reading of exposed reagent
spots, the vendor suggested that instructions could be added to the cards requiring that visual
readings always be made in indirect sunlight (i.e., in shade). To assure consistency of indoor
and outdoor readings, "indirect sunlight" might mean in shade outdoors and near a window
indoors.
Finally, regarding the cost of the Ozone Indicator Cards, the vendor's standard price list
shows a cost per card of approximately $1.60 per card when purchased in packages of 100,
with lower prices per card for larger quantities. Individual cards can also be purchased from
the vendor at a cost of approximately $3.00 per card.
44
-------
Chapter 7
Performance Summary
Table 7-1 summarizes the performance observed from the Ozone Detector Cards in the
laboratory and field testing. Shown in Table 7-1 are the results for each performance
parameter determined in the laboratory testing, and in the field testing performed by
SCAQMD and by BCLA. When ozone levels are known and sufficient Ozone Detector Card
data are available, performance results are broken down by ozone level, in terms of the
expected Ozone Detector Card range (1, 2, 3, or 4) that corresponds to the ozone level.
Table 7-1 shows that in both laboratory and field testing the Ozone Detector Cards
exhibited lower accuracy and duplication of readings at ozone concentrations
corresponding to the highest range reading of the cards (i.e., range 4). In laboratory
testing the accuracy, variability, and duplication of the Ozone Detector Cards were
sometimes worse when exposed to an ozone concentration near the boundary of a
detection range than when exposed to a concentration near the middle of a range, but this
effect was not consistently observed. The Ozone Detector Cards are relatively
inexpensive (approximately $1.60 or less per card, when purchased in lots of 100 or
more). Users reported the cards were easy to use, but users had difficulty in matching the
color developed in the reagent spots with the color index printed on the cards. This
difficulty may have contributed to the observed variability of card readings (e.g., in
laboratory testing the reported card readings fell into three different card ranges
regardless of the expected range reading). Bright simulated sunlight during laboratory
ozone exposure, and temperature, RH, and wind speed during field use had little effect on
Ozone Detector Card accuracy, but users reported that the lighting conditions under
which exposed reagent spots were read could affect their reported readings.
Additional performance information obtained in initial SCAQMD field testing is included in
Appendix A.
45
-------
Table 7-1. Performance Summary for Ozone Indicator Cards
Performance
Parameter
Accuracy (%)a
Variability (number of
card ranges reported (4
ranges is maximum
variability))3
Intra-Card Duplication
(%)•
Inter-Card Duplication
(%)"
User Agreement (%)a
Effect of Light
Intensity on Color
Development during
Ozone Exposure
Effect of Ambient
Conditions on
Accuracy
Operational Factors
Testing Effort
Laboratory
Range" 1: 95.6
2: 64.1
3: 7.8
4: 3.1
Range 1: 3
2: 3
3: 3
4: 3
Range 1: 93.8
2: 84.4
3: 93.8
4: 76.6
Range 1: 91.7
2: 59.4
3: 85.4
4: 47.4
Range 1: 91.2
2: 53.1
3: 81.3
4: 54.7
No effect on accuracy,
variability, or user
agreement; reduced
intra-and inter-card
duplication with Bright
light condition
NA
Easy to use; Difficult
to match reagent spot
color to color wheel;
Presence of cards
reduces ozone in test
chamber; Development
of color in reagent
spots during storage of
cards
SCAQMD Field
Range 1: 16.7
2: 98.1
3: 64.9
4: 10.0
NA
Range 1: 100
2: 100
3: 97.3
4: 60.0
100d
NA
NA
Full data set suggests
higher accuracy with
higher temperature and
lower RH; breakdown
by expected range
shows no conclusive
effects.
Easy to use; Difficult
to match reagent spot
color to color wheel;
Visual reading depends
on ambient lighting
conditions; Develop-
ment of color in
reagent spots during
storage of cards
BCLA Field
NAC
NA
NA
NA
100e
NA
NA
Easy to use; Difficult
to match reagent spot
color to color wheel;
Development of color
in one reagent spot
during storage of cards
a: Performance shown for expected ranges of card readings, when information is available.
b: Range 1 = 10 to 45 ppbv; Range 2 = 45 to 75 ppbv; Range 3 = 75 to 105 ppbv; Range 4 = >105 ppbv.
c: NA = not applicable as no data on this parameter were collected from this effort.
d: Based on 23 total cases (insufficient data for breakdown by expected range).
e: Based on 10 total cases.
46
-------
Chapter 8
References
1. Federal Reference and Equivalent Methods established by Code of Federal
Regulations, Title 40, Part 53; a list of commercial devices designated as Reference
or Equivalent Methods as of August 24, 2010 is available at
http://www.epa.gov/ttn/amtic/criteria.html.
2. Spicer, C.W., D.W. Joseph, and W.M. Ollison, J. Air Waste Manage. Assoc., 60,
1353-1364,2010.
3. Test/QA Plan for Verification of Ozone Indicator Cards, Version 1, prepared by
Battelle, Columbus, Ohio, November 2009. Available at
http://www.epa.gov/nrmrl/std/etv/vt-ams.htmltfoic.
4. Quality Management Plan (QMP)for the ETV Advanced Monitoring Systems Center,
Version 7.0, Environmental Technology Verification Program, Battelle, November
2008.
5. Statistics in Quantitative Analysis, in Chemical Analysis, H.A. Laitinen, McGraw-
Hill, New York, 1960.
6. Ultraviolet Radiation and Human Health, Fact Sheet No. 305, World Health
Organization, Geneva, Switzerland, December 2009.
47
-------
Appendix A
Results from Initial Field Tests of Ozone Detector Cards by SCAQMD
October and November, 2009
Al.l Field Data
Table Al-1 shows the ozone card indications and associated data obtained at SCAQMD field
sites in initial field tests in the fall of 2009. These data were collected prior to final EPA
approval of the test/QA plan, so they are considered to be supplemental data to the
verification test. These initial tests were conducted between October 1 and November 2,
2009, at SCAQMD's Crestline, Riverside, and Santa Clarita monitoring sites. All card
exposures were made during daylight hours, and almost entirely between 10:00 am and 5:00
pm, consistent with normal schedules for the testing personnel. Ambient ozone levels were
always below 65 ppbv, and usually below 50 ppbv. Table Al-1 shows the Ozone Detector
Card number and spot number, the average FEM ozone concentration measured during the
Ozone Detector Card exposure, the expected Ozone Detector Card range based on the FEM
reading, the Ozone Detector Card ranges reported by the SCAQMD user(s) of each card, and
the ambient temperature, RH, and wind speed during the exposure. Because of the limited
range of ambient ozone levels, the expected Ozone Detector Card ranges based on the FEM
readings were always either 1 or 2. Cards and reagent spots that were exposed
simultaneously can be identified in Table Al-1 by the identical values for FEM ozone,
temperature, RH, and wind speed. Cases in which the user's visual reading does not match
the expected range are shaded gray in Table Al-1. In several cases, one SCAQMD user
recorded range indications as 2.5 when he found it difficult to choose between ranges 2 and
3. Those readings are shown as differing from the expected range only when the difference
is greater than 1 full range (i.e., any difference of 0.5 range is not considered significant).
Table Al-1 shows that the large majority of the Ozone Detector Card ranges reported by
SCAQMD personnel in these initial tests were higher than the expected range based on the
corresponding FEM ozone reading. In some cases the user readings were two ranges higher
than the expected range based on the FEM data (i.e., a reading of 3 when a reading of 1 was
expected). This observation raised concern that the reagent spots may have undergone a
color change during storage that artificially elevated the visual reading, i.e., that a color
change may have already occurred before the reagent spots were exposed to ambient air. To
explore this hypothesis, SCAQMD personnel were instructed to remove the foil from unused
spots on the Ozone Indicator Cards, and to immediately record their visual readings of the
color intensity before any exposure to ozone. A total of 17 such unexposed spots were read
by SCAQMD staff on November 2, 2009, and the resulting readings are shown in Table Al-
2. The results in Table Al-2 confirm that unexpected color development occurred in the
Ozone Indicator Cards, as all 17 unexposed spots showed a range of 2 when read by
SCAQMD testing personnel. As a result of the data in Table Al-2, subsequent SCAQMD
field testing included checks of unexposed reagent spots to assess color with no exposure to ozone.
48
-------
Table Al-1. SCAQMD Ozone Detector Card Data from Initial Field Testing.
Card
Number
SC07
SC07
SC08
SC08
SC07
SC07
SC08
SC08
SC13
SC13
SC41
SC41
SC13
SC13
SC41
SC41
SC25
SC25
SC26
SC26
SC25
SC25
SC26
SC26
SC27
SC27
SC28
SC28
SC27
SC27
SC28
SC28
SC42
SC42
SC49
SC49
SC42
SC42
SC49
SC49
SC29
SC29
SC30
SC30
SC29
SC29
SC30
SC30
SC43
SC43
SC44
SC44
SC43
SC43
SC44
Spot
Number
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
FEM Ozone
Concentration
(ppbv)
40.6
40.6
40.6
40.6
43.9
43.9
43.9
43.9
45.5
45.5
45.5
45.5
41.7
41.7
41.7
41.7
43.7
43.7
43.7
43.7
46.9
46.9
46.9
46.9
63.9
63.9
63.9
63.9
64.8
64.8
64.8
64.8
35.0
35.0
35.0
35.0
36.0
36.0
36.0
36.0
44.7
44.7
44.7
44.7
51.4
51.4
51.4
51.4
54.4
54.4
54.4
54.4
48.1
48.1
48.1
Expected
Range3
1
1
1
1
1
1
1
1
2
2
2
2
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
Userl
Range
Reading
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2
3
2
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
User 2
Range Temperature RH
Reading (°C) (%)
-
-
-
-
-
-
-
-
2
2
2
2
3
3
3
3
-
-
-
-
-
-
-
~
-
-
-
-
-
-
-
-
2
2
2
2
2
2
2
2
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
91.2
91.2
91.2
91.2
90.0
90.0
90.0
90.0
84.0
84.0
84.0
84.0
85.6
85.6
85.6
85.6
63.8
63.8
63.8
63.8
66.1
66.1
66.1
66.1
71.3
71.3
71.3
71.3
70.8
70.8
70.8
70.8
65.0
65.0
65.0
65.0
65.1
65.1
65.1
65.1
52.2
52.2
52.2
52.2
66.3
66.3
66.3
66.3
74.1
74.1
74.1
74.1
71.1
71.1
71.1
2
2
2
2
2
2
2
2
6
6
6
6
5
5
5
5
17
17
17
17
12
12
12
12
11
11
11
11
14
14
14
14
47
47
47
47
39
39
39
39
69
69
69
69
53
53
53
53
45
45
45
45
46
46
46
Wind
Speed
(mph)
3.0
3.0
3.0
3.0
12.5
12.5
12.5
12.5
10.0
10.0
10.0
10.0
8.0
8.0
8.0
8.0
3.4
3.4
3.4
3.4
4.4
4.4
4.4
4.4
3.6
3.6
3.6
3.6
3.3
3.3
3.3
3.3
6.8
6.8
6.8
6.8
3.4
3.4
3.4
3.4
5.1
5.1
5.1
5.1
8.0
8.0
8.0
8.0
15.1
15.1
15.1
15.1
12.8
12.8
12.8
49
-------
Card
Number
SC44
SC16
SC16
SC50
SC50
SC16
SC16
SC50
SC50
SC06
SC06
SC05
SC05
SC06
SC06
SC05
SC05
SCSI
SCSI
SC32
SC32
SCSI
SCSI
SC32
SC32
SC03
SC03
SC04
SC04
SC03
SC03
SC04
SC04
SC23
SC23
SC24
SC24
SC23
SC23
SC24
SC24
sen
sen
SC46
SC46
sen
sen
SC46
SC46
SC18
SC18
SC19
SC19
SC18
SC18
SC19
Spot
Number
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
FEM Ozone
Concentration
(ppbv)
48.1
32.5
32.5
32.5
32.5
36.7
36.7
36.7
36.7
55.1
55.1
55.1
55.1
54.9
54.9
54.9
54.9
2.0
2.0
2.0
2.0
8.0
8.0
8.0
8.0
41.0
41.0
41.0
41.0
40.4
40.4
40.4
40.4
45.0
45.0
45.0
45.0
44.8
44.8
44.8
44.8
36.0
36.0
36.0
36.0
37.8
37.8
37.8
37.8
41.2
41.2
41.2
41.2
39.2
39.2
39.2
Expected
Range"
2
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Userl
Range
Reading
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2.5
2.5
2.5
2.5
2.5
2.5
2.5
2.5
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
User 2
Range Temperature
Reading (°C)
71.1
68.1
68.1
68.1
68.1
67.0
67.0
67.0
67.0
78.2
78.2
78.2
78.2
77.7
77.7
77.7
77.7
41.1
41.1
41.1
41.1
51.8
51.8
51.8
51.8
97.3
97.3
97.3
97.3
98.2
98.2
98.2
98.2
97.7
97.7
97.7
97.7
97.8
97.8
97.8
97.8
67.0
67.0
67.0
67.0
67.2
67.2
67.2
67.2
75.4
75.4
75.4
75.4
75.0
75.0
75.0
RH
(%)
46
65
65
65
65
66
66
66
66
43
43
43
43
42
42
42
42
75
75
75
75
53
53
53
53
17
17
17
17
15
15
15
15
13
13
13
13
12
12
12
12
68
68
68
68
64
64
64
64
41
41
41
41
44
44
44
Wind
Speed
(mph)
12.8
11.5
11.5
11.5
11.5
12.9
12.9
12.9
12.9
3.9
3.9
3.9
3.9
7.0
7.0
7.0
7.0
0.3
0.3
0.3
0.3
1.1
1.1
1.1
1.1
5.0
5.0
5.0
5.0
7.1
7.1
7.1
7.1
13.5
13.5
13.5
13.5
10.0
10.0
10.0
10.0
8.6
8.6
8.6
8.6
10.6
10.6
10.6
10.6
4.5
4.5
4.5
4.5
5.7
5.7
5.7
50
-------
Card
Number
SC19
SC33
SC33
SC34
SC34
SC33
SC33
SC34
SC34
SC21
SC21
SC22
SC22
SC21
SC21
SC22
SC22
SC35
SC35
SC36
SC36
SC35
SC35
SC36
SC36
SC09
SC09
SC47
SC47
SC09
SC09
SC47
SC47
SC01
SC01
SC17
SC17
SC01
SC01
SC17
SC17
SC02
SC02
SC20
SC20
SC02
SC02
SC20
SC20
SC37
SC37
SC38
SC38
SC37
SC37
SC38
Spot
Number
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
FEM Ozone
Concentration
(ppbv)
39.2
45.0
45.0
45.0
45.0
46.5
46.5
46.5
46.5
40.3
40.3
40.3
40.3
40.7
40.7
40.7
40.7
42.9
42.9
42.9
42.9
48.3
48.3
48.3
48.3
49.1
49.1
49.1
49.1
51.8
51.8
51.8
51.8
53.4
53.4
53.4
53.4
51.3
51.3
51.3
51.3
49.0
49.0
49.0
49.0
50.0
50.0
50.0
50.0
49.5
49.5
49.5
49.5
52.5
52.5
52.5
Expected
Range"
1
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Userl
Range
Reading
3
3
3
3
3
3
3
3
3
2.5
2.5
2.5
2.5
2.5
2.5
2.5
2.5
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
^2
3
3
3
3
3
3
3
3
2.5
2.5
2.5
2.5
2.5
2.5
2.5
2.5
3
3
3
3
2
3
2
User 2
Range Temperature
Reading (°C)
75.0
56.1
56.1
56.1
56.1
58.4
58.4
58.4
58.4
83.0
83.0
83.0
83.0
82.3
82.3
82.3
82.3
61.4
61.4
61.4
61.4
66.0
66.0
66.0
66.0
82.1
82.1
82.1
82.1
79.8
79.8
79.8
79.8
88.8
88.8
88.8
88.8
85.3
85.3
85.3
85.3
90.6
90.6
90.6
90.6
89.9
89.9
89.9
89.9
71.4
71.4
71.4
71.4
72.5
72.5
72.5
RH
(%)
44
40
40
40
40
35
35
35
35
23
23
23
23
26
26
26
26
29
29
29
29
20
20
20
20
21
21
21
21
23
23
23
23
20
20
20
20
23
23
23
23
12
12
12
12
12
12
12
12
21
21
21
21
16
16
16
Wind
Speed
(mph)
5.7
6.5
6.5
6.5
6.5
8.0
8.0
8.0
8.0
4.8
4.8
4.8
4.8
7.0
7.0
7.0
7.0
3.2
3.2
3.2
3.2
2.8
2.8
2.8
2.8
14.3
14.3
14.3
14.3
14.6
14.6
14.6
14.6
3.0
3.0
3.0
3.0
7.7
7.7
7.7
7.7
8.6
8.6
8.6
8.6
5.4
5.4
5.4
5.4
3.2
3.2
3.2
3.2
3.5
3.5
3.5
51
-------
Card
Number
SC38
SC12
SC12
SC14
SC14
SC12
SC12
SC14
SC14
SC39
SC39
SC40
SC40
SC39
SC39
SC40
SC40
SC48
SC48
SC48
SC48
SC15
SC15
SC45
SC45
SC15
SC15
SC45
SC45
Spot
Number
4
1
2
1
2
3
4
3
4
1
2
1
2
3
4
3
4
1
2
3
4
1
2
1
2
3
4
3
4
FEM Ozone
Concentration
(ppbv)
52.5
40.3
40.3
40.3
40.3
37.6
37.6
37.6
37.6
45.9
45.9
45.9
45.9
46.0
46.0
46.0
46.0
36.6
36.6
35.1
35.1
36.6
36.6
36.6
36.6
35.1
35.1
35.1
35.1
Expected
Range"
2
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
Userl
Range
Reading
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
2
^2
2
2
2
2
2
2
2
2
2
2
2
2
User 2
Range Temperature
Reading (°C)
72.5
82.8
82.8
82.8
82.8
83.2
83.2
83.2
83.2
54.8
54.8
54.8
54.8
51.8
51.8
51.8
51.8
86.1
86.1
87.1
87.1
86.1
86.1
86.1
86.1
87.1
87.1
87.1
87.1
RH
(%)
16
5
5
5
5
4
4
4
4
32
32
32
32
35
35
35
35
12
12
11
11
12
12
12
12
11
11
11
11
Wind
Speed
(mph)
3.5
5.2
5.2
5.2
5.2
2.6
2.6
2.6
2.6
4.2
4.2
4.2
4.2
5.7
5.7
5.7
5.7
4.1
4.1
3.7
3.7
4.1
4.1
4.1
4.1
3.7
3.7
3.7
3.7
52
-------
Table Al-2. SCAQMD Readings of Ozone Indicator Reagent Spots
Immediately After Removing Protective Foil (No Exposure to Ozone)
Card
Number
SC09
sen
SC12
SC13
SC14
SC15
SC16
SC41
SC42
SC43
SC44
SC45
SC46
SC47
SC48
SC49
SC50
Spot
Number
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
Expected
Range"
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Userl
Range
Reading
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
a: Based on absence of ozone exposure. User readings that do not match
the expected range are shown in shaded cells.
A1.2 Accuracy
Accuracy of the Ozone Detector Cards was 0% when the expected range reading was 1(121
total reagent spots), and 54% when the expected range reading was 2 (92 total reagent spots).
This inaccuracy was likely caused by the bias introduced by the blank reagent spot color
development that is discussed in Section Al.l and shown in Table Al-2.
A1.3 Duplication
Ozone Detector Card duplication was assessed as described in Section 5.3. Intra-card
duplication was determined by comparing the readings made by a single user of two reagent
spots on a single Ozone Detector Card which had been exposed to ambient air over the same
time period. Table Al-1 includes 106 total cases in which a user read two reagent spots on
the same card that had been exposed simultaneously (98 cases for User 1 and 8 cases for
User 2). User 1 indicated different readings for the two reagent spots in 4 of the 106 total
cases. Therefore the intra-card duplication rate was 96.2%.
53
-------
Inter-card duplication was assessed by comparing readings from a single card user for
reagent spots on different Ozone Detector Cards exposed at the same time. In almost all the
initial SCAQMD field testing, two reagent spots on each of two cards (4 total reagent spots)
were exposed simultaneously. In those cases, each set of four spots results in 4 individual
comparisons for inter-card duplication. In total, Table Al-1 includes 192 such comparisons
from User 1 and 16 such comparisons from User 2. Of those 208 total comparisons for inter-
card duplication, there were 6 cases (all with User 1) in which different readings were
obtained with reagent spots exposed simultaneously on different cards. Therefore the inter-
card duplication rate was 97.1%.
User agreement in the initial SCAQMD field testing was also assessed as described in
Section 5.3. Table Al-1 shows that there were 16 cases in which two users read the same
reagent spots. In those 16 cases, the two users recorded the same reading in 12 cases.
Therefore the user duplication in the initial SCAQMD field testing was 75%.
A1.4 Effect of Ambient Conditions
The effect of ambient conditions (temperature, RH, wind speed) on Ozone Detector Card
readings could not be determined in the initial SCAQMD field testing. The apparent upward
bias in observed card readings, as described in Section Al.l, prevented assessment of the
effect of ambient conditions on the accuracy of the readings.
A1.5 Operational Factors
The SCAQMD field operators who used the Ozone Detector Cards in the initial field testing
had comments similar to those reported in Section 6.3 of this report. Although the Ozone
Detector Cards were easy to use, the field operators stated that the color of the reagent spots
after exposure to ozone did not match well with the colors of the color wheel printed on the
card. Operators noted the occurrence of spot colors that appeared to be in between the colors
of ranges on the color wheel. This occurrence is illustrated by the fact that one user recorded
readings of 2.5, rather than 2 or 3, in this initial testing (see Table Al.l).
The initial field testing demonstrated one potential problem with the Ozone Detector Cards
that was noted in Section 6.4: the possible development of color in reagent spots prior to
removal of the foil covering. Table Al-2 shows that reagent spots on several cards had
developed substantial color before their foil covering was removed. Whether this indicates a
lack of integrity in the foil covering, or a problem in the procedure of storing the cards in
their individual plastic sleeves in their original box, is not known.
54
------- |