Batteiie

The Bu sincss ii/ I iinov.it ion

Environmental Technology
Verification Program

Advanced Monitoring
Systems Center

Test/QA Plan for Verification of
Ozone Indicator Cards

ElVft/ ElV


-------
TEST/QA PLAN

for

Verification of
Ozone Indicator Cards

November 10, 2009

Prepared by

Battelle

505 King Avenue
Columbus, OH 43201-2693


-------
Ozone Indicator Cards
Test/QA Plan
Page 1 of 45
Version 1
November 10, 2009

SECTION A
PROJECT MANAGEMENT

A1 Vendor Approval Page

ETV Advanced Monitoring Systems Center
Test/QA Plan
for

Verification of
Ozone Indicator Cards

November 10, 2009

Oeoijc Indicator Cards
Test-QA Plan
Page J of45
Version I
November 10, 2009

VENDOR ACCEPTANCE:

Name

Company fcHiy/iVO

Ln. i i/l C

Date	OQ gj

Ayi

A_^|


-------
Ozone Indicator Cards
Test/QA Plan
Page 2 of 45
Version 1
November 10, 2009

A2 TABLE OF CONTENTS

Section	Page

A PROJECT MANAGEMENT	1

A2 Table of Contents	2

A3 Acronyms and Abbreviations	4

A4 Distribution List	5

A5 Verification Test Organization	6

A6 Background	12

A7 Verification Test Descrition and Schedule	14

A8 Quality Objectives	15

A9 Special Training/Certification	15

A10 Documentation and Records	16

B MEASUREMENT AM) DATA ACQUISITION	18

B1 Experimental Design	18

B2 Reference Sample Collection	30

B3 Sample Handling and Custody Requirements	31

B4 Reference Method	31

B5 Quality Control Requirements	31

B6 Instrument/Equipment Testing, Inspection, and Maintenance	32

B7 Instrument Calibration and Frequency	32

B8 Inspection/Acceptance of Supplies and Consumables	32

B9 Non-Direct Measurements	32

BIO Data Management	32

C ASSESSMENT AND OVERSIGHT	36

CI Assessments and Response Actions	36

C2 Reports to Management	38

D DATA VALIDATION AM) USABILITY	39

D1 Data Review, Validation, and Verification Requirements	39

D2 Validation and Verification Methods	39

D3 Reconciliation with User Requirements	39

E REFERENCES	41


-------
Ozone Indicator Cards
Test/QA Plan
Page 3 of 45
Version 1
November 10, 2009

Appendices	Page

APPENDIX A: Field Data Sheet for Use by SCAQMD	44

APPENDIX B: Field Data Sheet for Use by BCLA	46

Figure	Page

Figure 1. Organization Chart for the Verification Test	7

Figure 2. Examples of Ozone Indicator Cards	19

Tables	Page

Table 1. Planned Verification Schedule	16

Table 2. Summary of OIC Verification Samples	21

Table 3. Summary of Data Recording Process	34


-------
Ozone Indicator Cards
Test/QA Plan
Page 4 of 45
Version 1
November 10, 2009

A3 ACRONYMS AND ABBREVIATIONS

ADQ

audit of data quality

AMS

Advanced Monitoring Systems

BCLA

Breathe California of Los Angeles

EPA

U.S. Environmental Protection Agency

ETV

Environmental Technology Verification

FEM

Federal Equivalent Method

LRB

laboratory record book

Mn02

manganese dioxide

nm

nanometer

OIC

ozone indicator card

ppbv

parts per billion by volume

PE

performance evaluation

QA

quality assurance

QC

quality control

QMP

quality management plan

r2

coefficient of determination

RH

relative humidity

SCAQMD

South Coast Air Quality Management District

TSA

technical systems audit

T

temperature

UV

ultraviolet

VOC

volatile organic compounds


-------
Ozone Indicator Cards
Test/QA Plan
Page 5 of 45
Version 1
November 10, 2009

A4 DISTRIBUTION LIST

Vendors

Dieter F el db erg
EnviroScan, Inc.

689 Unionville Road
Kennett Square, PA 19348

EPA

Michelle Henderson

U.S. Environmental Protection Agency

ETAV/NRMRL/ORD

26 W. Martin Luther King Dr., MS 208

Cincinnati, OH 45268

Collaborators/Subcontractors

Dan Witzling

Environmental Program Manager
Breathe California of Los Angeles
5858 Wilshire Blvd, Suite 300
Los Angeles, CA 90036

Rudy Eden

Laboratory Services and Source Testing
South Coast Air Quality Management
District

21865 E. Copley Drive
Diamond Bar, CA 91765

John McKernan, ScD, CIH

Commander, USPHS

Senior Environmental Health Officer

U.S. Environmental Protection Agency

ETAV/NRMRL/ORD

26 W. Martin Luther King Dr., MS 208

Cincinnati, OH 45268

Battelle

Amy Dindal
Thomas Kelly
Brad Goodwin
Zachary Willenberg
Battelle
505 King Ave.

Columbus, OH 43201

Peer Reviewers

Dennis Mikel

U.S. Environmental Protection Agency
Office of Air Quality Planning and
Standards
E-143-02

Research Triangle Park, NC 27711
Will Ollison

American Petroleum Institute
1220 L Street

Washington, D.C. 2005-4070


-------
Ozone Indicator Cards
Test/QA Plan
Page 6 of 45
Version 1
November 10, 2009

A5 VERIFICATION TEST ORGANIZATION

The verification test will be conducted under the auspices of the U.S. Environmental Protection
Agency (EPA) through the Environmental Technology Verification (ETV) Program. It will be
performed by Battelle, which is managing the ETV Advanced Monitoring Systems (AMS)

Center through a cooperative agreement with EPA. The scope of the AMS Center covers
verification of monitoring technologies for contaminants and natural species in air, water, and
soil. This verification will address ozone indicator cards (OICs) that provide short-term semi-
quantitative measures of ozone concentration in ambient air. The performance of OICs will be
judged in part by comparison with measurements made with an EPA-designated reference
measurement method (the Federal Equivalent Method [FEM]) for ozone.

The day to day operations of this verification test will be coordinated and supervised by Battelle
personnel, with in-kind involvement from personnel of the California South Coast Air Quality
Management District (SCAQMD), Breathe California of Los Angeles (BCLA), and the vendors
who will be having the performance of their OICs verified. Laboratory testing will be conducted
at Battelle in Columbus, Ohio. Field testing will be carried out by SCAQMD and BCLA under
Battelle's oversight at multiple locations in southern California. The vendors will provide
Battelle with instructions for use of their OICs and enough OICs to make up to 1,000 ozone
readings.

The organization chart in Figure 1 identifies the responsibilities of the organizations and
individuals associated with the verification test. Roles and responsibilities are defined further
below. Quality Assurance (QA) oversight will be provided by the Battelle Quality Manager and
also by the EPA AMS Center Quality Manager, at her discretion.


-------
Ozone Indicator Cards
Test/QA Plan
Page 7 of 45
Version 1
November 10, 2009

Figure 1. Organization Chart for the Verification Test

A5.1 Battelle

Dr. Thomas Kelly is the AMS Center's Verification Test Coordinator for this test. In this role,
Dr. Kelly will have overall responsibility for ensuring that the technical, schedule, and cost goals
established for the verification test are met. Specifically, Dr. Kelly will:

•	Prepare the draft test/QA plan, verification reports, and verification statements.

•	Revise the draft test/QA plan, verification reports, and verification statements in
response to reviewers' comments.


-------
Ozone Indicator Cards
Test/QA Plan
Page 8 of 45
Version 1
November 10, 2009

•	Assemble a team of qualified technical staff to conduct the verification test.

•	Establish a budget for the verification test and manage staff to ensure the budget is
not exceeded.

•	Coordinate with the vendors for provision of OICs for testing.

•	Direct Battelle technical staff in the laboratory testing and coordinate with BCLA
personnel for performance of the field testing

•	Direct the team in performing the verification test in accordance with this test/QA
plan.

•	Hold a kick-off meeting approximately one week prior to the start of the verification
test to review the critical logistical, technical, and administrative aspects of the
verification test. Responsibility for each aspect of the verification test will be
confirmed.

•	Ensure that all quality procedures specified in this EPA Quality Level III test/QA
plan and in the AMS Center Quality Management Plan1 (QMP) are followed.

•	Serve as the primary point of contact for SCAQMD, BCLA, and vendor
representatives.

•	Ensure that confidentiality of sensitive vendor information is maintained.

•	Assist vendors as needed during verification testing.

•	Become familiar with the operation of the OICs through instruction by the vendors, if
needed.

•	Prepare a deviation report for any departure from the test/QA plan during the
verification, obtain the requisite EPA and vendor approvals, and distribute the
approved report as specified in the AMS Center QMP.

•	Respond to any issues raised in assessment reports, audits, or from test staff
observations, and institute corrective action as necessary.

•	Coordinate distribution of the final test/QA plan, verification reports, and verification
statements.

Ms. Amy Dindal is Battelle's Manager for the AMS Center. As such, Ms. Dindal will oversee
the various stages of verification testing. Ms. Dindal will:

•	Review the draft and final test/QA plan.

•	Attend the verification test kick-off meeting.

•	Review the draft and final verification reports and verification statements.


-------
Ozone Indicator Cards
Test/QA Plan
Page 9 of 45
Version 1
November 10, 2009

•	Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test.

•	Maintain communication with EPA's technical and quality managers.

•	Issue a stop work order if Battelle or EPA QA staff discover adverse findings that
will compromise test results.

Technical staff from Battelle will support Dr. Kelly in planning and conducting the verification
test. The responsibilities of the technical staff will be to:

•	Assist in planning for the test, and making arrangements for the receipt of and
training on the OICs.

•	Attend the verification test kick-off meeting.

•	Conduct verification testing using the vendors' OIC technology.

•	Conduct FEM measurements during the testing.

•	Perform statistical calculations specified in this test/QA plan on the technology data
as needed.

•	Provide results of statistical calculations and associated discussion for the verification
reports as needed.

•	Support Dr. Kelly in responding to any issues raised in assessment reports and audits
related to statistics and data reduction as needed.

Mr. Zachary Willenberg is Battelle's Quality Manager for the AMS Center. Mr. Willenberg will:

•	Review the draft and final test/QA plan.

•	Attend the verification test kick-off meeting.

•	Conduct a technical systems audit (TSA) at least once during the verification test, or
designate other QA staff to conduct the audit. That TSA will address both the
laboratory and field components of OIC testing.

•	Audit at least 10% of the verification data or designate other QA staff to conduct the
data audit.

•	Prepare and distribute an assessment report for each audit.

•	Verify implementation of any necessary corrective action.

•	Request that Battelle's AMS Center Manager issue a stop work order if audits
indicate that data quality is being compromised.


-------
Ozone Indicator Cards
Test/QA Plan
Page 10 of 45
Version 1
November 10, 2009

•	Provide a summary of the QA/quality control (QC) activities and results for the
verification reports.

•	Review the draft and final verification reports and verification statements.

A5.2 OIC Vendor

The responsibilities of an OIC vendor are as follows:

•	Review and provide comments on the draft test/QA plan.

•	Accept (by signature of a company representative) the final test/QA plan prior to test
initiation.

•	Initially provide enough units of their OIC technology to carry out up to 1,000
measurements of ambient ozone during the verification test, and additional units
should the need arise during testing.

•	Supply instructions on the use of the technology, and written consent for test staff to
carry out verification testing.

•	Review and provide comments on the draft verification report and verification
statement for their respective technology.

A5.3 South Coast Air Quality Management District

The primary point of contact for SCAQMD will be Mr. Rudy Eden. The role of SCAQMD is to

assist Battelle in conducting a quantitative comparison of OIC readings with FEM readings. Mr.

Eden will lead SCAQMD personnel in meeting the following responsibilities of SCAQMD in

this verification:

•	Review and provide comments on the draft test/QA plan.

•	Select established SCAQMD ambient ozone monitoring sites that exhibit a range of
ozone concentrations and are available for this verification.

•	Provide sufficient SCAQMD personnel to carry out ambient ozone measurements
with OICs at the selected sites.

•	Provide ozone and meteorological data from those sites.

•	Review and provide comments on the draft verification report and verification
statement for each tested OIC technology.


-------
Ozone Indicator Cards
Test/QA Plan
Page 11 of 45
Version 1
November 10, 2009

A5.4 Breathe California of Los Angeles

The primary point of contact for BCLA will be Mr. Dan Witzling. The role of BCLA is to assist
Battelle in conducting a qualitative evaluation of OIC use, including an assessment of OIC
duplication and user agreement. The operators of OICs in BCLA testing will be non-technically
trained adult volunteers who lead after-school programs in the Long Beach, CA area. The test
sites of interest will be elementary and/or secondary schools in Long Beach where potential
exposures of school children to ozone are of concern. Mr. Witzling will lead in meeting the
following responsibilities of BCLA in this verification:

•	Review and provide comments on the draft test/QA plan.

•	Identify and coordinate sufficient volunteer personnel to carry out ambient ozone
measurements with OICs at sites of interest to BCLA.

•	Provide the results of those OIC measurements to Battelle along with user
observations about the OICs.

•	Conduct wearer acceptance testing of the OICs when worn as a personal exposure
badge, and report wearer observations to Battelle.

•	Review and provide comments on the draft verification report and verification
statement for each tested OIC technology.

A5.5 EPA

EPA's responsibilities in the AMS Center are based on the requirements stated in the
"Environmental Technology Verification Program Quality Management Plan" (EPA ETV
QMP).2 The roles of specific EPA staff are as follows:

Ms. Michelle Henderson is EPA's AMS Center Quality Manager. For the verification test,
Ms. Henderson will:

•	Review the draft and approve the final test/QA plan.

•	Attend the verification kickoff meeting.

•	Review checklists, reports, report responses, and closure statements of TSA, PE
audits, and quality systems audits (QSAs) conducted by Battelle.


-------
Ozone Indicator Cards
Test/QA Plan
Page 12 of 45
Version 1
November 10, 2009

•	Perform at her option an external TSA of field and/or laboratory activities,
performance evaluation (PE) audit, and/or an audit of data quality during the
verification test.

•	Notify the EPA AMS Center Project Officer of the need for a stop work order if
evidence indicates that data quality is being compromised.

•	Prepare and distribute an assessment report summarizing results of the external audit
performed.

•	Review the draft and approve the final verification reports and verification
statements.

Dr. John McKernan is EPA's Project Officer for the AMS Center. Dr. McKernan will:

•	Review the draft test/QA plan.

•	Approve the final test/QA plan.

•	Attend the verification kickoff meeting.

•	Be available during the verification test to authorize any test/QA plan deviations by
phone and provide the name of a delegate to the Battelle AMS Center Manager
should he not be available during the testing period.

•	Review the draft verification reports and verification statements.

•	Oversee the EPA review process for the test/QA plan, verification reports, and
verification statements.

•	Coordinate the submission of verification reports and verification statements for final
EPA approval.

•	Post the test QA plan, verification reports, and verification statements on the ETV
web site.

A6 BACKGROUND
A6.1 Technology Need

The ETV Program's AMS Center conducts third-party performance testing of commercially
available technologies that detect or monitor natural species or contaminants in air, water, and
soil. The purpose of ETV is to provide objective and quality assured performance data on
environmental technologies, so that users, developers, regulators, and consultants can make
informed decisions about purchasing and applying these technologies. Stakeholder committees


-------
Ozone Indicator Cards
Test/QA Plan
Page 13 of 45
Version 1
November 10, 2009

of buyers and users of such technologies recommend technology categories, and technologies
within those categories, as priorities for testing. Verification reports from previous tests are
available at http://www.epa.gov/nrmrl/std/etv/verifiedtechnologies.html.

Ozone is a widespread pollutant that is formed by photochemical processes involving sunlight,
nitrogen oxides, and volatile organic compounds (VOC) in air. The U.S. Clean Air Act and its
Amendments established air quality standards for ozone, and pollution control strategies that
require state and local authorities to regulate for compliance with the standards. Ozone is
regulated because of its effect on human health when air containing elevated concentrations of
ozone is inhaled. Because of the costs associated with emission control programs and penalties
for those regions that are not in compliance, it is essential that ozone measurements that
determine compliance with standards be accurate. For that purpose EPA has established Federal

"3

Reference and Equivalent Methods (FRM and FEM) of monitoring ozone. The method
currently widely used is the FEM, which makes use of the ozone molecule's strong absorption
band in the ultraviolet region of the spectrum, with a maximum coinciding with the strong
mercury vapor emission line at 254 nanometers (nm). The FEM has completely supplanted the
FRM for all compliance monitoring in the U.S., because of the greater complexity of the FRM
and its requirement for flammable ethylene gas. Commercial FEM instruments measure the
transmission of UV light through an air sample and compare the intensity with that obtained
along the same pathlength through air containing no ozone. A scrubber (typically Mn02or a
heated metal scrubber [HMS]) designed to selectively remove ozone from the air, is used to
determine the background absorption of UV light by species such as aromatic hydrocarbons.
Potential interferences in the FEM ozone measurement, due to removal of UV-absorbing
aromatic hydrocarbons by the ozone scrubber, have been identified and may be significant in
highly polluted conditions that produce elevated ozone levels of regulatory significance.4 For
assessment of human exposure to ozone in air, less expensive and complex measurements of
ozone can be useful. Qualitative methods that approximately indicate the ozone concentration in
air can be helpful to people with ailments that cause respiratory sensitivity to ozone.

A6.2 Technology Description

One simple and inexpensive approach to ozone measurement is the use of colorimetric indicator
cards for ozone, which incorporate a reagent that undergoes a color change when exposed to
ozone in air. To use the card, a protective film over a reagent spot is removed and the spot is
exposed to ambient air for a specified period of time (e.g., 10 minutes). Then the intensity of the


-------
Ozone Indicator Cards
Test/QA Plan
Page 14 of 45
Version 1
November 10, 2009

resulting color change in the reagent spot is visually compared to a color index printed on the
card, providing an estimate of the ozone concentration. Such OICs typically indicate ozone
concentrations in a few broad concentration ranges from near zero to over 100 parts per billion
by volume (ppbv). The ETV verification described in this test/QA plan will explore the
effectiveness and ease of use of OICs and quantify their response relative to the response of FEM
instrumentation in laboratory and outdoor testing.

A7 VERIFICATION TEST DESCRIPTION AND SCHEDULE
A7.1 Verification Test Description

The purpose of this test/QA plan is to specify procedures for a verification test applicable to
commercial OICs. One aspect of the verification test is to compare ozone readings made by the
OICs to those made by the FEM UV method for ozone. In performing the verification test,
Battelle will follow the technical and QA procedures specified in this test/QA plan and will
comply with the data quality requirements in the AMS Center QMP.1 This is an EPA QA Level
III project.

A7.2 Verification Schedule

Table 1 shows the planned schedule of activities for the verification testing and data analysis and
reporting. Both laboratory and field testing of OICs will be initiated upon final approval of this
test/QA plan, and will take place simultaneously. As shown in Table 1, testing activities are
planned to begin in the fall of 2009. Laboratory testing will require approximately one month.
Field testing will be split into fall and spring/summer episodes to assure capture of a range of
ambient ozone levels. Each of those two field periods will require approximately one month to
complete. A separate ETV verification report will then be drafted for each OIC tested, and the
report will be reviewed simultaneously by the technology vendor, SCAQMD, and BCLA, and
subsequently by peer reviewers. The final reports will be submitted to EPA for final signature,
and the final reports will be made publicly available on the EPA/ETV web site.

A7.3 Test Facility

Laboratory analyses will be conducted in Battelle laboratories in Columbus, Ohio.

In performing this verification test, Battelle will follow the procedures specified in this test/QA
plan and will comply with quality requirements in the AMS Center QMP.1 The air quality
laboratory that will be used is fully equipped for production, delivery, and calibrated
measurement of ozone in test atmospheres of controlled temperature and humidity.


-------
Ozone Indicator Cards
Test/QA Plan
Page 15 of 45
Version 1
November 10, 2009

A8 QUALITY OBJECTIVES

This verification test is designed to evaluate the performance of the OICs for determining ozone
in air. Laboratory evaluation will include a comparison of the OIC results to a commercial
ozone monitor designated by EPA as an ozone FEM (Thermo Environmental Model 49C, FEM
EQOA-0880-047). Ozone will be generated at known concentrations to challenge the OICs, and
will be monitored simultaneously by the FEM instrument. The ozone will be generated using an
Environics Model 6400 ozone generator that has itself been calibrated against a Dasibi Model
1800 UV calibration photometer. In field testing OIC readings will be compared to those of
ozone analyzers operating at SCAQMD air monitoring sites. To the extent possible, SCAQMD
sites will be used at which FEM monitors equipped with HMS technology, or FRM monitors, are
deployed. However, the choice of field sites and ozone monitors will be subject to SCAQMD
discretion. The quality of the ozone data will be documented by calibration data, and
documentation of the QA activities carried out by the relevant air quality agencies at the field
monitoring sites.

QA/QC requirements will include a TSA, a PE audit, and an audit of data quality (ADQ). These
QA procedures will be carried out by Battelle. The planned audit procedures are described in
Section CI. The EPA Quality Manager also may conduct an independent TSA, PE audits, and/or
ADQ at her discretion.

A9 SPECIAL TRAINING/CERTIFICATION

Documentation of training related to technology testing, field testing, data analysis, and reporting
is maintained for all Battelle technical staff in training files at their respective locations. The
Battelle Quality Manager may verify the presence of appropriate training records prior to the
start of testing. Battelle technical staff involved in this verification will have experience in
operation of ozone calibration and monitoring equipment, and specifically with the Environics,
Dasibi, and Thermo Environmental equipment noted in Section A8.

SCAQMD personnel who use the OICs at SCAQMD field sites will be site operators
experienced in calibration and performance of FEM ozone measurements at those sites. All
Battelle, SCAQMD, and BCLA personnel will receive training in use of the OICs based on the
instructions which are printed on every OIC.


-------
Ozone Indicator Cards
Test/QA Plan
Page 16 of 45
Version 1
November 10, 2009

Table 1. Planned Verification Schedule

Month and Year

Verification Activity



Testing

Data Analysis and Reporting

November 2009

Initiate laboratory and
field testing

Begin preparation of ETV report template

December 2009

Completion of
laboratory testing

Completion of first
phase field testing

Compile data from laboratory testing

Compile data from first phase field testing

Review and summarize laboratory operator observations

January-April 2010



Analyze laboratory test data
Analyze field test data

Review and summarize field operator observations
Prepare partial draft reports

May-June 2010

Completion of second
phase field testing

Compile data and operator observations from second phase
field testing
Analyze field test data

July 2010



Analyze field test data
Prepare draft reports
Internal review of draft reports

August 2010



Vendor/BCLA/SCAQMD/EPA review of draft reports
Revision of draft reports
Peer review of draft reports

September 2010



Revision of draft reports

Submission of final reports for EPA approval

A10 DOCUMENTATION AND RECORDS

The records for this verification test will be contained in the test/QA plan, laboratory record
books (LRB), data collection forms, electronic files (both raw data and spreadsheets), the final
verification report, or assessment reports. All of these records will be maintained in the
Verification Test Coordinator's office during the test and will be transferred to permanent
storage at Battelle's Records Management Office within two months of the finalization of the
verification reports. All Battelle LRBs are also stored indefinitely, either by the Verification
Test Coordinator or within two months of the finalization of the verification reports in Battelle's
Records Management Office. EPA will be notified before disposal of any files. Section BIO
further details the data recording practices and responsibilities.


-------
Ozone Indicator Cards
Test/QA Plan
Page 17 of 45
Version 1
November 10, 2009

All written records will be in ink. Any corrections to notebook entries, or changes in recorded
data, will be made with a single line through the original entry. The correction is then to be
entered, initialed, and dated by the person making the correction. In all cases, strict
confidentiality of data from the vendor's technology will be maintained, i.e., separate files
(including manual records, printouts, and/or electronic data files) will be kept for each vendor's
technology, and no intercomparison of data from different OIC technologies will be conducted.


-------
Ozone Indicator Cards
Test/QA Plan
Page 18 of 45
Version 1
November 10, 2009

SECTION B
MEASUREMENT AND DATA ACQUISITION

B1 EXPERIMENTAL DESIGN

An OIC consists of a playing-card-size piece of cardboard on which one or more spots of a solid
reagent are placed, each spot being approximately 5 mm in diameter. Before use each reagent
spot is covered and completely sealed with a thin strip of flexible foil. To use the OIC, the foil is
removed from a reagent spot and discarded, and the card is then left exposed to the ambient air
for a period of time specified by the vendor (e.g., 10 minutes for use in outdoor air). The reagent
reacts with ozone in the air and changes color, typically darkening from white or pale yellow
before exposure to darker yellow or brown. The extent of the color change depends on the ozone
concentration in the air and the duration of exposure; close adherence to the vendor-specified
exposure time allows the color change to be used as an indicator of the ambient ozone
concentration. Determination of the ozone concentration is done by visually comparing the
color of the exposed reagent spot to a printed color standard immediately after the end of the
exposure period. The color standard may be separate from the OIC or printed on the OIC itself,
and in the latter case can often be positioned over the exposed reagent spot by folding the OIC so
that the reagent spot shows through a hole around which the color standard is printed. The
correspondence of color with ozone concentration is qualitative, in that OIC ozone readings fall
into broad ranges. For example, one common OIC has four color comparison standards
(numbered 1 through 4) which correspond to ozone concentration ranges of 10 to 45, 45 to 75,
75 to 105, and >105 ppbv, respectively. Figure 2 shows examples of commercially available
OICs, illustrating the arrangement of reagent spots and the positioning of the color standard.
Instructions for use of the OIC are printed on the card.


-------
Ozone Indicator Cards
Test/QA Plan
Page 19 of 45
Version 1
November 10, 2009

(hone IMeetor Cord

Figure 2. Examples of Ozone Indicator Cards

This test/QA plan addresses the verification of OICs through laboratory testing and monitoring
of ozone in ambient air. Specifically the OICs will be evaluated for the following performance
parameters:

•	Accuracy

•	Variability of readings

•	OIC duplication and user agreement

•	Effect of light intensity on OIC color development

•	Effect of ambient temperature, relative humidity (RH), and wind speed on OIC
readings

•	Operational factors including ease of use, readability, and use of OICs as personal
monitors.


-------
Ozone Indicator Cards
Test/QA Plan
Page 20 of 45
Version 1
November 10, 2009

Accuracy will be determined by comparing OIC readings to reference ozone measurements made
in laboratory and field testing. Variability will be assessed by observing the spread of OIC
readings made at constant ozone concentrations. OIC duplication will be assessed by comparing
ozone readings made by a single user with duplicate OICs exposed simultaneously. User
agreement will be assessed by comparing visual readings made by two different users on the
same individual OIC reagent spots. Use of the OICs requires visual reading of the color of a
reagent spot; therefore all users will be asked to indicate if they are color-blind or have other
visual impairment before taking part in testing, and to remove sunglasses before making OIC
visual readings.

The effect of light intensity on OIC color changes will be assessed in the laboratory by
comparing OIC readings made by a single user on OICs exposed at a constant ozone level but
under varying degrees of illumination. Temperature, RH, and wind speed effects will be
estimated by recording the ambient conditions at ozone monitoring sites in the field at the time
that OIC and reference FEM measurements are obtained. Operator observations will be recorded
in all laboratory and field testing to assess operational factors of the OICs. In all testing, each
OIC and reagent spot will be uniquely numbered to facilitate data recording.

Bl.l Test Procedures

The following sections describe the test procedures that will be used to evaluate each of the
performance parameters listed above.

Table 2 summarizes the types and numbers of samples that will be used to verify performance of
each OIC. For laboratory testing, test atmospheres will be generated in a small environmental
chamber or continuous flow system. The OICs undergoing testing will be placed in the chamber
or flow system, and exposed to air of normal temperature and RH (i.e., approximately 22 °C and
50% RH) and controlled ozone content. The temperature, RH, and ozone concentration of the
test air will be monitored during each test, using standard commercial temperature and RH
sensors and a Thermo Environmental Model 49C UV ozone monitor (FEM EQOA-0880-047).
The signals from the ozone monitor and temperature and RH monitors will be recorded using a
Campbell Scientific datalogger or similar data acquisition system.


-------
Ozone Indicator Cards
Test/QA Plan
Page 21 of 45
Version 1
November 10, 2009

Table 2. Summary of OIC Verification Samples

Sample Type

Responsible
Organization

Number of Samples
or Readings"

Associated QC
Samples

Uses

Laboratory OIC
readings

Battelle

200
30

NA

Accuracy, variability,
OIC duplication user
agreement

Light intensity effect

Laboratory FEM
readings

Battelle

200
30

Initial multipoint
calibration, daily
zero/span check

Accuracy, variability,
OIC duplication user
agreement

Light intensity effect

Laboratory light
intensity readings

Battelle

30

Battelle
Instrumentation

Laboratory
calibration dark
(zero) checks

Light intensity effect

Laboratory
temperature and RH
readings

Battelle

Continuous

Battelle
Instrumentation
Laboratory calibration

Documentation of
laboratory test
conditions

Field OIC Readings

SCAQMD

120b

NA

Accuracy, OIC
duplication, user
agreement, effect of
ambient conditions

Field FEM
Readings

SCAQMD

120b

Per SCAQMD
compliance
monitoring program

Accuracy, OIC
duplication, user
agreement, effect of
ambient conditions

Field temperature,
RH, wind speed
readings

SCAQMD

120b

Per SCAQMD
maintenance

Effect of ambient
conditions

Field OIC Readings

BCLA

120b

NA

OIC duplication, user
agreement

User observations

BCLA/Battelle/S
CAQMD

As needed

NA

Operational factors

a: Estimated number of samples/readings per each OIC technology tested,
b: Rough estimate, testing will be subject to responsible organization's discretion.

Field testing will consist of use of the OICs by SCAQMD or BCLA representatives at multiple
sites in southern California. SCAQMD representatives will use OICs at sites at which
continuous ozone monitoring is carried out for compliance purposes using the FEM. At such
sites OIC readings will be recorded and compared to the average FEM readings over the time
period of the OIC exposure to ambient air. Temperature, RH, and wind speed data will be
recorded by SCAQMD staff during each OIC exposure interval in the field, and those data will


-------
Ozone Indicator Cards
Test/QA Plan
Page 22 of 45
Version 1
November 10, 2009

be evaluated to assess the effects of these conditions on OIC performance. BCLA staff will use
OICs at schools in Long Beach, CA where personal exposure of children to ozone may be of
particular concern. At both types of sites, multiple OICs will be exposed and read at the same
time, and to the extent feasible multiple users will record their visual judgment of the OIC
readings. All field testing will be conducted outdoors, and in all testing each OIC will be tested
with a single exposure duration (e.g., 10 minutes) specified by the respective vendor.

The total number of OICs required from each vendor for testing will be less than 200; the exact
number to be used will depend on field testing factors such as personnel availability, site issues,
etc., that cannot be entirely defined in this plan. Laboratory testing is expected to use
approximately 40 OICs if each OIC has five reagent spots, so approximately 50 will be required
to account for practice runs, damage, etc. The number of OICs to be used by SCAQMD for
accuracy testing similarly is planned to be about 25 at five reagent spots per OIC, but practice
runs, damage, and greater than expected availability of SCAQMD personnel may double that
number. The number of OICs to be used by BCLA will depend on the interest and involvement
of volunteer personnel; approximately 60 OICs will be provided to BCLA initially, and the
vendor will be prompted for more OICs if volunteer activities consume that initial allotment.

The quality and objectivity of the field evaluations conducted by SCAQMD and BCLA will be
assured by the efforts of Battelle technical and QA staff. The Battelle Verification Test
Coordinator (Dr. Kelly) will communicate the procedures and expectations for OIC evaluation to
both the SCAQMD and BCLA lead representatives (Mr. Eden and Mr. Witzling, respectively),
by verbal and written means including this test/QA plan. That communication will be
maintained before, during, and after the periods of field testing, and will continue in the data
analysis phase of the verification. In addition, the Battelle AMS Center QA Manager will
conduct on-site TSAs at both SCAQMD and BCLA field sites during OIC testing. In those
efforts he will compare test procedures and data recording efforts to the requirements of this
test/QA plan, and direct any necessary improvements.

Bl. 1.1-Accuracy

The accuracy of the OIC will be determined by comparing OIC readings determined visually to
simultaneous measurements made using the FEM. The comparison of accuracy will be made
graphically, by plotting OIC readings in indicated OIC ranges (e.g., 10 to 45, 45 to 75, 75 to 105,
>105 ppbv etc.) against the ozone concentration measured by the FEM.


-------
Ozone Indicator Cards
Test/QA Plan
Page 23 of 45
Version 1
November 10, 2009

In laboratory testing, ozone will be produced using an Environics Model 6400 ozone generator
(transfer standard) that has itself been calibrated against a Dasibi Model 1008 UV calibration
photometer. Dry high purity air will be used as the feed gas to the Environics Model 6400 ozone
generator. The dry air exiting the ozone source will then be mixed with humidified air to
produce the challenge ozone concentrations. The ozone concentrations delivered to the OICs
will be monitored in the OIC exposure chamber by the FEM instrument. The ozone
concentrations delivered to the OICs will be adjusted depending on the OICs being tested so that
ozone concentrations both in the middle and near the edges of the OIC indication ranges are used
in testing.

For example, with the OIC indication ranges noted in the previous paragraph, ozone challenge
concentrations of 30, 50, 70, 90 and 120 ppbv would be suitable. The use of ozone
concentrations near the outer limits of OIC ranges will provide information on OIC variability
that might not be evident with ozone concentrations centered in the ranges. Ozone
concentrations that coincide with the transition points from one OIC range to the next will not be
used.

The laboratory ozone challenges will be delivered in air at normal temperature and RH
conditions (i.e., approximately 22 °C and 50% RH). For each OIC tested, once reference method
readings have stabilized, four cards will be exposed in each laboratory testing step, and each OIC
will be read by two different users, who record their readings on separate sheets. Those users
will not know the ozone concentration being delivered to the OICs, i.e., the ozone concentration
will be changed between OIC exposures and will be known only by the Battelle test operator.
As Table 2 shows, 20 OIC readings will be recorded by each of two users at each of five ozone
levels, producing 200 total data points for comparison of OIC and reference method results.

In field testing, SCAQMD personnel will take OICs to established SCAQMD ozone monitoring
sites during normal site operations, and expose multiple reagent spots on multiple cards to
ambient air while at the sites. Personnel will visually judge each OIC reading, and record it on a
data sheet provided by Battelle (an example is shown in Appendix A), along with the date, time,
and location of sampling, and the FEM ozone reading, temperature, RH, and wind speed at the
site during the OIC exposure period. Potential sites to be used for this purpose by SCAQMD are
located in Azusa, Crestline, Glendora, Rubidoux, and Santa Clarita. These sites are all in


-------
Ozone Indicator Cards
Test/QA Plan
Page 24 of 45
Version 1
November 10, 2009

mountain foothill locations, and are frequently subject to elevated ambient ozone levels. The
relative locations of the FEM inlet and the OICs during exposure at each site will be
documented. In each such site visit up to five such readings will be taken with two OICs
simultaneously. The intent of this plan is that 12 such site visits will be conducted, producing up
to 60 ozone readings with duplicate OICs (120 OIC readings total). Duplicate OIC readings will
be obtained by exposing one or more spots on separate OICs over the same 10-minute period,
and recording the readings on pre-established data sheets (Appendix A). The unique OIC
number, the reagent spot number (1 through 5), and the visual readings of multiple users (if
applicable) will be recorded along with the start-stop times for every OIC reading taken. The
results will be entered from the data sheets into Microsoft Excel for evaluation of OIC duplicates
(i.e., different OIC numbers, same start-stop times, same user) and user duplicates (i.e., same
OIC number and reagent spot number, different users).

This effort will be carried out as an in-kind contribution to the verification by SCAQMD, so the
actual number of OICs used and readings taken may be subject to other demands on SCAQMD
personnel. At a minimum, 6 such site visits, producing up to 30 ozone readings with duplicate
OICs (60 OIC readings total) are planned. Also, it is desirable that the OIC visual readings be
made by more than one SCAQMD user at the field sites; the feasibility of this may similarly be
subject to the demands on SCAQMD personnel.

B. 1.1.2— Variability of Readings

Variability of OIC readings refers to the consistency, or lack thereof, in visually determined OIC
results with a constant ozone concentration. Variability will be assessed using the multiple
readings made by each of two users at five ozone concentrations, in the laboratory accuracy
testing described in Section B1.1.1. Variability will be expressed as the number of OIC
indication ranges into which the OIC readings fall at each of those concentrations.

Bl. 1.3—Duplication

The degree of agreement of ozone measurements made simultaneously on duplicate OICs will be
assessed using all data in which duplicate OICs are read by a single user. Similarly, the degree
of agreement of ozone measurements made by separate users will be assessed using all data in
which the same exposed OIC reagent spot is read by more than one user. These two measures of
performance are termed OIC duplication and user agreement, respectively, and are intended to
address two types of variation in OIC readings. OIC duplication addresses within-user variation,


-------
Ozone Indicator Cards
Test/QA Plan
Page 25 of 45
Version 1
November 10, 2009

whereas user duplication addresses between-user variation. Within-user variation arises from
differences in the color development of different reagent spots that have been exposed under
identical conditions. Between-user variation arises from visual perception differences in
readings made by different individual users on the same exposed reagent spot. These two forms
of duplication are distinguished in testing by unique numbering of OICs and reagent spots, and
by separate recording of readings made by different users.

In the laboratory testing, data to assess both OIC and user agreement will arise from the tests
conducted to assess accuracy (Section B1.1.1). In field testing, such data will arise whenever
either SCAQMD or BCLA representatives report visual readings for duplicate OIC cards, or
visual readings for the same OIC reagent spot from two different users. Such duplicate readings
will be recorded on the field data sheets to be used by SCAQMD and BCLA representatives (see
Appendices A and B, respectively). In all cases duplicate readings (whether of duplicate OICs
by a single user or of a single OIC by multiple users) will be taken in close succession and under
identical lighting conditions immediately after the OIC exposure period.

B. 1.1.4—Effect of Light Intensity

The effect of light intensity on OIC response to ozone will be evaluated in laboratory testing by
exposing OICs to a constant ozone concentration under different conditions of illumination. In
this test, all reagent spots on two OIC cards will be exposed simultaneously to approximately
60 ppb of ozone in air at 22 °C and 50% RH, with the test conducted under normal indoor
illumination and the OICs lying horizontal (flat) during testing. The total light intensity reaching
the OICs during the test will be monitored by appropriate commercial meters. The same test will
be repeated with the OICs in darkness during O3 exposure, and then repeated once more with the
test apparatus illuminated by sunlamps to achieve a light intensity of approximately 100
mW/cm , equivalent to a full overhead sun. The OIC results from the three trials (i.e., 10
readings at each of three light intensities) will be compared to assess whether light intensity
affects OIC reading. In all three tests, the same single user will make the visual readings on both
OICs under the same normal laboratory illumination conditions.

The light source used in the high light intensity test will be a Class A solar simulator (certified to
International Electrotechnical Commission 904-9 and American Society for Testing and
Materials E927-05 standards). This light source mimics the entire solar spectrum including the
low levels of UV-A (320-400nm) and UV-B (280-320 nm) in the solar spectrum at ground level.


-------
Ozone Indicator Cards
Test/QA Plan
Page 26 of 45
Version 1
November 10, 2009

A lamp that provides the solar spectrum and intensity characteristic of an overhead sun at sea
level (i.e., 1.0 sun intensity with air mass =1.0) will be used. Light intensity will be monitored
using Greenlee Model 93-1065F photometers, or similar, obtained from Battelle's
Instrumentation Services Laboratory.

Bl.1.5—Effect of Ambient Conditions

The effect of ambient temperature, RH, and wind speed will be evaluated based on information
provided by SCAQMD personnel from the field sites. The OIC and FEM field ozone results
used to determine OIC accuracy (Sections B 1.1.1 and B 1.2.1) will be segregated into those
showing agreement between OIC and FEM results and those showing disagreement. Then the
temperature, RH, and wind speed conditions associated with these two data sets will be
compared.

HI. 1.6—Operational Factors

Operational factors associated with use of the OICs will be evaluated based on the comments and
observations of all users (Battelle, SCAQMD, and BCLA) in the laboratory and field testing.

Such observations may address the convenience of the OICs, their readability under differing
conditions, the apparent consistency of OIC readings, and acceptability as a personal monitor. In
particular the observations of BCLA users will be important, as those users are likely to be the
relatively non-technically trained users for which the OICs are designed.

In a part of the field testing that is unique to BCLA's effort, BCLA will conduct trials of the use
of each OIC as a personal monitor. Exposure of an OIC worn as a personal monitor will be
conducted over the same time period as exposure of other OICs at a field site, so that
comparisons of OIC results can be made between personal and non-personal measurements.

Also, the BCLA representatives will report on the degree to which they could accept the OIC as
a personal monitor to be worn; comments on convenience, annoyance while worn, obstruction of
movement, conspicuous appearance, and any other user observations will be obtained. In testing
of OICs as personal monitors, the manner in which an OIC is worn by a BCLA user will be
controlled to avoid erroneous ozone readings. General recommendations for use of the OICs
include:

•	The OIC must be fully opened;

•	Foil covering must be fully removed from the reagent spot, and discarded;


-------
Ozone Indicator Cards
Test/QA Plan
Page 27 of 45
Version 1
November 10, 2009

•	The OIC must be fully exposed to ambient air without surface proximity that can
remove ozone from air;

•	The visual reading must be made immediately after the OIC exposure period.

Specific recommendations for use of OICs as personal monitors include:

•	Attach the OIC so that minimal interference results from surface contact;

•	Expose the OIC as a personal monitor simultaneously with exposure of other OICs
used as local (i.e., non-personal) air monitors.

•	Placement of the OIC in a horizontal (flat) orientation atop a hat or cap is preferable
to attachment in a vertical orientation hanging from clothing.

•	Recommended order of preference of location of an OIC worn as a personal monitor
(the first two provide horizontal orientation, the others are vertical):

top of cap > bill of cap > sleeve (shoulder) > shirt pocket > lapel .

B1.2 Statistical Analysis

The semi-quantitative nature of the OIC readings determines the types of statistical comparisons
that can be done to evaluate the performance parameters. The planned statistical comparisons
are described in the following sections.

B. 1.2.1—A ccuracy

The accuracy of the OIC with respect to the FEM will be assessed as a percentage of readings in
the correct OIC indication range, i.e.:

Accuracy = (iOIC in Range/Number of Trials) x 100	(1)

where OIC in Range is the number of total OIC readings reported as being in the OIC indication
range that encompasses the corresponding FEM ozone reading, and Number of Trials is the total
number of such comparisons for which FEM readings fell within that range. This calculation of
accuracy will be performed with the data from each of the four challenge ozone concentrations in
the laboratory testing, and with the data obtained by SCAQMD at field sites. Readings from
OICs exposed simultaneously, and readings from OICs read in the laboratory by more than one


-------
Ozone Indicator Cards
Test/QA Plan
Page 28 of 45
Version 1
November 10, 2009

user, will be included in this calculation as separate trials (i.e., all OIC/FEM comparisons will be
treated as independent data for this calculation).

Bl.2.2—Variability of Readings

Variability of the OIC readings will be evaluated using only the data from the repeated
laboratory trials at five ozone challenge concentrations. For each of those concentrations,
variability will be determined as the number of OIC indication ranges into which the user
readings fall. That is, at ozone concentration X (e.g.):

Variability X= (# of OIC Ranges with Readings at Ozone Concentration X) (2)

Bl. 2.3—Duplication

OIC duplication will be assessed in terms of the percentage of readings in which a single user
reports a result in the same OIC indicator range from two duplicate OICs exposed
simultaneously (regardless of whether that range agrees with FEM ozone results). OIC
duplication will be calculated as:

OIC Duplication = (Number Same Range J Number DuplicatesJ) x 100	(3)

where Number Same Ranged is the number of cases in which a user reading simultaneously
exposed reagent spots on duplicate cards reports the same OIC indication range from each card,
and Number Duplicatesd is the total number of cases in which one user reads such duplicate
exposed reagent spots. This calculation will be done for the data from the five ozone
concentrations in laboratory testing, and separately for any cases of duplicate cards used in the
field testing performed by SCAQMD and BCLA.

User agreement will be calculated in the same manner, except using data from multiple users
reading the same exposed reagent spot on a single OIC, i.e.:

User Agreement = (Number Same RangeJNumber Duplicatesu) x 100	(4)

where Number Same Rangeu is the number of cases in which two users reading the same exposed
reagent spot on a card report the same ODC indication range, and Number Duplicatesu is the


-------
Ozone Indicator Cards
Test/QA Plan
Page 29 of 45
Version 1
November 10, 2009

total number of cases in which two users read the same exposed reagent spot. This calculation
will be done for the data from the duplicate users with five ozone concentrations in laboratory
testing, and separately for any cases of duplicate users in the field testing performed by
SCAQMD and BCLA.

HI. 2.4—Effect of Light Intensity

The effect of light intensity on OIC performance will be assessed using the data from the
laboratory testing (section Bl.1.4), by calculating the accuracy, variability, and OIC duplication
of readings (Sections B 1.2.1 through Bl.2.3) of the test data at each of the three light intensity
conditions. Those results will be compared to indicate whether light intensity has any apparent
effect on the OIC performance at a constant ozone concentration. Accuracy or OIC duplication
results that differ by more than 20% accuracy or 20% duplication will be taken as evidence of a
significant light intensity effect. Similarly, variability results that differ by one or more OIC
indication ranges will be taken as evidence of a significant light intensity effect.

B.l.2.5—Effect of Ambient Conditions

The effect of temperature, RH, and wind speed on OIC readings will be assessed by segregating
the SCAQMD results into those that are accurate relative to the FEM (as defined in Section
B 1.2.1), and those that are inaccurate. The temperature, RH, and wind speed data provided by
SCAQMD for these two data sets will then be compared to assess whether the conditions
associated with inaccurate OIC readings are significantly different from those associated with
accurate OIC readings. Comparison of the data sets for significance of differences will be based
on t-test comparisons of means. Should data sets contain small numbers of samples (n < 10),
comparison of ranges will be used as an alternative procedure, based on tabulated values of Cn
and the equation:

H = x ± CnR	(5)

where |i is the population mean, x is the mean of a small set of samples, and R is the range of
values.5


-------
Ozone Indicator Cards
Test/QA Plan
Page 30 of 45
Version 1
November 10, 2009

B1.3 Reporting

The statistical comparisons described above will be conducted separately for each OIC being
tested, and user comments on the operational factors will be compiled and reported. The data for
each OIC will be kept separate from data for all other OIC's, and no intercomparison of the data
from different vendors' OICs will be performed at any time. A verification report will be
prepared for each OIC tested, that presents the test procedures and test data, as well as the results
of the statistical evaluation of those data.

Operational aspects of the OICs will be recorded by testing staff at the time of use, and
summarized in the verification report. The verification report will briefly describe the ETV
program, the AMS Center, the test equipment and test conditions, and the procedures used in
verification testing. The results of the verification test will be stated quantitatively, without
comparison to any other OICs tested or comment on the acceptability of the technology's
performance. The draft verification report will first be subjected to review by the technology
vendor, BCLA, and SCAQMD, and then revised and subjected to a review by EPA and/or other
peer reviewers. The EPA comments and the peer review comments will be addressed in further
revisions of the report, and the comments and responses will be tabulated to document the peer
review process. The reporting and review process will be conducted according to the
requirements of the AMS Center QMP.1

B2 REFERENCE SAMPLE COLLECTION

Reference sample collection in this verification consists of monitoring ozone in field and
laboratory testing using a properly calibrated FEM analyzer. OIC readings of ozone in air will
then be compared with the corresponding response of the FEM. The procedures and records of
reference method calibrations will be reviewed for both laboratory and field testing as part of the
Technical Systems Audit procedure (Section Cl.l).


-------
Ozone Indicator Cards
Test/QA Plan
Page 31 of 45
Version 1
November 10, 2009

B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS

Field data sheets (Appendices A and B) will be filled in by SCAQMD and BCLA
representatives. Copies of those sheets will be retained by the respective organization that
conducted the field testing, and the originals will be sent to Battelle by tracked shipment (FedEx
or similar) for compilation of the data.

B4 REFERENCE METHOD

The reference method used for ozone determination in this test is the EPA-established
UV absorption FEM, implemented for laboratory testing in the form of a Battelle-owned
commercial continuous real-time ozone analyzer. In laboratory testing the FEM analyzer will be
operated by Battelle staff according to the manufacturers' instructions, including those for warm-
up and stabilization time before testing. In field testing, comparable FEM analyzers will be
operated by appropriate state or local air quality agencies, with adherence to quality requirements
specified for such monitoring in the Code of Federal Regulations. The laboratory FEM ozone
analyzer will be operated according to requirements that exceed those for ambient ozone
monitoring.6 Specifically, the laboratory FEM ozone analyzer will be calibrated prior to testing,
and then subjected to an ozone zero/span check at the start of each day of testing, using the
Environics 6400 ozone transfer standard noted in Section A8. This transfer standard itself will
be calibrated against the Dasibi Model 1008 UV primary calibration photometer at the start of
testing. In that calibration ozone output of the Environics 6400 will be monitored using the
Dasibi photometer at several settings of the Environics 6400, and a calibration curve of output
ozone concentration versus Environics setting will be established.

B5 QUALITY CONTROL REQUIREMENTS

Quality of the laboratory reference ozone measurements will be assured by a calibration of the
FEM ozone analyzer before any testing, and a daily one-point check of the FEM analyzer at the
start of each day of testing, using the ozone transfer standard. The pre-testing calibration slope
must equal 1 ± 0.02 and the coefficient of determination (r ) must be greater than 0.95 before
testing can proceed. The FEM reading on the one-point daily check must be within 5% of the
ozone concentration from the transfer standard before testing can begin on that day. The FEM
will be adjusted if necessary to meet that daily requirement. Quality of the field reference
measurements will be assured by SCAQMD records of the calibration and maintenance of FEM
monitors at the field sites.


-------
Ozone Indicator Cards
Test/QA Plan
Page 32 of 45
Version 1
November 10, 2009

B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE

The Thermo Environmental FEM ozone analyzer owned by Battelle and used as the reference
ozone measurement will be tested, inspected, maintained, and calibrated per the manufacturer's
operating instructions, so as to meet the quality control requirements stated in Section B5. Other
equipment such as the temperature and RH monitoring instruments will be obtained from the
Battelle Instrument Services Laboratory and will have been calibrated within the past year.

B7 INSTRUMENT CALIBRATION AND FREQUENCY

Prior to the start of the OIC laboratory testing a multipoint calibration will be performed
on the Battelle FEM analyzer using an Environics Model 6400 ozone generator (transfer
standard) that has itself been calibrated against a Dasibi Model 1008 UV primary calibration
photometer. This calibration will include ozone concentrations in the range of 10 to 150 ppbv;
that range spans all the ozone challenge concentrations to be used in laboratory testing and the
full useful range of the OICs . The ozone calibration standards will be generated in dry high
purity air. Also, on each day of testing the FEM analyzer will be challenged with zero air and a
single point ozone span check. Consistent with CFR requirements,6 the daily ozone checks will
be in the range of 10 to 100 ppbv and will be supplied to the analyzer through the entire normal
sample flow path, including the inlet line, particle filter, and internal analyzer plumbing
including the ozone scrubber. However, to assure consistent data quality in laboratory testing of
OICs, the daily frequency of these ozone checks greatly exceeds the biweekly frequency required
for continuous ambient ozone monitoring.6

B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES

All materials, supplies, and consumables will be ordered by the Verification Test Coordinator or
designee. Where possible, Battelle will rely on sources of materials and consumables that have
been used previously as part of ETV verification testing without problems.

B9 NON-DIRECT MEASUREMENTS

Data published previously in the scientific literature will not be used to evaluate the vendor's
technology during this verification test.

B10 DATA MANAGEMENT

Various types of data will be acquired and recorded electronically or manually by Battelle,
SCAQMD, and BCLA during this verification test. Table 3 summarizes the types of data to be


-------
Ozone Indicator Cards
Test/QA Plan
Page 33 of 45
Version 1
November 10, 2009

recorded, how and by whom the data will be recorded, and how the data will be treated or used.
All observations relevant to the laboratory testing of the OICs will be documented by Battelle
staff in laboratory record books. Results from the laboratory FEM and temperature and RH
monitors will be recorded in electronic format. Preprinted data sheets (Appendix A) will be used
by SCAQMD to record OIC readings, ozone, temperature, RH, and operator observations in the
field. It will also be requested that electronic files be provided by SCAQMD to document ozone,
temperature, and RH. Preprinted data sheets (Appendix B) will be used by BCLA to record OIC
readings and operator observations in the field.

All Battelle LRBs, record books and files are stored indefinitely, either by the Verification Test
Coordinator or Battelle's Records Management Office. EPA will be notified before disposal of
any files.

Records generated by any Battelle staff during the verification test will be reviewed by a Battelle
staff member within two weeks of generation, before the records are used to calculate, evaluate,
or report verification results. This review will be performed by a Battelle technical staff member
involved in the verification test, but not the staff member who originally received or generated
the record. The review will be documented by the person performing the review by adding
his/her initials and date to the hard copy of the record being reviewed. Data entered from hard
copy records into spreadsheets will be reviewed in hard copy within the two-week window.
Spreadsheet entries will be checked as part of the ADQ (Section CI.3). In addition, any
calculations performed by Battelle staff will be spot-checked by another Battelle technical staff
member to ensure that calculations are performed correctly. All such calculations will be done
by incorporating Equations 1 to 5 from Section B1.2 into electronic spreadsheets as formulae,
and this spot-checking will be done by reviewing the correctness of the entered formulae. The
data obtained from this verification test will be compiled and reported independently for each
COA tested.


-------
Ozone Indicator Cards
Test/QA Plan
Page 34 of 45
Version 1
November 10, 2009

Table 3. Summary of Data Recording Process

Data to Be Recorded

Where Recorded

How Often Recorded

Disposition of Data

Dates, times, and details
of laboratory test events

Battelle record books

Start/end of test
procedure, and at each
change of a test
parameter

Used to organize/check test

results; manually
incorporated in electronic
spreadsheets as necessary

Laboratory FEM QA
results

Battelle record books, or
electronically

At FEM calibration or re-
calibration

Incorporated in verification
report as necessary

Temperature, RH, light
intensity data in
laboratory testing

Recorded electronically
and noted in Battelle
record books

Start/end of test
procedure, and at each
change of a test
parameter

Incorporated in verification
report as necessary

Laboratory OIC
readings

Battelle record books

At each reading

Converted to spreadsheet for
statistical analysis and
comparisons

OIC readings at field
sites

Field data sheets by
SCAQMD or BCLA

At each reading

Converted to spreadsheet for
statistical analysis and
comparisons

FEM readings,
temperature, RH, wind
speed at field sites

Field data sheets or
recorded electronically by
SCAQMD

Coincident with OIC
exposures to ambient air

Converted to spreadsheets for
statistical analysis and
comparisons

User observations at
field sites

Field data sheets by
SCAQMD or BCLA

At each use of OICs

Incorporated in verification
report as necessary

Field data sheets filled out by SCAQMD or BCLA representatives will be reviewed by the lead
SCAQMD or BCLA representative for completeness and consistency within one week after they
are filled out. SCAQMD and BCLA will then retain copies of their respective data sheets and
send the originals to Battelle. The Battelle Test Coordinator or Quality Manager will review all
such data sheets immediately upon receipt. Any corrections needed in how SCAQMD or BCLA
representatives fill in the forms will be implemented immediately.

Among the QA activities conducted by Battelle QA staff will be an audit of data quality. This
audit will consist of a review by the Battelle Quality Manager of at least 10% of the test data.
During the course of any such audit, the Battelle Quality Manager will inform the technical staff
of any findings and any immediate corrective action that should be taken. If serious data quality
problems exist, the Battelle Quality Manager will notify the AMS Center Manager, who is


-------
Ozone Indicator Cards
Test/QA Plan
Page 35 of 45
Version 1
November 10, 2009

authorized to stop work. Once the assessment report has been prepared, the Verification Test
Coordinator will ensure that a response is provided for each adverse finding or potential
problem, and will implement any necessary follow-up corrective action. The Battelle Quality
Manager will ensure that follow-up corrective action has been taken.

Field data sheets, laboratory record book entries, and other hard copy data will be converted to
Portable Document Format (.pdf) files and submitted to the EPA Quality Manager along with
compiled Excel files of test data within two weeks after Battelle's ADQ is completed. The EPA
Quality Manager may conduct an ADQ, at her discretion, and appropriate action will be taken in
response to any findings.


-------
Ozone Indicator Cards
Test/QA Plan
Page 36 of 45
Version 1
November 10, 2009

SECTION C
ASSESSMENT AND OVERSIGHT

CI ASSESSMENTS AND RESPONSE ACTIONS

Every effort will be made in this verification test to anticipate and resolve potential problems
before the quality of performance is compromised. One of the major objectives of this test/QA
plan is to establish mechanisms necessary to ensure this. The procedures described in this
test/QA plan, which is peer reviewed by a panel of outside experts, implemented by the technical
staff and monitored by the Verification Test Coordinator, will give information on data quality
on a day-to-day basis. The responsibility for interpreting the results of these checks and
resolving any potential problems resides with the Verification Test Coordinator. Technical staff
have the responsibility to identify problems that could affect data quality or the ability to use the
data. Any problems that are identified will be reported to the Verification Test Coordinator, who
will work with the Battelle Quality Manager to resolve any issues. Action will be taken to
control the problem, identify a solution to the problem, and minimize losses and correct data,
where possible. Independent of any EPA QA activities, Battelle will be responsible for ensuring
that the audits described below are conducted as part of this verification test.

Cl.l Technical Systems Audits

The Battelle Quality Manager will perform a technical systems audit (TSA) at least once during
this verification test, in both the laboratory and field testing locations. The purpose of this audit
is to ensure that the verification test is being performed in accordance with the AMS Center
QMP,1 this test/QA plan, and the published reference method. In the TSA, the Battelle Quality
Manager or a designee may review the reference method used, compare actual test procedures to
those specified or referenced in this plan, and review data acquisition and handling procedures.
In the laboratory, the Battelle Quality Manager may inspect the ozone challenge delivery system,
observe the ozone monitoring and test procedures, and review data records. He may also check
calibration certifications for test measurement devices. In the field the Battelle Quality Manager
may inspect SCAQMD site QA records, observe SCAQMD and BCLA test procedures, and
review data records. At his discretion, the Battelle Quality Manager may conduct a TSA at both
SCAQMD and BCLA sites in both the Fall 2009 and Summer 2010 field periods (see Section
A7.2). A TSA report will be prepared, including a statement of findings and the actions taken to


-------
Ozone Indicator Cards
Test/QA Plan
Page 37 of 45
Version 1
November 10, 2009

address any adverse findings. The EPA AMS Center Quality Manager will receive a copy of
Battelle's TSA report. At EPA's discretion, EPA QA staff may also conduct an independent
TSA during the verification test. The TSA findings will be communicated to technical staff at
the time of the audit and documented in a TSA report.

C1.2 Performance Evaluation Audit

A PE audit of the ozone measurements will be done to confirm the accuracy of the Battelle-
owned Dasibi 1008 UV photometer as the basis for the FEM calibration in this verification. A
side-by-side comparison will be done to establish the traceability of the Battelle Dasibi 1008 UV
photometer relative to the standard photometer owned by the Ohio EPA, which is also a Dasibi
1008, and which is traceable to the primary ozone standard reference photometer located at EPA
Region 5, Chicago, Illinois. In the side-by-side comparison, agreement of the Battelle and Ohio
EPA photometers within 2% or 1 ppbv ozone (whichever is larger) will be the target. Should
agreement fall outside this tolerance, the Battelle photometer data will be corrected to match the
Ohio EPA photometer readings.

C1.3 Audit of Data Quality

The Battelle Quality Manager will audit at least 10% of the verification data acquired in the
verification test. The Battelle Quality Manager will trace the data from initial acquisition,
through spreadsheet entry, reduction and statistical comparisons, to final reporting. All
calculations performed on the data undergoing the audit will be checked.

CI.4 QA/QC Reporting

Each assessment and audit will be documented in accordance with Sections 3.3.4 and 3.3.5 of the
AMS Center QMP.1 The results of all audits will be submitted to EPA. Assessment reports will
include the following:

•	Identification of any adverse findings or potential problems

•	Response to adverse findings or potential problems

•	Recommendations for resolving problems

•	Confirmation that solutions have been implemented and are effective

•	Citation of any noteworthy practices that may be of use to others.


-------
Ozone Indicator Cards
Test/QA Plan
Page 38 of 45
Version 1
November 10, 2009

C2 REPORTS TO MANAGEMENT

The Battelle Quality Manager, during the course of any assessment or audit, will identify to the
technical staff performing experimental activities any immediate corrective action that should be
taken. If serious quality problems exist, the Battelle Quality Manager will notify the AMS
Center Manager, who is authorized to stop work. Once the assessment report has been prepared,
the Verification Test Coordinator will ensure that a response is provided for each adverse finding
or potential problem and will implement any necessary follow-up corrective action. The Battelle
Quality Manager will ensure that follow-up corrective action has been taken. The test/QA plan
and final report are reviewed by EPA AMS Center QA staff and EPA AMS Center program
management staff. Upon final review and approval, both documents will then be posted on the
ETV website (www.epa.gov/etv).


-------
Ozone Indicator Cards
Test/QA Plan
Page 39 of 45
Version 1
November 10, 2009

SECTION D
DATA VALIDATION AND USABILITY

D1 DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS

The key data review requirements for the verification test are stated in Section BIO of this
test/QA plan. The QA audits described in Section CI of this document, including the audit of
data quality, are designed to assure the quality of the data. Data will be verified for
completeness, correctness, and compliance with the procedures as written in this test/QA plan.

D2 VALIDATION AND VERIFICATION METHODS

Section C of this test/QA plan provides a description of the validation safeguards employed for
this verification test. Data validation and verification efforts include the use of an EPA-certified
FEM and appropriate ozone calibration equipment, and the performance of TSA and data audits
as described in Section C. An audit of data quality will be conducted by the Battelle Quality
Manager to ensure that data review and validation procedures were completed, and to assure the
overall quality of the data. Any findings will be communicated to technical staff at the time of
the audit and documented in a report.

D3 RECONCILIATION WITH USER REQUIREMENTS

The purpose of this test is to evaluate the performance of OICs. The data obtained should
include thorough documentation to allow verification of the performance of each OIC. Potential
limitations of the data are primarily expected to take the form of incomplete recording of data on
test forms, e.g., card numbers, reagent spot numbers, site or user identification. Such limitations
will be minimized by instructions for use of the forms, review of the data forms, and prompt
correction of recording procedures. The data review and validation procedures described in the
previous sections will assure that data meet these requirements and are accurately presented in
the evaluation reports generated from this test. Data for which incomplete recording cannot be
resolved will be retained but will be flagged and not used for OIC verification.

This test/QA plan and the resulting ETV verification report(s) will be subjected to review by the
OIC vendors, BCLA, SCAQMD, EPA, and expert peer reviewers. These reviews will assure
that this test/QA plan and the resulting report(s) meet the needs of potential users of OICs. The


-------
Ozone Indicator Cards
Test/QA Plan
Page 40 of 45
Version 1
November 10, 2009

final report(s) will be submitted to EPA in Word and Adobe PDF compliant format and
subsequently posted on the ETV website.


-------
Ozone Indicator Cards
Test/QA Plan
Page 41 of 45
Version 1
November 10, 2009

SECTION E
REFERENCES

1.	Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center,
Version 7.0, Environmental Technology Verification Program, Battelle, November 2008.

2.	Environmental Technology Verification Program Quality Management Plan. Version 3.0,
EPA/600/R-08/009, January, 2008.

3.	Federal Reference and Equivalent Methods established by Code of Federal Regulations,
Title 40, Part 53; a list of commercial devices designated as Reference or Equivalent
Methods is available at http://www.epa.gov/ttn/amtic/criteria.html.

4.	Leston, A.L., W.M. Ollison, C.W. Spicer, and J. Satola, J. Air Waste Manage. Assoc., 55,
1464-1472, 2005.

5.	Statistics in Quantitative Analysis, in Chemical Analysis. H.A. Laitinen, McGraw-Hill,
New York, 1960.

6.	Code of Federal Regulations, Title 40 Chapter 1 Part 58 Appendix A - Quality Assurance
Requirements for SLAMS, SPMs, and PSD Air Monitoring, Environmental Protection
Agency, July 1, 2009 edition.


-------
Ozone Indicator Cards
Test/QA Plan
Page 42 of 45
Version 1
November 10, 2009

Appendix A
Field Data Sheet for Use by SCAQMD


-------
Ozone Indicator Cards
Test/QA Plan
Page 43 of 45
Version 1
November 10, 2009

Field Data Sheet for Use by SCAQMD in Verification of Ozone Indicator

Cards (OICs)

Date	 Field Site	User(s) Initials 1)	2)

OIC Reagent Spot Time	Reading FEM O3 Temp RH Wind Spd.

Number Number Start Stop Init/Fnla (ppb) (F) (%) (mph)

/

/

/

/

/

/

/

/

/

/

a: Init = initial reading when foil first removed from spot; Fnl = reading after spot exposed to air.
User Observations:


-------
Ozone Indicator Cards
Test/QA Plan
Page 44 of 45
Version 1
November 10, 2009

Appendix B
Field Data Sheet for Use by BCLA


-------
Ozone Indicator Cards
Test/QA Plan
Page 45 of 45
Version 1
November 10, 2009

Field Data Sheet for Use by BCLA in Verification of Ozone Indicator Cards

(OICs)

Date	 Field Site	User(s) Initials 1)	2)	

OIC Reagent Spot Time	Reading Personal Monitor	Reading

Number Number Start Stop	User la	Location	User 2a

				__/	/_

				__/	/_

				__/	/_

				__/	/_

				__/	/_

				__/	/_

				__/	/_

				_/	/_

				_/	/_

/ /

a: Record initial reading (when foil first removed from reagent spot) and final reading (after exposure to air).
Additional User Observations:


-------