c/EPA
United States
Environmental Protection
Agency
Office of Air Quality
Planning and Standards
Research Triangle Park NC 27711
EPA-450/2-84-008
December 1984
Air
National Air Audit System
Guidance Manual for FY 85
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
Office of Air Quality Planning and Standards
Research Triangle Par!v North Carolina 2771 1
18 DEC 1984
MEMORANDUM
SUBJECT: Distribution of the FY 1985 National Air
Audit. System (NAAS) Guidance^'Manual ^
'
FROM: Darryl D. Tyler, D i rector
Control Programs Development Div«fion (MD-15)
TO: Director, Air and Waste Management Division
Regions II, IV, VI-VIII, X
Director, Air Management Division
Regions I, III, V, IX
Attached is the FY 1985 NAAS Guidance Manual to be used in auditing
State and local air pollution control agencies. This year, all of the
audit material (guidelines, protocol, and questionnaires) has been combined
.Into one manual. This manual was developed jointly by STAPPA/ALAPCO and
EPA. I wish to thank the Regional Office personnel for their participation
and assistance in developing and reviewing this manual.
The NAAS was first implemented .in' FY 1984 and the results of that
effort are contained in a FY 1984 national report that will be published
separately. This year's audit manual is based on last year's manual,
with revisions to the questionnaires and more specific instructions for
reporting the audit results. A vehicle inspection and maintenance chapter
has also been added for FY 1985.
Last year a workshop was held for the Regional Office audit team members.
This year in lieu of a workshop, each of the EPA committee expediters
will hold conference calls in early January with the Regional Office
audit personnel to explain the changes in the FY 1985 guidelines, questionnaires,
and procedures. The tentative date for these conference calls will be
during the second week in January 1985. More information will be provided
as soon as the dates are set.
Several Regional Offices have asked for copies of the questionnaires
on a Lexitron disc. We are planning to accommodate this request where
possible. The air quality planning and SIP activity, new source review,
-------
-2-
and compliance assurance chapters will definitely be available on disc
and will be sent to the Regional Offices in the immediate future. If we can
be of further assistance, please contact Tom Williams or John Silvasi at
FTS 629-5665.
Attachment
cc: Regional Administrator, Regions I-X, w/o attachments
Chief, Air Branch, Regions I-X
Director, Environmental Services, Regions I-X
Audit Contacts, Regions I-X
Jack Farmer, ESED,
Richard Rhoads, MDAD
John O'Connor, SASD
Edward Reich, SSCD
Bill Barnard; ORD
John Calcagni, CPOB
Ron Campbell, OAQPS
Gerald Emison, OAQPS
Tom Helms, CPOB
Phil Lorang, OMS
John Rasnic, SSCD
-------
EPA-450/2-84-008
National Air Audit
System Guidance Manual
for FY 85
Control Programs Development Divisioji
U.S. ENVIRONMENTAL PROTECTION AGENCY
Office of Air and Radiation \
Office of Air Quality Planning and Standards
Research Triangle Park, North Carolina 27711
December 1984
-------
This report has been reviewed by the Control Programs Development Division
of the Office of Air Quality Planning and Standards, EPA, and approved for
publication. Mention of trade names or commercial products is not intended
to constitute endorsement or recommendation for use. Copies of this report
are available through the Library Services Office (MD-35), U.S. Environmental
Protection Agency, Research Triangle Park, North Carolina 27711; or, for a
fee, from the National Technical Information Services, 5285 Port Royal
Road, Springfield, Virginia 22161.
n
-------
FOREWORD
The second year of the National Air Audit System will continue the
program we started last year. Based on what we learned from last
year's experience, we have made considerable improvements in the program.
This System continues to be a unique example of a program conceived as a
State/local/EPA cooperative effort which has remained so throughout its
development and into the second year of implementation. I was very
pleased with the results of last year's audits and particularly gratified
to hear that the vast majority of agencies audited last year believed
that the the audit system met their expectations. I have every reason to
believe that these improvements will help this second year's audits to
surpass the benefits of last year.
This audit guidance manual--! ike last year's—was prepared under a
process that exemplifies a cooperative effort among all levels of government-
Federal, State, and local. In many ways, the continuation of a process of
developing and implementing a program is even more difficult than initiating
one. The movement from pilot program to established procedure is to be
commended. I believe that the phrase "State/local/EPA partnership" has
been shown to have true meaning in the continued commitment to the National
Air Audit System, and I continue to hope that this program can serve as a
model for the development of other shared programs.
I look forward with interest to reviewing the progress made in
implementing these National Audit Guidelines. We must all strive to
build the best possible air pollution control agencies at every level of
government. The National Air Audit System will be a useful tool to help
us all improve the management of our programs and provide the most effective
environmental protection for all of our citizens.
Joseph A. Cannon
Assistant Administrator for
Air and Radiation
i i i
-------
Table of Contents
Chapter Section Page
1. Introduction 1-1
Goals and Objectives of the National Audit System .... 1-3
Audit Protocol 1-5
Audit Reports and Use of Audit Data 1-7
Future NAAS Activities 1-8
2. Air Quality Planning and SIP Activity 2-1
Introduction 2-1
Protocol 2-1
Air Quality Evaluation 2-2
Emission Inventories 2-3
Modeling 2-3
SIP Evaluation and Implementation 2-3
Questionnaire 2-5
3. Mew Source Review 3-1
Introduction 3-1
FY 85 NSR Audit—New Features 3-1
Permit File Selection 3-2
Description of the Permit File Questionnaires 3-4
Appendix I—Reference Tables for Use With FY 35
NSR Questionnaires 3-15
Appendix 2—MSR Permit Summary Questionnaire. ...... 3-17
Appendix 3—1985 Permit File Questionnaire for Major
Sources Subject to PSD or Part D (Offsets) 3-19
Appendix 4---1985 Permit File Questionnaire for Sources
' not Subject to PSD or Part D (Offsets) 3-34
4. Compliance Assurance 4-1
Introduction 4-1
Periodic Review and Assessment of Source Data 4-1
File Review 4-5
Overview Inspection 4-8
5. Air Monitoring 5-1
Introduction 5-2
Regulatory Authority to Perform a System Audit 5-6
Guidelines for Preliminary Assessment and System Audit
Planning. . 5-10
Protocol for Conducting System Audits of State and Local
Agency Performance 5-16
Criteria for the Evaluation of State and Local Agency
Performance 5-31
System Audit Questionnaire (Short Form) 5-37
System Audit Questionnaire (Long Form) 5-57
Example Response to the National Air Monitoring
Questionnaire (Short Form) 5-144
IV
-------
Table of Contents (continued)
Chapter Section Page
6. Motor Vehicle Emissions Inspection Programs 6-1
Table of Contents 6-1
Introduction and Purpose 6-2
Audit Topics 6-3
Audit Process for I/M Programs 6-7
Appendix A--Description of I/M Program Elements 6-20
Appendix B--Inspection/Maintenance Program Audit
Questionnaire 6-30
Appendix ^-Description of On-Site Audit Activities . . . 6-57
Appendix D--Procedures Used in OMS Tampering Survey . . . 6-66
Appendix E--Instructions and Forms for Audit Activities . 6-85
Appendix F--Checklist for Completing the I/M Audit. . . . 6-125
-------
ACKNOWLEDGEMENTS
A note of thanks to the following people who contributed time and
energy to make the 1985 audit guidelines a reality.
PROJECT MANAGEMENT AND COORDINATION
Bill Becker
John Calcagni
Tom Helms
John Silvasi
Darryl Tyler
Tom Williams
STAPPA/ALAPCO
EPA, OAQPS
EPA, OAQPS
EPA, OAQPS
EPA, OAQPS
EPA, OAQPS
AUTHORS
Air Quality Planning
and SIP Activities
Bob French State of Delaware
Chairman
New Source Review
Robert Col lorn State of Georgia
Chairman
Bill Baker
Dave Calkins
Nick Nikkila
Bob Offutt
Dave Redic
Steve Rothblatt
Ray Vogel
Compliance Assurance
EPA, Region II
EPA, Region IX
State of Missouri
Louisville, KY
Dayton, OH
EPA Region V
EPA, OAQPS
Michael Bradley
Dan deRoeck
Norm Glazer
Lori Lawrence
John Paul
Roger Pfaff
Ron Van Mersbergen
Air Monitoring
NESCAUM
EPA, OAQPS
Philadelphia,
EPA, Region i
Dayton, OH
EPA, Region IV
EPA, Region V
Bill Meyer State of Virginia
Chairman
William Sylte State of California
Chairman'
Wayne Cropp
Jack Divita
Frank Giaccone
Larry Kertcher
Curt Marshall
Tom Maslany
Robert Miller
John Rasnic
Jim Mil burn
Chattanooga-Hami1 ton
Co., TN
EPA, Region VI
EPA, Region II
EPA, Region V
Dayton, OH
EPA, Region III
State of Michigan
EPA, OAQPS
EPA, Region IV
Don Arkell
Bill Barnard
Norm Beloin
Vic Guide
John Helvig
Jim Lents
Bob Ostrowski
Stan Sleva
Lane Co., OR
EPA, ORD, EMSL
EPA, Region I
EPA, Region III
EPA, Region VII
State of Colorado
Phil del phi a, PA
EPA, OAQPS
VI
-------
AUTHORS (continued)
Inspection and Maintenance
Jim Lents State of Colorado
Chairman
George Ferrari State of Maryland
Phil Lorang EPA, QMS
Robert Miller State of Michigan
Bob Offutt Louisville, KY
Herbert Wortreich State of New Jersey
-------
INTRODUCTION
-------
1. INTRODUCTION
This audit manual was developed through the joint efforts of the
State and Territorial Air Pollution Program Administrators (STAPPA), the
Association of Local Air Pollution Control Officials (ALAPCO), and the
Environmental Protection Agency (EPA). The project was directed by a
work group composed of members from the three organizations. The audit
guidelines and questionnaires in this manual were written and approved by
five subcommittees of this work group. The program areas selected by the
work group for development of audit guidelines were: (a) air quality
planning and SIP activity, (b) new source review, (c) compliance assurance,
(d) air monitoring, and (e) vehicle inspection and maintenance (I/M).
The subcommittees were chaired by State agency personnel with EPA
staff serving as expediters. All State agencies, local agencies, EPA
Regional Offices, and EPA Headquarters were provided an opportunity to
comment on a draft version of these audit guidelines. This manual, from
concept to finished product, is the result of a cooperative effort between
State agencies, local agencies, and EPA. While it necessarily reflects a
compromise between the competing interests of depth of analysis, breadth
of review, and resources to accomplish the audit, it does represent the
"real world" ideas of people.who actually implement the duties.
The National Air Audit System (NAAS) was first implemented in 1984;
the results of that effort are contained in an FY 84 National Report,
published spearately. This FY 85 audit manual has been based on last
year's manual, but has been revised to include new emphases and to upgrade
the quality of the audit effort over last year. As with last year's
manual, this manual has undergone review by STAPPA, ALAPCO, and EPA
Regional Offices and headquarters.
Last year, EPA set forth overall policy for the NAAS, based on
agreement with STAPPA/ALAPCO and the Regional Offices. The portions of
that policy that will continue to apply this year, together with new
emphases, follow:
0 Program Coverage - Coverage would consist of the same four
topics (Air Quality Planning and SIP Activities, New Source
Review, Air Monitoring, and Compliance Assurance) as last year.
In addition, a fifth topic—Vehicle Inspection/Maintenance--
Should be added.
o Audit Teams - EPA Regional Offices will select the composition of
the audit teams. A crossover team approach (State/local and
outside Regional Office representatives) is possible for FY 85
if Regions and affected States so desire. All audit team members
should have sufficient knowledge of the programs they audit; the
1-1
-------
audit is not intended as an on-the-job training program for
inexperienced personnel. Also, as a minimum, middle or senior
Regional management should personally participate in the exit
interview with the agency director.
Audit Coverage - Building on last year's results, all States and
selected locals should be audited during the year. As was the
case last year, each of the four existing audit topics should be
covered at each State agency. For local agencies, fanly those
topics for which the specific implementing authority.Crests with
the local agency should be audited. For I/M, only th'ose programs
which are listed in the Strategic Planning and Management System
(SPMS) guidance (New York, Delaware, Georgia, North Carolina,
Missouri, Texas, Nevada, and Oregon) will be required to be
audited this year. This restricted coverage is due to the
newness of the I/M audit guidelines and questionnaire and to
limited resources in Headquarters to support I/M audits.
Optional I/M Audits - With the mutual consent of the State, Regional
Offices are encouraged to conduct audits for other I/M programs
which have been implemented for at least one full year. Any
such optional audit in FY 85 should follow the NAAS guidelines;
Headquarters will support optional Regional audit activities to
the extent possible. Results of optional audits should be
included in the NAAS reports when the schedule allows, but may
also be reported separately.
On-site Visits and Pre-preparation - Regions should continue to
conduct on-site audit visits, but should maximize use of telephone
interviews and "on-hand" information to prepare for the visit
and to complete significant portions of the questionnaire.
Questionnaires should be exchanged and results studied and
evaluated prior to each on-site visit.
Schedule - Regions may begin their FY 85 audit as early as they
wish, so long as the audited agency is agreeable and the previous
item ("Onsite Visits and Pre-preparation") is accomplished .prior
to the on-site visit. All audits should be completed by May 31,
1985, and draft reports to EPA Headquarters by the end of
June 1985.
National Workshop - No national workshop will be conducted on
the four existing audit guidelines in FY 85. Sufficient informa-
tion currently exists on these such that a workshop is not
necessary. A workshop for the new I/M area was held in Ann
Arbor, Michiqan, on November 28 and 29, 1984, for EPA Regional
I/M audit staff.
1-2
-------
Corrective Actions - Actions to implement needed improvements
identified in the audits will be initiated through existing
mechanisms, i.e., 105 grants, State/EPA agreements, etc.
Questionnaires - All Regions must use the uniform questionnaires
contained in this manual. Additional questions may be appended only
with the mutual consent of the audited agency. Such additional
questionnaires and/or information requests should be clearly
identified as being independent of the NAAS.
National Report - The OAQPS will prepare a national report based
upon the resul ts of all Regional audits. The national report
will not rank agencies or focus on deficiencies of specific
agencies. Before they are finally issued, drafts of the national
report will be reviewed by EPA Regions and representatives of
STAPPA and ALAPCO.
Replacement for Other Audits - The NAAS will replace previous
Regional Office audit activities. It may not, however, replace
all portions of the 105 grant evaluations that are specified in
the grant regulations. Regions have the flexibility to expand
the audit to address additional topics or to explore specific
areas in more detail, subject to agreements reached with individual
States.
Other Oversight - When fully developed, the NAAS will eliminate
the need for much of the "item by item" Regional Office oversight
on certain State and local agency programs. The NAAS, however, will
not be a substitute for the necessary flow of communications be-
tween State/local agencies and EPA Regional Offices.
GOALS AND OBJECTIVES OF THE NATIONAL AIR AUDIT SYSTEM
The purpose of developing a national audit manual is to establish
standardized criteria for the EPA Regions' audit of State and local air
program activities. The primary goals of this program are to determine
the obstacles (if any) which are preventing the State or local air pollu-
tion control agencies from being as effective as possible in their air
quality management efforts and to provide EPA with quantitative information
on how to define more effective and meaningful national programs. States
are playing a larger role than ever in the planning and implementation of
complex, and often controversial, air pollution control strategies. EPA
oversight of these and related activities is necessary for ensuring
national consistency and for assisting the States in resolving identified
problems.
1-3
-------
The EPA and States can also use these audit results to ensure that
available resources are being focused toward identified needs (e.g.,
attainment and maintenance of standards, adoption of regulations, implemen-
tation of regulations and technical analyses to support control strategy
development).
The EPA also hopes to share the results of these audits in a manner
that permits the "cross-fertilization" of innovative approaches and
systems across States and Regions. Only through this national exchange
can we hope to benefit from the invaluable experiences gained to date by
control agencies in carrying out the requirements of the Clean Air Act.
This audit guideline outlines a program which EPA, State, and local
air pollution control agencies can jointly use to--
o Meet statutory requirements;
° Assist in developing at least a minimally acceptable level of
program quality;
o Allow an accounting to be made to the Congress and the public of
the achievements and needs of air pollution control programs;
° Enable EPA, States, and local agencies to agree on needed technical
support and areas where program improvements (including regulatory
reform) should be made; this includes improvements to both EPA
and State/local programs; and
° Maximize and effectively manage available resources within the
State and local agencies and EPA, resulting in expeditious attain-
ment and maintenance of ambient air quality standards as soon as
possible.
o Promote a better understanding of the problems facing air pol-
lution control agencies, thereby fostering mutual respect among
EPA, State, and local agency staff.
State, local, and EPA Regional Offices, working together, may identify
items in addition to those of the national program that are worthy of
further audit attention. In identifying these, the EPA Regional Office
and the State/local agency should understand in advance what the reasons
are and what the objectives and expected result(s) of this expanded
review will be. Also, the NAAS is not intended to preclude EPA Regions
from dealing, on a case-by-case basis, with significant deficiencies
which are identified during the course of the audit.
The NAAS builds upon oversight procedures that were already in use
in EPA Regional Offices before the FY 84 audit. In addition, the guide-
lines and the audit itself are not a substitute for the necessary daily
flow of communication between State/local agencies and EPA Regional
Offices. It is expected that the preparation for the audit (see Audit
Protocol) will utilize currently available reports from State and local
1-4
-------
agencies to EPA as fully as possible, and that EPA will work to minimize
the demands on State or local staff time in conducting the audit. It is
also expected that each State and local agency will cooperate with EPA
during the onsite audit visit.
EPA, State, and local agencies should keep in mind that the audit is
intended to improve the overall quality of air pollution control programs.
This intent of improving overall performance needs to be clearly understood.
The standards of performance outlined by these guidelines are not so
rigid that they eliminate the flexibility afforded by the Clean Air Act.
Also, these guidelines should not be construed to establish performance
standards which must absolutely be achieved in practice. Moreover, while
participating agencies will use the audit to point out where opportunities
exist for State or local improvements, it is not expected that the audit
will address every problem. EPA, however, will continuously search for
and disseminate information about better ways of consistently, effectively,
and efficiently implementing a comprehensive air pollution control program.
This includes possible reforms of EPA's requirements where feedback from
the audits suggests that certain requirements detract from program
effectiveness.
AUDIT PROTOCOL
Each Regional Office must tailor the structure of the audit according
to the particular characteristics of the State and local agencies in the
Region and its own operating procedures. Certain elements and procedural
steps, however, appear necessary or useful in FY 85 based on the FY 84
experience. These are discussed below.
Advance Preparation
EPA should send a letter to the control agency well in advance of
the audit. The letter should confirm the date and time of the audit (see
"Onsita Visit"), and describe what resources the State is expected to
provide, such as office space and staff time. This letter should also
identify the name and title of each EPA individual who will participate
in the audit.
With the exception of file audits and similar questionnaires, the EPA
Region will provide the control agency with the nationally prepared ques-
tionnaire. These should be sent to the State or local agency about a
month in advance of the audit and, thus, will allow the agency to better
prepare for the audit. The State or local agency should fill out the
questionnaire in advance; this audit information should be readily available,
although it may require time to gather and compile.
The State or local agency should return a copy of the completed
questionnaire to the Regional Office at least a week before the on-site
visit. The Regional Office subject experts should review the completed
1-5
-------
questionnaire and use it to prepare the audit team for the on-site visit.
Returning the completed questionnaire to the Regional Office before the visit
should serve to minimize wasted effort and time reviewing the questinnaire
during the on-site visit and to prepare the audit team for discussions
that focus on any problems uncovered in the questionnaire. Because of
time limitations, however, it may not be possible in all cases to return
the completed qustionnaire to the Regional Office before the on-site
visit; in these cases, the State or local agency will have to make the
completed questionnaires available to the audit team during the on-site visit.
The chapter for each of the audit topics presents the specific
protocol for that audit topic. This include? procedures such as advance
tailoring of the questionnaires for the air quality planning and SIP
activities audit, selection of files for new source review audits, and
instructions for use of the individual questionnaires.
Onsite Visit
The primary purposes of the onsite visit are to--
° engage in a broad discussion with agency staff to gain insight
into any recent changes in the structure of the organization, discuss
specific problem areas of the agency, and become acquainted with
the staff in order to better open up channels of communication;
° discuss and clarify answers to the questionnaire;
* review onsite documents that are too cumbersome to transmit such
as permits, modeling runs, and supporting files; and
° audit by observing the agency's daily operations of programs for
air monitoring, compliance assurance, new source review, planning
and SIP activities, and (where appropriate) vehicle inspection and
maintenance.
Typically, the audit is conducted in three phases:
° The EPA auditors for all programs meet with the State agency
director and top staff to discuss the goals of the audit and to
"break the ice." This meeting usually sets a cooperative tone for
the visit.
° The EPA auditors conduct a discussion of the questionnaire with
the person(s) in charge of each of the four or five activities
to be audited.
° An exit interview is held as a wrap-up session .to inform agency
management of the preliminary results of the audit. This promotes
harmony between EPA and the State by giving immediate feedback of
the results in a face-to-face meeting between the people actually
performing the audit and those responsible for the programs being
audited. EPA middle or upper management from the Regional Office
should participate in a least this portion of the on-site visit.
1-6
-------
The time which the audit team spends to complete the various phases
of the onsite visit should not exceed three days. This general rule,
however, will be difficult to adhere to in certain instances, such as
when satellite facilities of the agency being audited must also be visited.
In any event, the duration of the onsite visit should be mutually agreed
upon in order not to create an undue burden to the agency being audited.
AUDIT REPORTS AND USE OF AUDIT DATA
Each State or local agency audit report should contain the findings
for each of the four or five audited areas. The audit report must include
a copy of the completed questionnaire(s) (except file audit or similar
questionnaires—see specific instructions in appropriate chapters of tin's
manual) for each of the audit topics to enable national compilation of
audit results. Since the questionnaires will be included, the audit
report should not merely reiterate the answers on the questionnaires, but
present the Regional Office's overall findings. The Region should give
the State or local agency an opportunity to comment on a draft of the
report before it is released outside of EPA. This allows misunderstandings
and errors to be discovered before the report is made final.
The audit report should contain an executive summary that provider.
the Regional Office's overall assessment of the audited agency's program
after reviewing all the questions as a whole.
Major deficiencies identified during the audit should also be high-
lighted in the executive summary. This enables the Region to detail all
the findings of the audit without causing the reader to confuse minor
points with major problems. It also identifies to the audited agency
those deficiencies considered most serious.
Where an agency disagrees with the conclusions of the audit, it
should provide to EPA written comments outlining their perspective.
These will be incorporated as an appendix into the final report. The
report should also highlight outstanding and/or innovative program pro-
cedures that are identified.
The audit would be of limited use without some mechanism for
rectifying identified deficiencies. Therefore, it is important that the
report recommend measures or steps to treat the causes determined to be
responsible for these inadequacies. Lead agencies responsible for imple-
menting these recommendations and anticipated resource requirements
should also be considered.
In the FY 1984 audits, a number of questions were not directly
answered, but the respondents referred to reports attached to the ques-
tionnaire. These reports were generally not received in OAQPS. Even
when they were, time and resources did not permit EPA to read and evaluate
them for purposes of compiling the national report. For FY 1985, therefore,
each question should be answered on the questionnaire itself; attachments
should be avoided unless a question specifically requests them. If attach-
ments are requested and included, the attachments should be forwarded to
OAQPS along with the questionnaire.
1-7
-------
A recommended format for preparation of the audit report is given in
Table 1-1. Following a standardized format will not only enable reviewers
to easily find material in the text, but will also facilitate compilation
of information into the national report.
A national report compiling the findings of the audits conducted by
the EPA Regions will be prepared by EPA Headquarters. This analysis will
be based on the reports prepared by the EPA Regions discussed above. It
will present the status of implementation of the audit programs throughout
the nation and, if possible, will highlight areas where innovations have
resulted in substantially superior performance. It will not rank agencies
or focus on specific deficiencies in individual programs. While it will
address areas of conflict between EPA guidance and action of implementation
experience, it will not be a forum for addressing unresolved issues
between audited agencies and States. The specific content of the national
report will be similar to the content of the FY 84 national report, but will
allow for new topics.
FUTURE NAAS ACTIVITIES
The implementation of the MAAS in FY 84 was a first step of a "phased-
in" procedure. The FY 84 site visits were an opportunity to discover and
identify differences between the EPA policy as reflected in the audit
guidelines and actual practices in the field, and, as such, established a
baseline. The FY 85 and subsequent site visits should now be used to
measure progress in achieving an efficient and effective program.
EPA expediters plan to hold conference call presentations that
highlight the audit procedures for each of the audit topics for the
benefit of Regional auditors. These presentations are intended to promote
inter-Regional consistency in implementing the FY 35 audits. Around
February or March 1985, EPA plans to issue a more detailed program plan
and schedule for preparation of the draft and final audit reports.
The NAAS initiative is designed as a partnership effort to help EPA
and State and local agencies each do their respective jobs better. It is
our hope that it can become the foundation which all involved can use to make
solid progress in protecting and enhancing the quality of our Nation's air.
1-8
-------
Table 1-1
RECOMMENDED FORMAT FOR AUDIT REPORTS
° Introduction--Purpose of audit, for benefit of potential layman
readers; identify persons on EPA audit team and persons interviewed
in audit visit.
0 Executive Summary—Out!ine major findings, major deficiencies,
and major recomrniendations. As the name implies, it is a summary
designed for the chief executive—the director—of the audited
agency. The summary should cover all audit topics:
°° Air Quality Planning and SIP Activities.
00 New Source Review.
00 Compliance Assurance.
00 Air Monitoring.
00 Vehicle Inspection and Maintenance (where audit is conducted).
° Air Quality Planning and SIP Activities—Follow recommended format
in Chapter II of Audit Manual.
° New Source Review—Follow recommended format in Chapter III of
Audit Manual.
° Compliance Assurance—Follow recommended format in Chapter IV of
Audit Manual.
° Air Monitoring—Follow recommended format in Chapter V of Audit
Manual.
° Vehicle Inspection and Maintenance—Follow recommended format in
Chapter VI of Audit Manual.
° Appendices
oo Completed Air Quality Planning and SIP Activities Questionnaire
00 Completed Mew Source Review Questionnaires (except for permit
file questionnaire)
00 Completed Compliance Assurance Questionnaire
00 Completed Air Monitoring Questionnaire
00 Completed Vehicle Inspection and Maintenance Questionnaire
(where audit is conducted)
00 Audited Agency's comments on draft report (for final report).
1-9
-------
AIR QUALITY PLANNING
AND SIP ACTIVITIES
-------
2. AIR QUALITY PLANNING AND SIP ACTIVITY
INTRODUCTION
The same four FY 1984 major program areas in air quality planning and
State implementation plan (SIP) activity have been carried over for
evaluation in the FY 1985 National Air Audit System (NAAS). These areas
are:
°Air Quality Evaluation
°Emission Inventories
"Modeling
°SIP Evaluation and Implementation
Audit criteria are provided below for each of these program areas.
Each topic is prefaced by a brief discussion of what activities it
encompasses and what we generally hope to accomplish through the audit
review. The audit guidelines are organized to address the broader
aspects of each program area in the "numbered" questions. For FY 1985,
the format of this chapter has been restructured to provide for more
detail and less narrative than in FY 84.
Although most questions are worded to reflect elements of current or
proposed EPA guidance, some questions are intended as surveys of State
and local agency practices. The fact that the questionnaire contains
some of the latter kinds of questions is not intended to imply the
existence of an established (or planned) EPA requirement for the practice
on which the question is based.
PROTOCOL
In most questions, the State or local agency will respond to the
question and the EPA Regional Office will provide comments or evaluation
in the space provided at the end of the question.
In a departure from the FY 1984 audit procedure, however, there are
several questions where the EPA Regional Office must respond or provide
lists prior to any action on the questionnaire by the State or local
agencies. These questions are identified below: items in column 1 should
be completed prior to sending the questionnaire to the State/local
agency. Questions in column 2 should be completed after the State/local
agency has responded to their part of the question.
2-1
-------
± 2^ Regional Office Action
A2 Complete parts a and b under "EPA
Regional Office Response"
C4 Complete columns A, B, and C as appropriate
Dl(Table) Provide State/local agency a list of
revisions/strategies due
D2 Provide State/local agency a list of
studies due
D3 Complete part a. (two columns)
D4 Provide State/local agency a list of TCM's
in SIP
D5 Provide State/local agency a list of 03/CO
extension areas where RFP tracking is
required
D8 Complete parts a, b, and c under "EPA
Regional Office Response"
Procedurally, Regional Offices will, upon receiving the questionnaire
from OAQPS , add to it the items listed in column 1 before sending it to the
State and local agencies. These agencies will then complete the questionnaire
and if time permits, return it to the Regional Office. Regional EPA
staff would then review the State/local agency responses prior to the
on-site audit. If time is short, the questionnaires could be kept by the
State or local agency until the on-site audit. EPA's comments would be
added to the individual questions after the on-site audit. Regional
Office staff should be prepared to discuss all audit items, particularly
questions A2 (Section 107 designations) and D8 (generic bubble rules),
which require completion by EPA after completion of the questionnaire by
the State/local agency.
The draft report from the Regional Office should consist of an overall
executive summary and highlights of each part (i.e.: Air Quality Evaluation,
Emissions Inventories, Modeling and SIP Evaluation and Implementation). Regional
Office comments on individual questions should be given in the space provided on
the questionnaire.
AIR QUALITY EVALUATION
States are continuously gathering air quality data to satisfy various
statutory requirements and air management needs. The adequacy of these
ambient measurements is addressed in a separate audit area dealing with
monitoring requirements. This audit topic is concerned with the demonstrated
2-2
-------
State capabilities to perform air quality evaluations. This includes the
ability to systematically consider available air quality data for the
purpose of trends analyses, Section 107 redesignations, prioritization of
air program activities, and public information. The specific areas that
should be evaluated in this audit category are discussed in the questionnaire
that follows this section.
EMISSION INVENTORIES
The emission inventory provides information concerning source
emissions and defines the location, magnitude, frequency, duration, and
relative contribution of these emissions. An inventory is useful in
designing air sampling networks, predicting ambient air quality, designing
control strategies, and interpreting changes in monitored air quality
data. Plans for attaining and maintaining NAAQS1 are dependent on a
complete and accurate emission inventory.
In the implementation of a nationwide program of air quality
management, consistent methods of inventory compilation are essential.
An adequate emissions inventory must be accurate, complete and up-to-
date, and provide for consistency in planning between metropolitan areas,-
States, and Regions.
MODELING
Air quality models are being used more extensively in the conduct
of day-to-day activities in the planning and SIP program area. These
activities include such things as attainment demonstrations, major source
compliance determinations, new source review, evaluations of "bubbles,"
and assessing attainment status. Most State agencies should have the EPA
reference models on-line that are available for use in these and other
types of applications. The modeling audit is intended to gather information
regarding the agency's demonstrated expertise and capability to perform
necessary air quality modeling analyses consistent with accepted EPA procedures.
In accomplishing this objective, this year's audit reviews the various
kinds of modeling applications performed or evaluated at the State/local
program. Because the Region will have already reviewed certain site-specific
modeling analyses which the State/local agency has submitted (such as
bubbles, new source permits, etc.), the questionnaire asks the Regional
Office to list the results of these evaluations.
SIP EVALUATION AND IMPLEMENTATION
An evaluation of SIP development and implementation activities is
necessary to assess whether State plans for attainment and maintenance of
standards remain responsive to identified needs and are being reasonably
carried out. Certain key element (see audit questions) of specific SIP
strategies have been singled out for evaluation. As discussed in the
2-3
-------
preamble to this document, focusing the NAAS review on these areas is not
intended to preclude the auditor from evaluating other aspects of major
SIP activity in an air program. Nor does it imply that all other SIP
responsibilities are considered less important. Rather, it represents an
attempt to isolate a manageable number of SIP-related efforts for national
review and comparison.
2-4
-------
National Air Audit System
FY 1985 QUESTIONNAIRE
AIR QUALITY PLANNING AND SIP ACTIVITIES CHAPTER
A. Air Quality Evaluation.
A. 1. Air Quality Reports
a. Does your agency regularly make reports available to the public
on air quality data?
State/local agency response:
Yes No
b. If "yes," does the report—
Yes No
0 cover all criteria pollutants?
0 cover all monitors within the agency's
jurisdiction?
0 enable easy comparison to the national
ambient air quality standards?
0 compare current air quality to historical
trends in air quality?
0 show effect of control regulations in
reducing emissions?
0 make use of bar charts, trend line figures,
or similar visuals in presenting the data?
0 present maps depicting the locations of the
ambient monitors?
0 describe each of the pollutants?
0 briefly describe each of the sampling techniques?
c. If "a" is "yes," what is the time period between the end of
the data collection period and publication of the data? months.
EPA Regional Office Response:
2-5
-------
A.2. Does your agency perform a systematic review of Section 107 primary
or secondary attainment status designations and submit proposed changes
to EPA?
State/Local Agency Response:
Yes (go to a, b, c, and d below).
No (briefly explain):
a. Were any Section 107 reviews completed that did not result in a
request for redesignation?
Yes if so, how many in FY 1984?
No
b. In the first column, indicate the geographical unit (county, census,
tract, etc.) for which redesignation was requested.Tn the other columns,
indicate the number of redesignations initiated and the number completed
and submitted to EPA for primary and secondary attainment.
Geographical Initiated Completed & Submitted
Unit Primary Secondary Primary Secondary"
(Number of Redesignations)
0 attainment or
unclassifiable to
nonattainment
0 nonattainment to
attainment or
unclassifiable
0 unclassifiable
to attainment
Indicate below the methods used to review attainment status:
Yes No
staff notification of violations to agency management
consideration of air quality data for past 2-3 years ~
consideration of modeled exceedances ^] ~
consideration of control and emission changes
investigation into cause of violations "
2-6
-------
d. How often does your agency perform this review?
less more
than 123 than
1 yr year years years 3 yrs
TSP
S02
CO
03
EPA Regional Office Response (confirmation or comment):
a. On how many of the above completed and submitted designations (part a,
of this question) has the Regional Office taken final rulemaking?
to nonattainment: primary secondary
from nonattainment: primary secondary
from unclassifiable to attainment: primary secondary^
b. For those cases where final rulemaking has not been taken, please
describe current status and associated problems or issues:
2-7
-------
A.3. The FY84 audit asked if SIP updates occurred to take into account
the results of special air quality monitoring studies. During the year
ending September 30, 1984, please indicate below whether any source-related
enforcement activities, SIP revisions, permit modifications, etc., resulted
from monitoring data that revealed any newly detected violations of a
national ambient air quality standard.
State/Local Agency Response:
a. During FY 1984, did any air quality monitoring sites reveal any newly
detected violations of a national ambient air quality standard?
Yes If so, how many violating sites?
No
b. If "yes," please indicate below the latest action taken for each of the
above violations in each of the following categories (indicate only one
action for each newly-detected violating site):
(Number of Actions)
Initiated Submitted
Only Compl eted to EPA
Enforcement action based on
existing SIP regulations
Change in an individual source permit
Local, county, or areawide SIP revisions
(specify )
Statewide SIP revision (specify )
Other kinds of source-specific SIP revision
(s pec i fy )
Additional monitoring
Verification by modeling studies
(specify )
Data ignored--a documented exceptional event
Other (describe )
EPA Regional Office Response (confirmation or comment):
2-8
-------
B. Emissions Inventories
B.I. General
Last year's audit asked whether your agency maintained an inventory
of emission estimates and related essential data for the criteria pollutants
listed below. Please provide an update this year by answering the following
questions.
State/local agency response
a. When was the complete inventory last updated?
Year of last update by pollutant.
TSP S02 CO
VOC NOX Pb
b. Is your agency required to track reasonable further progress (RFP) for
CO or 03? Yes No
If yes, on an annual basis, does sufficient updating of the emissions
inventory take place to enable the agency to track and demonstrate RFP?
Yes No (Comments:)
c. How does your agency inventory sources or source categories for which
no EPA (AP-42) emission factors exist?
Develop own (if so, please describe:
Ignore source/source category (comments/description:
Other (explain)
d. Are reports prepared which describe the procedures and assumptions used
in preparing the emission inventory? Yes No
If yes, are the procedures and assumptions reviewed by external agencies,
industry or other groups? Yes No
Are they available in your files? Yes No
EPA Regional Office Response (confirmation or comment)
2-9
-------
B.2 Point Sources
a. Please indicate below the size cutpoints used by your agency to distinguish
between point and area sources. (If area sources are not inventoried the
cutpoint is the lower threshold used to inventory point sources.)
State/Local Agency Response
Cutpoint by pollutant in tons/year:
TSP S02 CO
VOC NOX Pb
b. Plese indicate below the size cutpoints used by your agency to distinguish
between major and minor point sources:
Cutpoint by pollutant in tons/year:
TSP SO? CO
VOC NOv Pb
EPA Regional Office Response (confirmation or comments):
2-10
-------
B.3. Area Sources
State/Local Agency Response:
a. Does your agency inventory area sources for any of the following
pollutants? (answer yes or no for each pollutant)
TSP S02 CO
VOC NOX Pb
b. If yes was entered for any of the pollutants in a. above, Indicate
which of the following features characterize the area source emissions inventory,
By pollutant: (indicate by an "x" for all applicable items)
Area El Features TSP S0_£ CO VOC NOX Pb
Statewide coverage
Non-attainment area coverage only ~
County resolved ~
Grid resolved
Annual only ^^ ~^_ _ ~^^ ~
Annual and shorter term
Includes fugitive -
open source emissions (TSP only)
Nonreactive emissions excluded
(VOC only)
Miscellaneous solvent use included
(VOC only)
Waste oil combustion emissions included
(for lead only)
EPA Regional Office (confirmation or comment):
2-11
-------
B.4. Mobile Sources
State and Local Agency Response:
The State or local agency should answer the following questions even
if a transportation agency such as an MPO or State DOT is responsible for
the mobile source inventory:
a. Does your agency maintain a mobile source emission inventory?
Yes No (If "no" please indicate the responsible agency:
b. What is the base year for the mobile source inventory in the
current SIP?
c. Which emissions model (Mobile 1, 2, 2.5, or 3) was used to
generate the base year mobile source inventory?
d. What pollutants are maintained in the base year mobile source
i nve nto ry ?
e. What pollutants are contained in the projected year mobile source
i nventory?
f. What is the projected attainment year in the SIP for the mobile
sources for CO: for 03: ?
g. Which emissions model is currently used to generate updates to
the mobile source emission inventory?
EPA Regional Office response:
2-12
-------
B.5. Emissions Inventories - Update Frequency
a. Please indicate which of the following status changes would
result in a change or update of records in the emission inventory:
State/Local Agency response:
Point Sources:
Source inspection
Source permit expiration
Source reporting
Source test
Source permit application
Continuous emission monitor report
Other: (specify )
Most of
Always the time Sometimes
Never
Does updating for point sources account for:
Changes in emission controls
Shutdown of sources
New sources
Malfunctions
Change in source activity or production
New inspection or enforcement information
Area and Mobile Sources:
Change in census or population estimates
Change in employment estimates
Change in fuel use estimates
Change in VMT estimates
Other: (specify .)
Most of
Always the time Sometimes Never
Most of
Always the time Sometimes Never
2-13
-------
b. Please indicate the frequency of update (c = continually, 1=1 year
or less, 2=2 years or more) for the complete emission inventory in each
of the following categories:
Major Minor
Pollutant Point Sources Point Sources Mobile Sources Other Area Sources
voc
so2
PM
Pb
NESHAP
EPA Regional Office response:
2-14
-------
Emission Inventories - Documentation and Verification
B.6. This question is a survey of the procedures used by your agency in
your efforts to improve the accuracy, consistency, and completeness of
emission estimates.
State/local Agency Response:
a. Please indicate below the approximate percentage of sources for which
emission data are subject to the following types of cross-checks:
Major Minor
Point Point Mobile Area
Sources Sources Sources Sources
Limit checks, extreme values, etc.
Consistency check of emission factors
among common source types.
Data review by other agency staff or by
supervisor.
Data review by source or outside agency.
Check of source data thru laboratory analysis. "
Other (specify ) ^^^^
b. Please indicate the performance frequency (c = continually, 1 = 1 yr
or less, 2 = 2 yrs or more) of the following checks for missing sources:
Major Minor
Point Point
Sources Sources
Periodic check of industrial listings
Periodic check of yellow pages, bulding permits, etc.
Other (specify )
c. Indicate if internal agency review of emission inventory data by other
agency functions is performed as follows:
Yes No Comments
Enforcement
Planning
Permits
Other: (list )
EPA Regional Office response, confirmation, or comment:
2-15
-------
B.7. Emission Inventories - Computerization and Report Formats
State/Local Agency Response
- Characterize the data handling system used to maintain the point and
areas source emission inventory.
Totally manual (paper files only)
Semi automated
Fully automated (if so, please indicate whether the following
capabilities are available or needed in a fully computerized system)
Col limn Description
A Available
B Not available, but needed
C Not available, not needed
D Explanation: required for Column C, optional for A or B
ABC
Storage of point source data
Storage of area source data
Computerized editing of input data ~
Capable of producing annual MEDS report
Stores enforcement/compliance data in
same file with emissions data
Batch data updating by control office
Interactive data updating by control
office or regional office
Batch retrieval of data
Interactive retrieval of data by
central office and regional office
Ability to store historical emissions
data as well as current data for
tracking of RFP
"User friendly" procedures to permit
personnel not trained in ADP to
use the system.
EPA Regional Office (Confirmation or Comment):
2-16
-------
B.8. In the past few years, many agencies have initiated and maintained emissions
inventories for noncriteria pollutants including NESHAPS and potentially toxic
substances. The following questions address the scope of these inventories.
State/Local Agency Response
a. Is your agency delegated Federal authority to regulate NESHAPS sources?
Yes No_
If yes, which ones?
Hg Be Asb VC1 Benzene
For those checked, above how many sources exist in your agency's
i nventory?
Hg Be Asb VC1 Benzene
b. Does your agency maintain an inventory of lll(d) pollutant sources?
Yes No
Flouride from phosphate fertilizer plants
Sulfuric acid mist from sulfuric acid plants
Total reduced sulfur from kraft pulp mills ~~~ ~
Flouride emissions from primary aluminum plants
c. Are other sources of potentially toxic substances or non-criteria
pollutants maintained in your emissions inventory? Yes No
If yes, how many pollutants are inventoried?
List is attached: Yes No
Does the inventory of potentially toxic substances include:
Point sources only
Point and area sources
EPA Regional Office (confirmation or comment)
2-17
-------
C. Modeling
C.I. Air quality modeling expertise varies among agencies according
to their respective responsibilities, the number of modeling problems that
need to be analyzed, the degree of difficulty of these modeling problems,
and perhaps nontechnical factors. Please summarize your agency's staff
levels and modeling experience.
State/Local Agency Response:
a. How many persons in your agency perform modeling? persons
Of these, how many have:
- A college level degree in meteorology or
atmospheric sciences persons
A college level degree in engineering, mathematics
or science with:
- No formal meteorological or dispersion modeling
training persons
- Some formal meteorological or dispersion modeling
training persons
Any other educational background with:
- No formal meteorological or dispersion modeling
training persons
- Some formal meteorological or dispersion modeling
training persons
(Any comments:
b. Indicate the number of modeling staff with the following years
of experience in meteorology and/or modeling (total should equal
total modeling staff):
less than 2 years
2-5 years
6-10 years
more than 10 years
persons Comments:
persons
persons
persons
2-18
-------
c. Can the agency staff run EPA models In house as follows?
Yes No
Screening models only
Refined models in simple circumstances
Refined models in all circumstances
d. Identify:
0 The most complex air quality model your staff has run for
regulatory applications and the circumstances:
The most commonly used models run by your staff
e. Has your staff used EPA's "Interim Procedures for Evaluating Air
Quality Models" for evaluating nonguideline models?
Yes No
If yes, briefly describe the model(s) and the circumstances.
f. Has your staff developed nonguideline models in-house?
Yes No
If yes, briefly describe model(s) and the circumstances,
EPA/Regional Office Response (confirmation or comment):
2-19
-------
C.2. The ability of an agency to effectively deal with modeling
problems that may arise also depends on the facilities (hardware and
software) available to the staff to perform or confirm modeling analyses.
Please summarize below your agency's accessibility, expertise and usage
of computer-based air quality models.
State/Local Agency Response:
a. To which air quality models does your staff have access?
Also indicate whether your staff is capable of running the model
and the approximate number of applications during FY 1984.
1. UNAMAP specify version no.:
2. Specific Models
EPA Guide-
line
Model
Not in
Guideline
Yet(2)
Other
Models*3)
Access
(yes or no)
In-house
Expertise to Use
(yes or no)
FY 1984 Usage
(Estimated Number
of Applications)
APRAC-1A
AQDM
COM
RAM
CRSTER
TCM
TEM
HI WAY
ISCLT
ISCST
MPTER
CDMQC
EKMA
AIRSHED
PAL
PLUVUE
Screening PT MAX
Techniques(4)pT DIS
PTMTP
PTPLU
VALLEY
COMPLEX I
2-20
-------
b. Is access generally by:
Telephone line to State/local agency mainframe computer?
Telephone line to private or subscription computer?
In-house dedicated computer?
Telephnone line to EPA computer?
c. Does your staff have the capability to modify software
for the above models? Yes No
If yes, which models have been mollified?
Where modified guideline models have been used, State/local
agency should answer appropriate part of question C.3.
Footnotes:
(1) Models recommended in the "Guideline on Air Quality
Models" (1978)
(2) Models recommended in the "Regional Workshops on Air
Quality Modeling: A Summary Report" (1981)
(3) In addition to the examples given, list the nonguideline
models available to you and indicate whether any have
been used on a case-by-case basis. Include long range
transport models, photochemical models, complex terrain
models and any other models for situations for where EPA
has not provided guidance,
(4) In addition to the EPA screening techniques listed,
indicate the accessibility and usage of any other screening
techniques available to you.
EPA/Regional Office Response (confirmation or comment):
2-21
-------
C.3. EPA modeling guidance recommends specific models and data bases to
be used in regulatory modeling. However, the guidance also indicates that
an alternative model or data base may be used in an individual case if it
can be demonstrated that the alternative technique is more appropriate
than the recommended technique. For FY 1984, describe the number of, and
circumstances related to, modeling analyses where it was necessary to
use alternative techniques from those specifically recommended in EPA
guidance.
State/Local Agency Response:
a. In approximately what number of the modeling analyses performed
by your agency in FY 1984 was it necessary to use techniques not
specifically recommended in EPA guidance?
0 times out of modeling analyses performed.
b. If an alternative model was used, indicate the reason(s)
for its usage, using the list below: (alternative data
bases is covered in part c of this question)
Number of Reason(s)
Cases for Use*
0 Use of nonguideline model
0 Modification of guideline model
0 Use of nonrecommended option in
a guideline model
0 Use of guideline model outside
its stated limitation
0 Other (describe )
*Reasons for use:(List one or more of the following numbers as applicable
in the space above)
1. The alternative technique was judged, for technical reasons, to be
more appropriate for the situation.
2. Lack of access to the guideline model recommended for the situation.
3. The alternative model was judged, through a performance evaluation,
to be more appropriate for the situation.
4. No EPA guidance applies to the situation.
5. Other (specify below).
2-22
-------
c. If there are cases where the selection/usage of data bases
for models are different from those recommended in EPA guidance, please
indicate the number and circumstances surrounding each case.
Number of cases Reason (brief statement)
0 Use of less than 5 years
of off-site or less than
1 year of on-site
meteorological data.
0 Use of techniques other
than those contained in
the "Guideline on Air
Quality Models" for
determining background
0 Use of techniques other
than those contained in
EPA poli cy on treatment
of calms
0 Use of techniques other
than those contained in
the EPA policy on design
of receptor network
d. In those cases where your agency used models and/or data bases
not specifically recommended in EPA modeling guidance, indicate below if
EPA ultimately concurred with your choice.
0 Number of cases where EPA concurred without major reanalysis
0 Number of cases where EPA concurred with major reanalysis
0 Number of cases where EPA did not concur
0 Number of cases currently awaiting an EPA decision
0 Number of cases in which EPA concurrence was not sought
EPA Regional Office Response (confirmation or comment):
2-23
-------
C.4 NOTE: EPA REGIONAL OFFICES WILL RESPOND TO QUESTION, STATE/LOCAL
AGENCY WILL PROVIDE RESPONSE BELOW:
State and local agencies will normally conduct and submit to EPA
modeling analyses to support certain actions. EPA will review these
analyses case-by-case and approve or disapprove them in the Federal
Register. Using the list below, please indicate in the appropriate boxes
the number of analyses where EPA approval or action is required, the
modeling analyses has been performed, and the analyses has been submitted
to EPA either formally or informally.
EPA Regional Office Response:
Please indicate in column A approximately how many of modeling
analysis described above were submitted to EPA during FY 1984, in each of
the program areas listed below. In column B, indicate the number of
analyses where EPA has required the State/Local Agency to revise the
analysis. In column C, use the codes provided to describe the factors
contributing to the number given in column B.
of analysis
reviewed
B
# of analysis
requiring revision
Contributing
factors*
Bubble (emission
trades)
Section 107
redesignations
New source review
(including PSD)
Nonattainment area
SIP analyses
Lead SIP's
Other SIP modeling
*Factors that contributed to requiring a revised analysis:
Agency did not know or ask for EPA guidance
Agency misinterpreted or misunderstood EPA guidance
Agency misapplied or did not follow EPA guidance
No guidance available from EPA
The analysis was judged technicaly inadequate by EPA
1
2
3
4
5
6. Other (describe)
State/Local Agency Response (confirmation or comment)
2-24
-------
C.5. In those instances where the modeling analysis supporting a
particular action is performed by industry and/or other governmental
entity, the agency will review and approve or disapprove the modeling
analyses.
State/Local Agency Response:
Indicate in column A approximately how many of the modeling analyses
described above were reviewed during FY 1984, in each of the indicated
program areas. In column B, indicate the number of the analyses where
the agency required the responsible party to revise the analysis. In
column C, use the codes provided to describe the factors contributing to
the number given in column B.
# of analyses
reviewed
IB
# of analyses
requiring revision
Contributing
factors*
Bubble (emission
trades)
Section 107
redesignations
New source review
(including PSD)
Lead SIP's
Other source specific
SIP modeling
I I
I I
P I
I I
I I
I I
I I
I I
I I
I I
I I
I I
I I
I I
I I
*Factors that contributed to requiring a revised analysis:
1. Responsible party did not know or ask for your agency's
guidance
2. Responsible party misinterpreted or misunderstood your
agency's guidance
3. Responsible party misapplied or did not follow your agency's
guidance
4. No guidance available from your agency
5. The analysis was judged technicaly inadequate by your agency
6. Other (describe)
EPA Regional Office Response (confirmation or comment)
2-25
-------
D. SIP Evaluation and Implementation
D.I. Timeliness of Regulation Development—
One of the purposes of the audit is to identify SIP supplements,
revisions, addenda, etc. that have not to date been submitted, or
SIP strategies that have not been implemented; then to determine if
the agency is addressing them, and where they are not, identify and
implement corrective actions. The response should be structured in
three parts (Part D, non-Part D, and other required submittals),
corresponding to the form of Table D-l, with the EPA Regional Office
providing a list of items in each part.
State/Local Agency Response:
a. Has the agency adopted or submitted revisions and/or implemented
strategies related to SIP's or lll(d) plans? Use Table D-l for
response.
b. How does the agency track implementation of these activities?
0 Management system with key dates and periodic reports
0 Assigned staff person to track all dates on periodic basis
0 Other (describe)
c. Have there been delays.in State action due to required approval
at a level in State or local government higher than the air
pollution control agency (e.g., legislative or executive branch
oversight)?
0 Yes (describe
0 No (proceed to Question D.2.)
d. If (c) is yes, how long are such delays generally?
0-2 3-6 7-12 over 12
mos. mos. mos. mos.
If delays are generally longer than 2 months, complete part (e)
below; if delays are 2 months or less, proceed to Question D.2.
2-26
-------
e. Which of the following actions, if implemented, would help reduce
the above delays?
0 EPA official's discussion with appropriate State officials
0 Change in States internal procedures
0 Change in State regulation
0 Change in State law
0 Change in State constitution
0 Other (describe)
EPA Regional Office Response (confirmation or comment):
2-27
-------
Table D-l
Timeliness of Regulation Development
D-l Has the agency adopted or submitted required revisions or implemented
required strategies related to SIP's or Section lll(d) plans?
STATE/LOCAL AGENCY RESPONSE
Compl eted
Submitted Not lOn Not on
Submitted*|Schedule Schedule *
Regional Offices will provide to the
agency a list of revisions
or strategies due or overdue by 12-31-84
for each of the categories listed below:
Required Activity
a. Part D Plans (submissions or
implementation of measuresT:
1.
2.
3.
•
b. Non-Part D Plans (submissions
or implementation of measures:
1.
2.
3.
c. Other SIP requirements:
1.
2.
3.
*Provide brief explanation, status, and corrective action needed:
EPA REGIONAL OFFICE RESPONSE (CONFIRMATION OR COMMENT):
Not Completed
2-28
-------
D.2. Timeliness of Studies--
Have additional studies (e.g., nontraditional TSP, CO hot-spots)
approved as part of SIP control strategies been carried out?
STATE/LOCAL AGENCY RESPONSE:
Completed
I Not Completed
EPA-Supplied List of
Studies Due**
2.
3.
Objective
Of Study
Submitted Not* [On Not on
Submitted|Schedule Schedule*
* Provide explanation, status, and corrective action needed:
EPA REGIONAL OFFICE RESPONSE (CONFIRMATION OR COMMENT):
** EPA Regional Offices must provide on the questionnaire given to the
audited agency a list of additional studies that were due by 12-31-84.
(This date, rather than the end of FY 1984, was selected to improve the
timeliness of responses to this question.)
2-29
-------
D.3. Regional Consultation
NOTE: EPA REGIONAL OFFICES WILL RESPOND TO PART a, STATE/LOCAL AGENCY
WILL RESPOND TO PART b.
a. In approximately how many source-specific SIP revisions did the
agency consult with the EPA Regional Office before formally submitting
them?
EPA Regional Office Response:
Total number Total number
submitted with consultation
Variances
Bubbles
Other source-specific
SIP Revisions
(describe:
State/Local Agency Comment on Part a:
b. Does-each agency staff person responsible for developing source-specific
SIP revisions have access to all current EPA policy and criteria on the
above-listed kinds of source-specific SIP revisions?
State/Local Agency Response:
Yes No
How is guidance disseminated within your agency?
Copies of guidance
Policy manual
Summary memo
Other (describe)
EPA Regional Office Comment on Part b:
2-30
-------
D.4. Transporation Control Measures:
State/Local Agency Response:
a. If applicable, are transportation control measures being implemented
in accordance with the SIP?
Being Implemented?
EPA/RO-supplied list of TCM's** Ye?flo*
1.
2.
*Explain and indicate progress being made toward implementation:
b. Indicate how the air pollution control agency tracks implementation
of TCM's:
Reporting by implementing agency
Membership by State/local agency on MPO committee_
Other (list)
EPA/Regional Office Response (confirmation or comment):
** EPA Regional Offices must provide on the questionnaire given to the audited
agency a list of TCM's in the SIP.
2-31
-------
D.5. Reasonable Further Progress (RFP) Toward Attainment
The Clean Air Act [§172(b)(3) and (4)] requires SIP's to provide for
revision and resubmission of emission inventories to ensure RFP. The
purpose of this question is to ascertain the degree to which agencies are
implementing this requirement.
State/Local Agency Response:
a. In FY 1984, did the agency compare changes in the emission inventory
with projected changes in emissions given in the 03 or CO RFP curve in
the SIP? This question applies only to those areas specified by the Regional
Office (CO and 03 exension areas and areas where EPA recently called for
SIP revisions). NOTE: Tracking RFP means compiling a realistic estimate
of an individual year's emissions and comparing this to the appropriate year
in the SIP RFP curve. Compiling of air quality data trends as an alternative
to compiling emissions data is not normally accepted as RFP tracking
Areas where 03 RFP should be tracked* Yes No**
1.
2.
3.
Areas where CO RFP should be tracked* Yes No**
1.
2.
3.
*EPA Regional Offices must provide on the questionnaire given to the audited
agency a list of extension areas for CO and 03 and areas where EPA has
recently called for SIP revisions
**Provide an explanation:
2-32
-------
b. If yes, did the agency--
--prepare a report of these comparisons?
Yes No
—submit the report to EPA?
Yes No
--make it available for public comment?
Yes No
EPA Regional Office Response (confirmation or comment):
a. Has the Regional Office received the above reports?
Yes No
b. If yes, has the Regional Office commented or otherwise responded
to the State/local ageancy on the report?
Yes No
If yes to part a, please explain how the report is used by the
Regional Office:
2-33
-------
D.6. State/Local Agency Response:
a. Does the agency periodically compare actual source activity (growth
increase or decreases) with past projections of source activity?
If so how often?
Once in 1-2 years
Once in 2-5 years
Less frequently
b. If major discrepancies are found, what action does the agency take?
Do nothing
Undertake study
Revise SIP
Other (describe)
EPA/Regional Office Response (confirmation or comment):
2-34
-------
D.7. Part 51 of EPA's SIP guidance requires States to adopt continuous
emission monitoring (CEM) regulations for selected categories of existing
stationary sources.
State/Local Agency Response:
a. Does the SIP contain these continuous emission monitoring (CEM)
requirements? (Use Table D-7 for response).
b. For any "Yes-partly" or "no" response, provide:
0 an explanation of why the requirement has not been met.
0 a schedule for coming into compliance with the requirement.
EPA/Regional Office Response (confirmation or comment):
2-35
-------
Table D-7
CEM REQUIREMENTS UNDER 40 CFR 51.19
Does the SIP contain CEM requirements that meet 40 CFR 51.19(e)?
(check boxes that apply)
Yes-
(Completely)
1. Fossil fuel-fired steam generators
greater than 250 MMBTU/hr heat input,
with a capacity factor greater than
30 percent
- Coal fired: -- Opacity
-- 862 (only for units
with S02 controls)
-- N02 (only for units
>1000 MMBtu/hr in
TfQCR's w/NOx control
strategy and NOX
emissions >0.1 times
N02 emission standards)
-- 02 or
Oil or oil/gas-fired with violations
or controls -- Opacity
2. HN03 plants >300 ton/day of
100 percent HMO? in AQCR's W/N02 control
strategy: -- N02
3. H2S04 plants >300 ton/day
of 100 percent ^$04: -- S02
4. Fluidized bed catalytic cracking units
with feed >20,000 bbls/day:
-- Opacity
I—I
I I
I I
Yes* |
(Partly) I
I I
I I
I I
No*
I I
I I
I I
No
Sources**
I I
Provide explanation:
** No sources of this category in State or local jurisdiction.
2-36
-------
D.8. Generic Bubble Rules
State/Local Agency Response:
a. Does the agency have a generic emission trading ("bubble") rule?
Yes Identify the rule _ ; then go to part b.
No The remainder of this question may be ignored.
b. If "a" is "yes," has it been approved by EPA in the SIP?
Yes_ No
c. If "b" is "yes," how many individual bubbles have been approved by
the agency under the generic bubble rule? . Please supply to the
Regional Office before the onsite audit visit a list of the sources
affected, the pollutants, and the amounts of emissions involved in
each of these emission trades.
d. How many bubbles approved under the generic bubble rule were submitted
to the EPA Regional Office for its information in FY 1984?
EPA RO Response: The Regional Office audit team will select from the
agency's list a sample of individual bubbles and evaluate them using the
format below. If only five or fewer bubbles were approved by the agency,
all should be audited; if more than five bubbles were approved, at least
five should be audited, preferably at least one from each major type
(e.g., each pollutant-source category type), if possible.
a. Does the bubble ensure that—
i. Applicable net baseline emissions will not increase?
Yes No . If no, explain reasons:
ii. The procedures approved by EPA in the generic rule were followed?
Yes No . If no, explain reasons:
iii. The emission limits produced under the rule will not interfere
with attainment and maintenance of the ambient air quality
standards and PSD increments?
Yes No . If no, explain reasons:
b. Did the agency follow EPA modeling guidance in approving the bubble?
Yes No . If no, explain reasons:
c. Did the agency subject the bubble to reasonably adequate notice and
comment before approval?
Yes No . If no, explain reasons:
2-37
-------
NEW SOURCE REVIEW
-------
3. NEW SOURCE REVIEW
INTRODUCTION
Last year's NSR audit effort resulted in the establishment of a useful
national data base that documents the policies, procedures, and actual
permitting practices (the latter based on information from a sampling of
permit files) used by State and local agencies to implement a preconstruction
review program for new and modified sources. The FY 85 audit has been
designed to build ufon the knowledge gained from last year's effort. In
order to do this, certain changes from last year's approach are being made
to avoid duplicating that phase of the audit involving the collection of
information pertaining to the agencies' ongoing NSR policies and procedures.
For FY 85, the NSR audit will focus almost exclusively on the examination
of permit files. Associated with this change, a number of new features
have been established. The new features of the FY 85 audit are described
in the next section, below.
Under some circumstances, it may be appropriate to reexamine certain
program areas if significant changes have occurred since the last audit.
Such a program chanye might include the assumption of PSD authority, or it
could be a substantial revision of existing procedures. In the event that
a reexamination of an agency's NSR program should be necessary, as agreed
upon by both the affected agency and the EPA Regional Office, the EPA
headquarters should be notified so that copies of last year's audit question-
naire can be provided. This will be necessary because of the fact that new
questionnaires have been prepared for the FY 85 NSR audit.
As was the case last year, some of the audit subjects covered in this
section continue to involve, in whole or in part, issues that could be
affected by proposed EPA rulemaking or ongoing litigation [e.g., CMA agreement
rulemaking proposed on August 25, 1983 (48 FR 38742)]. These particular
items are potentially impacted by regulatory amendment. Should changes to
the affected requirements be promulgated, EPA will issue revised guidance
as to how the audit should handle them. Until such time that the existing
Federal requirements and the State rules developed pursuant to these 40 CFR
Part 51 provisions can be changed, this guideline will assume that all
rules will continue to be implemented under the EPA requirements presently
in effect.
FY 85 NSR AUDIT - NEW FEATURES
Although the seven major audit topics used in the FY 84 audit, as well
as most of the questions asked, will be retained in FY 85, the questionnaire
format will be noticeably different. The format changes include:
(1) A breakdown of the FY 84 questions into more detailed step-by-step
questions that, in effect, guide the reviewer through each file audit. (This
is most noticeable in the section on applicability.)
(2) Conversion of the questions, wherever possible, to a multiple choice
format.
3-1
-------
(3) Addition of worksheet features at appropriate points to facilitate
the collection of the data from the file needed to respond to the audit
questions.
(4) Conversion of the questions from program procedures orientation to
an individual permit file orientation.
As a result, the questions used for the FY 85 audit are more detailed, even
though the actual number of NSR areas explored has decreased as some of the
FY 84 general questions were deleted.
The original questionnaire is being replaced by three distinct types
of questionnaires as follows:
(1) An NSR Permit Summary Questionnaire (see Appendix 2) that will be
filled out by the audited agency. This brief questionnaire is intended to
provide an assessment of the permit workload handled annually by each
audited agency. The permit summary is divided into major PSD/Part D permits
and "other" source permits, with a separate section concerning PSD monitoring
requirements. Since the concepts of "source" and "permit" vary, a definition
is provided in the instructions on the questionnaire.
(2) Two types of permit file questionnaires (see Appendices 3 and 4)
that will be filled out by the auditor for certain permit files selected
for review. These questionnaires are discussed in greater detail beginning
on page 3-4.
(3) An overall findings form that will be completed at the conclusion
of the on-site audit by the auditor based on the outcome of the file reviews.
[Note: This form is still under development by the NSR Audit Committee and
will be made available in early 1985.]
Unlike last year's procedure, the permit file questionnaires are to be
filled out solely by the auditor. This means that it is not necessary
to send copies to the agencies to be audited in advance of the onsite audit.
On the other hand, copies of the NSR Permit Summary Questionnaire should
always be sent in advance to the appropriate agencies. The agencies should
be asked to complete this brief questionnaire before the onsite audit so
that it can be returned to the auditor during the audit.
Copies of all completed forms are to be submitted to EPA Headquarters.
It should be noted, though, that the questionnaires may include sensitive
information which should not be widely distributed, so the questionnaires
should not be included as part of the formal audit report. They should
instead be forwarded to Headquarters as a separate part of the overall
package submittal.
PERMIT FILE SELECTION
The most important aspect of the FY 85 audit process is the file
review which occurs during the onsite visit. For the new source review
audit, permit files should be selected on the basis of permit action
type, source type and size, source location, public concern, and other
3-2
-------
factors geared to ensuring review of a variety of permitting actions and
decisions by the agency. Criteria to consider include:
0 Review both large (major) and small (minor) sources;
0 Review both new plants and plant modifications;
0 Review a PSD source for which preconstruction monitoring data
requirements apply;
0 Review a PSD source near a Class I area;
0 Review sources that avoided PSD or Part D review because of
restrictions on their operation or capacity;
0 Review some sources in nonattainment and sanctioned areas,
if applicable:
0 Review some of the most common source types in that State
(for example, boilers and asphalt plants), but also review
a variety of other source types; and
0 Review a controversial permit, a permit of high public
interest, or one that would be of particular interest for
reasons other than those described above.
By combining several of these factors in one permitting action, it
may be possible to satisfy the criteria above with only a relatively few
permits. Generally, however, in order to obtain a reasonable sampling of
permits, the auditor should randomly select at least 5 PSD/Part D and
10 other permits issued since the FY 84 audit. If this random selection
does not seem to represent the variety of criteria indicated above, the
auditor should note this and specifically select additional permits for
review which do reflect the missing criteria.
For the 1985 NSR audit, the auditor is NOT required to fill out the
file audit questionnaires for every permit file selected. Instead, after
selecting the permit files that will be examined, in accordance with the
criteria described above, audit questionnaires should be filled put for
a minimum of ONE PSD, ONE Part D (offsets) and THREE other (non-PSO/Part D)
permits. The following procedures should be used:
1. For completion of the PSD/Part D permit questionnaire—
Select the most recent PSD permit and the most recent Part D (offset)
permit and fill out the major source questionnaire, to the extent applicable,
for each one. In case the agency did not issue at least one of each of
these types of permits, select and review at least two of the other type,
preferably the most recent new plant and the most recent modified plant.
2. For completion of the non-PSD/Part D questionnaire—
Beginning with the most recently issued permits, select three permit
files that represent a variety of permit review situations as described
3-3
-------
in the criteria above. A file should be rejected only if it too closely
resembles a file already selected for evaluation using the questionnaire
except that the auditor should try to include at least one permit where
major review was avoided because of restrictions to the source's operation
or capacity.
3. All remaining major and minor source permits should be examined as time
allows using the appropriate questionnaire as a guide to ensure that the
applicable audit topics are adequately addressed. The auditor may complete
the questionnaire for any or all of the remaining permit files, but he or
she is not required to do so. It is understood that permit file review
time may vary greatly. Auditors are encouraged to review as many of the
selected files as possible, but, should time run short, are advised to
conduct only a limited review of these remaining files, concentrating on
problems identified by the completed questionnaires to determine whether
the problem is common to several permits or only to an isolated case. Of
course, if time is available, auditors should conduct a more detailed
review of the remaining permits (as well as others not originally selected)
to see if any additional observations can be made.
During the file review, it is advisable that agency persons responsible
for reviewing permit applications be available to answer questions that the
auditor might have. Ideally, the individual permitting engineer(s) most
familiar with the specific files selected should be available. However,
while the information obtained by agency personnel can be quite valuable,
it is assumed throughout the questionnaire that most information should be
readily available from the files themselves and. if such is not the case,
then the lack of documentation should be noted by marking "CBD" (cannot be
determined from information in the file), even when another answer is
marked as a result of discussions with agency personnel.
Auditors will find it useful to take with them a calculator and a
few basic documents:
(1) Copies of 40 CFR 51.18 and 51.?4,
(2) AP-42, for checking whether all emissions units were included, and
(3) A copy of the reference sheet included in this guidance.
DESCRIPTION OF THE PERMIT FILE QUESTIONNAIRES
The auditor will gather data for the FY 85 NSR audit primarily from
selected permit files. Depending on the permit file selected, the auditor
will use either the PSD/Part D or the non-PSD/Part D questionnaire in
accordance with the procedures described in the previous section. The
choice of questionnaires should be based on the type of preconstruction
review that the reviewing agency actually carried out in each case. The
first questionnaire (Appendix 3) is designed to be used to evaluate permit
files for which the reviewing agency consiclered^ the proposed source subject
to PSD or Part D (nonattainment area/offTet) requirements.
The second questionnaire (Appendix 4) was designed to evaluate permit
files where the reviewing agency determined that the proposed source was
3-4
-------
r\ot_ subject to PSD or Part D (offset) requirements. This would include
cases where a major source underwent a modification involving insignificant
emissions increases, as well as sources that were allowed to avoid PSD or
Part D review by restricting their potential to emit. This questionnaire
includes questions that will help to determine whether the agency followed
the correct procedures in subjecting a source to a non-PSD/Part D source
review rather than a PSD/Part D source review.
Source Information--(Section I)
The basic data needed to identify the permit*file reviewed is requested
in Section I of both questionnaires. The questions pertain to the type of
source, its location (Region and State), the auditor, and the unique identifying
file number assigned by the agency to this permit.
Public Participation and Notification--($ection II)
Public participation requirements for review of new and modified
sources are set forth under 40 CFR 51.18(h) and 51.24(q). These requirements
call for the issuance of a public notice which informs the public of a
pending permit action and of the opportunity for public comment or hearing
prior to final agency action on a source application.
The FY 84 audit results indicate that some States require public.
notification for all permits issued, but many do not. This year's audit
seeks further information as to what specific sources the public was
notified of, and how adequate the notification was.
Both questionnaires ask for the same information that was requested
last year. Because of concerns pertaining to the usefulness of public
notices versus the costs of providing such notices, the FY 85 audit asks
what it costs to issue a public notice. The answer should indicate the
amount charged by the newspaper or other media to publish the notice. If
this information is not available in the file (e.g., a copy of the receipt),
the auditor may wish to determine an approximate cost from the audited
agency, but it is not recommended that too much time be spent trying to
obtain this cost.
The public notice should inform the public of the availability for
their inspection of the application submitted by the source, the estimated
impact of the source on ambient air quality, and the agency's proposed
action to approve or disapprove the permit. Instructions for submitting
comments, as well as the opportunity for a public hearing, should also be
addressed. The auditor should verify that notices issued by the agency
adequately inform the public of the permit being considered and of their
opportunities to provide input to the final determination.
In addition to providing adequate notice to the public in general,
certain parties are to receive specific notification of proposed permit
actions where those parties would be directly affected by the proposed
source. The auditor should verify that the agency has, and uses, a mechanism
for notifying the appropriate government officials when the proposed
source may affect their jurisdiction. The auditor should particularly
note, in the case of PSD sources, whether and at what point in the process
3-5
-------
the Federal Land Manager (FLM) is notified of any pending agency action
on a source locating within 100 km of a Federal Class I area. In. addition,
the auditor should identify, for information gathering purposes, any
other criteria used to determine whether FLM notification is necessary.
Applicability Detertninations--(Section III)
State and local governments are expected to regulate not only PSD and
Part D sources but also construction of other air pollution sources. The
agencies are, however, particularly expected to strive for the level of
consistency needed to satisfy the minimum Federal requirements for subjecting
new and modified PSD and Part D sources to preconstruction review.
A. Definition of Source
The auditor should verify, through the review of selected permit
files, that the appropriate levels and detail of review are being made.
The listing of emissions units provides a basis for determining the answers
to several questions, so it should be as complete as possible.
Agencies must use, as a minimum, the appropriate Federal definitions
of "source" to make applicability determinations. The number of definitions
used by any particular agency will depend upon the specific Federal NSR
requirements being implemented by the audited agency under an approved SIP
or delegated authority. The auditor should consider the following:
For PSO, the agency should use a reasonable grouping of emissions
units as one stationary source, classified according to its primary activity,
i_.je_., same two-digit SIC code. The. industrial grouping will determine
t~he appl icable emissions threshold (100/250 TPY) governing major source
status, and therefore whether PSD applies.
For nonattainment areas, including areas where the construction ban
(40 CFR 52.24) is in effect, one of two possible definitions of source
should apply. That is, either the plantwide definition, as described for
PSD above, or the dual definition which considers a "source" to be both the
plant and each of its individual pieces of process equipment. The auditor
must know which definition is actually being used by the agency in order to
determine that it is being correctly applied.
For NSPS and NESHAPS, the applicable "source" is defined by various
subparts of 40 CFR Parts 60 and 61, respectively. The auditor should verify
that the NSPS/NESHAP applicability determinations are made independently of
the PSD or Part D (offset) determinations. .This is particularly important
where the PSD or Part D requirements do not apply, e.g, "minor" sources,
major sources which have de mini mis net emissions increases for the pollutant
of concern, or sources where exemptions from the PSD or Part D requirements
are otherwise, granted by the agency.
B. Fugitive Emissions
Fugitive emissions, to the extent they are quantifiable and emitted by
any of the listed source categories, should be included in the emissions
calculations for determining whether a source is major and subject to PSD
3-6
-------
or Part D review. For the auditors' convenience, the listed source categories
have been included in the Reference Table (see Appendix 1) which is to be
used with the FY 85 audit questionnaire. For other source categories,
i.e., those not listed, the source must first be evaluated as to whether
it is major without using fugitive emissions. However, fugitive emissions
should be included in the ambient impact analysis and other review requirements
whether the source is major or minor. The auditor should verify that the
emissions factors used to calculate fugitive emissions are documented and
reviewed by the agency independently from any use of such factors by the
applicant.
C. Potential to Emit
The status (PSD/Part D or non-PSD/Part D) of new or modified sources
must be determined on the basis of the source's potential to emit. "Potential
to emit" is a source's maximum capacity to emit a pollutant under its
physical and operational design. In order for any physical or operational
limitations to be considered as part of the source's design (to restrict
the maximum capacity of the source), the limitations must be made an enforceable
part of the permit. Moreover, the limitations must be Federally enforceable,
which requires that the permit condition(s) be identified in the construction
permit or an operating permit that has been specifically incorporated in
the approved SIP.
As was the case last year, the auditors this year are to determine
whether the audited agency correctly applies the concept of "potential to
emit" when making applicability determinations. Both questionnaires ask
questions concerning the use of acceptable, well-documented emissions
factors as well as the use of special limitations to define a new source's
potential to emit. The auditor should determine whether restrictions to a
source's potential to emit are properly applied, particularly when they are
used to allow the source to avoid PSD or Part D review.
For modified sources, it is important to note that major source status
in terms of potential emissions of the exjhstjuig source must be taken into
account. This involves the existing source"'s maximum capacity, which may
take into account all control equipment and operating restrictions that are
Federally enforceable. As was revealed in last year's audit, there may be
a tendency on the part of the air pollution control agency to overlook the
potential to emit of the existing source, particularly when actual emissions
are significantly less than the applicable major source cutoff size. The
auditor, by completing the non-PSD/Part D source questionnaire, should be
able to determine whether any problems exist with this aspect of the audited
agency's applicability procedures.
D. Emissions Netting
For modifications to existing sources, once the major or minor status
of the existing source has been affirmed, the applicability review of
proposed modifications should be based on the net change in actual emissions
on a tons-per-year basis. For example, emissions changes occurring from
retiring equipment or other methods of emissions reduction generally will
be credited on the basis of the difference in the emissions unit's actual
emissions before and after the reduction. Actual emissions estimates
3-7
-------
generally should be based on either: (l) reasonable engineering assumptions
regarding actual emissions levels and representative facility operation
over a two-year period, or (2) permitted allowable emissions determined on
a site-specific, case-by-case basis so as to be representative of actual
source emissions. Where an emissions unit has not begun normal operations,
the potential to emit of the unit should be used.
Any net change in actual emissions that would result in a significant
emissions increase at an existing major stationary source must generalTy"TSe
reviewed as a major modification. However, for this to be true in a nonattain-
ment area, the existing source must also emit in major amounts the nonattainment
pollutant(s) for which a significant net increase would occur. For proposed
new major sources subject to PSD, PSO review applies to all criteria and
noncriteria pollutants that would be emitted in significant amounts.
For the auditors' convenience, the EPA-defined significant emissions.
rates for criteria and noncriteria pollutants regulated under the Clean Air
Act have been included in the Reference Table (Appendix 1) attached for use
with the FY 85 source audit questionnaires. The auditor should check all
applicability determinations carefully with respect to significant emissions.
Last year's audit indicated that some agencies do not use the EPA significance
values to trigger review of major modifications. Instead, they used some
uniform cutoff point that tended to be more restrictive than the EPA significance
values for some pollutants but less restrictive for other pollutants.
The worksheet provided in Section 11 I.E. of both questionnaires can be
used for new and modified sources. For modifications, it should be noted
that EPA policy requires the emissions change resulting from the proposed
modification itself to be significant before considering other contemporaneous
emissions increases and decreases that may have occurred before the proposed
modification. If the proposed modification does not result in a significant
emissions increase, then a major modification is said not to occur regardless
of how previous contemporaneous emissions changes would alter the net
emissions change. State and local agencies may implement a more stringent
policy if they wish to do so. Where this is the case, the auditor should
note such policy and evaluate the permit in accordance with that more
stringent policy.
Adequate safeguards should be taken by the agency to prevent the use
of contemporaneous decreases in actual emissions if the decreases are not
creditable. The auditor must know how "contemporaneous" is defined by
each audited agency. Contemporaneous emissions decreases should be surplus
and should not credited more than once. No decrease previously relied on
by a PSD source can be considered again in determining the net change of
a current or future modification. In addition, any emissions reduction
that has occurred or is scheduled to occur pursuant to the attainment date
contained in the SIP control strategy cannot be counted for netting purposes.
Finally, in nonattainment situations, no reduction relied on previously
to meet the reasonable further progress requirement of Part D of the
Clean Air Act can be used for calculating emissions. If necessary, the
auditor should inquire about the agency's policy and procedure for preventing
double counting, but documentation in the file which specifically states
that the decrease was not relied on or counted elsewhere is preferable
3-8
-------
and should be encouraged. Should documentation not be readily available,
the auditor should so indicate.
E. Emissions Limits
Agencies may vary in the number of permits that they issue to a source
having more than one emissions unit. No Federal requirements exist to
govern the number of permits which may apply to any source. What is important,
however, is that each emissions unit is identified clearly, along with its
allowable emissions rate, or design, equipment, work practice or operational
standard, as may be appropriate for the specific source. It is particularly
important, when an agency issues one permit to a large complex, that each
emissions unit is identified separately, along with its allowable emissions
rate, as opposed to a single composite emissions rate for each pollutant.
The auditor should verify that, for each permit issued, there is separate
and clear identification of the affected emissions units and their corresponding
allowable emissions.
In addition to identifying the allowable emissions, equipment or other
standard for each separate emissions units, it is important that such
limitations be addressed adequately in conditions on the permit(s) for a
new or modified source. The auditor should examine the adequacy of the
conditions in terms of their clarity and enforceability. The auditor
should pay close attention to the use of clear and precise averaging periods
over which the various pollutant emissions are to be measured. [Note: In
many cases, averaging periods may be a part of the required test method and
may not be specifically stated on the permit. In such cases, auditors- should
discuss this with the audited agency and verify that the agency regulations
do require proper averaging periods by reference. Also, some agencies may
incorporate by reference the test method as well as the averaging period.]
Finally, the emissions rates must be consistent with acceptable measurement
procedures; otherwise, compliance will be difficult if not impossible to
ascertain and the conditions would be unenforceable.
Test methods used to determine compliance of the source with its
allowable emissions rates should be clearly defined or referenced as condi-
tions to the final permit. These compliance tests should be specific to
the individual emissions units to which they apply. The auditor should
verify the documentation of the compliance test methods and their adequacy
for covering each applicable emissions unit for which allowable emissions
rates are defined. Where test methods are not specified in the permit, the
auditor should determine whether the SIP specifications are otherwise
applicable and sufficient.
BACT/LAER Determinations-^Section IV. PSD/Part D source questionnaire only)
The primary objective is to assure that good, wel1-supported, BACT/LAER
determinations are made across the nation. Secondary objectives are to
measure the frequency of BACT/LAER determinations set equal to existing
new source performance standards, and to determine the amount of legitimate
attention being given by review agencies to the requirement for the
application of LAER on new and modified major sources constructing in
nonattainment areas.
3-9
-------
Pollutants regulated under the Clean Air Act are subject to a BACT
analysis if they would be emitted in significant amounts by a source whose
construction is subject to PSD. A pollutant subject to regulation under the
Clean Air Act generally has had a standard of performance under §111 or 112
and/or NAAQS promulgated for it. The analysis for the subject source
should address both fugitive and nonfugitive emissions. The auditor should
verify that the BACT analysis considers all significant emissions increases
rather than being restricted to criteria pollutants or major emissions
changes.
In selecting BACT, the applicant generally should consider more than
one control strategy, unless it can be demonstrated that the single proposed
strategy clearly represents the highest degree of continuous emissions
reduction available. In all cases, the control strategies considered
should be technically feasible and should address the economic, energy and
environmental impacts of the particular alternative. Quantifiable impacts
should be identified. The auditor should verify that adequate alternative
control strategies are included where appropriate.
In each case, the BACT analysis submitted by the applicant must be
reviewed independently by the permit agency. In particular, candidate
control equipment should be assessed to ensure that reasonable performance
claims, including consideration of continuing compliance, are being made.
Where the alternative representing the most stringent emission reductions
is not selected, the permit agency should review carefully the alternatives
to ascertain that the most appropriate one was used. The agency should
routinely check to see whether any technically feasible alternatives were
not considered, and why. The auditor should verify that the agency performs
an adequate independent review of the BACT analysis submitted by the applicant.
For each permit reviewed which was subject to BACT or LAER, the auditor
should note the regulatory baseline assumed by the review agency. In how
many instances do the agency's BACT/LAER determinations conform exactly to
existing SIP, NSPS, or NESHAP requirements? The auditor should verify that
adequate documentation is provided for those determinations which simply
meet the minimum requirements. For cases where LAER determinations conform
exactly to NSPS, the auditor should examine the reasons why LAER was not
determined to be a more stringent limitation.
Air Quality Monitoring Data (PSD)--(Section V. PSD/Part D source questionnaire
only)
Every PSD source with the potential to emit significant amounts of a
particular criteria pollutant, where both the existing air quality and the
estimated impact of the source or modification are significant, must meet
the requirements for preconstruction air quality monitoring data, unless
exempted under provisions for temporary emissions or compliance with the
Offset Policy. In the latter case, which applies only to VOC emissions, if
the source satisfies all conditions of the Offset Rule, postapproval monitoring
may be provided in lieu of providing preconstruction data [40 CFR 51.24(m) (l)(v)]
Only PSD sources are required to submit such data. For PSD sources not
required to submit data, the applicable exemption should be clearly stated
in the preliminary determination.
3-10
-------
The requirement for air quality monitoring data may be met in one of
two ways. First, the permitting agency may require that the PSD applicant
establish a monitoring network designed to collect the appropriate air quality
data. Second, if existing air quality data is representative of the air quality
in the area where the source would have an impact, then such representative
data may be provided in the place of monitoring data collected by the applicant.
The Ambient Monitoring Guidelines for PSD contain minimum quality
assurance requirements that must be met by the applicant when monitoring
must be performed. The detailed criteria for quality assurance generally
should not be audited by the new source review auditors. Instead, the
Regional ambient monitoring staff is usually better able to audit the quality
assurance procedures. It is important that the two groups discuss in
advance the division of responsibility of audited areas, to avoid overlap
or omissions. The new source review auditor should determine: (1) whether
a monitoring plan was submitted by the source and evaluated by the permitting
agency; (2) whether a quality assurance plan was submitted by the applicant;
and (3) whether the permitting agency evaluated the data for compliance
with 40 CFR 58 Appendix B.
Use of representative data is restricted by "the criteria described in
the Ambient Monitoring Guidelines for PSD. Generally, only new sources in
remote areas may use existing data gathered at sites greater than 10 km
away. For all sources in flat terrain, monitors within 10 km are acceptable.
For complex terrain, the guidelines are very difficult to meet, and new data
are almost always required. In addition to the monitor location criteria,
there are also restrictions concerning data currentness and quality. The
auditor should be familiar with the guidelines concerning representative
data and verify that the audited agency is following them.
PSD Increment Ana1ysis--(Section VI. PSD/Part D source questionnaire only)
Before a permit is granted, the permit agency must determine that no
national ambient air quality standards will be violated. In the special
case of a PSD permit, the agency must further verify that no allowable
PSD increment will be exceeded by the source under review. In both cases,
the ambient impact analysis must be reviewed carefully by the permit agency
responsible for managing the ambient air quality levels. The auditor should
determine the adequacy of the ambient air analysis performed as part of the
preconstruction review.
Allowable PSD increments exist only for S02 and TSP at the present
time. There are a number of important considerations that the permit
agency must routinely take into account in order to ensure that the maximum
allowable increments are not exceeded. The permit agency must give the
proper attention to such things as the baseline concentration, baseline
area and baseline date(s): the appropriate emissions changes for increment
consumption purposes; long and short-term increment averaging periods; and
special Class I area impacts.
3-11
-------
The baseline concentration generally reflects actual emissions occurring
at the time of receipt of the first complete PSD application in the §107
attainment or unclassifiable area. This ambient concentration is adjusted
to include projected emissions of major sources commencing construction
before January 6, 1975, but not in operation as of the baseline date, and
to exclude the impacts of actual emissions changes resulting from construction
at a major stationary source commencing after January 6, 1975.
Changes in emissions contributing to the baseline concentration from
any source subsequent to the baseline date and from any major source construc-
tion commenced after January 1, 1975, can either consume or expand the PSD
increment. Where actual emissions cannot be used, e.g., the source has not
yet begun to operate or sufficient operating data is not available, then
allowable emissions must be used. The auditor should verify that the
agency considers the appropriate emissions changes relative to the baseline
concentration.
The date of receipt of the first complete application for a major new
source or major modification subject to PSD becomes the baseline date, and
the area in which the baseline is triggered is known as the baseline area.
The analysi-s cart become somewhat complicated when the baseline area for the
proposed source includes more than one Section 107 attainment or unclassified
area, particularly if the baseline date has already been triggered in some,
but not all, of the Section 107 areas within that baseline area. Auditors
should be familiar with the PSD increment analysis process to help alleviate
some of the potential confusion that could occur during the review of the
PSD permit files. A good presentation of the increment analysis is contained
in the EPA PSD Workshop Manual (EPA-450/2-80-081, October 1980).
Both TSP and S02 have long and short-term averaging periods for which
PSD increments have Seen established. These maximum allowable increases are
not to be exceeded more than, once per year for other than an annual averaging
period. The auditor should verify that the application considers each
averaging period with complete documentation in the permit file.
For sources proposing to locate near a Class I area, an increment
analysis may be required under conditions that would not trigger an analysis
in any other locations. Any emissions from a proposed source should be
considered significant when the source would locate within 10 km of the Class I
area and cause an ambient impact equal to or greater than 1 ug/m3 (24-hour
average). Generally, sources locating within 100 km of a Class I area should
be screened to determine their impact on the Class I area.
NAAQS Protection--(Section IV. Non-PSD/Part D source questionnaire;
Section VII. PSO/Part D source questionnaire)
States may differ as to the emissions baseline used to protect the
NAAQS. In some cases, the allowable emissions from all sources must be
used for modeling air quality. In other cases, the modeled allowable
emissions from the proposed source or modification are added to the back-
ground air quality which is based solely on monitoring data. The auditor
should identify the emissions baseline required by the agency and gain an
understanding of the specific approach utilized to estimate the impact of
3-12
-------
a new or modified source. This information will be used to assess current
practices and for consideration of future policy development.
In evaluating the NAAQS, the ambient impact analysis should determine
the maximum long-term and short-term impacts of the proposed new source
or modification. However, maximum ambient impact may actually occur at
other locations when the impacts of other sources and background data are
taken into account. Hot spots may also occur where growth resulting from
minor sources or sources otherwise exempted from detailed permit review are
not subjected to a rigorous ambient analysis. The auditor should verify
that the agency performs a detailed analysis of a source's maximum ambient
impact beyond those areas of maximum impact of the source alone.
EPA has recommended the use of a number of models for specific types
of applications and has stated its preference for certain new models for
analyzing the impact of sources on ambient air quality. However, utilization
of any particular model should be consistent with the design and intent
of the model itself. Some models are very specific as to terrain and
applicability. The auditor should verify that impact analyses are being
performed with the appropriate models, and that the permit agency conducts
its own independent review of the source's analysis (including the replication
of modeling results when appropriate) to ensure conformance to accepted
procedures. EPA guidance is provided in "Guideline on Air Quality Models,"
EPA-450/278-027, April 1978. This report is currently undergoing revision.
Additional guidance is also provided in "Regional Workshops on Air Quality
Modeling: Summary Report," OAQPS, April 1981 and "Guideline for Use of
City-Specific EKMA in Preparing Ozone SIP's," EPA-450/4-80-027, March
1981. EPA's "Guideline on Air Quality Models" includes, among other things,
guidance on the selection of air quality dispersion models.
Emissions Offset Requirements—(Section VIII. PSD/Part D source questionnaire
only)
Part D of the Clean Air Act intends that certain stringent requirements
be met by major sources approved for construction in nonattainment areas.
One such stringent requirement calls for the proposed source or modification
to get emission reductions (offsets) from existing sources in the area such
that there will be reasonable further progress toward attainment of the
applicable NAAQS. The specific objectives are: (1) to assure that reviewing
agencies are requiring, where appropriate, adequate emissions offsets as a
condition to authorizing major construction in designated nonattainment
areas; and (2) to assure that emissions offsets are being obtained in a
manner consistent with RFP.
All emissions reductions used to offset proposed new emissions must
be made enforceable. This is true whether the offsets are obtained from
another source owned by the applicant or from a source not under common
ownership. In either case the offsets should be fully agreed upon and
documented, preferably within the permit of the source from which the
offset is obtained. In addition, Federal enforceability requires that
an external offset be made a part of the applicable SIP. This would require
a specific SIP revision if the offset is not made part of a permit issued
pursuant to the State's construction permit requirements approved pursuant
to 40 CFR 51.18 or 51.24. Conditions to State or local operating permits
3-13
-------
are not always considered to be part of the applicable SIP(s). The
auditor should verify that all offsets are well documented, which includes
well-defined emissions limits pertaining to the emissions offset.
The proposed emissions offset cannot be otherwise needed to show RFP
toward attaining the NAAQS. To use the same emission offsets for two
different purposes would result in "double counting" those emissions with
the net result being subsequent deterioration of air quality. The auditor
should seek assurance from the agency that compliance with annual RFP
increments is independent of the offsets being obtained from proposed new
or modified sources. In addition, the permit file should be checked to
determine whether any documentation is provided to address this issue.
In order for the system for getting offsets to be consistent with the
demonstration of reasonable further progress, they must both be expressed
in the same emissions terms, i.e., actual or allowable emissions. Section
173(1)(A) of the Clean Air Act sets the emissions offset baseline as the
"allowable" emissions of the source, but also requires that the offsets must
be sufficient to represent RFP. Consequently, where the RFP demonstration
is based on an actual emissions inventory, EPA requires that offsets to be
attained by a proposed new or modified source also be based on actual
emissions. The auditor should verify that there is consistency in the
emissions baseline for offsets and the RFP demonstration.
In order to comply with the Act requirement that emissions offsets
must be sufficient to represent RFP, any increases in area and minor source
growth not considered in the approved RFP demonstration must be covered by
offsets required of the proposed new or modified source. Failure to account
for these emissions increases would result in air quality deterioration
just as in the case of "double counting." The auditor should verify that
area and minor source growth considerations are made in order to establish
the offset level, particularly when more than one-year has passed since the
last offset.
Section 173(1)(A) of the Clean Air Act requires that offsets be obtained
and in effect "by the time the [new or modified] source is to commence
operation." No specific guidance is available to identify when a source
has officially "commenced" operation. Some agencies may allow a shakedown
period similar to the shakedown provision allowed for net emissions increases
in 40 CFR 51.18(j)(l)(vii)(f). The auditor should focus primarily on
whether offsets were sought to be in effect in a timely manner, which may
include, for replacement facilities, a shakedown period not to exceed 180
days. The auditor should also determine whether the effective date for the
offsets is documented in the permit file(s).
In general, an agency should not allow emissions reduction credit
for source shutdowns or curtailments which occur before the date of the new
source application. The main exception would occur in cases where the new
unit is a replacement for a unit shut down or curtailed at the same plant.
The auditor should also be aware of other possible allowances for shutdowns
under any applicable EPA-approved State emissions banking or trading rules.
The auditor should attempt to verify that offsets are actually granted in
accordance with all specific criteria governing offsets for shutdowns.
3-14
-------
APPENDIX 1
REFERENCE TABLES
FOR USE WITH FY 85 NSR AUDIT QUESTIONNAIRES
I. Questionnaire abbreviations:
0 CBD = cannot be determined from information available in permit file
0 NA = not applicable
0 PSD = prevention of significant deterioration
0 Part D = nonattainment area provisions applying to sources which emit
a nonattainment pollutant and locate within that nonattainment area.
II. Pollutant Criteria
Use this
Pollutant Abbreviation
C Carbon monoxide
R Nitrogen oxides
I Sulfur dioxide
T Parti cul ate matter
E Ozone (as volatile
R organic compounds)
I Lead
A
R Asbestos
E Beryllium
G Mercury
U Vinyl chloride
L Fluorides
A Sulfuric acid mist
T Hydrogen sulfide
E Total reduced sulfur
D Reduced sulfur compounts
CO
NOX
S02
TSP
VOC
PB
AS
BE
HG
VC
FL
SAM
H2S
TRS
RSC
Significant
Emissions
Levels, TPY
100
40
40
25
40
0.6
0.007
0.0004
0.1
1.0
3
7
10
10
10
Significant Air Quality
Impacts (for Monitoring
Determinations), ug/m3
575, 8-hr avg
14, annual ayg
13, 24-hr avg
10, 24-hr avg
(100 TPY of VOC)
0.1, 3-month avg
No monitoring required
0.001, 24-hr avg
0.25, 24-hr avg
15, 24-hr avg
0.25, 24-hr avg
No monitoring required
0.2, 1-hr avg
No monitoring required
No monitoring required
NOTE: For each regulated pollutant, any emissions rate is significant
that causes an air impact of 1 ug/m3 (24-hr) or greater in any Class I area
located within 10 km of the source.
III. The following source categories are major if _>100 TPY, including fugitive
emissions. (One exception exists: see note for last source category in list.)
Coal cleaning plants (with thermal dryers)
Kraft pulp mills
Portland cement plants
Primary zinc smelters
Iron and steel mills
Primary aluminum ore reduction plants
Pimary copper smelters
Municipal incinerators > 250 TPY
Hydrofluoric, sulfuric, or nitric acid plants
Petroleum refineries
Lime PI ants
Phosphate rock processing plants
Coke oven batteries
3-15
-------
Sulfur recovery plants
Carbon black plants (furnace process)
Primary lead smelters
Fuel conversion plants
Sintering plants
Secondary metal production plants
Chemical process plants
Fossil-fuel boilers (or combination thereof)
totaling > 250 million BTU/hr heat input
Petroleum storage & transfer units with total
storage capacity > 300,000 bbls
Taconite ore processing plants
Glass fiber processing plants
Charcoal production
Fossil fuel-fired steam electric plants
> 250 million Btu/hr heat input
Any other NSPS or NESHAP source as of
August 7, 1980 [Note: for PSD, major
source status based on emissions >250 TRY.]
3-16
-------
APPENDIX 2
NSR PERMIT SUMMARY
QUESTIONNAIRE
I. GENERAL INFORMATION AUDIT PERIOD: _/_ to _/_
Mo.Yr. Mo.Yr.
REGION:
STATE:
|~| State |~| Local Agency
AGENCY NAME:
Please answer the questions below for the specified audit period based
on the number of construction permits that were issued to sources (major
and minor) in your jurisdiction using the instructions provided at the bottom
of this page as appropriate.
11. PERMIT SUMMARY
1. PSO and Part D (Offset) Construction "Permits"
a. Prevention of significant deterioration (>^ 100 or 250 TPY)
b. Major sources in nonattainment areas (_> 100 TPY)
c. TOTAL
2. Other Source Construction "Permits"
a. Non-PSD major sources in attainment/unclassified areas)
b. Minor sources
i. Minor sources undergoing ambient impact analysis
ii. Sources avoiding major source review via operational
(including fuel use) or design capacity restrictions
affecting source's potential to emit.
c. TOTAL
III. PRECONSTRUCTION MONITORING FOR PSD
No. of PSD sources subject to preconstruction monitoring requirements.
No. of PSD sources actually required to collect data via monitoring,
No. of PSD sources allowed to use existing representative data.
3-17
-------
Instructions:
1. The audit period should be filled out by the EPA Regional Office
before forwarding the questionnaire to the audited agency. This period
represents the 12-month (approximate) period from the time of the last audit.
?.. Major source permits: enter "NO" if you do not have program authority;
"0" if you have authority, but none were issued.
3. "Permit" should be defined in terms of the entire source or project
for which a particular construction approval (for a new source or modification)
was requested. Consequently, one application should generally be regarded
as a "permit" regardless of the number of agency permits (for individual
emissions units) actually issued. For cases where an application would
qualify for two or more permit groupings on the questionnaire (e.g., major
source review for both PSD and for offsets), the permit should be counted
as belonging to each group. In the example given, the one application
would count both as one PSD permit and as one major nonattainment area
permit.
4. All "permit" numbers reported should pertain to new construction •
(which may involve a completely new plant or a modification to an existing
one) or a new method of operation for which a permit analysis was required.
Permit extensions, minor revisions, etc., should not_ be included in this
questionnaire. If the exact number of "permits" is not known, please
provide your most reasonable guess and place an "(E)" after the value
provided.
3-18
-------
APPENDIX 3
1985 PERMIT FILE QUESTIONNAIRE
FOR MAJOR SOURCES SUBJECT TO PSD OR PART D (OFFSETS)
[NOTE: Unless otherwise indicated place an "X" in the box beside each statement or
response which applies. Many of the questions will allow more than one response.]
SECTION I. SOURCE INFORMATION
A.I.Company/Source Name:
2.Size/Source Category (e.g., 250 MW coal fired boiler):
1
C. Date Complete I D. Date Permit to
Application rec'd:| Construct Issued:
1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
mo day yr I mo day yr
1
E. This permit was reviewed for (list
pollutants):
1. [ ] PSD for:
2. [ ] Offsets for:
B. 1.
2.
3.
4.
5.
6.
Region I I I
State I I |
Agency
a. [ ] State
b. [ ] Local:
Auditor
Permit #
Source-Type
a. [ ] New Major
b. [ ] Major Modi
Source
fi cation
F.I. [ ] This source is located within 10 km of a Class I area.
2. C ] This source is located within 100 km of a Class I area.
3. C ] Source is in an attainment area and significantly impacts a designated
nonattainment area or any area where a NAAQS violation exists.
4. [ ] None of the above.
SECTION II. PUBLIC PARTICIPATION REQUIREMENTS
A. Public Notice:
1. C ] was published in a newspaper (approx. cost, if available:
2. [ ] provided opportunity for public hearing
3. [ ] provided opportunity for written comment
4. [ ] described agency's preliminary determination
5. [ ] included estimated ambient impact
6. [ ] indicated availability of additional information for public inspection
7. [ ] was not provided
B. The following other affected government agencies were notified:
1. [ ] other agencies and officials within the State
2. [ ] other States
3. [ ] Federal Land Manager
4. [ ] EPA
5. C ] none of the above
3-19
-------
C. Documentation for parts A and B consists of: Notification of
Public Notice Other Agencies
1. copy of notice/correspondence in file a.[ ] b.[ ]
2. indication on processing checklist a.[ ] b.[ ]
3. no documentation provided a.[ ] b.[ ]
4. other, explain:
Section III. APPLICABILITY DETERMINATIONS
A. Definition of Source.
l.a. The new source or modification for which this permit application was made
was considered by the reviewing agency to consist of the following new and
modified pollutant-emitting activities associated with the same industrial
grouping, located on contiguous or adjacent sites, and under common
control or ownership (if more than 10, list only 5 largest):
New Modif. Emissions Unit/Size Pollutant
C ] C ] (2)
[ ] [ ] (3)
[ ] [ ] (4) ; .
[ 3 C ] (5)
b. CBD
2. Were any new or modified pollutant-emitting activities omitted which should
have been included:
a. C ] NO.
b. [ ] CBD.
c. C ] YES, as follows:
Emissions Emissions Reason given by agency for not
Unit/Size Pollutant TPY New Modif. considering as part of source
(1) [ ] C ]
(2) C 1 [ ]
(3) _____ C ] C ]
(4) C ] C ]
3-20
-------
3. Did this new source or modification escape any PSD or Part D analyses
for significant emissions as a result of the omission of the activities
listed in Section III.A.2.a. above? [ ] Yes [ ] No
B. Fugitive emissions. This source or modification:
1. [ ] Does not have any quantifiable fugitive emissions. (GO TO Section III.C.)
2. [ ] Has quantifiable fugitive emissions which were:
a. [ ] Included in determining whether the source/modification was major
for the following pollutants: .
b. [ ] Not included in determining whether the source was major for the following
pollutants:
because the source is neither one of the 28 PSD source categories
nor regulated under Section 111 or 112 of the Act.
c. [ ] Not included in determining whether the source was major for the following
pollutants: , although they should have been.
3. [ ] Has too little documentation in the file to determine whether
quantifiable fugitive emissions occur or were used.
C. Potential to Emit (PTE). Applicability determinations should be based
on PTE rather than actual emissions.
[ 3 Mark box if agency used, or accepted use of, actual emissions instead of
PTE; if so, respond to the following questions on the basis of how actual
emissions were determined.
1. The PTE of this source or modification was determined:
a. [ ] Using emissions rates based on emission factors which were:
(i) [ ] Well established (e.g., AP-42) or well documented in the file.
(ii) [ ] Not well established and lacking adequate documentation for the
following units and pollutants:
b. [ ] CBD
c. [ ] Using another method. Explain:
2. The PTE of this source was determined on the basis of:
a. [ ] CBD
3-21
-------
b. [ ] No limitations or restrictions on operation, materials or equipment.
c. [ ] The following limitations and restrictions:
If used, was limitation identified on:
Reconstruction Permit Operating Permit
(i) C
(1D C
HID C
Hv) C
(v) C
If Section
Limitation Yes No Yes No
] Control equipment [] [] [] []
] Operating hours [] [] [] []
] Operating rate [] [] [][]
] Fuel /material [] [] [] []
restriction
] [] [] [][]
II I.C. 2. b. above is marked, please describe the limitations:
c
C
c
c
c
N)
•3
]
]
]
]
3. Was PTE determined correctly?
a. C ] YES
b. [ ] CBD
c. [ ] NO. Explain:
D. Emissions Netting.
FOR MODIFICATIONS ONLY. [ ] Mark here if not applicable, THEN GO TO Section 11 I.E.
1. The determination of significant emissions can be a complex process when
emissions netting occurs. The following table should be used to determine
whether the proper procedure was followed:
Pollutant HT
a. TSP
b. S02
c. NOX
Proposed Emission
Changes, TRY
"
Other creditable,
contemporaneous
Overal1
Did agency
correctly identify
whether emissions
emissions changes, TPY Net Change, were significant?
Wet
T+T
TPY
Yes No
CBD
[ ] C ] C ]
[ ] [ ] C ]
[ ] C ] C ]
3-22
-------
d.
e.
f.
g-
h.
03 (VOC)
CO
C ] [
C ] [
C 1 C
C ] C
C ] C
For questions 2 through 5, mark and complete the applicable quesetions.
2. [ ] Emissions netting was based on actual emissions of existing units
were were not CBD
[ ]
C ]
C ]
C ]
3. C ]
a. [
b. [
4. [ ]
a.
b.
5. C ]
Were not
[ ]
C ]
C ]
[ 3
[ ] [ ] (a) Representative of normal unit operation
[ ] [ ] (b) Based on a two-year average
[ ] [ ] (c) Expressed in TRY
[ ] [ ] (d) Contemporaneous
Emissions netting was based on another approach which was:
] Acceptable. Explain:"
] Incorrect. Explain:
Emissions reduction credits were used which are:
[ ] Federally enforceable. Explain how:
[ ] Not Federally enforceable. Explain:
Creditable contemporaneous emissions decreases were used which:
Were CBD
[ ] [ ] a. Previously relied on by a PSD source in
determining net change.
[ ] [ ] b. Counted as part of the SIP attainment
strategy.
[ ]' [ ] c. Relied on to meet the reasonable further
progress requirement of Part D
[ ] [ ] d. Made Federally enforceable as permit
] C ]
] [ ]
] C ]
] C ]
] [ ]
which:
conditions
3-23
-------
E. Emissions limits.
1. If allowable emissions limits for each pollutant from each emissions unit
were included in the preconstruction permit, the number of limitations appearing
in the preconstruction permit would be (e.g., for a boiler emitting TSP, S02,
NOX, and VOC, there would be four limitations).
2. The number of limitations actually appearing in the preconstruction
permit is .
3. How many of the limitations: Number
Ye? flo"
a. Include clear and concise averaging periods?
b. Establish emissions limitations consistent with
acceptable measurement techniques?
c. Consist of design, equipment, work practice,
or operational standards?
d. Appear Federally enforceable?
e. Include stated or referenced compliance test
methods?
SECTION IV. BACT/LAER DETERMINATIONS (L J Mark here if not applicable, THEN
GO TO SECTION V.)
A. BACT Analysis.
Other (list)
TSP SQ-2 VOC NOX CO
1. Pollutant emitted in significant amounts. [] [] [] [] [] [ ] r~J
2. BACT determination made. [][][][][] C ] C ]
3. BACT was specified in permit. [][][][][] [ ] C ]
YES NO
4. a. Did the application address more than one
control option for BACT? [ ] [ ]
b. If NO, explain
c. If YES, did each option address the economic,
and environmental impacts associated with that option? [ ] [ ]
5. Does the file contain documentation to show
show that the reviewing agency verified the [ ] [ ]
applicant's calculations and assumptions for BACT?
-------
B. BACT/LAER Stringency.
(Mark'this line if either BACT [ ] or LAER [
does not apply. Respond below for each
determination which does apply.)
The Agency's BACT/LAER determination
compared to NSPS/NESHAP is:
"A" -- more stringent
"B" -- equal
"C" -- less stringent
"D" -- did not address,
but should have
"NA" -- not applicable
1.
2.
3.
4.
5.
6.
7.
8.
TSP
S02
HC
NOX
CO
BACT
C ]
C ]
C ]
C ]
C ]
LAER
C ]
C ]
C ]
C ]
C ]
C ]
SECTION V. AIR QUALITY MONITORING DATA -- PSD ( I J Mark here if not subject^
PSD, THEN GO TO Section VI) _
A. Applicability.
Source was:
1. C ] not required to address air quality monitoring data requirements
because:
a. [ ] existing air quality for all pollutants is de minimi's
for all pollutants (GO TO SECTION VI).
b. [ ] proposed ambient concentration increases for all pollutants
are de mini mis (GO TO SECTION VI).
c. [ ] cannot be determined. Explain:
(GO TO SECTION VI.)
2. [ ] required to address air quality monitoring data requirements for
at least one pollutant.
B. Air Quality Monitoring Worksheet:
1.
2.
3.
4.
5.
6.
7.
Pollutant
TSP
S02
CO
NOX
VOC/Oa
(a) Are
potential
emissions
For each
following:
"yes"
in (a), complete the
(b) Are modeled^
concentrations
significant? significant?
YES
C ]
[ ]
C ]
C ]
C ]
C ]
NO
[ ]
C ]
C ]
YES NO CBD
C ]
C ]
C ]
C ]
C 3 C 3
C ]
C ]
[ ]
C ]
C ]
[ ]
C ]
C ]
C ]
(c) Is existing air
quality significant?
YES
C ]
C ]
C ]
C ]
C ]
NO
C ]
C ]
C ]
C 3
[ ]
C ]
C ]
CBD
C ]
C ]
If (b) and (c) are
"yes", complete
the foil owing:
(d) What did the
agency require?
Source Repre-
Moni- sent.
toring Data Neither
L j n rr~
[ ] C ] C ]
C ] [ ] [ ]
C ] C ] C ]
[ 3 C 3 C 3
[ 3 C 3 C 3
C 3 C 3 C3
If (b) and (c) are not
both Yes Tor at least one
I pollutant, go to
I Section VI.
3-25
-------
C. Source Monitoring.
1. Did the applicant submit a monitoring plan, including quality
assurance (QA) procedures?
a. C ] YES, for )
b. [ ] NO, for ) Go to Section V.C.3
c. [ ] CBD, for _) Go to Section V.C.3
2. Is the monitoring plan in the permit file?
a. C ] YES, for
b. c ] NO, for jnnzzumziiii
3. For how long did the monitors collect air quality data?
a. [ ] 12 months or more for:
b. [ ] 4 to 12 months for:
c. C ] less than 4 months for: r_^__T__
If less than 12 months of data were submitted, summarize explanation:
D. Representative Data ([ ] Mark here if no representative data used, THEN GO
TO SECTIOM V.E.)
1. Is the basis for allowing the use of existing representative data
documented in the permit file?
a. [ ] YES, for
b. C ] NO, for
2. D.id the agency's determination of representativeness adequately consider:
CBD for: YES for: NO for:
a. Location of existing monitors [ ] [ ] [ ]
b. Quality of the existing air
quality data [ ] [ ] [ ]
c. Currentness of existing air
quality data [ ] [ ] [ ]
If "NO" for any pollutant, please explain:
E. If neither monitoring nor representative data were required for a pollutant,
explain what, if anything, the agency did require, and whether this was an adequate
requirement: ^
SECTION VI. PSD INCREMENT ANALYSIS _
CLASS I CLASS II
T5P ST52 T5P
A. Was a PSD increment analysis performed?
C ] NO, GO TO VI-H. C ] YES, as follows: C ] C ] MM
3-26
-------
How was the analysis performed?
[ ] By the applicant with adequate review (including
replication of results, if appropriate) by the
agency for:
[ ] By the applicant, without adequate review by the
agency for:
[ ] By the reviewing agency for:
[ ] Not applicable for:
C. Identify the dispersion model(s) used
to perform the increment analysis:
Model Used
[ ] C ]
C ] C ]
Class I
TSP
2.
3.
Pollutant/Area Classification
[ ] S02 for [ ] Class I [ ] Class II
[ ] TSP for [ ] Class I [ ] Class II
C ] SOg for [ ] Class I [ ] Class II
C ] TSP for [ 3 Class I [ ] Class II
[ ] S02 for [ ] Class I [ ] Class II
[ ] TSP for [ ] Class I [ ] Class II
C ] C ]
C ] C ]
Class II
TSP §02"
C ] C ]
Averaging Times
"3n~r24hr Annual
C ]
C ]
[ 3
C 3
C 3
C 3
C 3
C ]
4. C ] CBD
D.I. Did the agency select appropriate model(s)?
FOR MODEL:
C.I C.2 CTT
[ 3 a. Yes, and documentation supports use of each [3 C 3 C 3
model as being appropriate.
C 3 b. Yes, each model was appropriate, but inadequate [3 [3 C 3
documentation was available to explain its selection.
[ 3 c. Cannot be determined. Documentation not provided [3 C 3 C 3
to justify model selection.
[ 3 d. No, documentation failed to address appropriate [3 C 3 C 3
considerations. Explain:
. Did the agency exercise the appropriate model options (urban/rural,
receptor network design, wind speed profiles, building wake effects,
final/gradual plume rise, etc.)?
OPTION:
C.I C72~ C.3
[ 3 a. Yes, and documentation supports use of each [3 C 3 C 3
model as being appropriate.
[ 3 b. Yes, each model was appropriate, but inadequate [3 C 3 C 3
documentation was available to explain its selection.
[ 3 c. Cannot be determined. Documentation not provided [3 C 3 [ 3
to justify model selection.
[ 3 d. No. Explain: [3 C 3 [ 3
3-27
-------
E. Did analysis consider the appropriate meteorological data?
] 1. Yes, five consecutive years of the most recent representative
sequential hourly National Weather Service data.
] 2. Yes, five years of on site data subjected to quality assurance proce
Yes, at least one year of hourly sequential on site data, including
lorst-case conditions and subjected to quality assurance procedures.
YP« crrppninn data wprp uspri t.n nhf.ain rnn^prvati UP rp«nlt.«.
] 2
] 3
worst
] 4. Yes,
No.
5.
screening
Explain:
data were used to obtain conservative results.
F. Baseline Area.
The baseline area for either TSP or S02 (or both) is defined as one or more
designated attainment/unclassified (§107) areas and will include the §107 area
the source will locate in, plus any other §107 areas where the pollutant
impact exceeds 1 ug/ro3 annual average.
S02
C ]
2.
3.
Were §107 areas properly applied? TSP
[ ] a. Yes, the baseline area consists of all portions of the
designated attainment/unclassified .areas as listed in [ ]
40 CFR 81 Subpart C.
[ ] b. No, the baseline area consists of only portions of the
designated attainment/unclassified areas. [ ] [ ]
Explai n:
[ ] c. Cannot be determined from available information. [ ] [ ]
Did the baseline area include any other areas besides the area in which
the source would construct? (Mote: All other areas where an annual impact
of 1 ug/m3 or greater occurs should be included in the baseline area.)
[ ] a. No, file documentation demonstrated no significant impact
beyond area where source would locate.
[ ] b. No, but documentation was not provided to indicate whether
other areas should have been included.
C ] c. Yes, (TSP)/ ($02) attainment and unclassified areas
were included.
[ ] d. CBD.
Did the source trigger the baseline date?
[ ] a. Yes, for entire baseline area.
C ] b. Yes, for (TSP)/ ($02) of the areas within the baseline area.
[ ] c. No, baseline date(s) for all areas within baseline area
previously triggered.
[ ] d. CBD.
3-28
-------
G. Increment Consumption
Did the analysis include, where appropriate (or explain why not):
TSP S02
a. emissions from major sources commencing construction
after 1/6/75 in determining increment consumed (PSD
Workshop Manual, Pt I, Sec. C.2)?
(i) Yes, within both impact and screening area. [ ] [ ]
(ii) Yes, but only in impact area. [ ] [ ]
(iii) Yes, but only in screening area. [ ] [ ]
(iv) No. [ ] [ ]
(v) CBD C ] [ ]
b. emissions from minor sources occurring after the
applicable baseline date(s) within the impact area
in determining increment consumed?
(i) Yes [ ] C ]
(ii) No. Explain: [ ] [ ]
(iii) Not applicableSource triggered baseline date. [ ] [ ]
c. What impact concentrations were used for the short-term increments?
Concentration Used
Highest Of the
Pollutant Highest 2nd highest Other (explain)
TSP [ ] C ] C ]
S°2 C ] M C ]
Is there any reason to believe that an increment analysis should
have been performed but was not?
[ ] No. Increment
[ ] Yes, as follows: [ ] TSP L J Class I L J Class II
[ ] S02 [] Class I [ ] Class II
Explain each "yes":
3-29
-------
SECTION VII. NAAQS PROTECTION
A. Was a NAAQS analysis performed?
[ ] NO, GO TO SECTION VII. S
[ ] YES, as follows:
Pollutant _ Averaging Time: Model used _
1. [ ] TSP - a[ ]24hr: __ b[ ]annual
2. [ ] SOe a[ ]3hr: b[ ]24hr: c[ ]annual
3. [ ] NOX C ]annual
4. [ ] CO a[ ]lhr: b[ ]8 hr:
5. [ ] 03 [
6. [ ] Pb [ ]3mos:
B. For the pollutants checked above, how was the analysis performed?
FOR THESE POLLUTANTS:
[ 3 1. by the applicant with adequate review
(including replication of results, if [ ] all C ]:
appropriate) by the agency.
[ ] 2. by the applicant without adequate review
(including replication of results, if [ ] all [ ]:
appropriate) by the agency.
Explain:
[ ] 3. by the agency. [ ] all [ ]:
C. Did the applicant/agency use the appropriate model(s) to complete the analysis?
FOR THESE MODELS:
[ ] 1. YES, and documentation supports use
of the model(s). [ ] all [ ]:
[ ] 2. YES, but inadequate documentation was
available to explain Its use. [ ] all [ ]:
[ ] 3. CANNOT BE DETERMINED, documentation not
provided to justify model selection. [ ] all [ ]:
[ ] 4. NO, documentation failed to address
appropriate considerations. C ] all [ ]:
Explai n: ;
3-30
-------
D. Did the applicant/agency use the appropriate model options (urban/rural,
receptor network design, wind speed profiles, building wake effects, final/
gradual plume rise)?
FOR THESE MODELS:
[ ] 1. YES, and documentation supports use of •
the option(s). [ ] all [ ]: .
[ ] 2. YES, but inadequate documentation was [ ] all [ ]:
available to explain their use.
[ ] 3. CANNOT BE DETERMINED, documentation not [ ] all C ]:
provided to justify model option selaction.
[ ] 4. NO, documentation failed ot address [ ] all [ ]: '
appropri ateconsi derati ons.
Explain:
E. Did the analysis consider appropriate meteorological data?
[ ] 1. YES, five consecutive years of the most recent representative
sequential hourly National Weather Service data.
[ ] 2. YES, five years of on site data subjected to quality assurance procedures.
[ ] 3. YES, at least one year of hourly sequential on site data, including
worst-case conditions and subjected to quality assurance procedures.
[ ] 4. YES, screening data were used to obtain conservative results.
[ ] 5. NO. Explain:
F. Is there sufficient information in the file to verify that emissions
from the following stationary sources (including sources with permits,'
but not yet in operation) were adequately considered?
YES NO
1. Existing major stationary sources.
Explain: [ ] [ ]
2. Existing minor and area stationary sources.
Explain: [ ] [ ]
6. Did the analysis provide adequate protection against the
development of "hot spots"?
C ] 1. YES, analysis adequately defined points of maximum impact determined
from consideration of all sources in the vicinity rather than the
maximum impact of the proposed source alone.
[ ] 2. NO, analysis ignored significant emissions from other sources in
the vicinity.
Explain:
[ ] 3. CANNOT BE DETERMINED. Analysis not adequately documented.
3-31
-------
H. Is there any reason to believe that a NAAQS analysis should have been
performed but was not?
[ ] NO.
[ ] YES. (Explain.) '
Vltl. EMISSIONS OFFSET REQUIREMENTS
A. Emissions offsets:
[ ] 1. were not applicable to this source. (GO TO SECTION IX.)
[ ] 2. were applied to the following pollutants: .
[ 3 3. should have been applied to
the following pollutants:
[ ] 4. use cannot be determined from information in the file. Explain:
B. For each pollutant requiring offsets, record whether
documentation indicates:
(Note: You may need to consult the file for the source(s) from which the
offsets are being obtained to be able to respond to the following questions.)
QUESTION
For these six questions
fill in pollutants, then
use: "Y" for yes, "N" for
no, "C" for CBD
1. that emissions offsets
obtained by this source
are expressed in the same
terms (i.e. actual or
allowable) as are those
emissions used in the
RFP demonstration.
2. that minor/area source
growth taken into account
in determining the amount
of emissions
offsets needed.
3. that offsets are
surplus and would not
interfere with RFP.
4. that the offsets are
Federally enforceable.
POLLUTANT
B
Information obtained from:
Offset
This source Other.
permit permit Explain:
[ 1 C ] [ ]
[ ] C ] C ]
[ ] C ]
C ] [ ]
3-32
-------
5. that the offsets were
required to occur on or
before the startup of the [ ] [ ] [ ]_
new or modified source..
6. that offsets were not
utilized from early source
shutdowns or production [ ] [ ] [ ]_
curtailments, except for
replacements.
C. Comments:
SECTION IX.COMMENTS. TOTES
3-33
-------
APPENDIX 4
1985 PERMIT FILE QUESTIONNAIRE
FOR SOURCES NOT SUBJECT TO PSD OR PART D (OFFSETS)
[NOTE: Unless otherwise indicated, place an "X" in the box beside each statement or
response which applies. Many of the questions will allow more than one response.]
SECTION I. SOURCE INFORMATION
A.I. Company/Source Name:
2. Size/Source Category (e.g., 250 MW coal
fired boiler):
B. 1. Region I
Date Complete
ppli cation rec'd:
1 1 1 1 1 1
o day yr
D. Date Permit to
Construct Issued:
1 1 1 1 1 1 1
mo day yr
E. Source Location
1. [ ] Attainment/unclassified for
all criteria pollutants
2. [ ] Nonattainment for:
2. State 1|T
3. Agency
a. [ ] State
b. [ ] Local:
4. Auditor
5. Permit #
6. Source-Type
a. [ ] New Source
b. [ ] Modification
3. [ ] Nonattainment area subject to a construction ban for:
4. [ ] Within 10 km of a Class I area
SECTION II. PUBLIC PARTICIPATION REQUIREMENTS
A. Public Notice:
1. [ ] was published in a newspaper (approx. cost, if available:__ )
2. [ ] provided opportunity for public hearing
3. [ ] provided opportunity for written comment
4. [ ] described agency's preliminary determination
5. [ ] included estimated ambient impact
6. [ ] indicated availability of additional information for public inspection
7. [ ] resulted in comments ( "0" if notice produced no comments)
8. [ ] was not provided because [ ] a. exempted by agency rules
[ ] b. processing error.
B. The following other affected government agencies were notified:
1. [ ] other agencies and officials within the State
2. [ ] other States
3. [ ] Federal Land Manager
4. [ ] EPA
5. [ ] none of the above
3-34
-------
C. Documentation for Section II parts A and B consists of: Notification of
Public Notice Other Agencies
1. copy of notice/correspondence in file a.[ ] b.[ ]
2. indication on processing checklist* a.[ ] b.[ ]
3. no documentation provided a.[ ] b.[ ]
4. other, explain:
* i.e., no copies, but some official indication in file that notice was provided
SECTION 111. APPLICABILITY DETERMINATIONS
A. Definition of Source. The source or modification for which this permit
application was made was considered by the reviewing agency to consist of:
1. [ ] The following new and modified pollutant-emitting activities associated
with the same industrial grouping, located on contiguous or adjacent
sites, and under common control or ownership (if more than 5, list
only 5 largest):
TRY/
Pollutant
New
C ]
C 1
.[ ]
C ]
C ]
Emissions
Modif. Unit/Size
C ]
C ]
C ]
C ]
[ ]
a.
b.
c.
d.
e.
2.a.[ ] The following new and modified pollutant-emitting activities should
also have been included:
Emissions Unit Emissions Reason given by agency for not
or Activity/Size Pol 1utant" TPY New Modif. considering as part of source
(1) [ ] [ ]
(2) [ ] [ ]
(3) C ] [ ]
(4) [ ] [ ]
b. Did this new source or modification escape PSD or Part D (major source
locating in a nonattainment area) review as a result of the omission of
the activities listed in Section 111.A.2.a.?
C ] Yes [ ] Mo [ ] CBD
3. [ ] CBD
3-35
-------
B. Fugitive Emissions. This source:
1. [ ] Does not have any quantifiable fugitive emissions. (Proceed to Section III.
2.a.[ ] Has quantifiable fugitive emissions which were:
(1) [ ] Included in determining whether the source was major for the following
pullutants:
(2) [ ] Not included in determining whether the source was major for the
following pollutants: because the
source is neither one of the 28 PSD source categories nor regulated
under Sections 111 or 112 of the Act.
(3) C ] Not included in determining whether the source was major for the
following pollutants: , although they
should have been.
b. Did this source escape PSD or Part D review as a result of the omission
of fugitive emissions?
[ ] Yes [ ] No [ ] CBD
3. [ ] Has too little documentation in the file to determine whether
quantifiable fugitive emissions occur or were considered.
C. Potential to Emit (PTE). Applicability determinations should be based on
PTE rather than actual emissions.
[ ] Mark here if agency used, or accepted use of actual emissions; if so,
respond to the following questions on the basis of how actual emissions
were determined.
1. The PTE of this source or modification was determined:
(a) Using emissions rates based on emission factors which were:
(i) [ ] Well established (e.g., AP-42) or well documented in the file.
(ii) [ ] Not well established and lacking adequate documentation for the
following units and pollutants:
(b) [ ] CBD
(c) [ ] Using another method. Explain:
2. The PTE of this source was determined on the basis of:
a. C ] No limitations or restrictions on operation, materials or equipment.
b. [ ] The following limitations and restrictions:
3-36
-------
If used, was limitation identified on:
Precpnstruction Permit Operating Permit
Limitation ~" TeT W Te? ¥o NA
(1) C ] Control equipment C ] [ ] [ ] [ ] [ ]
(11) [ 3 Operating hours [ ] [ ] [ ] C ] [ ]
(111) C ] Operating rate C ] C ] C ] C ] [ ]
(1v) [ ] Fuel/material [ ] [ ] [ ] C ] C ]
restriction
(v) [ ] C ] C ] MMM
If Section III.C.2.b. above is marked, please describe the limitations:
3. Was PTE determined correctly?
[ ] YES -
C ] CBD
[ ] NO, Explain:
D. Emissions Netting. (FOR MODIFICATIONS ONLY. [ ] Mark here if not applicable,
THEN GO TO Section III.E.)
1. The determination of significant emissions can be a complex process when
emissions netting occurs. The following table should be used to determine
whether the proper procedure was followed:
Did agency
Other creditable, correctly identify
Proposed emission contemporaneous Overall net whether emissions
changes, TPY
Pollutant (+) (-) Net
a. :
b. :
c. :
d. :
e.
f. :
g- :
h. :
emissions changes, TPY change, were
1+) (-) TPY Yes
C ]
C ]
C ]
C ]
c ]
C ]
C ]
C ]
signi
Mo
C ]
C ]
C ]
C ]
f leant?
CBD
C ]
C ]
C ]
C ]
C ]
3-37
-------
2. [ ] Emissions netting was based on actual emissions changes from existing
units, which:
were were not CBD
[ ] [ ] [ ] (a) Representative of normal unit operation
[ ] [ ] [ ] (b) Based on a two-year average
[ ] [ ] [ ] (c) Expressed in TRY
[ ] [ ] [ ] (d) Contemporaneous
3. C ] Emissions netting was based on another approach which was:
(a) [ ] Acceptable. Explain:
(b) [ ] Incorrect. Explain:
4. [ ] Emissions reduction credits which are:
(a) [ ] Federally enforceable. Explain how:
(b) [ ] Not Federally enforceable. Explain:
5. [ ] Creditable contemporaneous emissions decreases were used which:
Were not Were CBD
[ ] [ ] [ ] (a) Previously relied on by a PSD source in
determining net change.
[ ] [ ] [ ] (b) Counted as part of the SIP attainment
strategy.
[ ] [ ] [ ] (c) Relied on to meet the reasonable further
progress requirement of Part D
[ ] [ ] [ ] (d) Made Federally enforceable as permit
conditions
E. Definition of Major. This new source (or EXISTING source, in the case of a
modification) was NOT subjected to PSD or Part D provisions because:
1. C ] Emissions of each pollutant from new or existing source were less than
100 TPY.
2. [ ] Emissions of all PSD pollutants from new or existing source were less
than the:
a. [ ] 250 TPY threshold for non-listed source categories; or
b. [ ] 100 TPY threshold for the 28 listed PSD source categories.
3-38
-------
3. [ ] Emissions of each Part D pollutant from new or existing source were
less than 100 TRY.
4. [ ] Existing source was major, but emissions increases resulting from
modification were not significant.
5. [ ] Review agency erred in determining applicability threshold and source should
have been subject to:
a. [ ] PSD review for the following pollutants: .
b. [ ] Part D review for the following pollutants:
6. [ ] CBD
7. [ ] Other. Explain:
E. Emissions limits.
If allowable emissions limits for each pollutant from each emissions unit
were included in the preconstruction permit, the number of limitations appearing
in the preconstruction permit would be (e.g., for a boiler emitting
TSP, S02, NOX, and VOC, there would be four limitations).
2. The number of limitations actually appearing in the preconstruction
permit(s) is .
Number
3. How many of the limitations: YesHJoCBDTotal
a. Include clear and concise averaging periods? -
b. Establish emissions limitations consistent with
acceptable measurement techniques?
c. Consist of design, equipment, work practice,
or operational standards?
d. Appear Federally enforceable?
e. Include stated or referenced compliance test ,_
methods?
F. Applicabi1ity Summary. Is there any reason to believe that this application
should have been subject to PSD or Part D provisions?
1. [ ] NO, GO TO Section IV.
2. [ ] YES. Explain:
3-39
-------
SECTION IV. AMBIENT AIR QUALITY AMALYSIS
A. Was an ambient impact analysts performed?
[ ] NO, GO TO SECTION IV-G.
[ ] YES, as follows:
Pollutant Averaging Time: Model used (list)
1.
2.
3.
4.
5.
6.
[
C
C
[
]
]
]
]
TSP
S02
NOX
CO
03
Pb
[ ]3hr:
[ ]lhr:
[ ]lhr:
[ ]24hr:
[ ]24hr:
[ ]8hr:
[
[
[
C
]annual
]annual
]annual
]3mos:
•
•
.
B. For the pollutants checked above, how was the analysis performed:
C ] 1. by the applicant with adequate review
(including replication of results, if [ ] all [ ]:
appropriate) by the agency.
[ ] 2. by the applicant without adequate review
by the agency [ ] all C ]:
Explain:
[ ] 3. by the agency. C ] all [ ]:
C. Did the applicant/agency use the appropriate model(s) to complete the analysis?
[ ] 1. YES, and documentation supports use
of the model (s). C ] all [ ]:
[ ] 2. YES, but inadequate documentation was
available to explain its use. C ] all C ]:
[ ] 3. CBD, documentation not
provided to justify model selection. [ ] all C ]:
[ ] 4. NO. C ] all [ ]:
Explain:
D. Did the agency exercise the appropriate model options (urban/rural, receptor
network design, wind speed profiles, building wake effects, final/gradual
plume rise, etc.)?
OPTION:
C. 1 CT2~ C. 3
[ ] a. Yes, and doumentation supports use of each option [ ] [ ] [ ]
as being appropriate.
[ ] b. Yes, each option was appropriate, but inadequate [ ] C ] [.]
documentation was available to explain its selection.
3-40
-------
[ ] c. Cannot be determined. Documentation not provided [ ] [ ] [ ]
to justify option selection.
[ ] d. No, documentation failed to address appropriate [ ] [ ] [ ]
considerations. Explain:
E. Did the analysis consider appropriate meteorological data?
[ ] 1. YES, five consecutive years of the most recent representative
sequential hourly National Weather Service data.
[ ] 2. YES, five years of on site data subjected to quality assurance procedures.
[ ] 3. YES, at least one year of hourly sequential on site data, including
worst-case conditions and subjected to quality assurance procedures.
[ ] 4. YES, other. Explain:
[ ] 5. NO; Explain:
E. Is there sufficient information in the file to verify that emissions
from the following stationary sources (including sources with permits,
but not yet in operation) were adequately considered?
YES NO
1. existing major stationary sources.
If no, explain: [ ] [ ]
2. existing minor and area stationary sources.
If no, explain: [ ] [ ]
F. Did the analysis provide adequate protection against the
development of "hot spots"?
C ] 1. YES, analysis adequately defined points of maximum impact determined
from consideration of all sources in the vicinity rather than the maximum
impact of the proposed source alone.
[ ] 2. NO, analysis ignored significant emissions from other sources in
the vicinity.
Explain:
[ ] 3. Cannot be determined from information available in file.
G. Is there any reason to believe that the proposed project should have
been subjected to an ambient impact analysis that was not performed?
C ] 1. YES, source was within 10 km of a Class I area but its
ambient impact was not considered.
[ ] 2. YES; Explain:
[ ] 3. NO.
SECTION V.ADDITIONAL REVIEW
Was this source subject to any of the following additional reviews?
Yes No
BACT
[ ] [ ] LAER
C ] [ ] PSD/Increment analysis
Comments:
3-41
-------
COMPLIANCE
ASSURANCE
-------
4. COMPLIANCE ASSURANCE
INTRODUCTION
In an effort to build on a successful first year audit program, the
major parts of the compliance assurance element in FY 1985 will be the same
as FY 1984: periodic review and assessment of source data, file reviews,
and overview inspections. It is clear from the results of last year's
review that a more detailed format is essential for improved consistency in
the compliance assurance program in FY 1985. Guidance for the FY 1984
audit was of a more general nature with an outline of topics and few specific
questions (except for the file review checklist). This approach left too
much of the audit implementation open to Regional interpretation, leading
to a variety of audit results both between and within Regional offices.
To correct this problem, the specific set of questions which follow
were developed -for use by all ten Regions to ensure consistency in the NAAS
effort, and provide a common basis for comparison nationally of State and
local compliance programs. These questions are based on the general compli-
ance assurance guidelines from last year and do not represent any significant
expansion of the program for FY 1985. All questions must be answered, and
procedures followed for each audit, without exception.
PERIODIC REVIEW AND ASSESSMENT OF SOURCE DATA
To assess the adequacy of State and local compliance programs in meeting
Clean Air Act requirements, the EPA Regional offices continually review
source compliance status and inspection information submitted by State and
local agencies for the SIP, NSPS, and NESHAPs programs. This information
is contained in the Compliance Data System (CDS). In order to draw upon
this knowledge, the Regions prior to each annual audit visit are to obtain
CDS retrievals for Class A SIP, NSPS, and NESHAP sources on inspection
frequency, compliance rates, and enforcement activity. These data must
then be analyzed by answering the following questions to form a picture of
the status of the audited Agency compliance program:
1) What percentage of sources have received the required inspections
as specified in EPA's inspection frequency guidance (sometimes documented
in the Section 105 grant agreements)? Prepare an inspection summary for
each audit as follows:
Progress in Meeting Inspection Commitments
Class Al SIP NSPS* NESHAP*
Total Sources
Number Inspected
in FY 1984
Percentage
*Assumes program delegated to State or local agency. Provide
audited agency comment on the validity of data..
4-1
-------
2) What is the compliance-status breakdown of sources in each air
program? Prepare compliance chart for each audited agency as follows:
Compliance Status of Operating Sources (Provide Date of Retrieval)
In (a) Meeting (b) In (c) (d)
Program Compliance Schedule Violation Unknown Total
Class Al SIP
Class A2 SIP*
NSPS
NESHAP
*Subset of Class A SIP
Also provide audited agency comment on validity of data.
Definitions
m
a. "In Compliance" includes sources reported to EPA by States and locals
as being in compliance with Clean Air Act requirements.
b. "Meeting Schedule" includes sources reported by States and locals as
not in compliance with one or more emission limitations, but which are
in compliance with an enforceable schedule to achieve compliance with
the emission limitation(s).
c. "In Violation" includes sources which are reported by the States and
locals as being in violation of Clean Air Act requirements and not
subject to a compliance schedule, or in violation of an established
schedule (have failed to meet one or more increments of progress).
d. "Unknown" includes sources where compliance status information is not
available (possibly due to lack of a recent inspection), or a decision
has not yet been made by EPA or the State on which of the other three
categories a source belongs in.
3) How many operating NSPS sources have not had their required 180 day
start-up test? Prepare summary chart for NSPS sources operating with and
without required performance test as follows (if program not delegated
indicate such):
4-2
-------
NSPS Sources Operating With and Without
Performance Test
Number of Number Operating Number Operating
Operating NSPS With Test >180 Days Without Test
4) Based on CDS, list the number of Class Al SIP, NSPS, and NESHAP
long-term complying sources (defined as being in-compliance four consecutive
quarters or more) that have not recently been inspected. Specify results
for each program separately. Show sources per year that have not been
inspected in: 2, 3, 4, etc., years.
5) This question relates to implementation of the "Timely and Appropriate"
guidance. The Region, either through the monthly consultations with the
States or during this pre-visit audit preparation, must trace each violator's
progress in accordance with the "Timely and Appropriate" guidelines, Part
II (A)-(E) Timelines for Enforcement Action contained in Joseph Cannon's
memorandum of June 28, 1984, and reproduced below:
A. 1. The clock starts (i.e.. day zero) 30 days after the date of the
inspection or receipt of a source self-monitoring report which
first identifies the violation. This provides sufficient time for
an evaluation of the inspection or source report data to determine
if a violation exists. If, during this 30-day period, the State
determines that a stack test or a sample analysis is required to
determine or confirm the violation, the clock does not start
until the date of receipt of the stack test or sample analysis
report.
2. Any serious problems occurring earlier in the process would be
identified and addressed in the National Air Audit System process
rather than under these timeliness.
B. By day 45, the source should be notified of the violation and its
need to remedy it by the State in writing or in a documented conver-
sation (in any form the State feels is appropriate).
C. By day 120, the source shall either be in compliance, on a legally-
enforceable expeditious State administrative or judicial order, be
subject to a referral to the State attorney general or for a State
adjudicatory enforcement hearing, or be subject to a proposed SIP
revision which has at least been scheduled for a State hearing and
which EPA staff-level review shows is likely to be approved. For
cases where penalties are required (see IV below), penalties must
also be addressed as part of the State action if it is to be sufficient
to obviate further EPA action.
4-3
-------
D. If a schedule is established, the State will monitor compliance
with that schedule and report on progress in accordance with estab-
lished reporting requirements. If a referral is made, EPA will
continue to monitor the progress of the case to and after filing.
If a SIP revision is initiated, EPA will monitor the progress of
the revision through the State administrative process. If a case
or SIP revision becomes unduly delayed, EPA will discuss this with
the State and may choose to initiate a parallel Federal action. No
formal timelines are being established for this stage of the enforcement
process, however.
E. If none of the actions specified in C. have occurred by day 120,
EPA will discuss with the State the status of the State's actions
and its expectations. If discussions with the State suggest that
the State is close to resolving the violation or that further
deferral is otherwise appropriate, EPA will continue to defer to
enable the State to complete its action. If EPA determines that
further deferral is not justified, it will proceed with its own
action at this point.
Based on the results of this review, the following questions must be answered:
Are violators being addressed according to "Timely and Appropriate"
guidelines?
What procedures have been established for identifying sources subject
to the "Timely and Appropriate" guidance and inputing these sources
into a tracking system?
What procedures have been established for the communication of data by
the EPA/State and local agencies? Have these procedures been followed?
Be specific.
Give specifics of State and local actions and results in cases where
EPA deferred NOV issuance beyond day 120.
Have the State and local agencies satisfied penalty requirements where
applicable under the "Timely and Appropriate" guidance?
Explain any deviations from the guidance.
6) How many long-term violators (shown in CDS as in violation two
consecutive quarters or more) have not been subject to enforcement activity
or additional surveillance activity? Provide a source-by-source listing of
the Agencys1 Class Al SIP, Class A2 SIP, NSPS, and NESHAP long-term violators
including type and date of most recent surveillance or enforcement activity.
Following the investigation into each of these questions, findings are
to be presented for each audited program under the heading of "Pre-visit
Assessment of Compliance Program." These findings should be clear and
concise statements on what CDS reflects about the program and what is known
4-4
-------
through the monthly consultations about "Timely and Appropriate" implementa-
tion. Conclusions can then be drawn about the condition of the audited
agency's compliance program, and summarized in paragraph form. These
results should also be used in the selection of files to be reviewed during
the on-site visit.
This pre-visit assessment should be sent to the State or local agency
at least 15 work days prior to the on-site visit. The Region should also
include in this transmittal any other air compliance related items requiring
discussion during the audit-even items of concern which are not directly
related to the pre-visit assessment.
FILE REVIEW
An effective State and local compliance program must have a well documented
file on each source. This file should be available for use by management
and field personnel. The structure and location of files are optional as
long as any needed data can be supplied upon request. The files must
contain information supporting the compliance status of each source.
The audit team is to review a representative sample of files from the
three air programs (SIP, NSPS, NESHAP) in each State or local agency. In
most cases, each audit will consist of 15-20 files. Sources selected for
review will be a cross-section of sources, but will include some sources
based on time in violation, NSPS sources with CEM requirements, recently
reported compliance changes, citizen or congressional inquiries, problems
surfaced in the CDS pre-visit program analysis, or personal knowledge of
the source.
For each file reviewed, the following questions must be answered. The
file review checklist which follows may be used as is, or altered as desired
to serve as a worksheet for each file.
Documentation
1. Can the reviewer, from information available in the file, determine
the programs to which the source is subject? If not, why not? The various
programs are SIP, PSD, NSPS and NESHAP. List any facilities where determination
was not possible.
2. From the information available in the files, can the source's
compliance status be determined for all regulations to which it is subject?
3. Does the file contain documentation supporting the source's compliance
status? *(At a minimum, the date of the last visit or excess emission
report review, person making the compliance determination, and what that
determination was).
4. Are all major emission points identified (i.e., in an inspection
report, operating permit, etc.) and each point's compliance status indicated?
4-5
-------
5. Does the file identify which emission points are subject to NSR,
NSPS, PSD, and NESHAP requirements? If yes, are regulated continuous
emission monitoring (CEM) requirements or permit conditions shown to be in
compliance and documented? Are required start-up performance tests included?
6. Does the file identify special reporting requirements to which a
source may be subject (i.e., excess emission reports from a malfunction or
CEM requirements) and are any such reports found in the file?
7. Does the file include technical reviews, source tests, CEM performance
specification tests, permit applications, correspondence to and from the
company, and other supporting documentation?
8. What methods of compliance documentation are used (e.g., source
test, CEM, fuel-'sampling and analysis, inspection, certification, engineering
analysis, etc.)?
9. Was the method used to ascertain compliance the most appropriate
one for the type of source being documented? Is the method prescribed by
NSPS, NESHAPs or SIP? If not, explain.
10. If the documentation includes an inspection, does the report
contain control equipment parameters observed during the inspection (pressure
drops, flow rates, voltages, opacities)? Were observed control equipment
operating parameters or CEM emission levels compared to specified permit
conditions, design parameters, or baseline observations? Were plant operating
parameters recorded?
11. If documentation is by a stack test, were visual emission observations
or CEM emissions levels and operating parameters recorded during the test?
Was there a quality assurance procedure used with a stack test? Was the
test observed and reviewed by an Agency representative? Was the test
conducted in accordance with the reference test protocol and procedures?
12. Are enforcement actions contained in the file?
13. Are compliance actions initiated according to T&A Guidance? List
any determined to take longer than 30 days from the time of violation
detection. If yes, how long?
14. What documentation is there to support the enforcement action
(reinspection, letter, etc.)?
*For the purposes of this report, the term "source"is synonymous with
facility and consists of one or more emission points or processes.
4-6
-------
15. Is there documentation to show follow-up to the enforcement action
(reinspection, letter, etc.)?
16. Are citizens' complaints documented in the file and followed up on?
17. What action does the Agency take with respect to excess emission
reports?
*
The review team should summarize their findings following the file
review. The summary must at a minimum, address the following questions:
1. Do all files reflect a reasonable profile of the source (meaning
that the files contain inspection reports, stack test reports, CEM data,
enforcement actions, etc.). If not, explain.
2. Do all files contain adequate written documentation to support the
compliance status reported to EPA? If not, explain.
3. Are violations documented and pursued to return the source to
compliance expeditiously (generally in conformance with "Timely and Appropriate"
guidance)?
Checklist For File Reviews
1. Source Name/Location Compliance Status
State Identification Number
2. Documentation in Files
0 multiple files? all programs identified?
0 chronological index to file (document name/date)?
missing documents:
Sequence of Actions Yes No
0 Is there a logical sequence, e.g., inspection, 30-day
follow-up inspection, State NOV, State order? Litigation? _
0 Periods of delay? _
0 Multiple final compliance date extensions? _
0 Repetitive actions? Increased intensity of enforcement
response?
4. Adequacy of Inspection Report(s)
0 Process and operating parameters defined?
4-7
-------
0 Regulation and emission limit specified?
0 Compliance method or inspection procedure specified?
0 Quality assurance identified? •
0 VE readings in accordance with Method 9?
0 Actual emissions quantified? If yes, by what method?
0 Can compliance be determined?
For individual points
For different programs
For the entire source?
0 CEM monitor and recordkeeping procedures inspected?
5. Compliance Status
0 Does file information agree with historical CDS?
0 What is the current compliance status in the file?
6. State or Local Actions
0 Timeliness of State or local action?
Number of days to take formal action?
0 Action in conformance with formal State or
local procedures?
0 State or local action effective?
0 Is compliance achieved?
7. Enforcement Actions
0 What is the most prevalent enforcement response?
0 Are penalties considered?
0 Is there a reasoned basis for action or inaction?
8. Citizen Complaints
0 Adequate follow-up?
OVERVIEW INSPECTION
To provide quality assurance for compliance data in State or local
files furnished to EPA, and to promote effective working relationships
between EPA and State or local agencies, EPA will implement an overview
inspection program. It is envisioned that 2-3 percent of the Class A SIP,
NSPS, and NESHAP sources in the CDS inventory will be inspected by the
completion of the FY 1985 audits and that the results will be reflected in
the final reports.
EPA should notify the State and local agencies of its intent at least
30 days before each inspection is to take place to encourage their participa-
tion. Each inspection should be an independent verification of the source's
compliance status at a minimum, and should review the State and local
inspector's procedures for determining compliance if the inspection is
4-8
-------
jointly performed. Contractors will not be used for any initial inspection
efforts. However, EPA in responding to problems identified is not constrained
in its use of available resources.
To promote uniformity the following questions must be answered for the
overview inspection effort:
1) How were sources selected by the Region for the overview inspections?
2) How many inspections were performed?
3) What exactly did each inspection consist of? Specify inspection
procedures used as well as the degree and extent of involvement of
State personnel.
4) What was their purpose (that is, to independently verify State
reported compliance, to observe State inspection practices, or some
combination of these)? Other purposes?
5) What were the results of each inspection? Answer should relate to
purpose stated in item 4.
6) Discuss the important points overall of the overview inspection
findings. Give recommendations for resolution of any problems
discovered during the effort.
This presents the compliance assurance questionnaire for the FY 1985
MAAS. Hopefully, this will give more structure and uniformity to the audit
process and will allow more substantive recommendations to be made where
the need exists.
Compliance Assurance Report Format
For each audit performed, the Region must prepare a compliance assurance
report that includes a complete summary of all audit activities (including
copies of the file checklists) and answers to all questions (by number)
outlined in the above questionnaire. The main body of the report should
follow the questionnaire exactly in each of the three areas: Pre-visit
Program Assessment, File Review, and Overview Inspections. In addition,
each report must include an overall summary of findings for each State
program including positive and negative points, and recommendations for
resolution. Each final report should be reviewed by the audited agency to
help eliminate misconceptions or misunderstandings.
4-9
-------
AIR MONITORING
-------
5. AIR MONITORING
This material is identical to:
SECTION 2.0.11
SYSTEM AUDITS CRITERIA
AND PROCEDURES FOR
AMBIENT AIR MONITORING PROGRAMS
of the
QUALITY ASSURANCE HANDBOOK FOR
AIR POLLUTATION MEASUREMENT SYSTEMS,
VOLUME II. EPA-600/4-77-027a
5-1
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 1 of 142
11.0 SYSTEM AUDIT CRITERIA AND PROCEDURES FOR AMBIENT :
AIR MONITORING PROGRAMS
11.1 Introduction
11.1.1 General - A system audit is an on-site review of an
agency's ambient air monitoring program to assess its compliance
with established regulations governing the collection, analysis,
validation, and reporting of data. A system audit is performed
annually by a member of the EPA Regional Quality Assurance (QA)
staff, for each state or autonomous agency within the Region.
The purpose of the guidance included here is to provide the
regulatory background and appropriate technical criteria which
form the basis for the air program evaluation, by the Regional
Audit Team. In order to promote national uniformity in the
evaluation of state and local agency monitoring program
performance, it is required that all EPA Regional Offices use at
least the short form questionnaire, corrective action
implementation request (CAIR), and the system audit reporting
format each year. Using section(s) of the long form questionnaire
is left to the discretion of the Regional QA Coordinator with
the concurrence of the State or local agency. The short form ques-
tionnaire is essentially the same as the monitoring audit question-
naire used in FY-84. No significant substantive changes have been
made; however, to improve the audit process, the questionnaire has
been reorganized to improve the information received and facilitate
its completion. In addition, requests for resubmission of data
already possessed by EPA have been deleted.
The scope of a system audit is of major concern to both EPA
Regions and the Reporting Organizations to be evaluated. A system
audit as defined in the context of this document is seen to
include an appraisal of the following program areas: Network
5-2
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 2 of 142
Management, Field Operations, Laboratory Operations, Data
Management, Quality Assurance and Reporting. The guidance pro-
vided concerning topics for discussion during an on-site
interview have been organized around those key program areas
(Section 11.5). The depth of coverage within these areas may
3
increased or decreased by use of section(s) of the long-form
questionnaire (Section 11.7) in conjunction with the short-form
questionnaire (Section 11.6). Besides the on-site interviews, the
evaluation should include the review of some representative ambient
air monitoring sites and the development of data audit trails
from field acquisition through reporting into the Storage And
Retrieval Of Air Data (SAROAD) computer system.
The system audit results should present a clear, complete and
accurate picture of the agency's generation of ambient air
monitoring data. It should also enable the Regional QA Audit Team
to assess data quality in terms of precision, accuracy, representa-
tiveness, and completeness, in keeping with the objectives
of the National Air Program.
11.1.2 Road Map to Using this Section - This section contains
sufficient information to conduct a system audit of an agency
responsible for operating ambient air monitoring sites, as
part of the State and Local Air Monitoring Stations (SLAMS)
network, and to report the results in a uniform manner. The
following topics are covered in the subsections below:
o A brief sketch of the regulatory guidance which dictates
that system audits be performed indicating the regulatory
uses to which the audit results may be put (Section 11.2);
o A discussion of
1) the requirements on the agency operating the SLAMS
network;
2) a delineation of the program facets to be evaluated
by the audit; and
3) additional criteria to assist in deciding the required
extent of the forthcoming audit; (Section 11.3)
5-3
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 3 of 142
o A recommended audit protocol for use by the Regional Audit
Team, followed by a detailed discussion of audit results
reporting (Section 11.4);
o Criteria for the evaluation of State and local agency
performance including suggested topics for discussion during
the on-site interviews (Section 11.5);
o A short-form questionnaire, based on the National Air
Monitoring Audit Questionnaire prepared by the STAPPA/ALAPCO
Ad Hoc Air Monitoring Audit Committee. (10-20-83)
(Section 11.6);
o A long-form questionnaire, organized around the six key
program areas to be evaluated (Section 11.7); and
o A bibliography of EPA guideline documents which provides
additional technical background for the different program
areas under audit (Section 11.8).
The guidance provided in this section is addressed primarily
to EPA Regional QA Coordinators and members of the Regional audit
teams to guide them in developing and implementing an effective
and nationally uniform yearly audit program. However, the
criteria presented can also prove useful to agencies under audit
in guiding them with respect to descriptions of the program areas
which will be evaluated.
Clarification of certain sections, special agency
circumstances, and regulation or guideline changes may require
additional discussion or information. For these reasons, a list
of contact names and telephone numbers is given in Table 11-1.
5-4
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 4 of 142
TABLE 11-1. LIST OF KEY CONTACTS AND TELEPHONE NUMBERS
Assistance
Area
Office/Laboratory
Name
Telephone
Number
EPA Location
Laboratory William J. Mitchell
Areas and NPAP
General QA William F. Barnard
Guidance
Monitoring Stanley Sleva
Objectives/Si ting
PARS System Gardner Evans
SAROAD Jake Summers
System/NADP
(919) 541-2769
FTS 629-2769
(919) 541-2205
FTS 629-2205
(919) 541-5651
FTS 629-5651
(919) 541-3887
FTS 629-3887
(919) 541-5694
FTS 629-5694
EMSL/QAD/PEB
EMSL/QAD/PEB
OAQPS/MDAD/MRB
EMSL/MAD/DRB
OAQPS/MDAD/NADB
5-5
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 5 of 142
11.2 Regulatory Authority to Perform a System Audit
11.2.1 General Regulatory Guidance - The authority to perform
system audits is derived from the Code of Federal Regulations (40
CFR). Specifically: 40 CFR Part 35 which discusses agency grants
and grant conditions, and 40 CFR Part 58 which deals specifically
with the installation, operation and quality assurance of the
SLAMS/NAMS network.
The regulations contained in 40 CFR Part 35 mandate the
performance of yearly audits Agency Air Monitoring Programs by the
Regional Administrators or their designees. Pertinent regula-
tory citations are summarized in Table 11-2. All citations are
quoted directly from the regulations and are intended as an
indication of the context within which system audits are performed
and the impact that audit results may have on a given agency.
Even though this is the regulatory authority to conduct such audits,
for the SLAMS network the specific authority is derived from 40
CFR Part 58. Three specific citations from 40 CFR Part 58 are also
quoted in Table 11-2. These citations bring the requirements of
40 CFR Part 35 fully into Part 58.
In addition to the regulations presented in Table 11-2, a
further requirement is imposed on Reporting Organizations submitting
data summary reports to the National Aerometric Data Bank (NADB)
through the SAROAD computer system. SAROAD acceptance criteria call
for at least 75% data completeness, which has been accepted as a
data quality objective for state and local agencies monitoring
operations. The Regional QA Coordinator may wish to use this
requirement together with information obtained by accessing the
SAROAD NA285 or NA288 Computer Programs, discussed on page 12. The
percent data completeness may be effectively used as an indica-
tor whether a rigorous system audit, using the long-form question-
naire, might be needed or not.
11.2.2 Specific Regulatory Guidance - The specific regulatory
requirements of an EPA-acceptable quality assurance program are to
5-6
-------
TABLE 11-2. SUMMARY OF REGULATORY AUTHORITY TO CONDUCT SYSTEM AUDITS
A. Highlighta of HO CFR 35
Section Number
Description
Grant Amount
en
\
Reduction In Grant Amount
Text
•In determining the amount of support for a control agency, the
Regional Administrator will consider
A. The functions duties and obligation assigned to the
agency by an applicable Implementation plan,
B. the feasibility of the program In view of the resources
to be made available to achieve or maintain EPA
priorities and goals
C. the probable or estimated total cost of the program
In relation to Its expected accomplishments
D. the extent of the actual or potential pollution problem
B. the population served within the agency's JurIndictton
P. the financial need, and,
0. the evaluation of the agency'a performance.*
•If the Regional Administrator's annual performance evaluation
reveals that the grantee will fall or has failed to achieve the
•ipected outputs described In his approved program, the grant
amount shall be reduced •
35.520
Criteria for (Grant) Award
35.520
Criteria for (Grant) Award
"Ho grant may be awarded to any Interstate or Intemunlolpal air
pollution control agency unless the applicant provides aaaurance
satisfactory to the Regional Administrator that the agency
provides for adequate representation of appropriate State,
Interstate, local and (when appropriate) International Interests
In the air quality control region, and further that the agency
has Ute capability of developing and Implementing a comprehenal
air quality plan for the air quality control region^1*
No grant may be awarded unless the Regional Administrator has
determined that (1) the agency has the capability or will develop
the capability, to achieve the objectives and outputs described
In Its EPA-approved program, and (2) the agency has considered
and incorporated as appropriate the recommendations of the latest
EPA performance evaluation In its program. "
ifl
o
0 rt
hh O
D*
M 0)
N> ^
VO
00
o # tn
01 0) (t>
rt < O
H. rt
(0 H-
O H-0
" O 3
(D
to
-------
TABLE 11-2. SUMMARY OF REGULATORY AUTHORITY TO CONDUCT SYSTEM AUDITS
A. Highlights of 10 CFR 35 (Confd)
Section Number
Deaorlptlon
Text
Grant Conditions
ui
00
In addition to any other requirements herein, each air pollution
control grant shall be subject to the following conditions:
A. Direct cost expenditures for the purchase of..
B. The Bum of non-Federal recurrent expenditures.
C. The grantee shall provide such Information as the
Regional Administrator may from tine to time require
to carry out his functions. Such Information may
oontaln( but la not limited tot Air quality data,
emission Inventory data, data describing progress
toward compliance with regulations by specific sources,
data on variances granted, quality assurance Informa-
tion related to data collection and analysis and similar
.regulatory actions, source reduction plans~an3
procedures, real time air quality and control
activities, other data related to air" pollution
emergency episodes, and similar regulatory actions.
to air pol
' regulator
35.538-1
Agency Evaluation
35.110
Evaluation of Agency Performance
"Agency evaluation should be continuous throughout the budget
period. It la EPA policy to Unit EPA evluatlon to that which
Is necessary for responsible management of regional and national
efforts to control air pollution. The Regional Administrator
shall conduct an agency performance evaluation annually In
accordance with 357110."
"A performance evaluation, shall be conducted at least annually
by the Regional Administrator and the grantee to provide a basis
for measuring progress toward achievement of the apporved
objectives and outputs described In the work program. The
evaluation ahall be consistent with the requirements of 35.536
for air pollution control agenolea *
*O O 5d W
p) OJ fl) (D
iQ rr < O
(D (D H- ft
0) H-
O O D
0 rt 3
Hi O 25
D* ^ 0
M
-------
TABI
B. Highlights of 40 CFR 58
LE 11-2. SUMMARY OF REGULATORY AUTHORITY TO CONDUCT SYSTEM AUDITS
Seotlon Nunber
58^0
Description
Surveillance Plan
Content (SLAMS)
Text
"By, January 1, 1980 the State aha 11 adopt and submit to the
Administrator a revision to the plan which will:
A. Provide for the
B. Provide for meeting the requirements of Appendices
At C, D, and B, to this part
C. Provide for the operation of.
D. Provide for the review of the air quality aurvell-
lanoe system on an annual basis to determine IT the
ayBtea ceets the monltorlng objectives defined In
Appendix D to this part. Such review must •
SB. 23
Monitoring Network Completion
tn
i
us
•By January 1, 19831
A. Each station in the SLAMS network must be in operation,
be aited in accordance with the criteria In Appendix B
to thla part, and be located aa described on the
station's SAROAD alte Identification form, and
B. The quality assurance requirements of Appendix A to this
part must be fully implemented
58.31
NANS Network Completion
Appendix A
Section 2.4
National Performance and
System Audit Program
•By January 1, 1981i
A. Each NAKS must be In operation......
B. The quality aaauranca requirements of Appendix A
to thla part must be fully implemented for all HAMS
•Agenolea operating all or a portion of • SLAMS network are
required to participate In EPA'a national performance audit
program and to permit an annual EPA ayatern audit of their
ambient air monitoring program....•for additional information
about these programs. Agencies should contact either the
Ee EPA
appropriate EPA Regional Quality Control Coordinator or the
Quality Assurance Branch, EHSL/RTP, for instructions for
participation.*
M O /a CO
p CD n>
M \->
\O M
00
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 9 of 142
be found in Appendix A to 40 CFR Part 58. Section 2.2 of Appendix
A details the operations for which an agency must have written
procedures. The exact format and organization of such procedures
is not indicated however. Thus many approaches to appropriate
documentation have been, suggested by EPA, local agencies and other
groups.
One approach adopted by many EPA Regional Offices was the
organization of the required material into the framework
recommended by the EPA Quality Assurance Management Staff (QAMS
005/80, December 1980) in the document titled "Interim Guidelines
for the Preparation of Quality Assurance Project Plans". The
sixteen (16) elements described in the guideline document provide
the framework for organizing the required Air Program operational
procedures, integrating quality assurance activities and
documenting overall program operation. This approach is
consistent with the required eleven items of 40 CFR Part 58,
Appendix A. Table 11-3 illustrates this consistency and
demonstrates how each required program element will be evaluated
in the context of the program areas used in the organization of
the long-form questionnaire.
11.3 Guidelines for Preliminary Assessment and System Audit
Planning
By performing a system audit of a given agency, the Regional
QA Coordinator is seeking a complete and accurate picture of that
agency's current ambient air monitoring operations. Past
experience has shown that four (4) person-days should be allowed
for an agency operating 10-20 sites within close geographical
proximity. The exact number of people, and the time alloted to
conduct the audit, is dependent on the magnitude and complexity of
the agency and the EPA Regional Office resources. During the
alloted time frame, the Regional QA Audit Team should perform
those inspections and interviews recommended in Section 11.4.
This includes on-site interviews with key program personnel,
5-10
-------
01
i
TABLE 11-3. SPECIFIC REGULATORY REQUIREMENTS TO BE EVALUATED IN A SYSTEMS AUDIT
REGULATIONS
(40 CFR 58, Appendix A)
(1) Selection of Methods and Analyzers
(1) Selection of Methods, Analyzers
and Samplers
(11) Docunentation of Quality Control
Information
(2) Installation of Equipment
(3) Calibration
(7) Calibration and Zero/Span Checks
for Multiple Range Analyzers
Only applicable if other than
automated analyzers are used and
analyses are being performed on
filters - e.g., NOT or lead and
TSP
(10) Recording and Validating Data
GUIDELINES
(QAMS Document 005/80)
Project Description
Organization & Responsibility
QA Objectives
Sampling Procedures
Sample Custody
Calibration Procedures and
Frequency
Analytical Procedures
Data Reduction, Validation
and Reporting
CURRENT ORGANIZATION
OF QUESTIONNAIRE (11.7)
Planning
Planning
Field Operations
Field/Lab Operations
Field/Lab Operations
Lab Operations
Data Management
>0 O ?0 M
D> 0MD (D
ifl ft < 0
(l> (D M- ft
tfl H-
M O H-0
O O O 3
ft 3
MI D* 25 0
4^ to
10 H H .
O
\o
CD
-------
tn
ro
TABLE 11-3. SPECIFIC REGULATORY REQUIREMENTS TO BE EVALUATED IN A SYSTEMS AUDIT (cont'd)
REGULATIONS
(40 CFR 58, Appendix A)
(4) Zero/span checks and adjustments of
automated analyzers.
(5) Control Checks and their frequency
(6) Control Limits for Zero/Span
(7) Calibration and Zero/Span for
Multiple Range Analyzers
(9) Quality control checks for air
pollution episode monitoring
Appendix A - Sections 2.0, 3.0 and
4.0.
(8) Preventive and Remedial Maintenance.
Appendix A - Section 4.0.
(10) Recording and Validating Data
(4) Zero/Span checks and adjectments of
automated analyzers.
(6) Control Limits and Corrective Actions.
(11) Documentation of Quality Control
Information.
(10) Data Recording and Validation.
GUIDELINEk
(QAMS Document 005/80)
Internal Quality-Control Checks
Performance and Systems Audits
Preventive Maintenance
Specific Routine Procedures
used to Assess Data Precision,
Accuracy-and Completeness.
Corrective Action
Quality Assurance Reports to
Management
CURRENT ORGANIZATION
OF QUESTIONNAIRE (11.7)
Field/Lab Operations
QA/QC
QA/QC
Field/Lab Operations
QA/QC
Data Management
Field/Lab Operations
Reporting
*O D JO CO
0) 0» (D (D
iQ ft < O
rt> tt» H- ft-
0) H-
H O H. O
M O O 3
O O 55
H» tr ss o
ro o •
ro
to
«
O
VO M
03
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 12 of 142
evaluations of some ambient air monitoring sites operated by the
agency, and the development of data audit trails.
11.3.1 Frequency of Audits - The EPA Regional Office retains the
regulatory responsibility to evaluate agency performance annually.
Regional Offices are urged to use the short form questionnaire
(Section 11.6), the CAIR (Fig. 11-4), and the audit reporting
format (Section 11.4.4.). Utilizing the above to provide OAQPS
with this audit information will establish a uniform basis for
audit reporting throughout the country. For many well-established
agencies, an extensive system audit and rigorous inspection may
not be necessary every year. The determination of the extent of
the system audit and its rigor is left completely to EPA Regional
Office discretion. Therefore, the option is provided here that
extensive inspections and evaluations may be accomplished using
the short-form questionnaire (Section 11.6), and appropriate
section(s) of the long form questionnaire (Section 11.7). It is
suggested that a complete system audit using the long form
questionnaire be performed at least once every three years.
Yearly reports must still, however, include the short form, CAIR,
and the report completed according to Section 11.4.4.
The primary screening tools to aid the EPA Regional QA Audit
Team in determining which type of audit to conduct and its
required extent are:
(a) National Performance Audit Program (NPAP) Data—which
provide detailed information on the ability of participants
to certify transfer standards and/or calibrate monitoring
instrumentation. Audit data summaries provide a relative
performance ranking for each participating agency when
compared to the other participants for a particular
pollutant. These data could be used as a preliminary
assessment of laboratory operations at the different local
agencies.
(b) Precision and Accuracy Reporting System (PARS) Data—which
5-13
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 13 of 142
provide detailed information on precision and accuracy
checks for each local agency and each pollutant, on a
quarterly basis. These data summaries could be used to
identify out-of-control conditions at different local
agencies, for certain pollutants.
(c) National Aerometric Data Bank (NADB) NA285 Data Summaries—
which provide a numerical count of monitors meeting and
those not meeting information on monitoring data completeness
criteria on a quarterly basis, together with an associated
summary of precision and accuracy probability limits. An
additional program NA 288 will provide data summaries
indicating -the percent of data by site and or by state for
each pollutant.
11.3.2 Selection of.Monitoring Sites for Evaluation - It is
suggested that approximately five percent (5%) of the sites of
each local agency included in the Reporting Organization be
inspected during a system audit. For smaller local agencies, no
fewer than two (2) sites should be inspected. To insure that the
selected sites represent a fair cross-section of agency
operations, one half of the sites to be evaluated should be
selected by the agency itself, while the other half should be
selected by the Regional QA Audit Team.
The audit team should use both the Precision and Accuracy
Reporting System (PARS) and the SAROAD computer databases in
deciding on specific sites to be evaluated. High flexibility
exists in the outputs obtainable from the NADB NA285 computer
program; data completeness can be assessed by pollutant, site,
agency, time period and season. These data summaries would assist
the Regional audit team in spotting potentially persistent
operational problems in need of more complete on-site evaluation.
It is strongly recommended that data completeness, as defined by
SAROAD, be the overriding criteria in the selection of sites to be
evaluated.
5-14
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 14 of 142
If the Reporting Organization under audit operates many sites
and/or its structure is complicated and perhaps inhomogeneous,
then an additional number of sites above the initial 5% level
should be inspected so that a fair and accurate picture of the
state and local agency's ability to conduct field monitoring
activities can be obtained. At the completion of the site
evaluations, the Regional audit team is expected to have
established the adequacy of the operating procedures, the flow of
data from the sites and to be able to provide support to
conclusions about the homogeneity of the Reporting Organization.
11.3.3 Data Audits - With the implementation by many agencies of
automated data acquisition systems, the-data management function
has, for the most part, become increasingly complex. Therefore, a
complete system audit must include the development and
documentation of a data audit trail, which starts at the
acquisition stage and terminates at the point of entry into the
SAROAD computer system. The process of establishing the data
audit trail will be dependent on the size and organizational
characteristics of the Reporting Organization, the volume of data
processed, and the data acquisition system's characteristics. The
details of performing a data processing audit are left therefore
to Regional and Reporting Organization personnel working together
to establish a data audit trail appropriate for a given agency.
Besides establishing and documenting trails, data audits must
involve a certain amount of manual recomputation of raw data. The
preliminary guidance provided here, for the amount of data to be
manually recalculated, should be considered a minimum enabling
only the detection of gross mishandling of data:
(a) For continuous monitoring of criteria pollutants, the
Regional QA Coordinator should choose two 24-hour
periods corresponding to the peak and low seasons for
that particular pollutant per local agency per year.
(In most cases the seasons of choice will be Winter and
Summer).
(b) For manual monitoring, four 24-hour periods per local
agency per year should be recomputed.
5-15
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 15 of 142
The Regional QA Coordinator.should choose the periods for the
data audit while planning the system audit and inspecting the
completeness records provided by the NADB NA285 system. The
recommended acceptance criteria for the differences between the
data input into SAROAD and that recalculated during the on-site
phase of the system audit, is given in Table 11-4.
TABLE 11-4. ACCEPTANCE CRITERIA FOR DATA AUDITS
Data Acquisition
Mode
Pollutants
Measurement, . Tolerance
Range (ppm)*a' Limits
Automatic Data
Retrieval
Stripchart
Records
, O,, NO
SO,, 03, NO,
CO^ J *
0-0.5, or 0-1.0 +3
•0-20, or 0-50 Hh0.3~ppm
0-0.5, or 0-1.0 +20 ppb
0-20, or 0-50 Tl ppm
Manual
Reduction
TSP
Pb
+2 g/m° ^Dl
+0.1 g/m
(a) Appropriate scaling should be used for higher
measurement ranges.
(b) Specified at 760 mm Hg and 25°C.
System audits conducted on large Reporting Organizations
(e.g. four local agencies) require recomputation of eight 24-hour
periods for each of the criteria pollutants monitored
continuously. This results from two 24-hour periods being
recomputed for each local agency, for each pollutant monitored,
during a given year. For manual methods, sixteen 24-hour periods
are recomputed, consisting of four 24-hour periods per local
agency, per year.
11.4 Protocol for Conducting System Audits of State and Local
Agencies
A system audit should consist of three separate phases:
o Pre-Audit Activities
5-16
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984-
Page 16 of 142
o On-Site Audit Activities
o Post-Audit Activities
Summary activity flow diagrams have been included as Figures
11-1, 11-2 and 11-3, respectively. The reader may find it useful
to refer to these diagrams while reading this protocol.
11.4.1 Pre-Audit Activities - At the beginning of each fiscal
year, the Regional QA Coordinator or a designated member of the
Regional QA Audit Team, should establish a tentative schedule for
on-site system audits of the agencies within their region.
Six (6) weeks prior to the audit, the Regional QA Coordinator
should contact the Quality Assurance Officer (QAO) of the
Reporting Organization to be audited to coordinate specific dates
and schedules for the on-site audit visit. During this initial
contact, the Regional QA Coordinator should arrange a tentative
schedule for meetings with key personnel as well as for inspection
of selected ambient air quality monitoring and measurement
operations. At the same time, a schedule should be set for the
exit interview used to debrief the agency Director or his designee,
on the system audit outcome. As part of this scheduling, the
Regional QA Coordinator should indicate any special require-
ments such as access to specific areas or activities. The
Regional QA Coordinator should inform the agency QAO that he will
receive a questionnaire, precision and accuracy data, and complete-
ness data from NADB programs hA 273 and MA 288 which is to be
reviewed or completed. He should emphasize that the completed
questionnaire is to be returned to the EPA Region within one
(1) month of receipt. The additional information called for
within the questionnaire is considered as a minimum, and both
the Region and the agency under audit should feel free to include
additional information. The Regional Audit Team may use this initial
contact or subsequent conversations to obtain appropriate travel
information, pertinent data on monitoring sites to be visited, and
assistance in coordinating meeting times.
5-17
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 17 of 142
DEVELOP AUDIT SCHEDULE
«
t
CONTACT REPORTING ORGANIZATIONS
TO SET TENTATIVE DATES
REVISE SCHEDULE AS NECESSARY
CONTACT REPORTING ORGANIZATION TO
DISCUSS AUDIT PROCEDURE
INITIATE TRAVEL PLANS
FIRM DATES FOR ON-SITE VISITS
w
N
/
i SEND QUESTIONNAIRE AND REQUEST
! PRELIMINARY SUPPORT MATERIAL.
REVIEW MATERIAL DISCUSS WITH
REPORTING ORGANIZATION QA OFFICER
FINALIZE TRAVEL PLANS WITH INFORMATION
PROVIDED BY REPORTING ORGANIZATION
DEVELOP CHECKLIST OF POINTS
FOR DISCUSSION
CONTACT AGENCY TO SET SPECIFIC INTERVIEW
AND SITE INSPECTION TIMES
TRAVEL ON-SITE
Figure 11-1. PRE-AUDIT ACTIVITIES
5-18
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 18 of 142
AUDIT TEAM INITIAL INTERVIEW OF
REPOBIISG ORCANIZAnOH DIRECTOR
IBTEH7TIM VTTH K£X PERSONNEL
AUDIT CROC? i
ADPTT C80UP 2
I3TERVIEW PLANNING KANAGZR
unzavrzv moa OPEBATIOMS MANAGER
| IHTERVISW UBORA70BT D1HECIO*
V
f
VISIT SIRS (AGENCY SELECTED)
VISIT LABORATORT
WITNESS OPERATIONS
VISIT SITES (RECIOH SELECTED)
REVIEW SAMPLE RECSI7IHC AID CCSTCOT
^MM^wJ
f
VISIT AUDIT AND CALIBRATION FACILITT
! SELECT PORTION OP DATA XBITIAtZ ABDIT TXAZL
1
SELECT posnaa OF DATA IBITIATE ADD IT
'\ ESTABLISH DATA AUDIT TRAIL THROCGH LABOIATORT
! OPERATIONS TO DATA MAHACZMEST
MEET TO
BISCTSS. FTJTOIHCS
L
ESTABLISH TBAIL THROUGH FIELD OPERATIONS
TO DATA MAHACEMEHT
QA OFFICER
QITERVIEW DATA HAMACEMEHT PERSONMEL
nHALIZE ADDn TSATLS AND COMPLETE
DATA ADPIT
PREPARE Atrorr BESDLTS SUMURT OF
(*) ortraLl bpvraclotu
(t) d*ca uidlc flndlnw
(c) laboratory op«racion«
(d) ft«Id ootraclonj
OtlTIATZ SZQUZSTS FOR CORRECTIVE
ACTTOH DffLZMEHTATIOB REOOESTS (CAIRi
DISCUSS remises WITH KET PERSONNEL 1
S OA OFFICER !
.
/
EXIT nrrsavisu VTTH REPOKTBTC ORCAHIZATIOH
DTRECTOH TO OBtAIS SICSATPRES OK CAIR
Figure 11-2.
ON-SITE ACTIVITIES
OS-SITE AUDIT COMPLETE
5-19
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 19 of 142
TRAVEL BACK TO REGIONAL HEADQUARTERS
AUDIT TEAM WORKS TOGETHER TO PREPARE
REPORT
INTERNAL REVIEW AT REGIONAL HEADQUARTERS
INCORPORATE COMMENTS AND REVISE DOCUMENTS
ISSUE COPIES TO REPORTING ORGANIZATION DIRECTOE
FOR DISTRIBUTION AND WRITTEN COMMENT
INCORPORATE WRITTEN COMMENTS RECEIVED
FROM REPORTING ORGANIZATION
SUBMIT FINAL DRAFT REPORT FOR
INTERNAL REGIONAL REVIEW
\/
REVISE REPORT AND INCORPORATE COMMENTS
AS NECESSARY
PREPARE FINAL COPIES
DISTRIBUTE TO REPORTING ORGANIZATION
DIRECTOR, OAQPS AND REGION
Figure 11-3. POST-AUDIT ACTIVITIES
5-20
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 20 of 142
Once the completed questionnaire has been received, it should
be reviewed and compared with the criteria and information
discussed in Section 11.2 and those documents and regulations
included by reference in Section 11.5. The Regional QA Audit Team
should also use the PARS and MADB NA273 and NA288 to augment the
documentation received from the Reporting Organization under audit.
This preliminary evaluation will be instrumental in selecting
the sites to be evaluated and in the decision on the extent of the
monitoring site data audit. The Regional Audit Team should then
prepare a checklist detailing specific points for discussion .with
agency personnel.
The Regional Audit Team could be made up of several members
to offer a wide variety of backgrounds and expertise. This team
may then divide into groups once on-site, so that both audit
coverage and time utilization can be optimized. A possible
division may be that one group assesses the support laboratory and
headquarters operations while another evaluates sites and
subsequently assesses audit and calibration information. The team
leader should reconfirm the proposed audit schedule with the
Reporting Organization immediately prior to travelling to the
site.
11.4.2 On-Site Activities - The Regional QA Audit Team should
meet initially with the agency's Director or his designee to
discuss the scope, duration, and activities involved with the
audit. This should be followed by a meeting with key personnel
identified from the completed questionnaire, or indicated by the
agency QAO. Key personnel to be interviewed during the audit are
those individuals with responsibilities for: Planning, Field
Operations, Laboratory Operations, QA/QC, Data Management, and
Reporting. At the conclusion of these introductory meetings, the
Regional Audit Team may begin work as two or more independent
5-21
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 21 of 142
groups.
For the audit group which remains at the agency headquarters,
it is suggested that the first interview be with the individual in
charge of the agency's planning function. This will enable an
assessment of network design, network documentation, and yearly
network review. After establishing the current and planned
activities for the air monitoring network, this group may begin to
assess laboratory operations.
Laboratory operations should be reviewed as support to the
agency's ambient air monitoring activities and as a laboratory
capable of additional, specialized analyses. It is during this
phase of the system audit that the audit group should start to
develop the data audit trail. The selection of this data is based
on guidance provided in Section 11.3.3.
It is envisioned that, if the initial meetings with the
Director and planning personnel were handled in the morning, then
the interviews with laboratory personnel could be started the same
afternoon. Following the initial interviews, the audit group
should review sample receiving and handling and start establishing
a data audit trail up to the point where data leaves the
laboratory for the Data Management function.
During the same period of time a similar data audit trail
could be established by the second audit group. The second audit
group should also thoroughly interview the Field Operations
Manager concerning the specifics of network operations,
maintenance, calibrations, internal quality control, and
documentation. This audit group should also select a portion of
data to start establishing a data audit trail. The group should
then proceed with the data audit trails, while conducting the site
inspections. In order to increase the uniformity of site
inspections where many individual members of a Regional Audit Team
may simultaneously have this responsibility, it is suggested that
a site checklist be developed and used.
5-22
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 22 of 142
At this point both audit teams, having established data and
operational integrity for their respective areas, should meet to
discuss their, findings. During this meeting the responsibility
for completion of the data audit trail through the rest of the
data management operations should be given to one group, while the
other group assesses the agency's quality assurance program, i.e.,
Quality Assurance Plan implementation and ancillary quality
control information. Once the QAO and Data Manager have been
interviewed, the Regional Audit team resumes working together to
finalize the recomputations necessary as part of the data
a udi t.
The importance of the data audit cannot be overstated. Thus,
sufficient time and effort should be devoted to this activity so
that the audit team has a clear understanding and complete
documentation of data flow. Its importance stems from the need to
have documentation on the quality of ambient air monitoring data
for all the criteria pollutants for which the agency has
monitoring requirements. The data audit trail developed will
serve as an effective framework for organizing the extensive amount
of information gathered during the audit of laboratpry, field
monitoring and support functions within the agency.
The entire audit team should now prepare a brief written
summary of findings organized into the following areas: Planning,
Field Operations, Laboratory Operations, Quality Assurance/Quality
Control, Data Management, and Reporting. Problems with specific
areas should be discussed and an attempt made to rank them in
order of their potential impact on data quality. For the more
serious of these problems, Corrective Action Implementation
Request (CAIR) forms should be initiated. An example form is
provided in Figure 11-4. The forms have been designed such that
one is filled out for each major deficiency noted that requires
formal corrective action.
5-23
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 23 of 142
CORRECTIVE ACTION IMPLEMENTATION REQUEST (CAIR)
Reporting Organization
State or Local Agency
Deficiency Noted:
Agreed-upon Corrective Action:
Schedule for Corrective Action Implementation;
Signed
Director
QA Officer
Audit Team Member
Date
Date
Date
Corrective Action Implementation Report:
Signed
Signed
Director
QA Officer
Date
Date
Figure 11-4. Example of a CAIR Form
5-24
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 24 of 142
The format, content, and intended use of CAIRs is fully
discussed in Section 11.4.5. Briefly, they are request forms for
specific corrective actions. They are initiated by the Regional
QA audit team and signed off upon mutual agreement by the agency's
Director or his designee during the exit interview.
The audit is now completed by having the Regional Audit Team
members meet once again with key personnel, the QAO and finally
with the agency's Director or his designee to present their
findings. The audit team should simply state the audit results
including an indication of the potential data quality impact.
During these meetings the audit team should also discuss the
system audit reporting schedule and notify agency personnel that
they will be given a chance to comment in writing, within a
certain time period, on the prepared audit report in advance of
any formal distribution.
11.4.3 Post-Audit Activities - The major post-audit activity is
the preparation of the System Audit Report. The report format is
presented in Section 11.4.4.
To prepare the report, the audit team should meet and compare
observations with collected documents and results of interviews
and discussions with key personnel. Expected QA Project Plan
implementation is compared with observed accomplishments and
deficiencies and the audit findings are reviewed in detail.
Within thirty (30) calendar days of the completion of the field
work, the audit report should be prepared and submitted.
The System Audit Report is submitted to the audited agency
together with a letter thanking agency personnel for their
assistance, time and cooperation. It is suggested that the body
of the letter be used to reiterate the fact that the audit report
is being provided for review and written comment. The letter
should also indicate that, should no written comments be received
by the Regional QA Coordinator within thirty (30) calendar days
from the report date, it will be assumed acceptable to the
5-25
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 25 of 142
agency in its current form, and will be formally distributed
without further changes.
If the agency has written comments or questions concerning
the audit report, the Regional Audit Team should review and
incorporate them as appropriate, and subsequently prepare and
resubmit a report in final form within thirty (30) days of receipt
of the written comment. Copies of this report should be sent to
the agency Director or his designee for his internal distribu-
tion. Also, the appropriate number of copies should be sent
to the Office of Air Quality Planning and Standard (OAQPS) for
internal EPA distribution. The transmittal letter for the amended
report should indicate official distribution and again draw
attention to the agreed-upon schedule for Corrective Action
Implementation.
11.4.4 Audit Reporting - The System Audit Report format discussed
in this section has been prepared to be consistent with guidance
offered by the STAPPA/ALAPCO Ad Hoc Air Monitoring Audit
Committee. The format is considered as acceptable for annual
system audit reports submitted to the OAQPS. Regional audit team
members shall use this framework as a starting point and include
additional material, comments, and information provided by the
agency during the audit to present an accurate and complete
picture of its operations and performance evaluation.
At a minimum the system audit report should include the
following six sections:
Executive Summary — summarizes the overall performance of
the agency's monitoring program. It should highlight problem
areas needing additional attention and should describe any
significant conclusions and/or broad recommendations.
Introduction -- describes the purpose and scope of the audit
and identifies both the Regional Audit Team members, key agency
personnel, and other section or area leaders who were interviewed.
It should also indicate the agency's facilities and monitoring
5-26
-------
Section No. 2.0.11
Revision No. 1
Date October I, 1984
Page 26 of 142
sites which were visited and inspected, together with the dates
and times of the on-site audit visit. Acknowledgement of the
cooperation and assistance of the Director and the QAO should also
be considered for inclusion.
Audit Results — presents sufficient technical detail to
allow a complete understanding of the agency operations. The
information obtained during the audit should be organized using
the recommended subjects and the specific instructions given
below. It will be noted that the report format follows the
four-area organization of the short-form questionnaire.
A. Network Design and Siting
1) Ne twork Size Provide an overview of the network
size and the number of local agencies responsible to the
state for network operation.
2) Ne twork Design and Siting Describe any deficiencies
in network design or probe siting discovered during the
audit. Indicate what corrective actions are planned to
deal with these deficiencies.
3) Ne twork Review Briefly discuss the conclusions of
the last network annual review and outline any planned
network revision resulting from that review.
4) Non-criteria Pollutants Briefly discuss the agency's
monitoring and quality assurance activities related to
non-criteria pollutants.
B. Resources and Facilities
1) Instruments and Methods Describe any instrument
non-conformance with the requirements of 40 CFR 50, 51,
53, and 58. Briefly summarize agency needs for instrument
replacement over and above non-conforming instruments.
2) Staff and Facilities Comment on staff training,
adequacy of facilities and availability of NBS-traceable
standard materials and equipment necessary for the agency
to properly conduct the bi-weekly precision checks and
quarterly accuracy audits required under 40 CFR Part 58,
Appendix A.
3) Laboratory Facilities Discuss any deficiencies of
laboratory procedures, staffing and facilities to conduct
the tests and analyses needed to implement the SLAMS/NAMS
monitoring and Quality Assurance plans.
5-27
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 27 of 142
C. Data and Data Management
1) Data Processing and Submittal Comment on the adequacy
of the agency's staff and facilities to process and submit
to SAROAD air quality data as specified in 40 CFR 58.35
and the Reporting requirements of 40 CFR 58, Appendix F.
Include an indication of the timeliness of data submission
by indicating the fraction of data which are submitted
more than forty-five (45) days late.
2) Data Review A brief discussion of the agency's
performance in meeting the 75% criteria for data
completeness. Additionally, discuss any remedial actions
necessary to improve data reporting.
3) Data Correction Discuss the adequacy and documentation
of corrections and/or deletions made to preliminary ambient
air data, and their consistency with both the agency's QA
Manual and Standard Operating Procedures, and any revised
protocols.
4) Annual Report Comment on the completeness, adequacy
and timeliness of submission of the SLAMS Annual Report
which is required under 40 CFR 58.26.
D. Quality Assurance/Quality Control
1) Status of Quality Assurance Program Plan Discuss
the status of the Agency's Quality Assurance Plan. Include
an indication of its approval status, the approval status
of recent changes and a general discussion of the
consistency, determined during the system audit, between
the Agency Standard Operating Procedures and the Quality
Assurance Plan.
2) Audit Participation Indicate frequency of participation
in an audit program. Include as necessary, the agency's
participation in the National Performance Audit Program
(NPAP) as required by 40 CFR Part 58. Comment on audit
results and any corrective actions taken.
3) Accuracy and Precision As a goal, the 95 percent
probability limits for precision (all pollutants) and TSP
accuracy should be less than +15 percent. At 95 percent
probability limits, the accuracy for all other pollutants
should be less than +20 percent. Using a short narrative
and a summary table, compare the Reporting Organization's
performance against these goals over the last two years.
Explain any deviations.
Discussion — includes a narrative of the way in which the
audit results above, are being interpreted. It should clearly
identify the derivation of audit results which affect both data
5-28
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 28 of 142
quality and overall agency operations, and should outline the
basis in regulations and guideline documents for the specific,
mutually-agreed upon, corrective action recommendations.
Conclusions and Recommendations — should center around the
overall performance of the agency1s monitoring program. Major
problem areas should be highlighted. The salient facts of
mutually agreed upon corrective action agreements should be
included in this section. An equally important aspect to be
considered in the conclusion is a determination of the homogeneity
of the agency's Reporting Organizations and the appropriateness of
pooling the Precision and Accuracy data within the reporting organ-
izations. The checklist in Figure LI-5 should be included and sub-
mitted with the supporting documentation.
Appendix of Supporting Documentation— contains a clean and
legible copy of the completed short-form questionnaire and any
Corrective Action Implementation Request Forms (CAIR). Additional
documentation may be included if it contributes significantly to a
clearer understanding of audit results.
11.4.5 Follow-up jnd Corrective Action Requirements - An effective
corrective action procedure for use by the Regional QA Audit Team
follows. As a means of requesting corrective actions identified
during the on-site audit, the auditor completes one copy of
the form, shown in Figure 11-4, for each major deficiency
noted. These CAIR forms are presented to, and discussed with,
the agency's Director, or his designee, and its QAO during
the exit interview. Once agreement has been reached both the
auditor and the Director sign off the form. The original is
given to the agency Director or his designee and a copy is
retained by the auditor. A photocopy of the completed
CAIR is included in the audit report. It is taken to be the
responsibility of the agency to comply with agreed-upon corrective
action requests in the specified time frame.
5-29
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 29 of 142
REPORTING ORGANIZATION HOMOGENEITY CHECKLIST
Yes No
1. Field operations, for all local agencies,
conducted by a common team of field
operators?
2. Common calibration facilities are used
for all local agencies?
3. Precision checks performed by common
staff for all local agencies?
4. Accuracy checks performed by common
staff for all local agencies?
5. Data handling follows uniform procedures
for all local agencies?
6. Central data processing facilities used
for all reporting?
7. Traceability of all standards established
by one central support laboratory?
8. One central analytical laboratory handles
all analyses for manual methods?
Figure 11-5. Example of Reporting Organization
Homogeneity Checklist
5-30
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 30 of 142 •
11.5 Criteria for the Evaluation of State and Local Agency
Performance
This section is designed to assist the Regional Audit Team in
interpretation of the completed questionnaire received from the
agency prior to the on-site interviews. It also provides the
necessary guidance for topics to be further developed during the
on-site interviews.
This section is organized such that the specific topics to be
covered and the appropriate technical guidance are keyed to the
major subject areas of the long form questionnaire (Section 11.7).
The left-hand side of the page itemizes the discussion topics and
the right-hand side provides citations to specific regulations and
guideline documents which establish the technical background
necessary for the evaluation of agency performance. A more
complete bibliography of EPA guideline documents is presented in
Section 11.8.
5-31
-------
11.5.1. Planning -
Topics for Discussion
o General information on
reporting organization and
status of Air Program, QA
Plan and availability of SOPs
o Conformance of network design
with regulation, and
completeness of network
documentation
o Organization staffing and
adequacy of educational
background and training of key
personnel
o Adequacy of current facilities
and proposed modifications
11.5.2. Field Operations -
Topics for Discussion
o Routine operational practices
for SLAMS network, and
conformance with regulations
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 31 of 142
Background Documents
o State Implementation Plan
O U.S. EPA QAMS 005/80
o Previous System Audit
report
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods,
Section 2.0.1.
o 40 CFR 58 Appendices D
and E
o OAQPS Siting Documents
(available by pollutant)
o QA Handbook for Air
Pollution Measurement
Systems, Vol. I -
Principles, Section 1.4
Vol. II - Ambient Air
Specific Methods,Section
2.0.5
o Types of analyzers and samplers
used for SLAMS network
Background Documents
o 40 CFR 58 Appendix C -
Requirements for SLAMS
analyzers
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods,
Section 2.0.2
o 40 CFR 50 plus appendices
A through G (potentially
K for PM10)
o 40 CFR 53 Reference and
Equivalency of analyzers
5-32
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 32 of 142
o Adequacy of field procedures,
standards used and field
documentation employed for
SLAMS network
o Frequency of zero/span checks,
calibrations and credibility
of calibration equipment used
o Traceability of monitoring and
calibration standards
o Preventive maintenance system
including spare parts, tools
and service contracts for major
equipment
o Record keeping to include
inspection of some site log
books and chain-of-custody
procedures
o Data acquisition and handling
system establishing a data
audit trail from the site to
the central data processing
facility
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.2 - TSP
Section 2 .'3 - NO2
Chemiluminescence
Section 2.5 - SO, FPD
Section 2.6 - CO NDIR
Section 2.7 - O,
Chemiluminescence
Section 2.9 - SO2
Fluorescence
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.9
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.7
o 40 CFR 58 Appendix A
Section 2.3
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.6
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.3
5-33
-------
11.5.3. Laboratory Operations -
Topics for Discussion
o Routine operational practices
for manual methods used in
SLAMS network to include
quality of chemical and
storage times.
o List of analytical methods
used for criteria pollutants
and adherence to reference
method protocols
o Additional analyses performed
to satisfy regional, state
or local requirements
o Laboratory quality control
including the regular usage
of duplicates, blanks, spikes
and multi-point calibrations
o Participation in EPA NPAP and
method for inclusion of audit
materials in analytical run
o Documentation and traceability
of laboratory measurements
such as weighing, humidity and
temperature determinations
o Preventive maintenance in the
laboratory to include service
contracts on major pieces of
instrumentation
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 33 of 142
Background Documents
o 40 CFR 50 Appendices A
through G
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.1 - SO,
Section 2.2 - TSP
Section 2.4 - NO,
Section 2.8 - Pb^
o Refer to locally available
protocols for analysis of
aldehydes, sulfate,
nitrate, pollens, hydro-
carbons, or other toxic
air contaminants
O U.S. EPA APTD-1132
"Quality Control Practices
in Processing Air
Pollution Samples"
o 40 CFR 58 Appendix A
Section 2.4
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.10
5-34
-------
Section No. 2.0.11
Revision No. 1
Date October I, 1984
Page 34 of 142
o Laboratory record keeping and
chain-of-custody procedures
to include inspection of
logbooks used
o Adequacy of Laboratory
facilities. Health and Safety
practices and disposal of
wastes
o Data acquisition, handling
and manipulation system
establishing data flow in
the laboratory, data back-up
system and data reduction
steps.
o Data validation procedures,
establishing an audit trail
for the laboratory to the
central data processing
facility
11.5.4. Data Management -
Topics of Discussion
o Data flow from field and
laboratory activities to
central data processing
facility
o Extent of computerization of
data management system and
verification of media changes,
transcriptions and manual
data entry
o Software used for processing
and its documentation; to
include functional description
of software, test cases and
configuration control for
subsequent revisions
o System back-up and recovery
capabilities
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.6
o Handbook for Analytical
Quality Control in Water
and Wastewater
Laboratories
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.3
o Annual Book of ASTM
Standards, Part 41, 1978.
Standard Recommended
Practice for Dealing with
Outlying Observations
(E 178-75)
Background Documents
5-35
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 35 of 142
o Data screening, flagging and
validation
o Data correction procedures
and key personnel allowed to
correct ambient air data
o Reports generated for in-house
distribution and for submittal
to EPA
o Responsibility for preparing
data for entry into the SAROAD
and PARS systems and for
responsibility for its final
validation prior to submission
11.5.5 QA/QC Program -
Topics for Discussion
o Status of QA Program
o Documentation of audit
procedures, integrity of
audit devices and acceptance
criteria for audit results
o Participation in the National
Performance Audit Program
For what pollutants and
ranking of results
o Additional internal audits
such as document reviews or
data processing audits
o Procedure and implementation
of corrective action
o Validation of Air
Monitoring Data, EPA-
600/4-80-030
o Screening Procedures for
Ambient Air Quality Data,
EPA-450/2-78-037
o Aeros Manual Series,
Vol. II, Aeros User's
Manual, EPA-450/2-76-029
Background Documents
o 40 CFR 58 Appendix A
and QAMS 005/80
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.12
o 40 CFR 58 Appendix A
5-36
-------
o Frequency of performance and
concentration levels for
precision checks for each
criteria pollutant
11.5.6. Reporting -
Topics for Discussion
o Preparation of precision and
accuracy summaries for the
PARS system
o Other internal reports used
to track performance and
corrective action
implementation
o Summary air data reports
required by regulations
o Completeness legibility and
validity of P & A data on
Form 1
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 36 of 142
o 40 CFR 58 Appendix A
Background Documents
o PARS User's Guide
(in preparation)
o 40 CFR 58 Appendices
F and 6
o 40 CFR 58 Appendix A
11.6 System Audit Questionnaire (Short-Form)
This short-form questionnaire has been designed specifically
for use annually in reviewing state and local agencies air
monitoring programs. If the Regional QA Coordinator decides that
a more rigorous system audit and site inspections are necessary,
he can utilize appropriate section(s) of the Long Form
Questionnaire (Section 11.7). This questionnaire has been
designed around the format recommended by STAPPA/ALAPCO in the
National Ambient Air Monitoring Questionnaire and is organized
around four (4) major topics consistent with the reporting format
outlined in Section 11.4.4. They are:
A. Network Design and Siting
B. Resources and Facilities
C. Data Management, and
D. Quality Assurance and Quality Control
5-37
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 37 of 142
NATIONAL AIR MONITORING AUDIT
QUESTIONNAIRE
Agency
Address
Telephone Number (Area Code) Number
Reporting Period (beginning-ending dates)
Organization Director
Air Program Supervisor
Data Management Supervisor
Quality Assurance Officer
Questionnaire Completed
(date) (by)
On-Site Visit
Date: Audit Team Members:
Affiliation of Audit Team:
5-38
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 38 of 142
A. NETWORK DESIGN & SITING
1. NETWORK SIZE
(a) Complete the table below for each of the criteria pollutants
monitored as part of your air monitoring network. Include only
those sites that are presently operating. Do not include
additional monitors which are collocated or index sites.
Number of Monitors
NC CO 03 SP Pb
NAMS
SLAMS
(excluding NAMS)
SPM
TOTAL
(b) SLAMS Network Description
1. What is the date of the most current official SLAMS Network
Description?
2. Where is it available for public inspection?
5-39
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 39 of 142
3. Does it include for each site the following?
YES NO
SAROAD Site ID#
Location
Sampling and Analysis Method
Operative Schedule
Monitoring Objective and Scale
of Representativeness
Any Proposed Changes
(c) For each of the criteria pollutants, how many modifications
(SLAMS including NAMS) have been made since the last system
audit? (List the total SLAMS and NAMS) Date of Last audit _
Number ^f Monitors
Pollutant Added Deleted Relocated
Sulfur Dioxide
Nitrogen Dioxide
Carbon Monoxide
Ozone
Total Suspended
Particulates
Lead
(d) Briefly discuss changes to the Air Monitoring Network planned
for the next audit period.
5-40
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 40 of 142
2. NETWORK DESIGN AND SITING
Indicate by SAROAD Number any non-conformance with the
requirements of 40 CFR 58, Appendices D and E.
Site ID
Monitor (SAROAD) Reason for Non-Conformance
S02
03
CO
N02
TSP
Pb
5-41
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 41 of 142
3. NETWORK REVIEW
Please provide the following information on your previous
internal Network Review required by 40 CFR 58.20d.
Review performed on: Date
Performed by
Location and Title of Review Document
Briefly discuss all problems uncovered by this review.
5-42
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 42 of 142
4. NON-CRITERIA POLLUTANTS
Does your agency monitor and/or analyze for non-criteria and/or
toxic air pollutants? Yes No
If yes, please complete the form below.
Pollutant
Monitoring
Method/Instrument
SOP Available
Yes/No
5-43
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 43 of 142
B. RESOURCES AND FACILITIES
1. INSTRUMENTS.AND METHODS
(a) Please complete the table below to indicate which analyzers
do not conform with the requirements of 40 CFR 53 for NAMS,
SLAMS, or SIP related SPM's.
Site Comment on
Pollutant Number Make/Model Identification Variances
CO
S02
N02
03
TSP
Pb
(b) Please comment briefly on your currently identified
instrument needs.
5-44
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 44 of 142
2. STAFF AND FACILITIES
(a) Please indicate the number of people available to each
of the following program areas:
Program Area
Number
Comment on Need for
Additional Personnel
Network Design
and Siting
Resources and
Facilities
Data and Data
Management
QA/QC
(b) Comment on your agency's needs for additional physical
space (laboratory, office, storage, etc.).
5-45
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 45 of 142
3. LABORATORY OPERATION AND FACILITIES
(a) Is the documentation of Laboratory Standard Operating
Procedures complete? Yes No .
Please complete the table below.
Analysis Date of Last Revision
TSP
Pb
so4
NO.
'3
SO
2 (bubblers)
N02
Others (list by pollutant)
(b) Is sufficient instrumentation available to conduct your
laboratory analyses? Yes No .
If no, please indicate instrumentation needs in the
table below.
Instrument New or Year of
Needed Analysis Replacement Acquisition
5-46
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 46 of 142
4. STANDARDS AND TRACEABILITY
(a) Please complete the table for your agency's laboratory
standards.
Parameter
Primary
Standard
Secondary Recertification
Standard Date
CO
NO,
SO,
Weights
Temperature
Moisture
Barometric
Pressure
Flow
Lead
5-47
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 47 of 142
(b) Please complete the table below for your agency's site standards
at each site audited (up to 7% of the sites; however, no°t to
exceed 20 sites).
PrimarySecondaryRecertification
Parameter Standard Standard Date
CO
so2
°3
5-48
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 48 of 142
C. DATA AND DATA MANAGEMENT
1. TIMELINESS OF DATA.
For the current calendar year or portion thereof which ended
at least 135 calendar days prior to the receipt of this
questionnaire, please provide the following percentages for
required data submitted.
% Submitted on Time
Monitor ing Sol CO Ol NOT TSP Pb
Qtr ^ J *
1
(Jan. 1-March 31)
_ ,
(Apr. 1-June 30)
3
(July 1-Sept. 30)
_
(Oct. 1-Dec. 31)
* "On-Time" = within 135 calendar days or after the end of the
quarter in which the data were collected.
5-49
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 49 of 142
2. DATA REVIEW
What fraction of the SLAMS sites (by pollutant) reported less
than 75% of the data (adjusted for seasonal monitoring and
site start-ups and terminations)?
Percent of Sites
Pollutant <75% Data Recovery
1st 2nd 3rd 4th
Quarter Quarter Quarter Quarter
Ozone
Nitrogen Dioxide
Sulfur Dioxide
Carbon Monoxide
Total Suspended
Particulates
Lead
5-50 .
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 50 of 142
3. DATA CORRECTION
(a) Are changes to submitted data documented in a
permanent file? Yes No
If not, why not
(b) Are changes performed according to a documented
Standard Operating Procedure or your Agency
Quality Assurance Project Plan? Yes No
If not according to the QA Project Plan, please attach
a copy of your current Standard Operating Procedure.
(c) Who has signature authority for approving
corrections?
(Name)
(Program Function)
5-51
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 51 of 142
4. ANNUAL REPORT
(a) Please provide the dates annual reports have been
submitted in the last two years.
(b) Does the agency's annual report (as required in 40 CFR
CFR 58.26) include the following?
YES NO
1. Data summary required in Appendix F.
2. Annual precision and accuracy information
described in Section 5.2 of Appendix A.
3. Location, date, pollution source and duration
of all episodes reaching the significant
harm levels.
4. Certification by a senior officer in the
State or his designee.
(c) Describe any deficiencies which cause the answer to part (b)
of this question to be No.
5-52
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 52 of 142
D. QUALITY ASSURANCE/QUALITY CONTROL
1. STATUS OF QUALITY ASSURANCE PROGRAM*
(a) Does the agency have an EPA approved quality
assurance program plan?
Yes No .
If yes, have changes to the plan been approved
by the EPA? Yes No
Please provide:
Date of Original Approval
Date of Last Revision
Date of Latest Approval
(b) Do you have any revisions to your QA Program Plan still
pending?
Yes No
* If answer is No give a brief summary of the deficiencies.
5-53
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 53 of 142
2. AUDIT PARTICIPATION
(a) Date last system audit was conducted?
By whom?
(b) Does the agency participate in the National Performance
Audit Program (NPAP) as required under 40 CFR 58
Appendix A?* Yes No
(c) Please complete the table below.
Parameter Audited
Date of Last NPAP
SO2 (continuous)
CO
Pb
ReF Device
S02 (bubbler)
N02 (bubbler)
No/ give a brief summary of deficiencies.
5-54
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 54 of 142
3. PRECISION AND ACCURACY GOALS
As a goal, the 95 percent probability limits for precision
(all pollutants) and TSP accuracy should be less than +15
percent. At 95 percent probability limits, the accuracy for all
other pollutants should be less than +20 percent. Using a short
narrative and a summary table, compare the Reporting
Organization's performance against these goals over the last
year. Explain any deviations.
Precision and accuracy are based on Reporting Organizations;
therefore this question concerns the Reporting Organizations
which are the responsibility of the agency. A copy of a computer
printout has been provided which contains the precision and
accuracy data submitted to EMSL for each of the agency's
Reporting Organizations. The printout, containing at least the
last four completed calendar quarters of precision and accuracy
data, was obtained using the NADB program NA273. This data
should be verified using agency records. If found in error,
please initiate corrections. Based on the data provided or
corrections thereto, complete the table in part "a" below
indicating the number of Reporting Organizations meeting the
goals stated above for each pollutant by quarter.
5-55
-------
(a) Precision Goals
# of Reporting
Pollutant Organization
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 55 of 142
Precision
Qtr/Yr Qtr/Yr Qtr/Yr Qtr/Yr
N02
so2
CO
TSP
Pb
(b) Accuracy Goals
Pollutant
f of Reporting
Organization
Accuracy
Qtr/Yr Qtr/Yr Qtr/Yr Qtr/Yr
so2
CO
TSP
Pb
(c) To the extent possible, describe problems preventing the
meeting of precision and accuracy goals.
5-56
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 56 of 142
11.7 System Audit Questionnaire (Long-Form)
The long-form system audit questionnaire which follows is
intended to provide a complete picture of agency ambient air
monitoring operations and quality assurance implementation. The
following instructions might prove helpful in completing this
survey questionnaire.
1. For ease in completing the questionnaire, it is not necessary
to type. Filling it out legibly in black ink is acceptable.
2. Feel free to elaborate on any point or question in the form.
Use additional pages as necessary to give a complete
response.
3. Mien necessary, submit copies of documents which will aid in
understanding your response.
4. Please pay careful attention in completing the questionnaire.
The information supplied will have a direct bearing on the
conclusions drawn and recommendations made concerning the
evaluation of your organization's program.
5. The Regional Quality Assurance Coordinator or a member of his
staff may be contacted for assistance in completing the
questionnaire.
5-57
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 57 of 142
SYSTEM AUDIT QUESTIONNAIRE (LONG FORM)
GENERAL INFORMATION
Questionnaire completion date_
On-site system audit date
Reporting period
Agency name and address_
Mailing address (if different from above)
Telephone number (FTS)
Commercial ( )
Agency Director
Agency QA Officer
Reporting Organizations Making Up This Agency
System audit conducted by_
Affiliation of audit team
Key Personnel: Completed Questionnaire Interviewed
Planning
Field Operations
Laboratory Operations
QA/QC
Data Management
Report ing
Persons Present during exit interview
5-58
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 58 of 142
A. NETWORK MANAGEMENT
1. GENERAL
(a) Provide an organization chart clearly showing the agency's
structure and its Reporting Organizations. (Attach sheet(s)
as necessary.)
(b) What is the basis for the current structure of the agency's
Reporting Organizations?
Common team of field operators?
Common calibration standards?
Common laboratory facilities?
Common calibration equipment?
• ^^—^^—^^
Common audit team?
Common audit systems (devices and standards)?
Common data handling and processing?
Other rationale.
(c) Does the agency feel that the data for the Reporting
Organizations it contains can be pooled?
Yes No Please comment on either answer
(d) Briefly describe any changes which will be made within the
agency the next calendar year.
5-59
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 59 of 142
(e) Complete the table below for each of the criteria pollutants
monitored as part of your air monitoring network.
Number of Monitors
SO2 NO2 CO O3 TSP Pb
NAMS
SLAMS
(excluding NAMS)
SPM
TOTAL
(f) What is the date of the most current official SLAMS Network
Description?
(i) Where is it available for public inspection?
(ii) Does it include for each site the following?
YES NO
SAROAD Site IDf
Location
Sampling and Analysis Method
Operative Schedule
Monitoring Objective and Scale
of Representativeness
Any Proposed Changes
5-60
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 60 of 142
(g) For each of the criteria pollutants, how many modifications
(SLAMS including NAMS) have been made since the last system
audit? (List the total SLAMS and NAMS)
Number of Monitors
Pollutant Added Deleted Relocated
Sulfur Dioxide
Nitrogen Dioxide
Carbon Monoxide
Ozone
Total Suspended
Particulates
Lead
(h) Briefly discuss changes to the Air Monitoring Network planned
for the next audit period.
5-61
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 61 of 142
(i) Does an overall SLAMS/NAMS Monitoring Plan exist?
Ye s NO
(j) Has the agency prepared and implemented Standard Operating
Procedures for all facets of agency operation? Yes NO
If no list subject of any missing SOPs
(k) Do the Standard Operating Procedures adequately address at
least the eleven (11) item quality control program required
by Appendix A to 40 CFR 58? Yes No Comment
(1) Clearly identify by section number and/or document title,
major changes made to documents since the last on-site
review.
Title/Section # Pollutant(s) Affected
(m) Does the agency have an implemented plan for operations
during emergency episodes? Yes No Indicate latest
revision, approval date and current location of this plan.
Document Title
Revision Date
Approved
5-62
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 62 of 142
(n) During episodes, are communications sufficient so that
regulatory actions are based on real-time data?
Yes No
(o) Identify the section of the emergency episode plan where
quality control procedures can be found.
5-63
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 63 of 142
2. NETWORK DESIGN AND SITING
(a) Indicate by SAROAD Number any non-conformance with the
requirements of 40 CFR 58, Appendices D and E.
Monitor
Site ID
(SAROAD)
Reason for Non-Conformance
SO,
CO
NO,
TSP
(b) Please provide the following information on your previous
internal Network Review required by 40 CFR 58.20d.
Review performed on: Date
Performed by
Location and Title of Review Document
Briefly discuss all problems uncovered by this review.
5-64
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 64 of 142
(c) Have NAMS Hard Copy Information Reports (NHCIRs) been
prepared and submitted for all monitoring sites within
the network? Yes No
(d) Does each site "have the required information including:
YES NO
SAROAD identification number?
Photographs/siides to the four cardinal
compass points?
Startup and shutdown dates?
Documentation of instrumentation?
Reasons for periods of missing data?
(e) Who has custody of the current network documentation?
(Name) (Title)
(f) Does the current level of monitoring effort, site placement,
instrumentation, etc., meet requirements imposed by current
grant conditions? Yes No Comment
(g) How often is the network design and siting reviewed?
Date of last review
5-65
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 65 of 142
(h) Please provide a summary of the monitoring activities
conducted as the SLAMS /NAMS network by the agency as
follows:
(i) Monitoring is seasonal for (indicate pollutant and
calendar quarters of high and low concentrations).
Calendar Quarters
High Low
Pollutant Concentration Concentration Collocated
Y/N
Y/N
Y/N
. Y/N
Y/N
Y/N
(ii) Monitoring is year-round for (indicate pollutant)
Pollutant Collocated
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
5-66
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page €6 of 142
(i) Does the number of collocated monitoring sites meet the
requirements of 40 CFR 58? Yes No Comment
(j) Does your agency monitor/and or analyze for non-criteria air
and/or toxic air pollutants? Yes No
If yes, please complete the form below.
Monitoring SOP Available
Pollutant Method/Instrument Yes/No
5-67
-------
3. ORGANIZATION, STAFFING AND TRAINING
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 67 of 142
(a) Please indicate the key individuals responsible for
the following:
Agency Director
SLAMS Network Manager
Quality Assurance Officer
Field Operations Supervisor
Laboratory Supervisor
Data Management Supervisor
SLAMS Reporting Supervisor
(b) Please indicate the number of people available to each
of the following program areas:
Program Area
Number
Comment on Need for
Additional Personnel
Network Design
and Siting
Resources and
Facilities
Data and Data
Management
QA/QC
5-68
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 68 o'f 142
(c) Does the agency have an established training program?
Yes No
(i) Where is this documented? ^
(rev date)
(ii) Does it make use of seminars, courses, EPA sponsored
college level courses? Yes No
(iii) Indicate below the three (3) most recent training events
and identify the personnel participating in them?
Event Dates Participant s)
5-69
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 69 of 142
(d) Does the agency subscribe to recognized publications? Please
provide a list of periodicals. Are periodicals available to
all personnel? •
Periodical Title Distribution
5-70
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 70 of 142
4. FACILITIES
(a) Identify the principal facilities where the work is performed
which is related to the SLAMS/NAMS network? (Do not include
monitoring sites but do include any work which is performed
by contract or other arrangements).
Facility Location Main SLAMS/NAMS Function
(b) Please review the entries on the above table. Are there any
areas of facilities which you believe should be upgraded?
Please identify by location.
(c) Are there any significant changes which are likely to be
implemented to agency facilities before the next system
audit? Comment on your agency's needs for additional
physical resources.
Facility Function Proposed Change - Date
5-71
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 71 of 142
B. FIELD OPERATIONS
1. ROUTINE OPERATIONS
(a) Is-the documentation of Monitoring Standard Operating
Procedures complete? Yes No .
Please complete the table below.
Pollutant
Monitored Date of Last Revision
__
Pb
SO
NO
2
(continuous)
'2
S0
(bubblers)
°3
CO
Others (list by pollutant)
(b) Are such procedures available to all field operations
personnel? Yes No Comment
(c) Are procedures prepared and available to field personnel
which detail operations during episode monitoring?
Yes No Comment
5-72
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 72 of 142
(d) For what does each Reporting Organization within the Agency
monitor? Provide the list requested below.
•
Reporting Organization # of Sites Pollutants
(e) On the average, how often are most of your sites visited
by a field operator? per
(f) Is this visit frequency consistent for all Reporting
Organizations within your agency? Yes No
If no/ document exceptions
(g) On the average, how many sites does a single site operator
have responsibility for?
(h) How many of the sites of your SLAMS/NAMS network are
equipped with manifold(s)? *
(i) Briefly describe most common manifold type.
(ii) Are manifolds cleaned periodically? Yes No_
If yes, how often? per
5-73
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 73 of 142
(iii) If the manifold is cleaned, what is used?
(iv) Are manifold(s) equipped with a blower? Yes No
(i) Is there sufficient air flow through the manifold at all
times? Yes No
Appoximate air flow is .
(flow units)
(j) Is there a conditioning period for the manifold after
cleaning? Briefly comment on the length of time the
conditioning is performed.
(k) What material is used for instrument lines?
(1) Has the agency obtained necessary waiver provisions to
operate equipment which does not meet the effective
reference and equivalency requirements? Yes No
Comment on Agency use of approved/non-approved
instrumentation.
5-74
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 74 of 142
(m) Please complete the table below to indicate which analyzers
do not conform with the requirements of 40 CFR 53 for NAMS,
SLAMS, or SIP related SPM's.
Site Comment on
Pollutant Number Make/Model Identification Variances
CO
S0
°3
TSP
(n) Please comment briefly on your currently identified
instrument needs.
5-75
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 75 of 142
2. QUALITY CONTROL
(a) Are field calibration procedures included in the documented
Standard Operating Procedures? Yes No
Comment on location of such procedures
(b) Are multipoint calibrations performed? Indicate both the
frequency and pollutant.
Reporting Organization Pollutant Frequency
(c) Are calibrations performed in keeping with the "guidance
offered in Section 2.0.9. Vol II of the Quality Assurance
Handbook for Air Pollution Measurement Systems? Yes No
If no, why not?
(d) Are calibration procedures consistent with the operational
requirements of 40 CFR 50 standard methods? Yes No
If no, briefly explain deviations
5-76
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 76 of 142
(e) Have changes been made to calibration methods based on
manufacturer's suggestions for a particular instrument?
Yes No Are these also documented? Yes No
(f) Do standard materials used for calibrations meet the
requirements of the appendices to 40 CFR 50 (EPA reference
methods) and Appendix A to 40 CFR 58 (traceability of
materials to NBS-SRMs or CPMs)? Yes No Comment on
deviations
(g) Are all flow-measurement devices checked and certified?
Yes No Comment
(h) What are the authoritative standards used for each type of
flow measurement? Please list them in the table below,
indicate the frequency of calibration standards to maintain
field material/device credibility.
Flow Devices Primary Standard Frequency of Calibration
(i) Where do field operations personnel obtain gaseous standards?
Are those standards certified by:
YES NO
(i) The agency laboratory?
(ii) EPA/EMSL/RTP standards laboratory?
(iii) A laboratory separate from this
agency but part of the same
Reporting Organization?
(iv) The vendor?
5-77
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 77 of 142
(j) Does the documentation include expiration date of
certification? Yes No
Reference to primary standard used? Yes No_
What traceability protocol is used?
Please attach an example of recent documentation of
traceability (tag, label, log sheet).
(k) Is calibration equipment maintained at each site? Yes_
No
For what pollutants?
(1) How is the functional integrity of this equipment
documented?
5-78
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 78 of 142
(m) Are zero and span (z/s) checks made for all continuous
monitoring equipment and flow checks made for TSP samplers?
Yes No
Please complete table below:
Span Cone'.
Pollutant (ppm) Frequency
(i) Continuous analyzers .
Flow Rate Frequency
(ii) TSP Samplers
(n) Does the agency have acceptance criteria for zero/span
checks? Yes No Comment
(i) Are these criteria known to the field operations
personnel? Yes No
(ii) Are they documented in standard operating procedures?
Yes No If not, indicate document where they can be
found.
5-79
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 79 of 142
(iii) Do the documents discussed in (ii) above indicate when
zero/span adjustments should and should not be made?
Yes No Indicate an example
(o) In keeping with 40 CFR 58 regulations are any necessary
zero and span adjustments made after precision checks?
Yes No If no, comment on why not
(p) Who has the responsibility for performing zero/span checks?
(q) Are precision checks routinely performed within concentration
ranges and with a frequency which meet or exceed the
requirements of 40 CFR 58, Appendix A? Yes No
Please comment on any discrepancies.
(r) Please identify person(s) with the responsibility for
performance of precision checks on continuous analyzers?
Person(s)
Title
5-80
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 80 of 142
3. PREVENTIVE MAINTENANCE
(a) Has the field operator been given any special training in
performing preventive maintenance? Briefly comment on back
ground and/or courses
(b) Is this training routinely reinforced?
If no, why not? _
Yes
No
(c) If preventive maintenance is MINOR, it is performed at
(check one or more) : field site , headquarters
facilities
equipment is sent to manufacturer
(d) If preventive maintenance is MAJOR, it is performed at
(check one or more) : field site , headquarters
facilities
equipment is sent to manufacturer
(e) Does the agency have service contracts or agreements in
place
with instrument manufacturers? Indicate below or attach
additional pages to show Which instrumentation is covered.
(f) Comment briefly on the adequacy and availability of the
supply of spare parts, tools and manuals available to the
field operator to perform any necessary maintenance
activities. Do you feel that this is adequate to prevent
any significant data loss?
5-81
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 81 of 142
(g) Is the agency currently experiencing any recurring problem
with equipment or manufactureds)? If so, please identify
the
equipment and/or manufacturer, and comment on steps taken to
remedy the problem.
5-82
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 82 of 142
4. EECORDKEEPING
(a) Is a log book(s) maintained at each site to document
site visits, preventive maintenance and resolution of
site operational problems and corrective actions taken?
Yes No Other uses
(b) Is the logbook maintained currently and reviewed
periodically? Yes No_
Frequency of Review
(c) What happens once entries are made and all pages filled?
Is the logbook sent to the laboratory for archiving?
Yes No If no, is it stored at other location (specify)
(d) What other records are used? YES NO
Zero/span record?
Gas usage log?
Maintenance log?
Log of precision checks?
A record of audits?
Please describe the use and storage of these documents.
(e) Are calibration records or at least calibration constants
available to field operators? Yes No Please attach
an example field calibration record sheet to this
5-83
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 83 of 142
questionnaire.
5-84
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 84 of 142
5. DATA ACQUISITION AND HANDLING
(a) With the exception of TSP, are instrument outputs (that
is data) recorded to (a) stripcharts, (b) magnetic tape
acquisition system (c) digitized and telemetered directly
to agency headquarters? Please complete the table below
for each of the Reporting Organizations.
Reporting Organization
Pollutants
Data
Acquisition Media
(b) Is there stripchart backup for all continuous analyzers?
Yes No
(c) Where is the flow of high-volume samplers recorded at the
site?
(i) For samplers with flow controllers? Log sheet ,
Dixon chart , Other (specify).
(ii) On High-volume samplers without flow controllers?
Logsheet , Dixon chart , Other (specify).
5-85
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 85 of 142
(d) What Kind of recovery capabilities are available to the
field operator after power outages, storms, etc? Briefly
describe below.
(e) Using a summary flow diagram, indicate below all data
handling steps performed at the air monitoring site.
Identify the format, frequency and contents of data
submittals to the data processing section. Clearly
indicate points at which flow path differs for different
criteria pollutants. Be sure to include all calibration,
zero/span and precision check data flow paths.
5-86
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 86 of 142
C. LABORATORY OPERATIONS
1. ROUTINE OPERATIONS
(a) What analytical methods are employed in support of your
air monitoring network?
Analysis Methods
__
Pb
so4
NO3
SO
(bubblers)
Others (list by pollutant)
N02
(b) Are bubblers used for any criteria pollutants in any of the
local agencies? Yes _ No _ If yes, attach a table
which indicates the number of sites where bubblers are used,
the agency and pollutant( s) .
(c) Do any laboratory procedures deviate from the 40 CFR 50
standard methods? Yes _ No _ If yes, are the deviations
for lead analysis _ , TSP filter conditioning _
or other --- (specify below)? _
(d) Have the procedures and/or any changes been approved by EPA?
Yes No Date of Approval
5-87
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 87 of 142
(e) Is the documentation of Laboratory Standard Operating
Procedures complete? Yes No . Please complete the
table below.
Analysis Date of Last Revision
___
Pb
so4
NO
'3
SO
2 (bubblers)
N02
Others (list by pollutant)
(f) Is sufficient instrumentation available to conduct your
laboratory analyses? Yes No . If no,
please indicate instrumentation needs in the table below.
Instrument New or Year of
Needed Analysis Replacement Acquisition
5-88
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 88 of 142
2. QUALITY CONTROL
(a) Please complete the table for your agency's laboratory
standards.
PrimarySecondaryRecertification
Parameter Standard Standard Date
CO
N02
SO2
°3
Weights
Temperature
Moisture
Barometric
Pressure
Flow
Lead
Sulfate
Nitrate
VOC
5-89
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 89 of 142
(b) Are all chemicals and solutions clearly marked with an
indication of shelf life? Yes No
(c) Are chemicals removed and properly disposed of when shelf
life expires? Yes No
(d) Are only ACS chemicals used by the laboratory? Yes No
(e) Comment on the traceability of chemicals used in the
preparation of calibration standards?
(f) Does the laboratory
(i) purchase standard solutions such as those for use with
lead or other AA analysis? Yes No
(ii) make them themselves? Yes No_
(iii) if the laboratory routinely makes its own standard
solutions, are procedures for such available?
Yes No
Where? Attach an example.
5-90
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 90 of 142
(g) Are all calibration procedures documented? Yes No
(i) Where?
(title) (revision)
(ii) Unless fully documented attach a brief description of a
calibration procedure.
(h) Are at least one duplicate, one blank, and one spike
included with a given analytical batch? Yes No
Identify analyses for which this is routine operation?
(i) Briefly describe the laboratory's use of data derived from
blank analyses? Do criteria exist which determine
acceptable/non-acceptable blank data? Please complete the
table below.
Pollutant Blank Acceptance Criteria
SO,
'2
NO,
so4
'2
)
N03
Pb
TSP
VOC
Other
5-91
-------
Section No. 2.0.11
Revision No. 1 f
Date October 1, 1984
Page 91 of 142
(j) How frequently and at what concentration ranges does the lab
perform duplicate analysis? What constitutes acceptable
agreement? Please complete the table below.
Duplicate Analysis
Pollutant Frequency Acceptance Criteria
SO,
'2
N02
so4
N03
Pb
TSP
VOC
Other
(k) How does the lab use data from spiked samples? Please
indicate what may be considered acceptable percentage
recovery by analysis? Please complete the table below.
Pollutant %^ Recovery Acceptance Criteria
SO.
'2
NO,
'2
)
NO
so4
3
Pb
TSP
VOC
Other
(1) Does the laboratory .routinely include samples of reference
5-92
-------
Section No. 2.0.11
•Revision No. 1
Date October 1, 1984
Page 92 of 142
material obtained from EPA within an analytical batch?
Yes No If yes, indicate frequency, level, and
material used.
(m) Are mid-range standards included in analytical batches?
Yes No If yes, are such standards included as a QC
check (span check) on analytical stability? Please
indicate the frequency, level and compounds used in the
space provided below.
(n) Do criteria exist for "real-time" quality control based on
the results obtained for the mid-range standards discussed
above? Yes No If yes, briefly discuss them below or
indicate the document in which they can be found.
(o) Are appropriate acceptance criteria documented for each
type of analysis conducted? Yes No Are they known to
at least the analysts working with respective instruments?
5-93
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 93 of 142
3. PREVENTIVE MAINTENANCE
(a) For laboratory equipment, who has responsibility for major
and/or minor preventive maintenance? Person
Title
(b) Is most maintenance performed:
(i) in the lab? Yes No
(ii) in the instrument repair facility? Yes No
(iii) at the manufacturer's facility? Yes No
(c) Is a maintenance log maintained for each major laboratory
instrument? Yes No Comment
(d) Are service contracts in place for the following analytical
instruments:
YES NO
Analytical Balance
Atomic Absorption Spectrometer
Ion Chroma tograph
Automated Colorimeter
5-94
-------
Section No. 2.0.11
Revision No. 1 *
Date October 1, 1984
Page 94 of 142
4. RECORDKEEPING
(a) Are all samples that are received by the laboratory:
(i) logged-in? Yes No
(ii) assigned a unique laboratory sample number?
Ye s No
(iii) routed to the appropriate analytical section?
Yes No
Discuss sample routing and special needs for analysis (or
attach a copy of the latest SOP which covers this).
(b) Are logbooks kept for all analytical laboratory instruments?
Yes No
(c) Do these logbooks indicate:
YES NO
(i) analytical batches processed?
(ii) quality control data?
(iii) calibration data?
(iv) results of blanks, spikes and duplicates?
(v) initials of analyst?
5-95
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 95 of 142
(d) Is there a logbook which indicates the checks made on
(i) weights? Yes No
(ii) humidity indicators? Yes No
(iii) balances? Yes No
(iv) thermometer(s)? Yes No
(e) Are logbooks maintained to track the preparation of filters
for the field? Yes No
(i) Are they current? Yes No
(ii) Do they indicate proper use of conditioning? Yes No
(iii) Weighings? Yes No
(iv) Stamping and numbering? Yes No
(f) Are logbooks kept which track filters returning from the
field for analysis? Yes No
(g) How are data records from the laboratory archived?
(i) Where?
(ii) Who has the responsibility? Person
Title
(iii) How long are records kept? Years
5-96
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 96 of 142
(h) Does a chain-of-custody procedure exist for laboratory
samples? Yes No
(i) Has chain-of-custody been documented and implemented as part
of standard laboratory procedures? Yes No If yes,
indicate date, title and revision number where it can be
found.
5-97
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 97 of 142
5. DATA ACQUISITION AND DATA HANDLING
(a) Identify those instruments which make use of computer
interfaces directly to record data? Which ones use
stripcharts? integrators?
(b) Are QC data readily available to the analyst during a given
analytical run? Yes No
(c) For those instruments which are computer interfaced,
indicate which are backed up by stripcharts?
(d) What is the laboratory's capability with regard to data
recovery? In case of problems can they recapture data or
are they dependent on computer operations? Discuss briefly.
(e) For automated data acquisition instrumentation has a user's
manual been prepared? Yes No Comment
(i) Is it in the analyst's or user's possession? Yes No_
(ii) Is it current? Yes No
5-98
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 98 of 142
(f) Please provide below a data flow diagram Which establishes,
by a short summary flow chart; transcriptions, validations,
and reporting format changes the data goes through before
being released to the data management group? Attach
additional pages as necessary.
5-99
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 99 of 142
6. SPECIFIC POLLUTANTS: TSP AND LEAD
(a) Are filters supplied by EPA used at SLAMS sites?
Yes No Comment
(b) Do filters meet the specifications in the Federal Register
40 CFR 50? Yes No Comment
(c) Are filters checked for surface alkalinity? Yes No_
Indicate frequency
(d) Are filters visually inspected via strong light from a view
box for pinholes and other imperfections? Yes No If
no, comment on way imperfections are determined?
(e) Are filters permanently marked with a serial number? Yes_
No Indicate when and how this is accomplished
(f) Are unexposed filters equilibrated in controlled
conditioning environment which meets or exceeds the
requirements of 40 CFR 50? Yes No If no, why not?
(g) Is the conditioning environment checked or calibrated?
Yes No Indicate frequency
5-100
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 100 of 142
(h) Is the balance checked with Class "S" weights each day it is
used? Yes No Indicate frequency of such checks
(i) Is the balance check information placed in a QC logbook?
Yes No If no, where is it recorded
(j) Is the filter weighed to the nearest milligram?
Yes No If not, what mass increment
(k) Are filter serial numbers and tare weigh\ts permanently
recorded in a bound notebook? Yes No Indicate
where
(1) Are filters packaged for protection while transporting to
and from the monitoring sites? Yes No
(m) How often are filter samples collected? (Indicate average
lapse time (hrs) between end of sampling and laboratory
receipt.)
(n) Are field measurements recorded in logbook or on filter
folder.
(o) Are exposed filters reconditioned for at least 24 hrs in the
same conditioning environment as for unexposed filters?
Yes No If no, why not
5-101
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 101 of 142
(p) Are exposed filters removed from folders, etc., before
conditioning? Yes No
(q) Is the exposed filter weighed to the nearest milligram?
Yes No
(r) Are exposed filters archived? Yes No When?
Where?
Indicate retention period?
(s) Are analyses performed on filters? Yes No Indicate
analyses other than Pb and mass which are routinely
performed.
(t) Are sample weights and collection data recorded in a bound
laboratory logbook? Yes No On data forms? Yes No
(u) Is analysis for lead being conducted using atomic absorption
spectrometry with air acetylene flame? Yes No
If not, has the agency received an equivalency designation
of their procedure.
(v) Is either the hot or ultrasonic extraction procedure being
followed precisely? Yes No Which?
5-102
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 102 of 142
(w) Is Class A borosilicate glassware used throughout the
analysis? Yes No
(x) Is all glassware scrupulously cleaned with detergent, soaked
and rinsed three times with distilled-deionized water?
Yes No If not, briefly describe or attach procedure.
(y) If extracted samples are stored, are linear polyethlyene
bottles used? Yes No Comment
(z) Are all batches of glass fiber filters tested for lead
content? Yes No_ At a rate of 20 to 30 random filters
per batch of 500 or greater? Yes No Indicate rate
(aa) Are ACS reagent grade HN03 and HC1 used in the analysis?
Yes No If not, indicate grade used
(bb) Is a calibration curve available having concentrations that
cover the linear absorption range of the atomic absorption
instrumentation? Yes No Briefly describe
5-103
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 103 of 142
(cc) Is the stability of the calibration curve checked by
alternately remeasuring every 10th sample a concentration
<.! ug Pb/ml; ^10 ug Pb/ml? Yes No
If not, indicate frequency
(dd) Are measured air volumes corrected to reference conditions
as given in CFR regulations (Qstd of 760 nun Hg and 25°C)
prior to calculating the Pb concentration? Yes No
If not/ indicate conditions routinely employed for both
internal and external reporting
(ee) In either the hot or ultrasonic extraction procedure, is
there always a 3O-min HjO soaking period to allow HNO3
trapped in the filter to diffuse into the rinse water?
Yes . No Comment
(ff) Is a quality control program in effect that includes
periodic quantification of (1) lead in 3/4" x 8" glass fiber
filter strips containing 100-300 ug Pb/strip, and/or (2) a
similar strip with 600-1000 ug strip, and (3) blank filter
strips with zero Pb content to determine if the method, as
being used, has any bias? Yes No Comment on lead QC
program or attach applicable SOP.
5-104
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 104 of 142
D. DATA AND DATA MANAGEMENT
. DATA HANDLING
(a) Is there a procedure, description, or a chart which shows a
complete data sequence from point of acquisition to point of
submission of data, to EPA? Yes No
Please provide below a data flow diagram indicating both
the data flow witliin the Reporting Organization and the data
received from the various local agencies.
5-105
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 105 of 142
(b) Are data handling and data reduction procedures documented?
(i) For data from continuous analyzers? Yes No
(ii) For data from non-continuous monitors? Yes No
(c) In what format and media are data submitted to data
processing section? Please provide separate entry for
each Reporting Organization.
Reporting Reporting
Organization Data Media Organization Data Media
(d) How often are data received at the processing center from
the field sites and laboratory?
(a) at least once a week?
(b) every 1-2 weeks?
(c) once a month?
(e) Is there documentation accompanying the data regarding any
media changes, transcriptions, and/or flags which have been
placed into the data before data are released to data
processing? Describe
5-106
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 106 of 142
(f) How is the data actually entered to the computer system?
Digitization of stripcharts? Manual or computerized
transcriptions? Other?
(g) Is a double-key entry system used for data at the processing
center? Are duplicate card decks prepared? Yes No
If no, why not?
(h) Have special data handling procedures been adopted for air
pollution episodes? Yes No If yes, provide brief
description
5-107
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 107 of 142
2. SOFTWARE DOCUMENTATION
(a) Does the agency have available a copy of the AEROS Manual?
Yes No Comment
(b) Does the agency have the PARS user's guide available?
Yes No Comment (provide guide #)
(c) Does the Data Management Section have complete software
documentation? Yes No Comment
If yes, indicate the implementation date and latest
revision dates for such documentation.
(d) Do the documentation standards follow the guidance offered
by the EPA Software Documentation Protocols? Yes No
If no, what protocols are they based on
5-108
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 108 of 142
(e) What is the origin of the software used to process air
monitoring data prior to its release into the SAROAD/NADB
database?
(i) Purchased? Yes No ; Supplier
Date of latest version
(ii) Written in house? Yes No ; Latest version
Date
(iii) Purchased with modifications in-house? Yes No_
Latest version Date
(iv) Other(specify)
(f) Is a user's manual available to data management personnel
for all software currently in use at the agency for
processing SLAMS/NAMS data? Yes No Comment
(g) Is there a functional description either:
(i) included in the users manual? Yes No
(ii) separate from it and available to the users? Yes No_
5-109
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 109 of 142
(h) Is the computer system contents including, ambient air
monitoring data, backed up reguarly? Briefly describe,
indicating at least the media, frequency, and backup-media
storage location?
(i) What is the recovery capability (how much time and data
would be lost) in the event of a significant computer
problem?
(j) Is test data available to test the integrity of the
software? Yes No Is it properly documented?
Yes No
5-110
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 11O of 142
3. DATA VALIDATION AND CORRECTION
(a) Have validation criteria, applicable to all pollutant data
processed by the Reporting Organization been established and
documented? Yes No If yes, indicate document where
such criteria can be found (title, revision date).
(b) Does documentation exist on the identification and
applicability of flags (i.e. identification of suspect
values) within the data? Yes No
(c) Do documented data validation criteria employed, address
limits on and for the following:
(i) Operational parameters, such as flow rate measurements
or flow rate changes.
(ii) Calibration raw data, calibration validation and
calibration equipment tests. '
(iii) All special checks unique to a measurement system
(iv) Tests for outliers in routine data as part of screening
process
5-111
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 111 of 142
(v) Manual checks such as hand calculation of concentrations
and their comparison with computer-calculated data
(d) Are changes to submitted data documented in a permanent
file? Yes No
If not, why not
(e) Are changes performed according to a documented
Standard Operating Procedure or your Agency
Quality Assurance Project Plan? Yes No
If not according to the QA Project Plan, please attach
a .copy of your current Standard Operating Procedure.
(f) Who has signature authority for approving
corrections?
(Name) (Program Function)
(g) Are data validation summaries prepared at each critical point
in the measurement process or information flow and forwarded
with the applicable block of data to the next level of
validation? Yes No Please indicate the points where
such summaries are performed.
5-112
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 112 of 142
(h) What criteria are applied for data to be deleted? Discuss
briefly
(i) What criteria are applied to cause data to be reprocessed?
Discuss
(j) Is the group supplying data provided an opportunity to review
data and correct erroneous entries? Yes No If yes, how?
(k) Are corrected data resubndtted to the issuing group for
cross-checking prior to release? Yes No
5-113
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 113 of 142
4. DATA PROCESSING
(a) Does the agency generate data summary reports? Yes No_
Are the data used for in-house distribution and use?
Yes No
Publication? Yes No
Other (specify)
(b) Please list at least three (3) reports routinely generated,
providing the information requested below.
Report Title
Distribution
Period Covered
(c) Have special procedures been instituted for pollution index
reporting? Yes No If yes, provide brief description
(d) Who at the agency has the responsibility for submitting data
to SAROAD/NADB? (name) (title)
Is the data reviewed and approved by an officer of the agency
prior to submittal? Yes No
(name) (title)
5-114
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 114 of 142
(e) Are those persons different from the individuals who submits
data to PARS? Yes No If yes, provide name and title of
individual responsible for PARS data submittal.
(name) (title) PARS
Data review and approval (name)
(title)
(f) How often are data submitted to:
( i) SAROAD?
(ii) PARS? '
(g) How and/or in what form are data submitted?
(i) TO SAROAD?
(ii) TO PARS?
(h) Are the recommendations and requirements for data coding and
submittal, in the AEROS User's Manual followed closely for
SAROAD? Yes No Comment on any routine deviations in
coding procedures
(i) Are the'recommendations and requirements for data coding and
submittal, in the PARS User's Guide, followed closely?
Yes No Comment on any routine deviations in coding
and/or computational procedures.
5-115
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 115 of 142
(j) Does the agency routinely request a hard copy printback on
submitted data;
(i) from SAROAD/NADB? Yes No
(ii) from PARS? Yes No
(k) Are records kept for at least 3 years by the agency in an
orderly, accessible form? Yes No
If yes, does this include raw data , calculations ,
QC data , and reports ? If no, please comment
(1) In what format are data received at the data processing
center? (Specify appropriate pollutant.)
(a) ppm ; (b) % chart (c) voltages (d) other ;
(m) Do field data include the following documentation?
Site ID? Yes No
Pollutant type? Yes No
Date received at the center? Yes No
Collection date (flow, time, date)? Yes No
Date of Laboratory Analysis (if applicable) Yes No
Operator/Analyst? Yes No
5-116
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 116 of 142
(n) Are the appropriate calibration equations submitted with the
data to the processing center? Yes No If not, explain
(o) Provide a brief description of the procedures and appropriate
formulae used to convert field data to concentrations prior
to input into the data bank.
S02 :
N°2-
CO
TSP
CH4/THC_
5-117
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 117 of 142
Pb
Other
(p) Are all concentrations corrected to EPA standard (298°K, 760
mm Hg) temperature and pressure condition before input to the
SAROAD? Yes No If no, specify conditions used
(q) Are data reduction audits performed on a routine basis?
Yes No If yes,
(i) at what frequency?
(ii) are they done, by an independent group?
(r) Are there special procedures available for handling and
processing precision, accuracy, calibrations and span checks?
Yes No If no, comment
If yes, provide a brief description:
Span check data
Calibration data
5-118
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 118 of 142
Precision data
Accuracy data
(s) Are precision and accuracy data checked each time they are
recorded, calculated or transcribed to ensure that anomalous
values are not submitted to EPA? Yes No Please
comment and/or provide a brief description of checks
performed.
(t) Is a final data processing check performed prior to
submission of any data? Yes No If yes, document
procedure briefly
If no, explain
5-119
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 119 of 142
5. INTERNAL REPORTING
(a) What reports are prepared and submitted as a result of the
audits required under 40 CFR 58 Appendix A?
Report Title
Frequency
(Please include an example audit report and, by attaching a
coversheet identify the distribution such reports are given
within the agency. )
(b) What internal reports are prepared and submitted as a
result of precision checks also required under 40 CFR 58
Appendix A?
Report Title Frequency
(Please include an example of a precision check report and
identify the distribution such reports receive within the
agency.)
5-120
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 120 of 142
(c) Do either the audit or precision reports indicated include
a discussion of corrective actions initiated based on audit
or precision results? Yes No If yes, identify
report(s) and section numbers
(d) Does the agency prepare Precision and Accuracy summaries
other than Form 1? Yes No Please attach examples of
recent summaries including a recent Form 1.
(e) Who has the responsibility for the calculation and
preparation of data summaries? To whom are such P and
A summaries delivered?
•
Name Title Type of Report Recipient
5-121
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 121 of 142
(f) Identify the individual within the agency who receives the
results of the agency's participation in the NPAP and the
internal distribution of the results once received.
(i) Principal Contact for NPAP is (name, title)
(ii) Distribution is
(name) (title)
5-122
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 122 of 142
6. EXTERNAL REPORTING
(a) For the current calendar year or portion thereof which
ended at least 135 calendar days prior to the receipt
of this questionnaire, please provide the following
percentages for required data submitted.
% Submitted on Time
Monitoring
Qtr
SO2 CO 03
NO2 TSP
Pb
(Jan. 1-March 31)
(Apr. 1-June 30)
(July 1-Sept. 30)
(Oct. 1-Dec. 31)
* "On-Time" = within 135 calendar days or after the end of the
quarter in which the data were collected.
(b) Identify the individual within the agency with the
responsibility for preparing the required 40 CFR 58
Appendix F and G reporting inputs, (name, title)
(c) Identify the individual with the agency with the
responsibility for reviewing and releasing (signing-off on)
(name, title)
5-123
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 123 of 142
(d) Does the agency regularly report the Pollutant Standard
Index (PSI)? Briefly describe the media, coverage, and
frequency of such reporting.
(e) What fraction of the SLAMS sites (by pollutant) reported
less than 75% of the data (adjusted for seasonal monitoring
and site start-ups and terminations)?
Percent of Sites
Pollutant <75% Data Recovery
1st 2nd 3rd 4th
Quarter Quarter Quarter Quarter
Ozone
Nitrogen Dioxide
Sulfur Dioxide
Carbon Monoxide
Total Suspended
Particulates
Lead
5-124
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 124 of 142
(f) Does the agency's annual report (as required in 40 CFR
CFR 58.26) include the following?
YES NO
(i) Data summary required in Appendix F.
(ii) Annual precision and accuracy information
described in Section 5.2 of Appendix A.
(iii) Location, date/ pollution source and duration
of all episodes reaching significant levels.
(iv) Certification by a senior officer in the State
or his designee.
5-125
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 125 of 142
E. QUALITY ASSURANCE/QUALITY CONTROL
1. STATUS OF QUALITY ASSURANCE PROGRAM
(a) Does the agency have an EPA approved quality
•
assurance program plan?
Yes No .
If yes/ have changes to the plan been approved
by the EPA? Yes No
Please provide:
Date of Original Approval
Date of Last Revision
Date of Latest Approval
(b) Do you have any revisions to your QA Program Plan still
pending?
Yes No
(c) Is the QA Plan fully implemented? Yes No Comment
(d) Are copies of QA Plan or pertinent sections available to
agency personnel? Yes No If no, why not?
(e) Which individuals routinely receive updates to QA Plan?
5-126
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 126 of 142
2. AUDITS AND AUDIT SYSTEM TRACEABILITY
(a) Does the agency maintain a separate audit/calibration support
facility laboratory? Yes No
(b) Has the agency documented and implemented specific audit
procedures? Yes No
(c) Have audit procedures been prepared in keeping with the
requirements of Appendix A to 40 CFR 58? Yes No
If no, comment on any EPA approved deviations
(d) Do the procedures meet the specific requirements for
independent standards and the suggestions regarding personnel
and equipment? Yes No Comment
(e) Are SRM or CRM materials used to routinely certify audit
materials? Yes No
(f) Does the agency routinely use NBS-SRM or CRM materials?
Yes No For audits only? For calibrations
only? For both? For neither, secondary standards
are employed .
5-127
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 127 of 142
(g) Please complete the following table to summarize auditing
methods for CO, NO2, SO2, Og analyzers, and High-Volume
Samplers.
Audit
Pollutants Audit Method Standard
CO
°
(continuous)
so2
NO,
* (bubblers)
so2
TSP
5-128
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 128 of 142
(h) Are SRM or CRM materials used to establish traceability of
calibration and zero/span check materials provided to field
operations personnel? Yes No
(i) Specifically for gaseous standards, how is the traceability
of audit system standard materials established? Are they:
(i) purchased certified by the vendor?
(ii) certified by the QA support laboratory which is part of
this agency?
(iii) Other? (Please comment briefly below).
(j) Are all agency traceability and standardization methods used
documented? Yes No Indicate document where such methods
can be found
5-129
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 129 of 142
(k) Do the traceability and standardization methods conform with
the guidance of Section 2.0.7 Vol II of the Handbook for Air
Pollution Measurement Systems?
(i) For permeation devices? Yes No
•
(ii) For cylinder gases? Yes No
(1) Does the agency have identifiable auditing equipment
(specifically intended for sole use) for audits?
Yes No If yes, provide specific identification
(m) How often is auditing equipment certified for accuracy
against standards and equipment of higher-authority?
(n) As a result of the audit equipment checks performed, have
pass/fail (acceptance criteria) been decided? Indicate what
these criteria are with respect to each pollutant? Where are
such criteria documented?
Pollutant Criteria
5-130
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 130 of 142
(o) Is there a procedure for corrective actions taken When an
audit system fails to satisfy the acceptance criteria?
Yes No Briefly discuss this
(p) What is the agency accepted recertification period for audit
materials? Please indicate them in the table below and
include any acceptance criteria which may be applicable.
Recertification
Pollutant Audit Standard Period
CO
°3
NO,
(continuous)
so2
NO,
2 (bubblers)
so2
TSP
5-131
-------
Section No. 2.0.11
Revision No. 1
'Date October 1, 1984
Page 131 of 142
3. NATIONAL PERFORMANCE AUDIT PROGRAM (NPAP) AND ADDITIONAL
AUDITS
(a) Identify the individual with primary responsibility for the
required participation in the National Performance Audit
Program.
(i) For gaseous materials? (name, title)
(ii) For laboratory materials? (name, title)
(b) Does the agency currently have in place any contracts or
similar agreements either with another agency or outside
contractor to perform any of the audits required by 40 CFR
58? Yes No Comment
(i) If yes, has the agency included QA requirements with
this agreement? Yes No
(ii) Is the agency adequately familiar with their QA program?
Yes No
(c) Date last system audit was conducted?
By whom?
(d) Does the agency participate in the National Performance
Audit Program (NPAP) as required under 40 CFR 58
Appendix A?* Yes No
5-132
-------
Section No. 2.0.11
Revision No. 1
Date October I, 1984
Page 132 of 142
(e) Please complete the table below.
Parameter Audited
Date of Last NPAP
SO2 (continuous)
CO
Pb
ReF Device
S02 (bubbler)
N02 (bubbler)
*If no/give a brief summary of deficiencies.
5-133
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984.
Page 133 of 142
4. DOCUMENTATION AND DATA PROCESSING REVIEW
(aT Does the agency periodically review its record keeping
activities? Yes No
Please list below areas routinely covered by this review, the
date of the last review, and changes made as a direct result
of the review.
Area/Function Date of Review Changes? Discuss Changes
Y/N
Y/N
Y/N
Y/N
Y/N
(b) Are data audits (specific re-reductions of stripcharts or
similar activities) routinely performed for criteria
pollutant data reported by the agency? Yes No
If no, please explain.
(c) Are procedures for such data audits documented?
Yes No
5-134
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 134 of 142
(d) Axe they consistent with the recommendations of Sections
2.3-2.9 of Vol. II of the QA Handbook for Air Pollution
Measurement Systems? Yes No If no, why not?
(e) What is the frequency and level (as a percentage of data
processed) of these audits?
Poll. Audit Freg. Period of Data Audited % of Data Rechecked
(f) Identify the criteria for acceptable/not-acceptable results
from a data processing audit for each pollutant, as
appropriate.
Pollutant Acceptance Criteria Data Concentration Level
(g) Are procedures documented and implemented for corrective
actions based on results of data audits which fall outside
the established limits? Yes No If yes, where are
such corrective action procedures documented
5-135
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 135 of 142
CORRECTIVE ACTION SYSTEM
(a) Does the agency have a comprehensive Corrective Action
program in place and operational? Yes No
(b) Have the procedures been documented? Yes No As a part
of the agency QA Plan? Yes No As a separate Standard
Operating Procedure? Yes No Briefly describe it or
attach a copy.
(c) How is responsibility for implementing corrective actions on
the basis of audits, calibration problems, zero/span checks,
etc. assigned? Briefly discuss
(d) How does the agency follow-up on implemented corrective
actions?
5-136
-------
" Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 136 of 142
(e) Briefly describe two (2) recent examples of the ways in which
the above corrective action system was employed to remove a
problem area with
(i) Audit Results:
(ii) Data Management:
5-137
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 137 of 142
6. AUDIT RESULT ACCEPTANCE CRITERIA
(a) Has the agency established and has it documented criteria to
define agency-acceptable audit results? Please complete the
table below with the pollutant, monitor and acceptance
criteria.
Pollutant
Audit Result Acceptance Criteria
CO
°3
N02
so2
N02
so2
TSP
(continuous)
(bubblers)
(b) Were these audit criteria based on, or derived from, the
guidance found in Vol. II of the QA Handbook for Air
Pollution Measurement System, Section 2.0.12? Yes No
(i) If no, please explain.
(ii) If yes, please explain any changes or assumptions made
in the derivation.
5-138
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 138 of 142
(c) What corrective action may be taken if criteria are exceeded?
If possible, indicate two examples of corrective actions
taken within the period since the previous system audit
which are based directly on the criteria disucssed above?
Corrective Action fl
Corrective Action 12
(d) As a goal, the 95 percent probability limits for precision
(all pollutants) and TSP accuracy should be less than +15
percent. At 95 percent probability limits, the accuracy for
all other pollutants should be less than +20 percent.
Using a short narrative and a summary table, compare the
Reporting Organization's performance against these goals
over the last year. Explain any deviations.
NOTE: Precision and accuracy are based on Reporting
Organizations; therefore this question concerns the Reporting
Organizations which are the responsibility of the agency. A copy
of a computer printout has been provided which contains the
precision and accuracy data submited to EMSL for each of the
agency's Reporting Organizations. The printout, containing at
least the last four completed calendar quarters of precision and
accuracy data, was obtained using the NADB program NA273. This
data should be verified using agency records. If found in error,
please initiate corrections. Based on the data provided or
corrections thereto, complete the tables below indicating the
number of Reporting Organization's meeting the goal stated above
for each pollutant by quarter.
5-139
-------
(i) Precision Goals
f of Reporting
Pollutant Organizations
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 139 of 142
Precision •
Qtr/Yr Qtr/Yr Qtr/Yr Qtr/Yr
so2
CO
TSP
Pb
(ii) Accuracy Goals
Pollutant
# of Reporting
Organization
Accuracy
Qtr/Yr Qtr/Yr Qtr/Yr Qtr/Yr
N02
so2
CO
TSP
Pb
5-140
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 140 of 142
(e) To the extent possi"ble, describe problems preventing the
meeting of precision and accuracy goals.
5-141
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 141 of 142
11.8 Bibliography
Guideline documents for the SLAMS Air Program, arranged in
descending chronological order, the most recent ones first.
Reference
EPA-600/4-83-023
June 1983
EPA-600/7-81-010
Traceability
May 1981
EPA-QAMS-005/80
December 1980
EPA-600/4-80-030
June 1980
EPA-600/4-79-056
September 1979
EPA-600/4-79-057
September 1979
EPA-600/4-79-019
March 1979
EPA-450/4-79-007
February 1979
EPA-600/4-78-047
August 1978
EPA-450/2-78-037
July 1978
EPA-450/3-78-013
April 1978
EPA-450/3-77-018
December 1977
Report Ti tle
Guideline on the Meaning and Use of
Precision and Accuracy Data Required
by 40 CFR Part 58 Appendices A and B
A Procedure for Establishing
of Gas Mixtures to Certified National
Bureau of Standards SRMs
Interim Guidelines and Specifications
for Preparing Quality Assurance Project
Plans
Validation of Air Monitoring Data
Tranfer Standards for Calibration of Air
Monitoring Analyzers for Ozone
Technical Asssitance Document for the
Calibration of Ambient Ozone Monitors
Handbook for Analytical Quality Control
in Water and Wastewater Laboratories
Guidance for Selecting TSP Episode
Monitoring Methods
Investigation of Flow Rate Calibration
Procedures Associated with the High
Volume Method for Determination of
Suspended Particulates
Screening Procedures for Ambient Air
Quality Data
Site Selection for the Monitoring of
Photochemical Air Pollutants
Selecting Sites for Monitoring Total
Suspended Particulates
5-142
-------
Reference
EPA-600/4-77-027a
•Measurement
May 1977
EPA-450/3-77-013
April 1977
EPA-450/2-76-029
December 1976
EPA-450/2-76-005
April 1976
EPA-600/9-76-005
Measurement
March 1976
EPA-450/2-76-001
February 1976
EPA-450/3-75-077
September 1975
APTD-1132
March 1973
DU-6-1982
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 142 of 142
Report Title
QA Handbook for Air Pollution
Systems, Vol. II - Ambient Air Specific
Methods
Optimum Site Exposure Criteria for SO.
Monitoring
Aeros Manual Series, Vol. II - Aeros
User's Manual
Aeros Manual Series, Vol. V - Aeros
Manual of Codes
QA Handbook for Air Pollution
Systems, Vol. I — Principles
Aeros Manual Series, Vol. I - Aeros
Overview
Selecting Sites for Carbon Monoxide
Monitoring
Quality Control Practices in Processing
Air Pollution Samples
Federal Register for TSP/Hi Vol;
Background, QA, and Calculations.
5-143
-------
Section No. 2.0.11
Revision No. 1
Date October 1. 1984
Page 37 of 142
Example Response
to the'
NATIONAL AIR MONITORING AUDIT
QUESTIONNAIRE
Agency Department of Air Quality State of Union
Address 421 Union Street, Union City, Union
Telephone Number (Area Code) 100 Number 555-5555
Reporting Period (begining-ending dates)
Organization Director Harry Union
Air Program Supervisor Joe Union
Data Management Supervisor Herbert Union
Quality Assurance Officer James Union
Questionaire Completed November 16, 1984 Harry Union
(date)(by)
On-Site Visit
Date: 12/1/84 Audit Team Members: George Herbert, Clara
Some, Bert Single ....
Affiliation of Audit Team: USEPA. Region II
5-144
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 38 of 142
A. NETWORK DESIGN & SITING
1. NETWORK SIZE
(a) Complete the table below for each of the criteria pollutants
monitored as part of .your air monitoring network. Include only
those sites that are presently operating. Do not include
additional monitors which are collocated or index sites.
Number of Monitors
MAMS
SO 2
17
32
N02
2
9
CO 03 TSP
4 16 37
15 30 61
Pb
4
18
SLAMS
(excluding NAMS)
5__ 0 7 6 10 13_
SPM
TOTAL 54 11 26 52 108 35
(b) SLAMS Network Description
1. What is the date of the most current official SLAMS Network
Description? 3/18/83
2. Where is it available for public inspection? DAQ Office
421 Union Street, Union City
5-145
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 39 of 142
3. Does it include for each site the following?
yes no
SAROAD site ID # X
Location X
Sampling & analysis method X
Operating schedule X
Monitoring objective and scale
of representativeness X
Any proposed changes X
(c) For each of the criteria pollutants, how many modifications (SLAMS
including MAMS) have been made since the last system audit? (List
the total SLAMS and NAMS). Date of last audit
Pollutant Added
Sulfur Dioxide 3
Nitrogen Dioxide 0
Carbon Monoxide 2
Ozone 4
Total Suspended
Particulates 15
Lead
Number of Monitors
Del eted
0
1
1
1
0
0
Relocated
4
0
1
3
2
0
(d) Briefly discuss changes to the Air Monitoring Network planned
for the next audit period.
Five new N0£ sites are planned, 1 in Unionville, 1 in Union Town, and 3
in the Northeastern portion of Union City.
5-146
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 40 of 142
2. NETWORK DESIGN AMD SITING
Indicate by SAROAD Number any non-conformance with the requirements
of 49 CFR 58, Appendices D and E.
Monitor
Site ID
(SAROAD)
Reason for Non-Conformance
SO 2
720100016F01
probe height of 72 meters
720111011F01
located too close to major
highway
CO
none
none
TSP
Pb
720120010F01
site is a population oriented
category (b) site whereas a category
(a) high concentration area site
is needed
5-147
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 41 of 142
3. NETWORK REVIEW
Please provide the following information on your previous internal
Network Review required by 40 CFR 58.20d.
Review performed on: date 2/18/84
Performed by Department of A.Q. Planning Office _
Location and Title of Review Document State of Union SLAMS Network Review *
Briefly discuss all problems uncovered by this review.
This review indicates that 5 new N0£ monitors need to be located in the
State and that the Union Town TSP monitoring network should be reevaluated no
later than 7/1/85 to consider monitoring location changes based on population
and industrial shifts in the area.
5-148
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 42 of 142
4. NON-CRITERIA POLLUTANTS
Does your agency monitor and/or analyze for non-criteria air and/or .
toxic air pollutants? Yes X No
If yes, please complete the form below.
Pollutant
HC
H2S
BXP
Monitoring
Method/I nstrument
Instrumental Flame lonization
Instrumental Flame Photometric
Hi-Yol Thin Layer Chromatography
SOP Available
Yes/No
Yes
Yes
No (under
development)
5-149
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 43 of 142
B. RESOURCES AND FACILITIES
1. INSTRUMENTS AND METHODS
(a) Please complete the table below to indicate which analyzers dp not
conform with the requirements of 40 CFR 53 for NAMS, SLAMS or SIP
related SPM's.
Pollutant
CO
SOa
N02
03
TSP
Number
0
1
0
0
1
Site Comment on
Make/Model Identification Variances
Philips 720111009F02 Not using approved
PVI9700 range
Telodyne 720111010F03 Shelter does not
conform-to current
required dimensions
Pb
(b) Please comment briefly on your currently identified instrument needs.
Need to replace 1 03 monitor and 3 S02 instruments, as all four instruments
are 14 years old and past their useful life.
5-150
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 44 of 142
2. STAFF AND FACILITIES
(a) Either indicate the number of people available to each of the following
program areas, or attach a copy of the agency's organizational chart.
Program Area
Number
Comment on need for
Additional Personnel
Network Design
and Siting
none
Resources and
Facilities
15
need one person for instrument
repair shop
Data and Data
Management
need one additional data clerk
QA/QC
need 2 additional people to keep
up with performance audits
(b) Comment on your agency's needs for additional physical space
(laboratoy, office, storage, etc.).
5-151
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 45 of 142
3. LABORATORY OPERATION AND FACILITIES
(a) Is the documentation of Laboratory Standard Operating Procedures
complete? Yes X No .
Please complete the table below.
Analysis
Date of Last Revision
Gravimetric
Atomic Absorption
Colorimetric
Specific Ion Electrode
N/A (no bubblers used)
(bubblers)
TGS method frit
Others (list by pollutant)
None
TSP
Pb
504
N03
S02
3/81
9/81, in process of revision.
12/82
7/81
10/82
5-152
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 46 of 142
STANDARDS AND TRACEABILITY
A. Please complete the table for your agency's laboratory standards.
Parameter
CO
N02
S02
03
Weights
Temperature
Moisture
Barometric
Pressure
Flow
Primary Recertifi cation
Standard Date
SRM Mo. ABS
CRN No. DFE 7/83
SRM Perm Tube
SRM
CRM
Type or Class of
Weights
NBS Thermometer
Chilled
Mi rror
Dewpoint Detector
Mercury
Barometer
We Test Meter
Dry Test Meter
Laminar Flow Elements
Bubble Flow Meters
Roots Meter
Other
Secondary Recertifi cation
Standard Date
Airco Gas
Perm Tube
Cylinder of No & GPT
•
S/N
03 Analyzer Converted
to Photometer
ASTM Thermometer
Sling
Psychrometer
Electronic
Barometer
Electronic
Flow Meters
Rotameters
Orifices
Other
Lead
AA or ICP standards
Fisher or other manufacturer standard
5-153
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 47 of 142
4. STANDARDS AND TRACEABILITY
B. Please complete the table below for your agency's site standards
at each site audited (up to 7% of the sites; however, not to exceed 20 sites.)
This could include cylinder gases, permeation tubes, uv photometers, etc.
PrimarySecondaryRecertification
Parameter Standard Standard Date
CO cylinder #2793 1/15/84
N02
SO 2
03
5-154
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 48 of 142
C. DATA AND DATA MANAGEMENT
1. TIMELINESS OF NAMS DATA.
For the current calendar year or portion thereof which ended at least
135 calendar days prior to the receipt of this questionnaire, please
provide the following percentages for required data submitted.
% Submitted
on Time*
502
92.6
CO 03
N02
TSP
90.0
Pb
The fraction of data listed as submitted late should be based on
that data which are submitted more than 45 days after the close of the
90 day reporting requirement or 135 days after the end of the quarter
in which the data were collected. For example purposes, assume that an
audit was performed on November 14, 1984 which was 137 days after the
end of the second quarter of calendar year 1984. The fraction of the
data to be counted late would be the number of observations not reported
to the National Air Data Bank (including missing observations) divided
by the total number of possible observations for all sites. The missing
observations and total observations may only be adjusted due to approved
seasonal operation and for sites which were started or discontinued
during a quarter. Continuous and non-continuous data should be calcu-
lated separately. For example, assume your NAMS network contains 10
continuous SOg monitors and lO.Hi-Vols. Since the audit was conducted o"n
November 14, 1983, the late or missing data of interest (more than 45
days late) would be data from the 1st and 2nd quarters of 1984. There-
fore, the nunber of possible observations for the continuous instruments
would be 181 days x 24 hours per day x 10 sites or 43440 observations.
Based on NADB printouts or your own agency records, you could determine
the number of missing or late observations. Suppose that 3210 observa-
tions had not been submitted and there were no data adjustments. The
fraction of late data would be 3210 divided by 43440 or 7.4 percent.
Similarly, there should be 300 Hi-Vol observations during the first and
second quarter from a 10 site network operating on a 1 in 6 day schedule.
If 30 observations had not been submitted by the audit date then 30
divided by 300 would mean 10% of the data were late.
Late - 100 percent = percent on time
CONTINUOUS S02 ANALYZERS 7.4% LATE - 100 = 92.6
TSP 10% LATE -100 = 90
Other pollutants should be calculated similarly. A copy of NADB NA288
has been provided to aid in this calculation. See example of NA 288
next 2 pages.
5-155
-------
NAHS CCMPLFTCr'FSS REPORT
Example of NA 288
s .'
.
"a A'MlKUSf POLL MTH
" "fitl*f igf^fff^i^fm^mti T 5 P ° 1
"rTRr^jTONE''HTGHrCMOOL 311 HA'4 P AH
"u EAST HITH SCHOOL ftU 0"ITTAIfl °D
** *^09 S FROAJUAYt SUBMIT (*fiiiMrv r.
1 »
" ?09 S. PROAOUAY. SUMMIT COUMY H
1 •• -
" FAST H I f* H c OH 00 L 60 PRITTAiy "^
1 ". _T
" EAST HIHH SCHOOL 80 BRITTAIM RO
1 »
* iz,.. i M cy ?i
" 800 PATTERSON AVE'.
» '
" «fl9 U. HOPACAN
k •'
» " _
" 2POO 13TH STRFLT <\A SOURET JIJ
b —
° M«O^C YOUT7 SCHOOL 1V11 NIH'^Y f.'
P *•
" i -»70 r APT Trip ri*
M
FRPfl
JAN
A
170
A
1 ?0
725
97
79T
97
704
95
X
731
QR
Ad 7
82
6
170
99
nn
120
657
n A
1?0
r
J A-l
? J
S
01
FEP
4
00
ft
80
669
9 A
9
99
687
99
X
688
94
82
14
Afll
98
4
no
60
691
OQ
80
M
1ES REPORTING DATA
TATE : ••»
/01/P4 1HRU 03/31/84
NUMOFR OF OBSERVATIONS 7-DAY GAP INDICATOR IXI
p£Rff;(ii1
MAR APR HAY JUNE JULY AUG SEPT OCT NOV DEC
5
. .,100
120 .
735
' 99
7,s
97
X
715
99
733
99
7T7
99
inn
T™
98
6
120
K.
100
658
BH
120
r
i (in
.
« OF
7f-D«Y
YEAR GAPS
15
100
16
107
2129
97
2135
98
1391 4
64
735 B
34
2152 S
99 •—
if)
1912
88
15
100
2146
98
14
93
14
93
2002
92
16
107
14
93
-------
Example of
TROM Jl/01/fl"4 THRU 03/31/fl«
_IO-IAL
T£P PB CO S02 N02 03
cr sues
_L5_
.2 LL
_5D_
MJ»-kTP CF 5ITt? ^EE
c nitM jo v ro TTr tf T
26
11
51
rfprrMT r.g «.nr<;
100 100 50 7J 100 81
67
CRTTFRI*
CF «HFS WITH AT
mcT i 7-nnv n tp
rr 7-n«v
_n n_
RUHIFR CF riTE! ».01
HAT*
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 49 of 142
2. DATA REVIEW
What percent of the SLAMS sites (by pollutant) reported less
than 75% of the data (adjusted for seasonal monitoring and
site start-ups and terminations)?
Percent of Sites
Pollutant <75% Data Recovery
1st 2nd 3rd 4th
Quarter Quarter Quarter Quarter
Ozone
* 5 10 7**
Nitrogen Dioxide
Sulfur Dioxide
Carbon Monoxide
Total Suspended
Particulates
lead
* None required to operate this quarter.
** Based on 1 month required operating time.
Response
Similar to Question #1 of this section, we will assume that an audit
occurred on November 14, 1984, which means that we are dealing with a partial
year of data. In response to this question, provide an answer for each
pollutant for each quarter. The percentage of data recovery is the hourly
data available for the quarter divided by the number of hours in a quarter
for all continuous instruments except ozone. For ozone, calculate data
5-158
-------
recovery as the percent of valid days of data recovered divided by the number
of days during the quarter. To obtain the percent data recovery for Hi-
Yols or bubblers, the number of 24 hour samples is divided by the number of
samples that should have been sampled during the quarter (15 or 16). For
those states which store their SLAMS data in SAROAD, an NADB program (NA288)
will produce a printout by site which provides the percentage of possiK1e
observations received for each quarter by site. Any state that does not
store their SLAMS data in SARAOD will have to use their own records to
establish the percentage of sites with less than 75% data submission. The
percentage of sites with less than 75% data submission is the number sites
with <75% divided by the total number of sites for a particular pollutant.
* Suppose there are 21 ozone monitors in the state, only 3 of which are
operated year-round. Nominally, the ozone season for this state occurs
during the 2nd and 3rd quarters. Therefore, the fraction of monitors with
<75% data recovery for the 1st and 4th quarters is based only on the number
of monitors scheduled to operate during that period which is 3. During the
2nd and 3rd quarter, all 21 monitors were scheduled to operate. Therefore,
the calculation is based on 21 monitors.
(b) Please provide an explanation if more than 20 percent of the sites
have a data recovery rate less than 75 percent.
5-159
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 50 of 142
3. DATA CORRECTION
(a) Are changes to submitted data documented in a permanent file? Yes No X
If not, why not not enough staff to perform paperwork
(b) Are changes performed according to a documented Standard Operating
Procedure or your Agency Quality Assurance Project Plan? Yes X No_
If not according to the QA Project Plan, please attach a copy of your
current Standard Operating Procedure.
(c) Who has signature authority for approving corrections?
John Doe Data Processing Section .
(name)(Program Function)
5-160
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 51 of 142
4. ANNUAL SLAMS SUMMARY REPORT
(a) Please provide the dates annual reports have been
submitted to the Regional office in the last two years.
6/9/83. 7/5/84
(b) Does the agency's annual report (as required in
40 CFR 58.26), include the following:
Yes No
1. Data summary required in Appendix F. X
2. Annual precision and accuracy information described in X
Section 5.2 of Appendix A.
3. Location, date, pollution source and duration of all X
episodes reaching the significant harm level.
4. Certification by a senior office in the State or his X
designee.
(c). Describe any deficiencies which cause the answer to part (b) of this
question to be NO.
5-161
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 52 of 142
D. QUALITY ASSURANCE/QUALITY CONTROL
1. STATUS OF QUALITY ASSURANCE PROGRAM*
(a) Does the agency have an EPA approved quality assurance program
plan? Yes X No .
If yes, have all changes to the plan been approved by EPA?
Yes No X
Please provide:
Date of original plan approval 1/2/82
Date of last revision 6/13/84
Date of latest approval 7/I/83
(b) Do you have any revisions to your QA program plan still pending?
v—- V ll —
Yes X No
5-162
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 53 of 142
1/9/83
2. AUDIT PARTICIPATION
(a) Date last system audit was conducted?
By whom? EPA Region II
(b) Does the agency participate in the National Performance Audit
Program (NPAP) as required under 40 CFR 58, Appendix A?*
Yes X No
(c) Please complete the table below.
Parameter Audited
S02 (continuous)
CO
Pb
ReF Device
S02 (bubbler)
(bubbler)
Date of Last NPAP
3/13/84
4/12/83
11/17/83
5/17/84
None used
5/21/82
5-163
-------
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 54 of 142
3. PRECISION AND ACCURACY GOALS
As a goal, the 95 percent probability limits for precision
(all pollutants) and TSP accuracy should be less than +15
percent. At 95 percent probability limits, the accuracy for all
other pollutants should be less than +20 percent. Using a short
narrative and a summary table, compare the Reporting
Organization's performance against these goals over the last
year. Explain any deviations.
Precision and accuracy are based on Reporting Organizations;
therefore this question concerns the Reporting Organizations
which are the responsibility of the agency. A copy of a computer
printout has been provided which contains the precision and
accuracy data submitted to EMSL for each of the agency's
Reporting Organizations. The printout, containing at least the
last four completed calendar quarters of precision and accuracy
data, was obtained using the NADB program NA273. This data
should be verified using agency records. If found in error,
please initiate corrections. Based on the data provided or
corrections thereto, complete the table in part "a" below
indicating the number of Reporting Organizations meeting the
goals stated above for each pollutant by quarter.
5-164
-------
a) Precision and Accuracy goals
# of Reporting
Section No. 2.0.11
Revision No. 1
Date October 1, 1984
Page 55 of 142
Precision
Pollutant
03
N02
S02
CO
TSP
Pb
(b) Accuracy
Pollutant
03
N02
S02
CO
TSP
Pb
Organization
2*
3
2
2
3
1*
Goals
# of Reporting
Organization
2*
3
2
2
3
L*
Qtr/Yr
3/83
1
2
1
2
2
1
Qtr/Yr
4/83
1
2
1
2
3
1
Qtr/Yr
1/84
0
2
1
2
2
1
Qtr/Yr
2/84
0
2
1
2
1
1
Accuracy
Qtr/Yr
3/83
2
3
2
1
3
0
Qtr/Yr
4/83
2
3
2
2
3
1
Qtr/Yr
1/84
2
2
2
2
3
1
Qtr/Yr
2/84
2
3
2
2
3
1
* No P&A data available for 1984 therefore, table represents 1-4 qtrs, of 1983
for ozone and lead.
_c) To the extent possible, describe problems preventing the meeting of
precision and accuracy goals.
5-165
-------
AUTONAKD
NATIONAl AEROMFTRIC DATA BANK
FMIPONMFMAL PPOKCTJON ACENCY
SA*OAO/PRECIS I ON-AC CURACY REPORT
PAGE: i
OCT 19. 1581)
NA273/NAPOOO
prcric
.*••»»*#
KG. ST. SO TVpJ £
DAT* HTY
OOJ C <4'101 83-1
Kei\lO»TUF »» P3-1
fl3-?
R2-4
flM-1
B4-2
8M-5
U01
SUIFI'R DI
83-1
•** 83-2
P3-3
S3-4
fl?-5
Ul
I
01
NITRnGfN OIOXIflE *
81-5
83-J
83-2
83-3
fl3-«
fl?-5
81-1
C qnL'ni 83-i
0?Of'L »*•*»*» )>3-?
83-5
PRfCTSTON 0 » T A
« *«* •***«*« »*»«****»»****»*
P fF HKftT5 PPOF LI*
ANLY2RS CHCCKi LO UP
010C
C102
on an
007P
0366
rnin
U092
Cl?''
JC9
any
uCb
008
U08
octi
021
0?1
O?!
0?1
021
UC4
00<»
or n
UC4
on*
UQ4
004
0?9
029
ACCURACY DATA
*•****»******»*•*
PROB
LO-L4-UP
»•*»••«
s oupcr
AUO GAS
F
F
F
F
F
F
F
F
F
F
F
F
F
F
F
F
F
F
F
0
0
0
•*•****
TRACE
ABLTY
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
»+*»*****»»•**»*»«»»*»»••»*»»
II AUUITS PROB LIM PROB
Ll-3 IH
002
003
002
002
0009
002
002
0004
005
006
005
006
0022
005
006
0011
002
001
002
002
0007
002
002
0004
008
007
006
009
0030
000
0000
0000
000
0000
0000
000
0000
0000
000
0000
LO-L1-UP
-07
-11
-01
-09
-07
-06
-08
-08
-12
-08
-08
-09
-09
-13
-10
-12
-13
-08
-06
-09
-06
-04
-05
-10
-08
-V>
-19
-II
-04
«12
*02
*09
*05
*13
-07
«03
*ll
*09
+ 15
*07
+ 11
+09
+ 14
+ 12
+ 09
+04
+09
+07
-00
+02
+01
+08
+03
+ 10
+ 09
+08
•*»**«
LIH
LO-L2-UP
-10
-02
-OH
-14
-08
-08
-07
-08
-08
-08
-02
-06
-06
-15
-07
-11
-01
-01
-00
-01
/•-F^
^•01
-u
-u
-08
-07
-13
-10
+ 01
+06
+ 07
+ 11
+ 06
+ 12
-01
• 06
+ 07
+ 12
+ 14
+ 10
+11
+ 11
+ 10
+ 11
-00
+ 01
+ 03
+01
+ 15
+07
+ 11
+ 03
+01
• 06
+05
+ 04
i *****
PROb
»***'
LIH
LO-L3-UP
-06
-01
-00
-15
-06
-07
-06
-07
-09
-10
-03
-05
-07
-12
-05
-09
-00
-00
+ 01
+ 00
-08
+ 01
-04
-12
-10
-08
-13
-11
-02
+03
+ 02
+08
• 03
+ 10
+ 04
+07
+ 06
• 11
+ 15
+ 12
+ 11
• 12
+07
+ 10
+02
+01
+ 03
+02
+ 09
+02
+ 06
• 03
+03
• 05
+04
+ 04
-------
PAMIAI ff Ti:."l;r
NM?OM».L fEFOKCTMC C»TA HANK
rNVIKOfoP.FWT AL PROTfCTlON AGENCY
5 AHO/iD/FRfUS 10H-ACCIIRACV REPORT
PAGE ?
OCT 19i 1984
NA273/NAPOOO
°ncr fion
««,«**,.«,..».,
RF ST pp TYP
;-' /ICY 'UTA MY
Yt<-C
fir
TON
DAT*
ccunr pror LIU
STTT« LO UP
CCLL f»«p
FCLOU LIH
VAL COLL
DATA
ACCURACY DATA
»»t»»»*»«**»*******»»*****»****«*»***»»»*
K AUDITS PRCe'lin PROB LIM PROB LIH
LE« 1-3 LO-Ll-UP LO-L2-UP LO-L3-UP
oOl I 111P1 »?-!
«««* PAr'TTCOLArC *•»• R3-?
"3-3
R3-4
\
HU-1
PH-2
fil-0
75
75
75
75
7h
75
76
75
0
0
0
0
0
0
0
0
54
56
5Z
54
216
47
47
94
019
OU
015
019
0069
021
015
0036
-07 *05
-OB *02
-05 +08
-07 *08
-07 +06
-09 +10
-OB +06
-09 +08
U1
Ol
-------
MAMIAl
V-ACCORACY DATA KKY
******•******»*****•»•»*•**
RH ST RO IYP ''DLL Y9-Q
N»l!Of.'/.L tEPOHETRTC CATA PANK
rNVIROr.rCHJl AL PROTFCTION AGENCY
f,».«iO*D/F-Rl CIS ION-ACCURACY RLPORT
ftl-
P.M-5
002 I 1?128 83-1
******* LTAO «•*«»•»* fl"«-2
«3-3
cn
III
00
t»
002 I 2
*• NITHOGFN OTOXint *
R3-1
"3-2
P3-3
R3-1
931-5
ftM-1
rrr cistoh fj A T A
*******1**f**************************************
H PF CC1.LOC Pf'OH LIH CCLL S»»P VAL COLL
SAM^LRS SITFS LO UP PELOU LIH DATA PRS
1 7
17
1 7
17
17
17
17
17
17
17
17
17
17
9
9
9
9
9
9
9
9
?
?
2
-12
-13
-Oq
-1 3
-13
-06
-10
+ 25
+ 13
+ 1 3
+ 11
+ 16
+ 05
+ 11
+ 08
-08 +10
-10 +06
-06 *05
-10 +07
-09 +07
0
0
0
0
n
0
0
0
0
0
0
2
2
0
0
2
0
2
0
1
1
30
27
30
26
113
26
27
55
15
15
15
15
60
28
29
30
27
114
28
25
53
A
#0$^^#^^* 4
fl AUDITS
LEV 1-3
005
003
003
004
0015
on
015
0026
003
003
003
003
0012
007
006
006
006
0025
005
005
0010
ecu
t $ $ 9t ^ 1
PRCB
R A
h * A tt 4
LIM
LO-L1-UP
-IT (
-05
+ 01
-03
-06
-03
-16
-02
-0«
-ot
-on
-01
-03
gg)
«06
+ 06
«12
+ 13
+02
*07
+03
4>OI|
«04
+ 06
*05
• 06
C V
k * A A A ^ t
PROP
OAT
* A A A A ^
LIN
LO-L2-UP
-06
-29
-06
-08
-12
-13
-10
-12
+ 07
-02
«03
-02
«02
-04
-16
-01
-01
-06
-02
+ 01
-01
+ 11
+ 19
+ 00
+ 11
+ 10
+ 10
-09
+ 01
+ 12
+ 02
+ 06
+ 03
• 06
+ 05
+ 06
+ 02
+ 03
+ 04
+ 02
+04
+ 03
PAGE 3
OCT 19t 1981
N*273/NAPOOO
A
k ft A £ £ A ft 4 ^ *
PROB LIH
LO-L3-UP
-08 +09
-16 +05
-02 +03
-01 +01
-07 +05
-01 +01
+06 +08
+03 +05
-------
NATIONAL *F_ROMETPIC DATA flANK
FNVIKONKFMAL PROTECTION APENCY
SA^OAD/PRECISION^ACCURACV REPORT
PAGE 4
OCT lit 116H
NA273/NAPOOO
T S T 0 N 0»TA
*4**»********»*****»
Pf-fCTT PPOF: LIH
S CHECKS LO UP
ACCURACY DATA
«•»»»»»**»»*»»***»»»*»*»•»****»«*»•*»»*•**•*»*•*
f CURCF TRACE tt AUDITS PROB tlH PROB LIH
AU 0 GAS ABLTY Ll-3 L4 LO-L1-UP LO-L2-UP
I*****
PROB LIH PROB LIH
LO-L3-UP LO-L4-UP
83-1
83-i
83-3
63-4
13-5
84-1
R4-2
84-5
B3-1
R3-i'
H3-3
83-4
83-5
84-1
84-2
84-5
83-1
83-2
83-3
8>3-4
83-5
8M-1
R4-2
P4-5
P3-1
83-2
83-3
83-4
63-5
005
DPS
005
OT5
i)T5
OC4
005
OC5
009
OC9
oo a
nob
0" 9
007
009
006
007
UC7
017
007
007
007
007
OC7
OC6
007
008
017
OC7
UT128
CC24
00?8
CC27
0107
f 0?t
0060
C06A
0048
CP40
0040
CC4?
0170
C044
0106
C150
0033
CC37
0033
0031
0134
C03?
0073
C105
CC27
0034
OP37
0027
C125
-1 3
-11
-12
-re
-i i
-05
-04
-05
-t 4
-25
XT<£>
T^v.
•11
/^Tp
^n
-?0
-?3
£5£z
-ce <
-17
-17
-10
-14
(ffi
^tl
-J2
-13
-14
+08
»07
«08
+ 08
+08
+ 08
+ 11
+ 10
+11
+ 19
+ 13
) +12
+ 19
+ 15
>»12
c^j5Z
£23?
+11
*ifl
»13
+ 17
\ B
P
B
P
e
B
B
e
B
P
E
B
.B
B
B
e
k
B
0
0
0
0
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
003 000
002
004
002
0011 0000
001
002
0003 0000
004 000
003
004
002
0013 0000
002
003
0005 0000
002 000
003
Oi)3
002
0010 0000
002
002
0004 0000
002 000
002 ^
007
002
0013 0000
-15
-10
(5fr>
^=r\
-15
-03
-03
-08
-27
-15
-14
-16
-11
-11
-11
-15
-09
-07
-06
-09
-12
-il
-12
-00
-05
-12
-14
-08
+ 20
-00
+05
+ 07
+08
+ 10
+ 10
• 01
+03
+ 10
+ 17
+08
*01
•09
•05
+07
*11
*07
*13
+ 10
+06
+08
+07
-00
+06
• 19
*13
+ 10
-05
-07
-IP
-' K
-09
-02
-02
-09
-08
-17
-12
-12
-06
-10
-08
-01
-07
-10
-02
-05
-11
-05
-08
-08
-08
-08
-15
-10
• 10
-01
+ 00
+ 14
• 06
• 06
+06
• 02
-06
• 10
• 13
• 05
+01
• 10
+06
• 01
+ 12
• 01
+03
• 04
•04
• 03
«04
• 03
• 05
+ 10
• 01
+05
-02
-07
-07
-14
-08
-01
-01
-08
-14
-16
-19
-14
-06
-11
-09
-01
-11
-09
-00
-05
-05
-06
-06
-11
-oa
-07
-19
-12
• 05
-03
• 01
• 14
• 04
• 05
+05
-01
-02
• 05
• 16
• 05
+03
+ 10
•07
+ 01
+ 13
• 04
•01
• 05
-02
• 02
+00
+02
+ 03
•06
+ 01
+03
-------
MAM'AI
fJMICtJ/.L tf^OKE^^: TATA
rNVIRONHEMT AL P^^CTION AGENCY
5*ro«ri/^t
-------
INSPECTION
AND MAINTENANCE
-------
6. MOTOR VEHICLE EMISSIONS INSPECTION PROGRAMS
Table of Contents
Section Title Page
1.0 INTRODUCTION AND PURPOSE ' 6-2
2.0 AUDIT TOPICS 6-3
2.1 Program Design 6-4
2.2 Program Operations 6-4
2.3 Economy and Efficiency 6-5
2.4 ' Program Effectiveness 5-6
3.0 AUDIT PROCESS FOR I/M PROGRAMS 6-7
3.1 Advance Preparation 6-8
3.1.1 Documentation Assembly 5-8
3.1.2 Program Questionnaire 6-10
3.1.3 Notice to Program Officials 6-10
3.2 On-Site Audit Visit . 6-11
3.2.1 Interviews • 6-13
3.2.2 Records Review 6-14
3.2.3 Inspection Station Visits 6-15
3.2.4 Special Surveys 6-17
3.3 Audit Report 6-18
3.4 Follow-up Activities 6-18
APPENDICES
A. Description of I/M ..Program Elements 6-20
B. Insepction/Maintenance Program Audit Questionnaire 6-30
C. Description of On-Site Audit Activities 6-57
D. Procedures Used in QMS Tampering Surveys 6-66
E. Instructions and Forms for Audit Activities 6-85
F. Checklist for Completing the I/M Audit 6-125
6-1
-------
Guidelines for Auditing Motor Vehicle
Emissions Inspection Programs - FY1985
1.0 INTRODUCTION AND PURPOSE
Motor vehicle emissions inspection/maintenance (I/M) programs
are currently required in over 30 States or localities under
Part D of the Clean Air Act. At present, 26 I/M programs are
in operation across the country affecting over 40 million
vehicles. With only a single exception, these programs all
measure the tailpipe emissions concentrations of subject
vehicles; many of them also inspect for evidence of emission
control system tampering or misfueling. In order to ensure
that these programs are administered and operated properly
and that they achieve the desired air quality benefits, the
responsible State and local agencies must continually monitor
and evaluate their respective programs. In addition, EPA has
a regulatory responsibility to review these programs in each
State for conformance to policy and (State Implementation
Plan) SIP commitments.
A major part of EPA's efforts to evaluate I/M programs will
be accomplished through periodic program audits (or program
evaluations). The primary purpose of this report is to
present I/M auditing guidelines for use by EPA personnel
involved in these in-depth audits. Both emissions
inspections and tampering/misfueling inspection elements are
covered by the guidelines. The auditing guidelines are
designed to ensure that EPA's I/M audits in the various
States are both comprehensive and consistent. The secondary,
but equally important, purpose of this report is to provide
additional information to State and local agencies which
would be useful in their internal I/M audits or program
evaluations.
The primary purpose of the I/M audit is to allow EPA to
ensure that each State or locality is implementing and
enforcing its I/M program in a manner consistent with its
SIP. This objective must be satisfied within the audit,
either during the audit visit or as .part of audit follow-up
activities. As a result of the audit, both EPA and the State
or local agencies involved will be able to determine what, if
any, program improvements may be required to allow SIP goals
and commitments to be met.
A secondary purpose of the I/M audit is to evaluate the
overall effectiveness and efficiency of each I/M program.
Because EPA reviews the design and operation of I/M programs
across the country, EPA has the opportunity to observe the
6-2
-------
strengths and successes of the various programs. EPA has
historically tried to serve as a clearinghouse for the
transfer of I/M information among States and localities in
order to allow each State/locality to learn from others'
experiences. As a result of the audit, EPA may be able to
suggest administrative or technical modifications which would
increase program effectiveness, program efficiency or both.
2.0 AUDIT TOPICS
The topics reviewed during the audit include, at a minimum,
all design and operational aspects of the I/M program which
affect the program's ability to meet legal requirements of
the SIP and the Clean Air Act. A certain uniform and
reasonably achievable degree of program effectiveness in
reducing vehicle emissions is required for each I/M program,
and each approved SIP contains a program design which EPA
determined would achieve at least that degree of
effectiveness. The starting point for the audit is to relate
the design of the program in the SIP to its actual operation
in order to evaluate whether applicable laws,
rules/regulations, etc. are being administered properly.
Evaluation of the actual program operation must include:
1. Whether the program is being adequately enforced, i.e.,
whether subject vehicles are being inspected.
2. Whether inspection standards are adequate. (This
applies in cases in which the approved SIP does not
contain specific standards, but does state that the
inspection program will operate with a specific failure
rate, stringency factor, or other characteristic which
depends on standards.) j
3. Whether vehicles are being inspected properly, i.e.,
according to established procedures, using the proper
cutpoints, etc.
4. Whether vehicles identified for repair are being
repaired effectively.
The audit may also consider factors relating to program
economy and efficiency. In many cases, the costs of an I/M
program can be at least partially offset by cost savings
associated with better fuel economy and other side benefits
of proper maintenance. Another cost consideration relates to
how efficiently repairs are being provided for failed
vehicles. Time lost by vehicle owners in complying with'
program requirements is a cost which should be minimized.
Also, ensuring that warranty coverage is provided and that
owners are made aware of their warranty rights can help to
reduce the costs of I/M repairs to consumers.
6-3
-------
Finally, the audit may address opportunities for improvements
in program effectiveness beyond that required in the SIP.
Program effectiveness is a function of both program design
and program operations. As stated above, the I/M audit will
be one means for EPA to determine whether each I/M program
achieves the level of effectiveness implied by the SIP
design. In some cases, the I/M audit may also identify ways
in which increased effectiveness beyond that anticipated in
the .SIP could be obtained through minor design or operational
changes.
The discussions which follow are intended to explain the need
for each part of the audit process described below in Section
3.0. The discussions also give some indications of the
considerations EPA will use in evaluating I/M programs.
2.1 Program Design
Program design parameters which should be reviewed during an
audit include inspection test procedures, emission standards
(cutpoints), inspection station licensing requirements,
analyzer specifications and maintenance/calibration
requirements, quality control procedures, audit/surveillance
procedures, internal control systems (quality assurance),
enforcement procedures, vehicle coverage considerations,
waiver procedures, consumer assistance and protection, and
mechanics training. Each of these elements contributes to
overall program effectiveness. Also, many of them determine
the availability of the emissions performance warranty in a
particular State or locality. Detailed descriptions of each
I/M design element are found in Appendix A.
2.2 Program Operations
One of the critical parts of the program audit is to
determine whether the various elements of the I/M program as
designed (and approved in the SIP) are actually being carried
out in the program. Field observations are necessary to
determine whether:
1. Vehicles are being rested properly and the results
are being reported correctly.
2. Emission standards are being properly applied.
3. Licensing requirements are being met.
4. Proper analyzers are being used.
6-4
-------
5. Analyzers are being calibrated and maintained.
6. Quality control procedures are being followed.
7. Inspection and other records are being kept properly.
8. Needed data analyses are being done and used
effectively.
/
9. Inspection stations are receiving proper
surveillance and supervision.
10. Waivers are being processed properly.
11. Owners of non-complying vehicles are being
identified and prosecuted according to procedures
outlined in the SIP.
12. Failed vehicles are being repaired effectively.
13. Consumer assistance and protection provisions are
being administered properly.
14. Mechanics training is being conducted appropriately.
2.3 Economy and Efficiency
There are several aspects of I/M programs which not only
contribute to effectiveness but which also affect program
economy and efficiency.
The primary area with economy/efficiency influences is the
quality of I/M repairs. To help ensure proper, cost-effective
I/M repairs, I/M programs should direct efforts at both vehicle
owners and mechanics. In order to protect themselves as
consumers, owners of failed vehicles need help in identifying
the kinds of repairs they need, what these repairs should cost,
the normal range of emissions levels expected after proper
repair, and their applicable warranty and waiver rights.
The I/M program must also direct efforts at the repair industry
in order to maximize the quality and minimize the costs of I/M
repairs. Mechanics training programs are a common part of many
I/M programs. However, usually only a small fraction of
practicing mechanics ever participate in the training program.
Therefore, to the extent possible, efforts should be directed
at those mechanics who do not go through training to give them
diagnostic information and information about who or where to
call to get more assistance. State/local agencies can also
help to improve I/M repairs by having a system for monitoring
repair costs and waiver rates at repair facilities.
6-5
-------
In some cases, some false ideas about economies or efficiencies
in I/M programs have evolved. For instance, there are many
State/local programs which have not adopted the quality control
or other procedures needed to allow owners to enforce their
emission performance warranty rights. Doing so might mean, in
some cases, spending slightly more funds for administering the
I/M program, but it would eliminate much of the consumer
expense for I/M repairs on 1981 and newer vehicles. Therefore,
providing warranty coverage is a cost efficient practice.
Other activities which can have benefits which exceed their
costs are:
L. Repair brochures for owners.
2. Mechanics training.
3. Repair information (manuals) for mechanics.
4. Hotlines for consumers and mechanics.
5. Newsletters for mechanics.
While each of these activities can be designed to be an
effective tool, they must be reviewed carefully to make sure
that they are serving the intended purpose.
2.4 Program Effectiveness
Perhaps the most important purpose of the program audit is to
assess overall program effectiveness.
The ultimate test of I/M program effectiveness is the air
quality benefit of the program. Unfortunately, the air quality
issue is much too complex to 'address in the program audit. The
program audit must instead deal with whether the program is
being managed effectively to produce the desired emissions
reductions, In other words, the audit should determine whether
adequate steps are being taken to enforce the program and to
ensure that vehicles are being inspected properly and
effectively repaired.
EPA policy interprets the Clean Air Act to require a certain
level of effectiveness (i.e., emissions reductions) from each
I/M program. A number of SIP design factors affect the ability
of a program to be effective in reducing emissions including:
vehicle class coverage, model year coverage, geographic
coverage, failure rates, frequency of inspections, precautions
to ensure accurate inspections, waivers, cutpoints, and
others. Apparent deviations from the SIP will be investigated
6-6
-------
and documented during the audit. Previously unidentified
deficiencies in the SIP design itself will not be specifically
sought but may also be discovered. Generally the product of
the audit will be a list of such problems and recommendations
for their correction; but not a quantification of the emission
reductions presently being obtained with the problems present.
Follow-up action by EPA separate from the audit itself may
include a quantification of the emissions reductions actually
being achieved, if such a quantification would be useful to
designing and obtaining program improvements or to reconciling
the SIP with actual practices. If apparent SIP deficiencies
are discovered, EPA will discuss with the State how they can be
addressed in a manner that is' fair, efficient, nationally
consistent, and mutually acceptable.
3.0 AUDIT PROCESS FOR I/M PROGRAMS
The audit process for I/M programs is comprised of four basic
elements:
1. Advance preparation.
2. Audit visit.
3. Audit report.
4. Follow-up actions.
The preparation for the audit allows the auditors to
familiarize themselves with the design and operations of the
program under review and to identify those particular aspects
of the program which may need special emphasis during the audit
visit. Proper preparation will allow the auditors to establish
priorities for various audit activities in order to make
efficient use of the limited time available during the audit
visit. It will also reduce the State/local resources needed to
cooperate with EPA in the audit and the disruption of the I/M
program itself.
The audit visit is for the purpose of verifying and documenting
whether the program is being properly administered, operated,
and enforced, i.e., according to established laws, rules and
regulations, and procedural requirements in the SIP.
Additionally, the audit visit will allow a better evaluation of
program effectiveness and efficiency.
The auditors assigned to perform the audit must collectively
possess as much knowledge as possible about the operations of
I/M programs in general and about the specific details of the
6-7
-------
program under review, The goal of audit preparation is to
learn the basic design of the I/M program, to identify as much
as possible about the operatiag characteristics of the program,
and to determine as much as possible the potential strengths
and weaknesses of the program.
The audit report must properly and objectively reflect the
findings of the audit, both positive and negative. The audit
report should specifically identify:
1. The strengths of the program.
2. Any identified operating deficiencies relative to the
approved SIP for which program improvements must be
implemented.
3. EPA recommendations for correction of these deficiencies.
4. Any aspects of the program which need further study; this
would include potential operating problems, aspects of
the SIP design itself which appear inadegjuate, or areas
where EPA can suggest optional minor modifications which
would improve program effectiveness or efficiency.
5. EPA efforts to coordinate the audit report with the
State/local agencies; this would include a discussion of
any improvements or commitments already made by the
State/local agencies as a result of the audit; official
comments of -the State/local agencies on the draft audit
report should be appended to the audit report.
EPA staff will plan follow-up activities as necessary to
encourage and assist State or local implementation of the
improvements discussed in the audit report. In cases where
deficiencies must be corrected, follow-up audit visits may be
necessary after corrective actions have been implemented.
3.1 Advance Preparation
To ensure a successful audit, it is extremely important that
the audit staff adequately prepare themselves for the audit.
3.1.1 Documentation Assembly
The first step in preparing for the audit visit is to
consolidate documentation which is immediately available in EPA
Regional and headquarters files and to obtain any missing
documentation. This would obviously include the applicable SIP
and related documentation. Each State's SIP must include a
complete description of its I/M program, the program's enabling
6-8
-------
legislation, and the I/M rules and regulations. In some cases,
the SIP may identify planned program changes, such as phasing
in tighter cutpoints in subsequent years of the program, and/or
I/M program enhancements as "extraordinary" measures. In many
cases, but not necessarily always, SIP I/M documentation will
include procedures manuals for testing and/or quality control,
discussions of data analyses and/or other quality assurance
procedures, and mechanics training programs. Auditors should
be careful not to overrely on SIP materials, however, for these
materials can sometimes be incomplete or outdated. The SIP may
not contain operating contracts, equipment specifications, or
procedures manuals which may need to be obtained separately.
In those cases where I/M programs are operated by independent
contractors or where contractors are performing special I/M
functions for State or local agencies, a situation could
potentially arise concerning the disclosure of trade secrets.
In such cases, the contractor's trade secrets are provided
protection from disclosure under two separate provisions of
federal law. First, 18 U.S.C. 19.05 prohibits a federal
employee from divulging trade secrets. Second, the federal
Freedom of Information Act contains a provision which exempts
the disclosure of trade secrets. To ensure adequate
protection, however, it is incumbent upon the State/local
agency and the contractor involved to clearly identify and
label all information which contains trade secrets. If EPA
agrees that the information includes trade secrets as defined
in the statutes noted above, the Agency will treat the
information accordingly.
Other important sources of information in preparing for an
audit visit are periodic operating reports on I/M programs,
public awareness and informational printings, EPA documentation
of previous audits or investigations, and routine
correspondence. Operating data on I/M programs are often
formally reported on a- quarterly, semi-annual, or annual
basis. These data can be helpful in determining whether the
program is operating properly and -effectively. For instance,
information on ' the number of vehicles inspected during the
period can be indicative of an enforcement problem. Also,
information on failure rates and waiver rates is useful and can
be indicative of program effectiveness. In addition to formal
reports to agency heads, governors, or legislatures, agencies
which operate I/M programs may be able to provide copies of
routine data summaries prepared for in-house use.
Routine correspondence to and from program officials, citizens,
and other interested parties should be reviewed to determine
what, if any, issues may have already been identified or
addressed. It is particularly important that auditors be aware
6-9
-------
of any special sensitivities in a specific State or locality
revealed in previous correspondence.
3.1.2 Program Questionnaire
To facilitate audit preparation, an. .I/M program questionnaire
has been developed and is included" in Appendix B. This
document contains questions covering the relevant aspects of an
I/M program. Completing the questionnaire will require EPA
staff, with State or local assistance as necessary, to learn
and document the most important design features and operating
experiences of the I/M program in question. In addition, for
the first round of in-depth audits the questionnaire will
document some seemingly less important design features. It
will be useful for EPA to have these features documented to
avoid misunderstandings during the audit and for future
reference, and they can be documented at this point with
minimal additional effort.
The questionnaire is in two sections, one dealing with design
and intended operating aspects and the other with actual recent
operating experience. EPA Regional Office staff and EPA
headquarters staff will collaborate on completing the
questionnaire to the best of their ability based on available
documentation or personal knowledge, referencing the source of
information or documentation for each question if it is not
obvious. (If agreeable to the Region and the State/local
agency, the State/local agency may complete this step.) The
Regional Office will then send the questionnaire to the
relevant State or local agencies and ask them to correct and
explain any errors on EPA's part and to provide answers where
EPA staff were unable to do so, if the answers are reasonably
available to the agencies themselves. The agencies will be
asked to describe how answers could be obtained for questions
that cannot be readily answered. These questions will be
discussed during the site visit, or earlier by telephone. It
is recommended that the Regional Office send the questionnaire
to the State/local agencies at least 90 days prior to the site
visit in order to allow ample time for review and coordination.
3.1.3 Notice to Program Officials
After audit preparation has progressed to the point where an
audit visit can be scheduled, the EPA Regional Office should
send a formal written notice of the audit co the appropriate
State and/or local program officials. The written notice
should give the State/local officials ample lead time (about 60
days should be sufficient in most cases) to prepare for the
audit visits, and the audit visits should be scheduled at a
mutually convenient time. The formal notice should also
6-10
-------
specify, whenever possible, those individuals who will comprise
EPA's audit team and what State/local organizations should be
represented. Finally, the formal notice should identify any
special issues raised during the advance preparation.
In all cases, a copy of the completed program questionnaire
will be sent to the State/local program officials for their
official review. The results of their review could be
discussed either prior to or during the audit visit, whichever
is felt to be the most constructive. In some cases, EPA may
request to have certain information not covered by the
questionnaire submitted for review prior to the audit visit in
order to complete audit preparations. These cases would
probably be unusual and involve unique program features not
addressed in the national questionnaire; however, if they
occur, the State/local agency obviously must be given adequate
time to submit the needed information. Any unanswered
questions will be subjects for discussion during the on-site
visit.
When agencies in addition to the air planning agency must be
included in the audit, the Regional Office will determine how
to secure their involvement and cooperation, and will notify
each of the upcoming site visit. In addition, the Regional
Office will provide notice to the Office of Mobile Sources of
scheduled audit visits at least 60 days in advance.
3.2 On-Site Audit Visit
The purpose of the audit visit is to verify information already
available and to gather new information as needed to satisfy
the objectives of the audit. One objective should always be to
identify those areas where EPA can provide assistance to
strengthen I/M programs. Sometimes such assistance may involve
specific aid to a particular State, and at other times it may
involve more general assistance aimed at resolving an overall
technical issue.
The audit visit should be adequately planned to ensure that all
needed activities are conducted within the time constraints
involved. Generally, a two- or three-person EPA audit team
should be able to complete the on-site visit in three or four
days, at times working independently, depending on the size and
complexity of the program. These days need not be consecutive,
and the Regional Office may find it desirable to separate
special surveys, records review, inspection station visits, and
interviews with officials.
6-11
-------
The audit must be planned and coordinated with State/local
agencies in order to minimize the level of intrusion and
disruption of normal program activities. The level of
State/local resources which will need to be devoted to the I/M
audit will vary according to the type and size of the program.
The greatest demand . on State/local staff time would be for
initial and exit meetings, each of which may be several hours
long and may involve several State/local personnel. Many audit
activities would involve observations of State/local personnel
performing their regular functions, such as auditing stations
or processing waivers. Therefore, the audit would not greatly
increase the demands on these persons' time. Except for very
large and complex I/M programs, it is expected that the audit
will require no more than about two State/local staff persons'
time for three to four days.
In States with I/M programs operating in multiple urbanized
areas, it will usually be necessary to visit each I/M area in
addition to the city in which program officials and records are
located. This is because each urbanized area will usually have
a separate manager, potentially different enforcement agency
practices, and potentially different repair industry competence
and enthusiasm for the I/M program. However, visits to the
other areas can usually be limited to one day, can be separated
in time from the main on-site visit, and may utilize different
EPA personnel. It may also be possible to utilize contractor
or other third-party support for the other areas, if acceptable
to State or local officials.
Because a written record of the auditors' work should be
retained in the form of working papers, it is requisite that
each auditor keep detailed notes as to persons contacted or
interviewed and information received, its source, and when
received. Sample forms for such notes are included in Appendix
E. These samples are intended to remind the user of the
important information to be recorded. Regionally prepared
substitutes may be used.
Auditors should be alert to situations which do not appear to
be in keeping with program procedures or regulations in the
SIP; such situations should be fully investigated during the
audit visit. Auditors should also be alert to situations that
could be indicative of fraud, abuse, or illegal acts; these
situations may be more appropriately reported to higher level
State or local program officials rather than being investigated
during the audit, except to identify the effect that such acts
have on the program. Finally, auditors should be alert to
signs that their visit may have resulted in observed behavior
not typical of normal program operations. Where practical,
auditors should select specific facilities and personnel for
observation without advance notification to the State or local
agencies.
6-12
-------
An important part of the. audit visit is an evaluation of the
internal control system applicable to the program,
organization(s), and activities under review. Each State or
locality operating an I/M program should have an established
methodology and capability for evaluating program operations in
order to assess program results. Through this system of
administrative controls, the State or locality should
continuously or periodically compare actual program operations
to intended design. The emphasis in this system should be to
identify those activities where deviations are occurring and
then to take corrective action. Internal audits are usually an
integral part of the internal control system.
In the following sections, specific activities of the I/M audit
visit are listed and discussed. A more detailed discussion of
these activities is found in Appendix C. The purpose of each
activity is to collect information related to one or more of
the audit topics listed in Section 2.0 and discussed in detail
in Appendix A. In some instances, the audit topics may have
been sufficiently clarified during audit preparation. However,
whenever time allows, it is advisable to discuss each of these
areas with program officials during the audit visit. Even if
no new information is gained, the discussions will at least
enable the auditors to verify that the information obtained
during audit preparation is accurate.
3.2.1 Interviews
The following State/local officials or their designees must be
interviewed diaring the audit visit: the air planning agency
officials with responsibility for mobile sources, the I/M
manager (whether employed by the air agency or another agency) ,
and operations personnel with close knowledge- of current
practices and experiences in the areas of enforcement, guality
control, repair waivers, and data analysis. The I/M manager
may be able to discuss all of these areas, or additional staff
may need to be included.
The agendas for these interviews should be based on the
questionnaire and the findings of any on-site activities which
have proceeded the interview. The interviews should seek
clarification of program design, intended procedures, and
observed procedures. They are not occasions for EPA
assessments of deficiencies or strengths. Such assessments, if
the Regional Office wishes to make them during the site visit,
should occur during the exit interview with agency officials.
Such assessments may be postponed to later correspondence or
meetings, but must be part of the formal audit report. The
exit interview should also be used to offer EPA assistance on
any matter of interest to the I/M officials.
6-13
-------
The starting point for the audit visit should be an initial
meeting(s) with State and/or local officials involved in air
quality planning for mobile sources. In addition to
discussions of program design and organization, this meeting
should focus on any problem areas in the program which State/
local officials have already identified and their plans for
resolving them. Also any planned program modifications,
improvements, and expansions should be discussed.
Interviews with the I/M manager and other program operations
personnel should focus on operational aspects of the program
including: compliance rates; failure rates; waiver rates;
repair costs; internal control efforts (surveillance, results
of station audits, data analyses, investigations using unmarked
vehicles, etc.); enforcement aspects (procedures, results,
etc.); mechanic training efforts; and complaints (types, how
resolved, etc.). Again, known problems and planned resolutions
should be discussed.
In addition to interviewing agency officials with direct
involvement in the I/M program, it may be desirable to
interview non-program individuals with knowledge of the I/M
program and differing perspectives on its operation. The
objective is not to solicit complaints, to promote the I/M
program to these public sectors, or to make the audit a press
event. Rather it is to compensate for the necessary brevity of
EPA's own audit and to be sensitive to suggestions that EPA's
audit was too brief to detect actual problems of importance.
Useful interviews might be held with contractor
representatives, auto club officials, service industry
association officials, auto dealers association officials,
consumer agency officials, instructors of mechanics, and
similar persons. These people can be interviewed by telephone
at a more convenient time than the site visit, either before or
after the visit.
3.2.2 Records Review
Another important phase of the audit visit is the review of
records relevant to the I/M program. These records include
enforcement records, inspection records, waiver records,
surveillance records, and complaint files.
Inspection records should be reviewed to determine what data
are collected and how the data are used. When inspection
records are kept manually, it is important to review a sample
of the records for completeness, legibility, accuracy of
inspection standards, reasonableness of test scores, and
accuracy of the pass/fail decision. Where retests are
involved, the emission reductions obtained from repairs would
also be of interest.
6-14
-------
In reviewing waiver records, it is important to determine:
1. How waivers are processed to ensure compliance with
waiver criteria.
2. How many waiver applications are being received,
approved, and denied.
3. The extent to which waivers are denied because of
inappropriateness of repairs.
4. The extent to which waiver transactions can be
tracked by repair facility.
Surveillance records should be reviewed to determine the data
collected during each type of surveillance activity and how
these data are used. Review of these records should also allow
a determination of the frequency of various surveillance
activities and how often problems are identified and resolved.
Reviewing cases where stations or inspectors have been
suspended for infractions is also .advisable.
Complaint files can be reviewed to determine the number and
kinds of complaints being received. However, reviewing
complaints should be given a low priority, unless observations
during the audit indicate that there are problems, such as
improper inspections or inadequate repairs, which could be
further evaluated by reviewing complaints. In such cases, it
is advisable to also review a sample of complaint reports to
determine how they were investigated and resolved.
3.2.3 Inspection Station Visits
During the audit visit, all types of inspection stations and
other licensed facilities should be visited, including regular
inspection stations, fleet stations, referee or waiver
stations, licensed or certified repair facilities, reinspection
stations, or any other type of station conducting initial
inspections or retests. Depending on the type of station, any
or all of the following activities may be part of the visit:
1. Audit or observe an audit of analyzers, using audit
span gas.
2. Observe inspections.
3. Observe waiver processing.
4. Check records.
5. Interview station personnel.
6-15
-------
In centralized systems, in order to prevent bottlenecks in the
lane, it is often preferable to observe how analyzer audits are
performed by State/local personnel rather than EPA's auditors
actually doing the analyzer audits. In centralized facilities,
it is always possible to observe vehicle inspections. In
decentralized programs, analyzers can be checked by the EPA
auditors or the EPA auditors can observe the procedures being
used by the State/local personnel. If the State/local analyzer
audit procedures do not include a calibration check through the
probe, the EPA auditor should conduct such a test. In
decentralized systems, there are often no vehicles waiting to
be inspected, so inspections cannot always be observed.
Decentralized stations are generally required to keep
inspection records and analyzer calibration records; therefore,
these records should be reviewed during the station visits.
The number of station visits in an audit visit will vary
according to the type and size of the I/M program under
evaluation. In the case of a centralized program with only a
few (less than five) inspection stations, there should be
sufficient time available during the audit to visit all or most
of the facilities. In larger centralized programs and in
decentralized programs, only a fraction of the stations can be
visited during the audit. The exact number will vary in each
case, however, the following guidelines should be used in
deciding how many and which stations to visit .
In centralized programs:
1. Visit at least three stations; more should be
visited whenever time allows, or if inconsistencies
are found, or when the program has an extremely
large number of centralized facilities (e.g., New
Jersey).
2. Choose stations with a high volume of inspections
(based on records review).
3. Choose stations that represent a reasonable cross
section of types of neighborhoods.
4. Choose at least one station, if possible, that has a
past record of quality control problems; i.e., low
failure rates, high level of analyzer audit
failures, etc. (based on records review).
6-16
-------
In decentralized programs:
1. Visit at least 10-20 stations, depending on the size
of the program, more if possible or when
inconsistencies are found.
2. Choose stations which represent a cross section of
types of neighborhoods.
3. Choose stations which cover several different State/
local agency field investigators.
4. Choose stations which are assigned to represent a
cross section of different types of businesses
(service stations, independent garages, auto
dealers, chain service centers, etc.).
5. Choose stations which represent a cross section of
types/makes of analyzers.
In decentralized programs, the EPA auditors will accompany at
least three State/local field auditors (unless, of course,
fewer than three State/local auditors are utilized in the
program) on their regular audit visits. The EPA auditors will
choose which State/local auditors are accompanied and must be
careful to get the right mix of types of stations. This can be
done by reviewing the planned audit schedules of the
State/local auditors.
Whenever they exist, waiver stations, mobile stations, referee
stations, fleet stations, reinspection stations, and roadside
pullover teams should also be visited.
3.2.4 Special Surveys
In some cases there may be a need for special surveys as part
of the audit visit (or at some more convenient time). In areas
that use window or compliance stickers for enforcement
purposes, a sticker compliance survey must be conducted. Such
a survey involves visiting a number of parking lots to observe
a sample of vehicles and collect data on sticker compliance or
none omp1i anc e.
Other potentially useful data gathering surveys are emissions
testing surveys and tampering surveys. Such surveys involve
collecting emissions short test data and/or tampering data on a
representative group of local vehicles. Emissions testing
surveys can be conducted in shopping center or other parking
lots as part of a voluntary testing program or as part of a
mandatory roadside pullover program. The latter method is far
preferable because it removes the biases associated with
6-17
-------
voluntary testing and ensures the collection of a certain
amount of data. For tampering surveys, the mandatory roadside
pullover program is the only survey method acceptable because
of these biases.
3.3 Audit Report
The EPA Regional Office will prepare an audit report which will
document the findings of the audit, present EPA conclusions,
and suggest possible improvements. The Office of Mobile
Sources will be given an opportunity to review and comment on
the Region's draft. Regions should also allow the audited
State and local agencies to review the reports in draft. All
official comments received from the State and local agencies
should be appended to the report. By agreement, the Region and
QMS may share the drafting of the report. There is no fixed
format for the report, but QMS may develop a suggested format
for audits conducted later in FY1985 based on early experiences.
Regions should prepare draft audit reports in time to give OMS
an opportunity to review them prior to the June 30, 1985
deadline established for the remainder of the National Air
Audit System report. OMS will discuss with the Regional
Offices and with STAPPA/ALAPCO at a later date the type of
national I/M audit report which is appropriate and when and how
such a report should be prepared.
3 . 4 Follow-up Activities
If design or operating problems which are serious enough to
constitute non-implementation of the SIP or to make the SIP
inadequate are identified in the audit report, the Regional
Office is expected to prepare a plan for giving the State or
locality reasonable opportunity and assistance in solving those
problems. This plan should expect faster State or local action
on more serious problems, and provide more time for
deliberations and alterations for less serious ones.
Generally, the Regional Offices will have the lead in obtaining
corrections from the States and localities. The Office of
Mobile Sources will provide technical support and track
progress in each I/M program.
If the audit report identifies improvements which are desirable
but not required under the SIP or the Clean Air Act, the
Regional Office will transmit these suggestions to the State or
locality also. Appropriate emphasis should be given to these
suggestions as they are likely to be the most significant
result of the audit in many I/M programs.
6-18
-------
The audits should be of sufficient scope and adequately
documented and reported in such a way that the record will
adequately support' EPA follow-up action in the form of a call
for SIP revisions or a finding of non-implementation of the
SIP, should that be necessary. In some cases, it may be
necessary to collect .additional data to support these or other
types of follow-up actions.
6-19
-------
APPENDIX A
DESCRIPTIONS OF I/M PROGRAM ELEMENTS
1. Inspection Test Procedures
2. Emission Standards (Cutpoints)
3. Inspection Station Licensing Requirements
4. Analyzer Specifications and Maintenance/
Calibration Requirements
5. Quality Control Procedures
6. Internal Control System (Quality Assurance)
7. Enforcement Procedures
8. Vehicle Coverage Considerations
9. 'Waiver Procedures
10. Consumer Assistance and Protection
11. Mechanics Training
6-20
-------
APPENDIX A
DESCRIPTIONS OF I/M PROGRAM ELEMENTS
1. Inspection Test Procedures
The goal is to provide a uniform, reliable, simple test to
identify high emitters. A retest is required to ensure that
failed vehicles receive adequate maintenance to reduce their
emissions. Inspection test procedures are usually one of the
following three basic I/M tests: 1) idle test, 2) two-speed
idle test, or 3) two-mode loaded test. Sometimes different
I/M tests are specified for pre-1981 model years and for 1981
and newer model years in order to increase program
effectiveness. Particular vehicle models may have special
test procedures. In a few cases, the test applies to either
carbon monoxide or hydrocarbons rather than both.
An additional variation among State/local programs is the
degree of automation involved with the test procedure. Some
programs have a fully automated procedure while others have a
completely manual system.
Other considerations with respect to test procedures are the
emission performance [Section 207(b)] warranty requirements.
The Section 207(b) warranty generally applies to 1981 and
newer (1982 and newer at high altitude) model year light-duty
vehicles and light-duty trucks. The Section 207(b) warranty
regulations have specific requirements for test procedures
which must be followed, if warranty coverage is to be made
available.
Many I/M programs require at least partial inspections for
tampering and/or misfueling as well as emissions checks. In
those cases, the anti-tampering/anti-misfueling inspections
must also have stipulated procedures.
2. Emission.Standards (Outpoints)
The goal is to achieve certain levels of emission reductions
while not overburdening repair facilities or jeopardizing
public acceptance. Emission standards can be chosen which
will equitably divide the repair burden for the program,
i.e., provide for similar failure rates across vehicle design
and age categories rather than overburdening owners of older
or newer vehicles. Evaluation of cutpoints should focus on
overall failure rate, differences in failure rates among age
groups, and similarity to cutpoints in other I/M areas.
6-21
-------
Usually program planners choose outpoints for pre-1981
vehicles by setting a target failure rate (e.g., 20%) of the
vehicles inspected. This procedure is reasonable since the
number . of high emitting vehicles which can yield
cost-effective emission reductions is high; the only
constraints on cutpoints are therefore the perceived public
acceptability of the resulting failure rate and the capacity
of the repair industry to handle the volume of repair
requests. For 1981 and newer vehicles, the 207(b) warranty
cutpoints are usually adopted. Data from existing I/M
programs and EPA testing indicate that the failure rates for
1981-1- vehicles are usually 3-5% when the 207(b) cutpoints are
used. (Failure rates may be temporarily higher for late
starting I/M programs, due to an accumulation of failures in
the 198H- fleet in the years proceeding the start of the
program.) Because this failure rate is so low, there is
little justification for States/localities using less
stringent cutpoints for 1981+ vehicles. If less stringent
cutpoints are used, a re-examination of whether the I/M
program design can achieve EPA's minimum emission reduction
requirements is in order. EPA should not hesitate to
recommend improvements to cutpoints even when existing
cutpoints are determined to meet the requirements of the SIP
and EPA's minimum requirements, but such recommendations
should be clearly identified as just that.
3. Inspection Station Licensing Requirements
The goals are to assure that all inspections are conducted by
properly trained and equipped inspectors and to provide a
mechanism for accountability of inspection facilities. In
centralized programs, both contractor-run and government-run,
where inspectors are under more or less direct control of the
State/locality, there may only be employee training
requirements rather than licensing requirements. In
decentralized programs, and in the fleet or other
decentralized stations usually involved in centralized
programs, EPA policy requires that there must be licensing
requirements which ensure that:
1. All stations employ trained inspectors.
2. All stations have approved analyzers.
3. All stations keep necessary records on
inspections, calibrations, and maintenance and
agree to make these records available to the State
or local agency.
6-22
-------
4. Analyzer Specifications and Maintenance/Calibration
Requirements
The goal is to adopt and enforce equipment requirements which
will provide for accurate and therefore consistent emission
measurements. In all existing centralized, contractor-run
systems, the analyzers are computer-operated and feature
automatic data collection. These systems usually have
elaborate maintenance/calibration networks associated with
them.
In existing centralized, government-run systems, the
analyzers are often computer-operated with automatic data
collection but may not be as sophisticated as the contractor
systems. In some cases, centralized government-run systems
use manually operated garage-type analyzers.
As one would expect, decentralized I/M programs have the most
variety with respect to analyzer requirements. Some programs
require computerized analyzers, and others require only
manual analyzers. Some programs have established a list of
approved analyzers, while others accept any analyzer
certified as meeting certain specifications.
The 207(b) warranty regulations include several analyzer
specifications. However, none of the 207(b) specifications
for the analyzer itself are very limiting and, consequently,
have had little effect on I/M program design. Nevertheless,
they should be considered in evaluating an I/M program.
Regardless of the type of analyzer or the type of program,
appropriate maintenance and calibration requirements are
needed to ensure that the analyzers are capable of yielding
accurate and repeatable measurements. These requirements
obviously differ for manual and computer systems, but are
equally important in both cases.
If an I/M program needs to upgrade its maintenance and
calibration practices to meet the requirements of the 207(b)
warranty regulations, EPA should assist the State/local
agencies in identifying needed improvements. In those cases
where it is appropriate, EPA should have the State/locality
request approval by EPA of non-standard but equivalent
quality control practices for 207(b) purposes.
6-23
-------
5. Quality Control Procedures
The goal is again to ensure accurate and repeatable
measurements, but in a broader context than the analyzer
specifications and maintenance/calibration requirements.
Quality control procedures should be prescribed to ensure
that analyzers are calibrated and maintained, that
inspections are conducted properly, and that inspection and
calibration records are completed properly. In computerized
systems, some, but not all, of these checks can be automated.
A comprehensive quality control system should address:
1. Analyzers
Periodic calibration checks
Periodic leak checks
Regular preventive maintenance
Accurately named calibration gases
2. Inspections
Assurance that analyzer is ready for testing
(warmed up, zeroed, and spanned)
Assurance that vehicle is warmed up
Assurance that there is ao excessive exhaust
system leakage
Assurance that proper probe insertion depth
is achieved
3. Pass/Fail Determinations
Proper vehicle and cutpoint identification
Safeguards to prevent mistake, fraud or abuse
6. Internal Control System (Quality Assurance)
The goals are to assure that prescribed regulations and
procedures are followed and to assure that the overall
program and each respective part are achieving their
purposes. The internal control system is totally dependent
on data to serve as feedback on hcv well the program and
parts thereof are working. Therefore, the first requirement
of internal control is to have an adequate and functioning
data collection system... To have maximum benefit, data
analysis must be accurate, reliable, complete, and timely.
Data analyses should be capable of identifying the level of
noncompliance among vehicle owners, the failure rate among
inspected vehicles of different age groups (initial
inspections separately from retests), the waiver rates, and
the quality of repairs (through comparison of initial and
retest emissions levels of failed vehicles). It is highly
desirable that failure rates and waiver rates be calculated
and per radically reviewed at the level of the individual
inspection station or repair facility.
6-24
-------
An integral part of the internal control system is the
auditing of inspection facilities. These audits should focus
on analyzer calibration and leak checks as well as record
checks, especially when manual records are kept. In
decentralized programs, investigations with unmarked vehicles
are especially useful if they test for both incorrect
failures and incorrect passes.
In those programs which rely on pre-printed windshield (or
other window) stickers or certificates of compliance for
enforcement, an accountability system is needed. Each
inspector or inspection station should be required to show
that the number of used stickers or certificates corresponds
to the number of inspection passes for the period in
question. This applies to both centralized and decentralized
programs, since improper diversion of stickers or
certificates is possible in both.
Another integral element of internal control is corrective
action, or at least the means for corrective action. In some
cases, this would involve penalties for infractions by
inspection stations or individual inspectors. In other
cases, corrective action might involve making an
administrative change in the way records are kept or a change
in the forms themselves. The ultimate test of internal
control is whether problems can be identified and resolved.
7. Enforcement Procedures
The goal is to assure 100% participation of subject vehicles
in the I/M program. There are three systems of enforcement
currently used in I/M programs (some areas use a combination
of these):
1. Registration denial systems.
2. Sticker based systems.
3. Registration data link systems.
The registration denial systems have historically provided
the most effective and efficient means to enforce I/M
requirements. In this system, vehicle registration is denied
unless a vehicle has complied with the I/M requirement.
Existing penalties for operating an unregistered vehicle
serve to deter noncompliance, and the State or locality has
an incentive (recovery of lost registration revenue) for
enforcing compliance.
6-25
-------
Sticker enforcement systems are another widely used
enforcement method, In this system, vehicles are issued a
window sticker as evidence of compliance. Vehicles with
expired stickers or without stickers are subject to citation
by police followed by some sort of penalty. Sticker
enforcement programs may suffer due to lack of the ability to
readily distinguish subject vehicles (especially in
regionalized programs) and due to a lack of priority for
sticker enforcement on the part of police agencies.
The registration data link system is a system used by several
regionalized programs which do not have authority for
registration denial and cannot effectively use stickers
alone. In the data-link system, vehicles are identified and
scheduled for inspection on the basis of registration data.
Inspection data is then reviewed by comparing the list of
vehicles scheduled for inspection in a particular period to
the list of vehicles actually inspected and passed. Through
this comparison, non-complying vehicles and their owners can
be identified for enforcement action. The chief drawbacks to
this system are the lag time in data analysis and the level
of resources which must be devoted to establish the data
link, operate it, and pursue prosecution of non-complying
owners. The use of window stickers is often combined with
the data-link system to provide an added enforcement
mechanism.
8. Vehicle Coverage Considerations
This design area covers those factors which affect the
fraction of total vehicles being inspected in the I/M
program. These factors include: L) geographic coverage area,
2) weight class or use exemptions, 3) exemptions related to
fuel type, and 4) model year exemptions.
It is difficult to state a goal for this design area because
it can be viewed from different perspectives. From EPA's
regulatory perspective, the vehicle coverage variables are
factors which affect program effectiveness. Obviously, other
things being, equal, the larger the fraction of vehicles
participating in the I/M program, the more effective the
program will be in reducing fleet emissions. The vehicle
coverage variables viewed alone cannot give a meaningful
picture of program effectiveness; however, these variables
can identify potential ways to strengthen an I/M program.
For instance, programs can be strengthened by increasing the
geographic coverage area to include all fringe areas of
development rather than limiting the I/M program to the
urbanized area.
6-26
-------
Similarly, weight class and use exemptions can be changed to
increase vehicle coverage. For instance, some programs only
require I/M for vehicles up to 6000 pounds GVW. Most I/M
programs now include vehicles up to 8500 pounds GVW, which
covers light-duty vehicles and light-duty trucks, both of
which are used almost exclusively for personal
transportation,- and many programs (over 10) require
inspections for even higher weight class vehicles.
Model year exemptions provide another potential area for
affecting program effectiveness and the distribution of the
program's impact among the vehicle owning population. Many
I/M programs inspect all vehicles or all 1968 and newer model
years, which will give essentially complete coverage in
1987. Programs with less coverage, especially those with 10
or less model years coverage, could potentially improve the
effectiveness of their programs by changing the model year
exemptions.
One important consideration when reviewing all exemptions is
whether there are loopholes. in the way they are
administered. It is desirable, for instance, to have
provisions to prohibit, and procedures to prevent, owners
from registering their vehicles outside the I/M area. Also,
where a weight limit exists, it is important to define the
exact basis for the limit (gross vehicle weight rating, empty
weight, or other), the way the official weight of a vehicle
is determined (reference book, examination, or owner
testimony), and whether evasion through misreported weight is
occurring. Similarly, use exemptions and fuel type
exemptions should be examined for loopholes.
9. Waiver Procedures
Many I/M programs have waiver provisions with the goal being
to limit the amount of expense an owner would have to incur
as a result of failing the I/M test. Most I/M waivers are
tied to repair cost ceilings. Repair cost ceilings between
$50 and $100 are the most common, but both lower and higher
limits are used in some programs. Since waived vehicles
represent reductions in program effectiveness, high waiver
rates can be particularly troublesome. Therefore, it is
important to review the number of waivers being issued as
well as the criteria for granting waivers and the procedures
used in processing waivers. All SIP's were approved under
the assumption that waiver rates would be very low, and SIP
adequacy may be re-examined as a follow-up to the audit, if
waiver rates are high and if better management of the waiver
process appears incapable of sufficiently lowering the waiver
rate.
Criteria should be set to prevent tampered vehicles from
getting waivers and to ensure that owners take advantage of
6-27
-------
warranty service. Procedures for processing waivers should
focus on verifying that all waiver criteria have been met,
including verification that all repairs claimed toward the
repair ceiling were appropriate and actually done. Another
desirable safeguard is to have the ability to track waiver
rates by repair garage in order to be able to identify both
abuses and simple lack of repair expertise.
Particular attention should be given to waiver criteria for,
and waiver rates among, 1981 and newer vehicles. These
vehicles are particularly susceptible to high waiver rates
because the repair industry is relatively unfamiliar with
them. They are becoming the majority of the fleet, and good
repair practices should be encouraged before bad habits
become entrenched.
10. Consumer Assistance and Protection
Other consumer assistance and protection aspects of I/M
programs, in addition to waivers, include the following:
1. Referee test - This is an EPA requirement for
decentralized programs but may be found in
centralized programs, too.
2. Complaint handling - Procedures should be
established for investigating complaints.
3. Repair information - Owners of failed vehicles
should receive a brief discussion of the possible
reasons their vehicle failed and approximate cost
estimates of these repairs; this information will
guide owners toward obtaining proper repair at a
reasonable cost and may serve to reduce abuses by
repairers and the resulting citizen
dissatisfaction. I/M programs can also go as far
as to publish average repair costs at different
facilities.
11. Mechanics Training
The goal of mechanics training is to improve program
effectiveness and enhance consumer protection by having a
supply of mechanics trained in proper emission related
repairs. Cost savings to the public from more efficient
repairs can more than offset the cost of delivering training.
The following aspects of mechanics training are important:
1. Course curriculum - What is the course content?
2. Course distribution-- How is the course delivered?
6-28
-------
3 . Course promotion - What is done to promote
interest and participation by mechanics?
4. Course followup - What is done to foster continued
support to trained mechanics?
The reasons for the first question are self-evident.
Obviously, course content must address proper analyzer use
and calibration, emission test procedures, procedures for
detecting tampering and misfueling, basic information on the
types of I/M failures, and diagnosis and repair of excessive
hydrocarbon and carbon monoxide emissions.
The second question's importance is in its relevance to the
third. Mechanics training courses are generally offered
through one or more of the following ways:
1. Community colleges, vocational/technical schools,
high schools, etc.
2. Independent training agents (private individuals
or firms licensed or certified to offer courses).
3. In-house training personnel.
Often the cost and geographic availability of the training
have a bearing on how many mechanics participate. Also, very
often little or no effort is expended by program officials to
promote mechanics training or to follow up with
participants. It must be realized that if repair cost
waivers are readily available, mechanics are not forced by
customer satisfaction to become competent in emission
repairs. I/M programs with waiver provisions, therefore,
carry a greater burden to encourage participation in training.
Training can be promoted through mailings to garages and
contacts with garage associations, service station dealers
associations, auto clubs and the like. A program to identify
incompetent and problem mechanics should be accompanied .by
efforts to get these mechanics to receive training. Finally,
legislation to allow only trained and certified mechanics to
qualify vehicles for repair cost waivers exists in some
States and should be considered by all.
Followup with trained mechanics can be useful to keep them
interested and informed of new issues or additional training
opportunities. Some ways to do this include:
1. Periodic newsletter.
2. Repair information hotline.
3. General mailings.
6-29
-------
APPENDIX B
INSPECTION/MAINTENANCE PROGRAM AUDIT QUESTIONNAIRE
Part I: Program Design
Part II: Operating Experiences
6-30
-------
I/M Audit Questionnaire
Part I: Program Design
1. Vehicle Coverage
2. Cutpoints
3. Test Procedures
4. Tampering and Misfueling Inspection Requirements and
Procedures
5. Analyzer Specifications
6. Analyzer Maintenance and Calibration
7. Station Licensing; Inspector Licensing
8. Record Keeping at Time of Inspection
9. Data Handling Subsequent to Inspection
10. Station Audit/Surveillance Activities
11. Challenge Mechanism
12. Repair Waivers
13. Enforcement
14. Mechanics Training and Licensing/Certification
15. Consumer Issues
16. Self Assessment Through Data Analysis
17. Other Data Collection Activities
18. Future Plans
6-31
-------
Instructions
(l) EPA staff should complete Part I of the questionnaire to the best
their ability. Documents and staff contacts should be used to
obtain necessary information prior to site visit.
(2) EPA staff should specifically note on the "EPA Comments" page* of
each section those questions which could not be answered or for
which there was conflicting information, and request a State/local
answer. The State/local agency may also be requested to provide
documentation for answers provided earlier by telephone. The
questionnaire should be forwarded to the State/local agency with the
request that it be completed and returned to EPA prior to the site
visit.
(3) The State/local agency should provide any lengthy answers and/or
explanations on the "State/local Comments" page*. Any brief
State/local answers, corrections or additions on the question/answer
pages should be identified by agency initials in the margin. -
(4) Additional comment pages should be inserted as needed.
*The space for some answers to questions and for EPA and State/local
comments has been deleted from the questionnaire as it appears in this
appendix.
6-32
-------
1. Vehicle Coverage
(a) Exemptions
Are exemptions made for:
Age? yes/no
Vehicle weight? yes/no
Fuel type?
(b)
Diesels
Other
Motorcycles?
yes/no
yes/no
yes/no
New vehicles? yes/no
(less than 1 yr. old)
Other factors? yes/no
Subject vehicles
Is the program statewide?
yes
Describe:
Describe:
Fuel Type:
Describe:
no
If yes, estimate no. of subject vehicles
statewide:
If no, estimate no. of subject vehicles in each
urbanized area:
urbanized area
no. of subject vehicles
Explain how this estimate was derived:
(c) Explain how vehicles are assigned weight classes, usage types,
fuel types or other exempting characteristics in the
registration/enforcement process:
6-33
-------
2. Outpoints*
model year vehicle class CO outpoint (%) HC cutpoint (ppm)
Were outpoints changed in the last 12 months? yes/no
If yes, give the date they were changed and the previous outpoints
Date:
model year vehicle class CO cutpoint (% ) HC cutpoint (ppm)
* Where this format is not appropriate, attach a detailed list of
outpoints, and the exact vehicle categories to which they apply.
3. Test Procedures
Attach an official copy of the current usual or primary test
procedure. Where not completely described in the test procedure,
provide the following information. If appropriate, distinguish
between official requirements for a valid test, contractual
requirements on the testing contractor, non-binding advice ^B
inspection stations, and mere common practice.
(a) Describe the test modes used (idle, 2500 RPM, loaded) and their
order in the test procedure. Indicate whether a tachometer is
required. For loaded mode, specify speed and horsepower:
(b) Describe any conditions under which modes are not performed:
(c) Describe any preconditioning mode and indicate whether it is
mandatory or optional:
(d) Is there a test for visible emissions? yes/no
If yes, describe:
(e) Is a C02 or (CO + C02 ) cutpoint used? yes/no
If yes, describe:
(f) Is some other exhaust leak check used? yes/no
If yes, describe:
6-34
-------
(g) Describe provisions for ensuring that:
(1) The vehicle is properly warmed up.
(2) The analyzer is properly warmed up.
(3) There is adequate probe insertion depth..
(4) All vehicle accessories are turned off during testing.
(5) Vehicles with multiple exhausts are tested properly.
(h) Indicate the required frequency for the following analyzer
quality control checks:
Electric zero/span check:
Hydrocarbon hangup check:
(i) Is the usual or primary test procedure consistent with the
207(b) Ford restart test? yes/no Explain:
(j) Describe any special test procedures used, including those for
Ford vehicles; indicate which model year vehicles they apply
to, and when they are used:
(k) Describe how standards for a specific vehicle are selected for
comparison to the vehicle's emission readings. (Does the
inspector select the outpoints or does the analyzer do it
automatically?)
(1) If vehicle standards are selected according to vehicle type or
weight, explain how the vehicle type or weight is determined
during the test:
(m) Is the pass/fail decision made by:
The inspector?
The analyzer?
If the pass/fail decision is made by the analyzer, what
prevents the inspector from ignoring the machine's decision?
6-35
-------
Tampering/Misfuel ing Inspections
Does the program have provisions for a tamper
inspection? yes/no
Attach an official copy of the tampering/misfueling inspection
procedures. Where not completely described in the inspection
procedures, provide the following information:
(a) Identify the vehicles subject to tampering/misfueling
inspections (including tampering inspections that are a
condition for repair waivers) on the basis of vehicle age,
vehicle type and any other relevant factors:
(b) Describe the established procedures for conducting the
tamper ing/mis fueling inspection:
(c) Describe the equipment, supplies and/or manuals required to
perform tampering/misfueling inspections:
(d) Describe the mechanism for updating reference manuals:
(e) List the original equipment emission control components that
are checked:
(f) Does the tampering/misfueling check include testing the
tailpipe with lead-sensitive (Plumbtesmo) test paper? yes/no
(g) Describe the repair requirements for vehicles which fail tm
tamper ing/misfueling inspection:
(h) Grace period: Vehicles failing the tampering/misfueling check
must have the necessary repairs done within days.
(i) Describe the procedures to enforce repair grace periods:
(j) Requirements for replacement parts:
No special requirements
Original equipment manufacturer parts only
Other:
(k) Describe the procedures used for reinspections to assure that
replacement parts are installed and working properly:
(1) Describe the mechanism for resolving issues of:
(1) Whether vehicles are/are not supposed to be equipped with
certain components.
6-36
-------
(2) Whether lead-sensitive test paper failures are due solely
to contaminated fuel purchase.
5. Analyzer Specifications
If available, attach an official copy of the approved analyzer
specifications and/or a list of approved analyzers. Where not
completely covered in the specifications sheet, provide the
following information:
(a) Identify the analyzer specifications required for each type of
facility in the program (i.e., contractor lanes, decentralized
stations, fleet stations, state-run stations, certified repair
facilities, etc.). If a standard specification (e.g., BAR* 74,
BAR 80, ETI, BAR 84) is used, only a reference is needed.
(b) In a decentralized program using a very limited number of
approved analyzers, identify the models and the approximate
number of each in service:
6. Analyzer Maintenance and Calibration
(a) Routine analyzer maintenance and calibration is performed by:
Inspector Contractor Program official
(Agency )
(b) Facility-gas span checks are performed:
One or more times per day
. Weekly
Monthly
Other Specify:
(c) List the components and concentrations of the. facility span
gas(es):
Component Concentration
(d) The accuracy of the facility span gas is:
NBS traceable ± 1% NBS traceable ± 2%
Other Specify:
6-37
-------
(e) When ordering facility span gas, does the order specify that
the gas must be named in accordance with EPA recommended
practice for naming I/M calibration gas? yes/no
(f) Facility-gas span checks are done:
Through the sampling probe
Through the port
(g) What tolerance is used as a basis for determining the, need for
adjustments? + % of span gas value
If a different tolerance is used, describe:
(h) Leak checks: Indicate the type and frequency of leak checks
performed, and the magnitude of detectable leaks:
Probe/port comparison
Frequency: Magnitude detectable: %
Vacuum leakdown
Frequency: Magnitude detectable: %
Low flow indicator
Frequency: Magnitude detectable: %
Other Specify:
Frequency: Magnitude detectable: %
Explain how these magnitudes were determined:
(i) Describe the frequency and details of other preventive
maintenance (e.g. less frequent multi-point calibration checks,
filter checks, hose cleaning):
(j) Are records required for all routine maintenance and calibra-
tion? yes/no
Describe:
Analyzers with automatic data collection: Describe frequency
and details of preventive maintenance for data collection
mechanism:
6-38
-------
(1) Describe any service agreements (with analyzer manufacturers
or other firms) for analyzer maintenance and calibration,
including the frequency of service visits and details of
service:
(m) What happens if an instrument fails any of the checks listed
above?
7. Station Licensing/Inspector Licensing
(a) For all decentralized programs and decentralized portions of
centralized programs, provide a copy of the inspection station
licensing requirements.
(b) Provide a copy of the training and licensing/certification
requirements for inspectors. Is periodic relicensing or
recertification necessary? yes/no
Describe:
(c) Are repair facilities registered, licensed or certified? yes/no
If yes, describe the details of this program:
(d) Describe the records required to be kept at stations, including
inspection records, analyzer maintenance/calibration records,
copies of audit reports or others:
(e) In decentralized stations, does the agency keep a registry of
inspectors 'and their employing stations? yes/no
8. Record Keeping at Time of Inspection
(a) Attach a copy of the inspection form.
(b) Indicate what data is collected on the inspection form:
Station identification
Inspector identification
Signature of inspector
Vehicle identification number
Vehicle license plate number
Vehicle make
Vehicle model year
6-39
-------
Vehicle odometer reading
Outpoints used
Emission readings
Pass/fail
Waiver information
207(b) compliance certification
Time and date of inspection
Other specify:
(c) How is the data collected?
Manually
Automatically
Combination Explain:
(d) Are repair data collected for failed vehicles? . yes/no
Describe:
(e) Sticker-enforced programs:
Is a sticker number recorded for each, vehicle? yes/no
Can a sticker number be traced to a specific vehicle?
yes/no
Explain:
(f) Non-sticker enforced programs:
Is a certificate of inspection serial number recorded
for each vehicle? yes/no
Can a certificate be traced to a specific vehicle?
yes/no
ExD.lain:
6-40
-------
Data Handling Subsequent to Inspection
(a) Are inspection data for each station summarized periodically?
yes/no
If yes, how often?
(b) Who prepares the data summaries?
Station personnel
Contractor
Program official
(c) Indicate which of the following data appear in the data summary:
Number of inspections
Number of passes
Number of initial failures
Number of stickers or certificates of compliance issued
Number of waivers issued
Other Specify:
(d) Are the data segregated for 1981 and newer vehicles? yes/no
(e) How are data from all stations consolidated?
(f) What periodic reports are prepared for distribution outside the
I/M agency?
(g) What data does the I/M program formally report to EPA? How
frequently?
(h) Provide copies of data summaries, reports, etc.
10. .Station Audit/Surveillance Activities
Attach an official copy of the station audit/surveillance
procedures. Where not completely described in the audit procedures,
provide the following information:
(a) How often are inspection stations audited by program officials?
(Give name of agency and identify which officials (title) will
participate in station audits )
6-41
-------
(b) How many program officials are directly involved in station
audits?
(c) Describe the training provided to auditors:
(d) Describe how internal data analysis is used to assist in
station audits:
(e) Are covert audits performed using unmarked vehicles? yes/no
o Are vehicles pre-set to pass or fail the I/M test?
o How frequently are garages checked with unmarked cars?
(Are garages selected for special audits on the basis of
complaints or other factors?)
Are penalties imposed for infractions? yes/no
Describe:
(f) Checks performed by auditor:
Audit gas span checks
Leak check-type:
Sample line integrity (including probe condition)
Observe quality control checks by station personnel
Check facility calibration gas for correct concentrations
Review station records
Observe inspector testing vehicles
Other (describe):
(g) List the components and concentrations of the audit span
gas(es):
component concentration
(h) The accuracy of the audit span gas is:
NBS traceable +1% NBS traceable +2%
____ Other Specify:
(i) When ordering audit span gas, does the order specify that the
gas must be named in accordance with the EPA recommended
practice for naming I/M calibration gas? yes/no
6-42
-------
(j) How are stickers/compliance certificates accounted for?
(k) Are records/stickers collected? yes/no Explain:
(1) In a decentralized program with manually collected data, do the
official procedures instruct the auditor to examine forms for
errors or discrepancies and provide feedback to the stations?
yes/no
(m) Are penalties for infractions specified? yes/no
(n) Does the auditor report the result of the audit in writing?
yes/no If yes, provide a sample audit form or report.
(o) Where are copies of the auditor's report filed"?
11. Challenge Mechanism
(a) Describe how owners can obtain an unbiased challenge test;
describe the procedures, equipment, etc.
(b) Is there a fee for this test? yes/no amount: $
(c) How are motorists notified of the availability of the challenge
test?
12 . Repair Waivers
(a) Describe provisions for repair waivers or other mechanisms to
get a sticker/certificate without passing cutpoints:
(b) How much must a motorist spend on emission-related repairs in
order to receive a waiver?
Can the cost to repair tampered components be included in the
cost limit? yes/no
(c) Are certain repairs required? yes/no
If yes, describe:
(d) Are minimum emission reductions required? yes/no
If yes, describe:
(e) Can tampered vehicles receive waivers? yes/no
If no, describe how tampered vehicles are prevented
receiving waivers:
6-43
-------
(f) Can vehicles with warranty rights receive waivers? yes/no
(g) Documentation required:
Repair receipts
Repair form (provide copy)
Other Specify:
How is documentation verified and by whom?
(h) Describe any methods used to track waivers by garage or
mechanic:
13. Enforcement
Identify the basic enforcement approach:
Registration-enforced
Sticker-enforced
Data link
Combination Describe:
For Registration-Enforced Programs:
(a) Describe methods used to prevent motorists from avoiding
inspection (by registering in another county, registering as a
commercial vehicle, registering as a higher weight class, etc.):
(b) Describe step by step how the enforcement system works for a
particular vehicle; identify the responsible office; describe
how documentation of passing the test is created and handled;
identify which steps are manual and open to error:
(c) Explain how blank certificates are accounted for to prevent
abuse:
For Sticker-Enforced Programs:
(a) Are exempted vehicles given stickers? yes/no
If yes, how are exemption stickers distributed to the correct
vehicles?
(b) Explain how police can distinguish between exempt, complying
and noncomplying vehicles:
6-44
-------
(c) Explain how stickers are accounted for to prevent abuse:
(d) In the case of a failed vehicle that is not repaired
immediately, describe the type of temporary sticker issued, its
period of validity, and how its expiration is determined:
(e) Can parked vehicles be ticketed by police? yes/no
If yes, on-street only, or in publicly owned lots, or on
private property?
(f) Identify which police agencies can give citations:
(g) What is police policy on I/M enforcement? (Only ticket cars
stopped for other violations? Active enforcement of stickers?)
(h) What is the fine for a sticker violation?
Is there a grace period? yes/no grace period:
Is a court appearance necessary? yes/no
Are these cases handled in traffic court? yes/no
Identify the prosecuting-office:
Explain:
(i) Does the fine go the State or local treasury?
(j) Are sticker surveys conducted? yes/no
By whom?
How often?
Survey size?
(k) Describe any other procedures used to assess the level of
sticker noncompliance:
(1) Is the total number of citations issued by all police agencies
monitored? yes/no
Data-Link Enforcement:
(a) How are vehicles scheduled for inspection?
(b) How are motorists notified?
(c) Explain how delinquent vehicles are identified, and the length
of time involved:
6-45
-------
(d) Describe the follow-up procedure for dealing with delinquent
vehicles, including the length of time involved:
(e) How are vehicles which have left the program area or ba^.
junked identified and purged from the list?
(f) What are the penalties for non-compliance?
(g) Is a court appearance necessary? yes/no .
Is there a grace period? yes/no grace period:
Are these cases handled in traffic court? yes/no
Identify the prosecuting office:
Explain:
(h) Is there a back-up enforcement method (some way to identify
non-complying vehicles on the road)? yes/no Explain:
(i) Is there any way to assess the level of compliance separately
from the data-link tracking system? yes/no
Explain: :
14. Mechanics Training and Licensing/Certification
(a) Provide a copy of the course curriculum outline, includi^^
names of the instructor(s), number of classes, course duration
(hours of instruction) and fees.
(b) Provide a copy of the registration, licensing and certification
requirements for mechanics. Is periodic relicensing or
recertification required? yes/no
(c) Describe any follow-up with training participants, including
the use of newsletters, mailing lists, etc:
(d) Explain how mechanics training is promoted:
(e) Is a repair assistance hotline provided? yes/no
Describe: .
(f) How are repair manuals made available to mechanics who have not
attended training courses?
15. Consumer Issues
(a) Provide samples of brochures and other materials on public
awareness that are available to motorists.
6-46
-------
(b) Indicate what information is provided to motorists at the time
of inspection:
Types of repairs to be expected for different types
of I/M failures
Expected range of repair costs for different repairs
Lists of certified repair facilities
Where to call to get information/assistance or
to file a complaint
Fuel economy indicators from emission readings
Normal emissions levels of tuned vehicles
Emission performance warranties
(c) Is a consumer assistance hotline used? yes/no
Describe:
(d) Describe any other I/M public awareness efforts:
16. Self Assessment Through Data Analysis
Describe agency efforts to internally monitor program activities
through data analysis, covering what is done in the following areas:
(a) Tracking failure rates, waiver rates, - etc. overall and by
inspection station or by inspector.
(b) Tracking performance of scheduled audits.
(c) Tracking audit results to ensure proper station performance.
(d) In sticker/certificate systems, maintaining sticker/certificate
accountability.
(e) Tracking number of inspections to assess compliance rates.
(f) Reviewing average emissions levels before and after repair to
assess effectiveness of repairs.
(g)' Tracking types of repairs and/or repair costs to assess
appropriateness of repairs, impact on the public, and adequacy
of existing repair cost limits.
(h) In sticker systems, conducting periodic field sticker surveys
to assess overall level of compliance as well as to identify
any local hotspots of non-compliance.
6-47
-------
17. Other Data Collection Activities
(a) Are roadside pullovers used to check emissions and/or check
tampering? yes/no
If yes, attach a description including equipment used,
calibration procedures and types of data collected.
(b) Are public opinion surveys performed? yes/no
If yes, attach a description of questions asked, data
collected, etc.
(c) Are repair cost surveys performed? yes/no
(d) Is data collected from motorists complaints? yes/no
Describe:
(e) Describe any other method of collecting data to evaluate the
program:
18 . Future Plans
Indicate any plans for changing basic parts of the program.
Cutpoint changes. Describe:
Changes in testing procedures. Describe:
Changes in vehicle coverage, i.e. more model year coverage,
higher weight class coverage, broader geographic coverage in
existing area, etc.
Describe:
Additions of program areas, i.e., new cities.
Describe:
Improvements or reductions in data analyses and/or quality
control.
Describe:
Additions of tampering or emissions checks.
Describe:
Changes in waiver criteria. Describe:
6-48
-------
Changes to accommodate the emissions performance warranty
requirements.
Describe:
Changes in program type, fees, and any other administrative
factors.
Describe:
6-49
-------
I/M Audit Questionnaire
Part II: Operating Experiences
1. Operating Statistics - Emissions Inspections/Tampering Inspections
2. Quality Control Statistics
3. Data Analyses
4. Consumer Protection
5. Repair Waivers
6. Enforcement
7. Mechanics Training
8. Self Assessment Through. Data Analysis
6-50
-------
Instructions
(1) EPA staff normally will not have access to the data required to
answer this portion of the questionnaire. If this is true, the
questionnaire should be forwarded to the State/local agency with a
request that it be completed and returned to EPA prior to the site
visit.
(2) If EPA has access to suitable reports from the I/M program, EPA
staff will complete the questionnaire to the extent possible and
make narrative comments on topics of special concern before
forwarding the questionnaire to the State.
(3) Additional answer sheets should be inserted as needed.
6-51
-------
Operating Statistics - Emissions Inspections/Tampering Inspections
(a) Identify:
(1) The number of licensed inspection stations (non-fleet).
(2) The number of fleet inspection stations.
(3) The number of vehicles required to be inspected.
(4) The number of inspectors involved in the program.
(b) Provide the following data for emissions inspections only,
tampering inspections only, and both emissions and tampering
inspections (joint statistics). If data as described are not
available, provide similar data which fit the description as
closely as possible. Where no similar data are available.
describe what transcription and/or processing would be required
to obtain similar data.
(1) Number of vehicles inspected at licensed inspection
stations and at fleet stations, and failure rate (by
month) for all vehicle ages combined over the previous 12
months. (If procedures or cutpoints are different for
different vehicle types (passenger cars, light trucks,
heavy duty trucks) and the data are kept segregated,
provide separate statistics for each type.)
(2) Failure rate by model year (or cutpoint group) and vehicj^
type (for previous L2 month period as a whole). ^^
(3) Failure rate as above by separate urbanized areas or other
regional groupings.
(4) Number of first, second and third emissions retests.
(5) If inspections and reinspections may be performed at
either centralized stations, or decentralized licensed
stations, identify the number of initial emissions
inspections and reinspections at each type of station.
(c) For previous years, identify for each year:
(1) The number of vehicles inspected.
(2) The failure rate.
(d) For tampering and misfueling inspections, identify:
(1) Failure rates oy component inspected for the previous 12
months (by month).
6-52
-------
(2). Failure rates by component inspected for previous years
(by year).
(3) The number of referee cases in the last 12 months (by
month) and in previous years where:
a. A referee was requested to determine whether a
vehicle was originally equipped with certain
components.
b. The determination (of (a)) was in favor of the owner
(i.e., it was determined that the vehicle was not
originally equipped with the component in question).
c. A referee was requested for a lead-sensitive
(Plumbtesmo) test paper failure.
d. The determination (of (c)) was in favor of the owner.
(e) Provide statistics from any tampering surveys.
Quality Control Statistics
(a) For the previous 12 months, provide the following data (by
month) for audits of inspection stations. Where audits were
performed at both centralized and decentralized inspection
stations, provide separate data for each:
(1) Number of overt audits performed.
(2) Number of covert audits performed.
(Provide tabular results of all covert audits.)
(3) Number of analyzers checked with span gas.
(4) Number of analyzers failing the span gas checks.
(5) Distribution of errors (i.e., percentage of analyzers
within +5%, +10% (etc.) of bottle value and percentage
within -5%, -10% (etc.) of bottle value).
(6) Number of analyzer leaks identified during audit.
(7) Number of temporary facility or analyzer shutdowns
resulting from audits.
(8) Number of penalties and warnings resulting from audits
(describe nature of penalties and/or warnings).
(b) Provide data on any quality control problems repeatedly
identified through audits and how they were investigated and
resolved. (Include any span gas problems and/or analyzer
problems. )
6-53
-------
3. Data Analysis
(a) For each of the last 12 months, identify:
. (1) Mean HC and CO readings of passing vehicles (by model year
(or outpoint group) and vehicle type).
(2) Mean HC and CO readings of failing vehicles (by model year
(or cutpoint group) and vehicle type) for both initial and
final tests.
(3) Number of stickers or certificates issued (by vehicle
type).
(b) Provide copies of the latest inspection data summaries and
repair cost summaries.
4. Consumer Protection
For the last 12 months identify:
(a) The number and type of motorists' complaints received.
(b) The number and type of consumer inquiries received.
(c) The number of challenge tests provided and how the results
compared to initial tests at program test stations,
5. Repair Waivers
For each of the last-12 months identify:
(a) Waiver rates (by model year (or cutpoint group) and vehicle
type).
(b) Mean HC and CO readings of waived vehicles at initial and final
inspection.
6. Enforcement
Registration Enforcement:
Estimate the percentage of motorists evading the inspection
requirement by registering outside the I/M program area.
Exnlain how this pstimate was derived.
L equ ± L eiueuL uy L t?y la L t?L xiiy uui_s>iu.t;
Explain how this estimate was derived.
6-54
-------
Sticker Enforcement:
For each of the past 12 months identify:
(a) The number of citations issued, number of fines
imposed and the average fine.
(b) The results of any local sticker surveys.
Data Link Enforcement:
For each of the past 12 months identify:
(a) The number of vehicles at each stage of the process
as of the first of the month.
(b) The number of penalties assessed and the average fine.
(c) The number of cases of failing to be inspected on
time that were resolved without penalties.
(d) The average number of days from missed inspection to
each subsequent step in the process.
Mechanics Training
(a) For the previous 12 months identify:
(1) The number and type of mechanics training courses offered
(include a general description of the course content).
(2) The number of course participants.
(3) The number of participants passing the course.
(4) The number and nature of inquiries from mechanics.
(b) For previous years identify:
(1) The number of mechanics trained each year.
(2) The number of mechanics passing the course each year.
(c) Identify:
(1) The total number of licensed or certified mechanics.
(2) The total number of licensed or certified garages.
(d) Describe the size and make-up of the circulation list for
periodic newsletters or other information distribution.
6-55
-------
Self-Assessment Through Data Analysis
(a) Describe results of recent internal assessments; provide copj
of data or narrative reports, if possible.
(b) Identify and describe significant problems identified through
the internal control/self assessment system, and how these
problems were resolved.
6-56
-------
APPENDIX C
DESCRIPTIONS OF ON-SITE ACTIVITIES
1. Surveys
2. Records Review
3. Procedures Observations
6-57
-------
APPENDIX C
DESCRIPTIONS OF ON-SITE AUDIT ACTIVITIES
l. Surveys
The activities denoted as surveys consist of information
collection activities that involve the examination of
vehicles and equipment. State or local cooperation and
participation should always be sought. This will make EPA's
job easier, assuage any public suspicions, and may provide a
learning experience for the State or local personnel who
accompany the EPA auditors.
Enforcement Survey
In I/M programs with sticker enforcement (or with another
form of enforcement but accompanied by a sticker which
indicates compliance) a sticker survey must be performed.
Instructions and sample recording forms are included in
Appendix E. The purpose of the survey is to determine what
percentage of subject vehicles are complying with the
inspection requirement as indicated by a valid sticker,
versus the percentage of vehicles without a sticker or with
an expired sticker.
A sticker survey must be conducted in each I/M urbanized area
with population over 200,000. The goal is to examine a
sample of about 1000 randomly selected vehicles in each
urbanized area. For practicality, it is acceptable to survey
vehicles which are parked on-street, in paid off-street
parking, or in free public lots such as at shopping centers.
Roadside pull-overs or toll booth observations would also be
acceptable. At- least 5 widely spaced locations of several
types in each urbanized area should be surveyed to get the
sample of 1000 vehicles, so that the sample better represents
the area as a whole. At least one location should be in the
central business district. The key in conducting the survey
is to avoid known sources of bias, but, at the same time, be
able to determine whether there are variations in
noncompliance rates in different parts of the urbanized area.
The sticker survey offers a convenient opportunity to get a
rough measure of the influence of non-subject (e.g.,
out-of-county) vehicles, since non-subject vehicles will have
to be identified in the- survey to avoid bias to the
compliance rate estimate. Consequently, non-subject vehicles
should be counted and recorded, rather than just passed by.
If there are a significant number of non-subject vehicles
operating in the I/M area, EPA may in the audit report
recommend expansion of program boundaries or vehicle coverage.
6-58
-------
The sticker survey is not intended to identify specific
vehicles or owners for adverse action, so vehicle identifiers
should not be recorded. The only exception would be when
subject vehicles cannot be identified without a registration
crosscheck (such as in a computer matching system) in which
case license plates should be recorded for unstickered
vehicles only.
Investigation into compliance rates is not required in
registration-enforced programs unless there is reason to
suspect that registrations are being processed without
required inspection documentation. Such suspicions should be
investigated on a case-by-case basis.
Data-linked enforcement programs should be investigated via
records review to determine how many vehicles are at each
stage of the enforcement sequence.
Idle Test Survey
In all I/M programs with decentralized inspections, or with
decentralized reinspections following failure at a
centralized station, an idle test survey is required.
Whenever possible, idle test surveys should be conducted in
centralized programs, especially if there is reason to
believe that there is widespread readjustment after
inspection.
The purpose of the survey is to verify that recently
inspected and stickered/certified vehicles generally do meet
State or local cutpoints, that the percentage which do not
meet cutpoints increases only gradually with time since
inspection, and that vehicles due for inspections very soon
have a failure rate consistent with the failure rate reported
by the decentralized inspection network. Failure to verify
these three expectations indicates the probable existence of
one or more operating problems: cheating or mistakes by
licensed inspection stations, inadequate repairs, excessive
waivers, widespread readjustment following inspection, or
inaccurate reporting of inspection results by stations.
Further investigation to identify the specific problems would
then be necessary.
The survey requires that vehicles be obtained, that an idle
test be performed with appropriate preconditioning and
instrumentation quality control, and that the date (month and
year) of the last inspection be obtained. Usually the latter
can be determined from the inspection sticker or license
plate expiration date. Ideally, vehicles would be randomly
selected with no opportunity to decline, so that bias is
avoided. Compromises from this ideal will often be
necessary; however, EPA will strive to conduct the idle test
6-59
-------
surveys in a manner at least as unbiased as the regular
tampering surveys conducted by EPA's Field Operations and
Support Division. A description of the survey procedures
used by FOSD in the past is found in Appendix D.
It is recognized that the idle test survey, along with the
tampering survey discussed next where needed, will be the
logistically most difficult part of the I/M audit. Special
equipment, special skills, and cooperation by the State and
local authorities will be needed. The EPA Office of Mobile
Sources anticipates that all idle test surveys can be
conducted in conjunction with QMS's regular program of annual
tampering surveys. Combining the idle test surveys with
OMS's tampering surveys will also eliminate the need for
additional clearance from the Office of Management and Budget
for conducting public surveys. It may be necessary to
postpone the idle test survey until after the rest of the
site visit, in which case another visit or telephone call may
be necessary to discuss the results with the State or local
agency.
Tampering Survey
In programs with required inspections of some or all vehicles
for tampering and misfueling and additional emissions
reduction credits are claimed for them in the SIP, a
tampering survey must be performed. . In other areas,
tampering surveys are strongly recommended. Its objectives
are similar to those of the idle test, and similar
statistical and logistical considerations apply. A tampering
survey will be more complicated because special inspection
skills are needed, more time is required to inspect each
vehicle, and more data need to be recorded and analyzed.
Because of these complexities, and because of the requirement
for clearance from the 'Office of Management and Budget,
tampering surveys must always be coordinated with OMS's Field
Operations and Support Division.
Analyzer Audit, Centralized Programs
In centralized programs all of the active analyzers in at
least three inspection stations should be audited. These
audits may be performed by EPA personnel or, if the State/
locality prefers, EPA personnel may watch State/local
personnel check the analyzers. State/local span gases can be
used for analyzer checks, as long as it is verified that the
State/local gas is named properly.
In centralized programs with multiple urbanized areas, the
site visit can be limited to one urbanized area (assuming
there are at least three stations there), as long as one of
the following conditions is met:
6-60
-------
1. There are independent routine audits of all stations
in the other urbanized areas conducted by an outside
group or agency other than the one which performs
routine calibration and maintenance, the results of
these audits can be reviewed through records, and the
EPA auditors have observed at least one audit by this
outside group or agency.
or,
2. The same State/local personnel perform the routine
calibration and maintenance in all urbanized areas.
Otherwise the EPA auditors must audit the analyzers in three
stations in each urbanized area.
An analyzer audit consists of a calibration check through the
probe and a low flow indicator check. State/local and
contractor cooperation will be needed to audit analyzers that
are in service in open inspection lanes. Instructions and a
recording form are found in Appendix E.
2. Records Review
In association with each of the following records reviews,
the EPA auditors should seek an understanding of how the
records are generated and handled by the I/M program. Where
it seems useful and practical, copies of records should be
requested.
Vehicle Records
Recent documentation (inspection forms, retest forms, repair
forms or receipts, and waiver forms) must be reviewed from at
least 500 vehicles. In a decentralized program, these
records must come from at least 10 different inspection
stations. In centralized programs with computer-printed
inspection forms, test records for passing vehicles need not
be reviewed and the number of vehicles may be reduced
accordingly. Care should be taken that the forms are from
typical cases and that they have not been pre-screened before
being provided to EPA. This review must be completed for
each urbanized area, but they may be shipped to any
convenient location for the review.
The auditors should examine the forms for completeness,
legibility, accurate application of inspection standards,
reasonableness of the test scores, correct pass/fail
determination, appropriateness of repairs, reductions in
emission levels from repairs, and adequacy of documentation
for a waiver if one was given. If severe deficiencies or
6-61
-------
repeated errors are noted for a licensed inspection station,
the EPA auditors should ask to be allowed to review the
records of past audits and of past and ongoing corrective
action towards that station.
Station Audit Records, Centralized
EPA auditors should review the records created by the routine
State or local audit of a few inspection lanes, to
familiarize themselves with the procedures used by the
auditors and the data available from their activities.
Station Audit Records, Decentralized
SPA auditors should review the records created by 3 full days
of decentralized station auditing by each State or locality
field investigator. EPA auditors should -look for audit
completeness, adherence to procedures, and indications that
inadequate performance by licensed stations is routinely
identified and corrected. . Instructions and a form are found
in Appendix E.
In centralized programs with authorized self-inspecting
fleets which together Account for 5 percent or more of annual
inspections, records :rom at least 20 fleet audits should be
reviewed using the same procedures and form as decentralized
programs.
Data Summaries
If the I/M program generates periodic data summaries not
previously made available, these should be reviewed on site
for the last few reporting periods. The manner in which the
summaries are produced and the meaning of all entries should
be understood.
Licensing/Suspension Records
Files relating- to the disciplining of licensed inspection
stations or fleets which do not adhere to procedures should
be examined to determine the general nature "of the State's or
locality's practices in such cases.
Consumer Inquiries and Complaints
If the I/M program keeps such records, they can be scanned to
determine the nature of such interactions with the public.
These activities should be considered a low priority" unless
other audit findings suggest a need to review these records.
6-62
-------
Enforcement Records
Statistics on recent and current enforcement activities
should have been obtained during the audit preparation.
While on-site, the EPA auditors should verify the enforcement
procedures and general level of activity by reviewing records.
Other Records
Unique program features or earlier findings during the audit
may suggest other records which should be reviewed.
3. Procedures Observation
Much, of the on-site visit will consist of observing I/M
officials or licensed inspectors perform their regular
functions to determine if actual operations are consistent
with the documented program design, questionnaire answers
supplied by the I/M program, and good engineering and
management practice. It will, of course, be impossible to
observe every individual and function, and problems may
escape notice. Suspicions raised' by record reviews, surveys,
and interviews may make it advisable to intensify the
observation of procedures compared to the minimums suggested
here.
Audits of Inspection Stations, .Centralized
The regularly scheduled audit of 1 centralized inspection
station by State/local personnel should be observed. If no
such audits are scheduled during the site visit, at least one
special audit should be requested. If more than one agency
routinely performs audits of the lanes, the agency with
authority to readjust analyzers and shut down lanes should be
observed. The audit "records of all auditing agencies should
be reviewed for consistency of findings.
Audits of Inspection Stations, Decentralized
EPA auditors .should accompany State or local officials as
they visit licensed inspection stations on regular audits.
The EPA auditors should observe how the State/local employees
conduct the audit: whether written procedures are followed by
the auditor, whether the auditor has the expertise to
correctly respond 'to questions from the station personnel,
whether the auditor performs other important functions not
documented in written procedures, and whether and how the
auditor reacts to equipment defects or misperformance by
station employees. Whenever possible, the auditors should
observe inspections being conducted and examine any records
being maintained by.the station. A sample form for observing
audits is found in Appendix E; it should be revised as
6-63
-------
necessary to reflect the specific content of the I/M
program's written audit procedures.
A total of 10-20 licensed inspection station audits divided
among at least 3 State/local auditors should be observed by
EPA. If possible, EPA staff should select the specific
State/local auditors without advance notice. This number of
station audits must be observed in each urbanized area.
In centralized programs with, authorized self-inspecting
fleets which together account for 5 percent or more of annual
inspections, at least 3 fleet audits should be observed using
the same procedures and form as for decentralized programs.
Inspections, Centralized
EPA staff should observe at least 30 inspections by
centralized inspectors, not all at one station. These
observations may be conveniently combined with the surveys of
centralized analyzers. If heavy-duty vehicles are inspected
at a separate location, several such inspections should be
observed if time permits. Observations of inspections should
include retests and waiver processing where they are integral.
Waiver Processing
Where waivers are granted separately from the retest, EPA
auditors should observe several waivers being processed. If
this observation suggests that the review of a larger number
of waiver records (as discussed in Section 2) will not
adequately characterize the true operation of the waiver
system, additional processing should be observed.
Spot Checks Using Unmarked Cars
Some decentralized I/M programs conduct spot checks using
unmarked cars at licensed inspection stations. Such checks,
particularly if they use vehicles adjusted to fail standards,
can be a very important part of the program's quality
assurance efforts. An EPA auditor should observe the
procedures used first hand by accompanying a State/local
official on a spot check. EPA should also discuss with
program officials what is done when a station "fails" a spot
check and how stations are selected for this surveillance.
Other Procedures
Other activities should be observed as necessary. For
example, where questionnaire answers or records review
indicates a shortfall of vehicle inspections in a
registration enforced system, it is recommended that the
registration renewal process be observed.
6-64
-------
6-65
-------
APPENDIX D
PROCEDITRES USED IN QMS TAMPERING SURVEYS
6-66
-------
APPENDIX D
PROCEDURES USED IN QMS TAMPERING SURVZTS
EXAMPLE PROJECT PLAN
MOTOR VEHICLE TA.1PERING A.VD FUEL SWITCHING SURVEY (1983)
I. Objectives
a. Update previously collected tampering and fuel switching
data.
b. Determine current tampering and fuel switching trends for:
(1) the most prevalent types of tampering and fuel switching,
(2) the effects of tampering and. fuel switching on vehicle
emissions, (3) the amount of ^tampering and fuel switching by
age and manufacturers.
II. Background
A significant part of the Nation's efforts to achieve the clean
air standards has been directed toward control of motor vehicle
emissions. Motor vehicles account for nearly three-quarters of
the total carbon monoxide, over one-third of the hydrocarbons, and
one-third of the oxides of nitrogen emitted to the atmosphere. To
reduce these emissions, automotive manufacturers have been required
to install control devices on certain classes of new vehicles.
Each manufacturer must certify that the control devices will meet
the established emission standards.
Congress has passed laws making it illegal to discount or modify
emission control devices. As of August 8, 1977, these laws were
broadened to prohibit all automobile service or repair facilities,
as well as dealers, from knowingly tampering with a car's emission
control devices. Thes laws are contained in the 1977 Amendments
to the Clean Air Act under sections 203(a)(3)(A) and 203(a)(3)(B).
Regulations issued pursuant to Section 211 of the Clean Air Act
make it illegal for retail outlets to introduce or allow the
introducton of leaded fuel into vehicles which require unleaded
fuel.
Ill. Required Studies
Studies shall be made at selected locations throughout the United
States. These locations represent geographic diversity with some
of the locations previously surveyed being repeated.
A minimum of 250 and a maximum of 300 vehicles at each location
shall be inspected. Inspections shall be limited to .1975 and newer,
light-duty vehicles. The vehicles shall be selected on a random
basis.
6-67
-------
It is expected that each vehicle inspection will require approxi-
mately three to five minutes to thoroughly check all., emission
control systems, record basic data, measure idle HC and CO
emissions, and obtain a fuel sample from vehicles requiring
unleaded gasoline and measure lead deposits in the tail pipes
of all vehicles. A detailed explanation of the data required and
recording procedures is contained in Attachment A. The procedures
for collecting-, labeling and shipping fuel samples', are contained
in Attachment B. Instructions for using the Plurabtesmo test paper
are contained in Attachment C. The procedures for emission.
testing are contained in Attachment D. Calibration requirements
for the HC/CO analyses are contained in Attachment E.
A four member team, 3 contractor personnel and at least one EPA
representative, will be used at each location. The EPA repre-
sentative will be responsible for communicating with any news
media and explaining the objectives of the survey as presented
in this project plan. EPA representatives will be responsible
for a log of inspection refusals showing date, time, make-model-year
of vehicle and the reason for refusing the inspection. EPA repre-
sentatives will also do a background report for each site which
will include the exact situation in which vehicles were procured,
a geographical description of the site, weather, and other circum-
stances that might affect refusals, who the inspectors are on a
particular site, how many and vhich observations were made at
each site, and any other circumstances that might bear upon the
representativeness of the data.
IV. Suggested Equipment
2 - HC-CO Gas Analyzers with sample lines, water trap and tailpipe
probe
1 - Calibration Gas ± 2% of listed concentration Nominal
18% CO
1560 ppm HC (Hexane equivalent)
4% CO
827 ppm HC (Hexane equivalent)
1.6% CO
320 ppm HC (Hexane equivalent)
3 - Inspection Mirrors
1 - Large long-handled mirror for exahust system inspection
2 - Flashlights .
2 - Vacuum Pumps
2 - Fender Covers
6-68
-------
2 - Fuel Sampling Pumps with 3 ft hoses
350 - Sample Bottles per site
2 - Power' Inverters for sites without power
1 pair - Battery Jumper Cables
2 - Leaded nozzles
2 - .One Gallon Gasoline Cans
1 50 ft Extension Cord
2 - Screwdrivers and pliers
350 - Plumbtesrao paper strips per site (Plumbtesmo oust not be
more than 12 months old and one piece from each box must
be tested within a week before use on a leaded vehicle or
by some other means to assure that the paper is still
sensitive to lead.)
200 pairs - Surgical gloves
200 - Wooden clothes pins with spring action
- Sufficient boxes, sample bottles, labels, packing and
shipping labels, and sample tags to label and ship up to
350 fuel samples per site. Sufficient data sheets to
process up to 500 cars per site.
V. Analytical Requirements
Gasoline samples collected during this survey will be analyzed
by EPA for lead content using x-ray fluorescence spectroscopy
procedures.
Samples shall be stored in an explosion-proof refrigerator. In
less than four weeks, the samples shall be brought to room
temperature and analyzed. Any sample with a lead content
>_0.05 g/gal shall be placed back in the refrigerator; other
samples .'after .analyses shall be stored in a dedicated solvent
cabinet.
Duplicate results that differ by more than 0.005 g/gal shall be
rejected.
Sample results will be reported in grams per gallon to three
significant figures.
6-69
-------
DATA COLLECTION' AXD RECORDING PROCEDURES
The following data will be recorded on a "Part A" data sheet (shown as
Figure A-l).
a. Vehicle identifying survey number - Vehicles shall be numbered
sequentially as they are inspected, and this number shall be
preceded by a site identifying letter.
b. Odometer mileage (in thousands) .
c. Dash Label - will be coded as follows:
0 - Not originally equipped
1 - Functioning properly
7 - Missing item
d. Make
e. Model
f. Model year - obtained from uaderhood emission label.
g. Vehicle type - car or truck.
h. License plate - state of registration.
i. Engine' family/CID (cubic inch displacement) as recorded from the
underhood emission label.
j. HC in ppm arid CO in percent with the engine at curb idle.
k. Pluabtesmo - Plumbtesrao paper is used to check for the presence
of lead in vehicle exhaust pipes. A positive indication will
be coded as "P" and a negative as "N".
1. Tank cap - Part of the evaporative system, the tank cap seals with
the filler neck to maintain a closed system. Tank caps will be
coded as follows:
1 - Functioning properly
7 - Missing item
9 - Malfunctioning item - Tank cap not sealing properly
6-70
-------
m. Tank label will be coded as follows:
0 - Not originally equipped
1 - Functioning properly
7 - Missing item
a. Filler neck inlet restrictor (unleaded vehicles only) - The
restrictor is designed to prevent the introduction of leaded
fuel into a vehicle requiring unleaded fuel. If will be coded
as follows:
0 - Not originally equipped .
1 - Functioning properly
4 - Mechanical disconnect - Widened to fit a leaded filler nozzle
7 - Missing item
o. Catalytic converter - Oxidizes the CO and HC to water and CO. in
the exhaust gases. The converter will be coded as follows:
0 - Not originally equipped
1 - Functioning properly
7 - Missing item
9 - Malfunctioning item - High temperature discoloration usually
light blue
p. Exhaust system is coded "P" if it is original equipment. If it is
afteraarket or non-stock a "6" is coded.
q. Exhaust system integrity shall be coded as:
1 - Functioning properly (no obvious leaks)
9 - Malfunctioning (leaks evident)
r. Oxygen sensor - look for this item imaediately upstream of the
catalyst. This item should be coded as follows:
0 - Not originally equipped
1 - Functioning properly
2 - Electrical disconnect
4 - Mechanical disconnect (unscrewed from exhaust system but still
connected electrically) ' .
6-71
-------
7 - Hissing item
s. Inspectors initials and date.
The following data will be recorded on the "Part B" data sheet (shown
in Figure A-2).
a. Vehicle identifying survey number - numbered the same as Part A.
b. Air cleaner, exhaust manifold, intake manifold, and distributor -
if these parts are original equipment a "?" is coded. If these
parts are aftennarket or non-stock a "6" is coded.
c. Turbochar.ger may be coded "P", "6", "A", or "0". The "A" is
used to indicate add-on equipment, the "0" for not equipped, and
"6" is used to indicate a non-stock replacement part.
d. Carburetor Type - A "P" is used to indicate that the carburetor
is a production unit (non-sealed original equipment). If the
carburetor is a sealed unit (without liaiter caps), an "S" is
recorded. If fuel injection is used, then a "F" is recorded. If
the carburetor has been replaced vith a non-stock'unit, then a "6"
is recorded.
e. Carburetor barrels - This item shall be recorded as to the number
of barrels:
0 - Not equipped (fuel injected)
1 barrel
2 barrels
4 barrels
f. Limiter caps - Plastic caps on idle mixture screws designed to
limit carburetor adjustments. Limiter caps will be coded as
follows:
0 - Not originally equipped
1 - Functioning, properly
4 - Mechanical disconnect - Tabs broken or bent
7 - Missing item
8 - Misadjusted item (sealed plugs have been removed)
6-72
-------
Positive crankcase ventilation system - A typical configuration
for a V-8 engine consists of the PCV valve connected to a valve
cover and then connected to the carburetor by a vacuum line. The
other part of the system has a fresh air tube running from the
air cleaner, to the other valve cover. The PCV will be coded as
follows:
0 - Not originally equipped
1 - Functioning properly
3 - Vacuum disconnect - When the line between the PCV valve and the
carburetor is disconnected
I* - Mechanical disconnect - When the fresh air tube between the
valve cover and the air cleaner is disconnected or removed
6 - Non-stock - When fuel economy device is installed in PCV line
7 - Missing item - When the entire system has been removed
Idle stop solenoid - This solenoid provides an idle stop for
maintaining idle speeds to the higher speeds needed to minimize
CO emissions. On some vehicles, it is used to close the throttle
and thus prevent run-on when the engine ignition is turned off.
On vehicles with air conditioning, it is used for increasing
engine idle speed to compensate for a decrease in idle speed
when the air conditioner is engaged.
With the air conditioner on (or in non-air conditioned vehicles),
the solenoid should activate and contact the throttle linkage.
With the air conditioner turned off, there should be a small
gap between the solenoid stop and the throttle linkage.
The idle stop solenoid will be coded as follows:
0 - Not originally equipped
1 - Functioning properly
2 - Electrical disconnect
7 - Missing itea
9 - Malfunctioning - If the gap between the solenoid and the
throttle plate is incorrect.
Heated air intake - Provides warn air to the 'carburetor during
cold engine operation. The heated air intake will be coded as
follows:
0 - Not originally equipped
6-73
-------
1 - Functioning properly
3 -.Vacuum disconnect - If the vacuum line to the vacuum override
motor is missing or disconnected.
4 - Mechanical disconnect - When the stovepipe is disconnected or
deteriorated. Also when the air cleaner has been unsealed,
i.e., inverted air cleaner lid, oversized filter element, or
holes punched into air cleaner.
6 - Non-stock equipment - Custom air cleaner.
7 - Hissing -item - Missing stovepipe hose
9 - Malfunctioning item - Problems with the vacuum override motor
Evaporative control system - Controls vapors from the fuel tank and
carburetor. Some systems have two lines, one from the fuel tank
to the canister, and one from the canister to the carburetor or
air cleaner to air purge the canister. Other systems have a third
line, usually connected to the carburetor. The ECS will be coded
as follows:
0 - Not originally equipped
1 - Functioning properly
3 - Vacuum disconnect - Line from canistep to carburetor or air
cleaner disconnected
4 - Mechanical disconnect - Line from fuel tank to canister
disconnected
5 - Incorrectly routed hose
7 - Missing item
9 - Malfunctioning item - When the purge line is connected to the
air cleaner and the air cleaner is unsealed
Air injection system - Consists of an air purap driven by a belt
connected to the crankshaft pulley or an aspirated air injection
system. In the forme_r case the piunp directs air through a control
valve and line connected to the exhaust manifold. An air injection
system may also consist of an aspirator located in the air cleaner
that supplies air to the exhaust manifold. A conventional air
injection.system is broken dovn into two parts which are coded as
follows:
Air pump
0 - Not originally equipped
6-74
-------
1 - Functioning properly
4 - Mechanical disconnect (other than belt removal)
7 - Missing item
9 - Malfunctioning
Air pump belt
0 - Not equipped
1 - Functioning properly
7 - Missing item
8 - Misadjusted itea - Loose pomp belt
For vehicles equipped with aspirated air systems, the air pump
belt and air puarp (above) should be coded "0" and the condition
of the system indicated with a category of "Aspirated air
injection system" - coded as follows:
Aspirated air injection system
0 - Not.originally equipped (if conventional system or none)
1 - Functioning properly
4 - Mechanical disconnect
7 - Missing itea
9 - Malfunctioning
Exhaust gas recirculation (ECR) system - The standard configuration
consists of a vacuum line from the carburetor to a sensor (used to
detect temperature to activate the EGR valve), and another vacuum
line from the sensor to the EGR valve. Some systems have multiple
sensors' and thus additional vacuum lines. The system directs a
portion of the exhaust gases back into the cylinders for the control
of oxides of nitrogen. This is one system, where a functional check
will be performed. Non-sealed ECR valve functional check will
consist of:
1. Visually inspecting to see if the valve, sensor(s) and hoses
are in place.
2. If the system is intact, revving the engine and checking
visually or by touch the EGR valve steal movement.
6-75
-------
3. If the stem falls to move, pulling off the vacuum .line to
the valve and checking for vacuum while the engine is revved.
If vacuum occurs, the valve is not functioning and the hose
aipple on the valve vill be checked for blockage. If vacuuo
does not occur, the line will be checked for blockage. If
it is not blocked, a hand vacuum puap will be connected to
the sensor outlet and the engine rewed. If a vacuum is
obtained, the sensor is functional. If no vacuum is obtained,
the lice from the sensor to the carburetor will be checked for
vacuum vhile the engine is rewed. If this line Las vacuum,
then the sensor is not functioning and will be checked for a
plugged port.
4. Some systems have a vacuum delay valve. If the EGR valve is
not functioning, checking the delay valve for plugs and that
it is not installed backvards.
Sealed EGR valve functional check will consist of:
1. Visually inspecting the system.
2. Disconnecting the vacuum hose to the EGR valve. The hand
vacuum pump will be connected to the valve and vacuum applied
with the engine running. If idle speed drops with the appli-
cation of vacuum, the valve is good. The vacuuo pump should
'then "be inserted into the line leading to the valve's vacuum
source. The engine will be rewed to determine if vacuum is
available. If vacuum is not available, the sensors and hosing
are checked using the same procedures described for the
non-sealed unit.
The EGH control valve and sensor are coded as follows:
EGR control valve
0 - Not originally equipped
1 - Functioning properly
3 - Vacuum disconnect - Disconnected, removed or plugged vacuum line
7 - Missing item (entire valve removed)
9 - Malfunctioning item
EGR sensor
0 - Not originally equipped
1 - Functioning properly
3 - Vacuum disconnect
6-76
-------
5 - Incorrectly routed hose
7 - Missing item
9 - Malfunctioning item
o. Vacuum spark retard - Adjusts the timing as RPM changes. It works
oo manifold vacuum which is a func&ion of RPM. The vacuum spark
retard will be coded as follows:
0 - Not equipped
1 - Functioning properl7
2 - Electrical disconnect
3 - Vacuum disconnect - Any removed, plugged, or disconnected
vacuum line
n. Inspectors initials and date.
6-77
-------
6-78
-------
FIGURE A-l
Tznpering Survey - ?»rt A
Col.»
.1 ID number ///'//
"5 Cdoneter COou.) / / / /
'8 Dash label
• 0 - not Orig. equipped
: 1 - Funct. properly
7 - missing iten
.9 Make / / / / /
13 Model / / / / /
17 Model year / / /
19 Vehicle type
1 - passenger car
• . 2.- light duty trucks(incl. vans }
20 License plate(State) / / /
. 22 Engine family
33 CID / / / /
26 Idle EC / / / / / (in ?
(justify rignt)
40 idle C0% / / / / / (in %)
(to V10 percent)
44 PluoibtesDO
P - positive
H - negative
45 Tank cap
; 1 - functioning properly
• . 7 - rnissing .item
9 - malfunctioning
Col.
46 Tank label
• 0 - not Orig. equipped
1 - Funct. properly
7 - sissing item
47 Finer neck restrictor
0 - not .Orig. equipped
1 - Fur.ct. properly
4 - Mech. Disc*, (widened)
7 - aissing iten
48 Catalytic converter
0 - not Orig. equipped
1 - Funct. properly"
7 - aissing item
9 - aalrunctioning
49 Exhaust system
P - Orig. equipment
6 - non-stock
50 Dchaust system integrity
1 - Funct. Prop, (no
ocvicus leaks)
9 - r-alfunctioning
(leaks evident)
51 Oxygen sensor
. .' 0 - not Orig. equipped
1 - Funct. prope'rly
2 - electrical disconnect
4 - Mech. Disc, (unscrewed)
• 7 - aissing item
Inspector's initials
6-79
/ / / / / / / / /
-------
A-2
Tampering Survey - ?»rt 3
Csl. 4
1 ID number / / / / /
: 5 Air cleaner
P - Orig, Equip.
6 - ncn-stcck
6 Exhaust manifold
P - Crig. Equip.
6 - non-stock
• 7 Intake rranifold
P - Orig. Equip.
6 - non-stock
8 Distributor
P - Orig. Equip.
6 - non-stock
'9 Tuztccharger
0 - not Orig. equipped
P - original equipment
6 - non-stock
A - add-on
10 Carburetor type
? - Orig. Equip.
(non-sealed)
S - sealed Carb.
(no caps)
F - fuel injection used
6 - non-stock Carb.
11 Carburetor barrels
0 - not Orig. equipped
.1
2
4
12 Limiter caps
0 - not Orig. equipped
1 - Funct. properly
4 - rtech. Disconn. (tabs
broken or bent)
7 - missing item
8 - misadjusted (sealed
plugs removed) '
Col.
13 PCV system
0 - net Crig. equipped
1 - Funct. properly
3 - vacuum disconnect.
4 - Mech. disconnect
S - non-stock (Incl. fuel
econony devices)
7 - missing item
14 Idle stop solenoid
0 - not Orig. equipped
1 - Funct. properly
2 - Elect, disconnect
7 - missing item
9 - malfunctioning
15 Seated air intake
0 - not Crig. equipped
1 - Funct. properly
3 - vacuum disconnect
4 - Mech. disconnect
6 - non-stock (custom
air cleaner)
7 - missing item
(stovepipe hcse)
9 - .Malfunct. it era
(Vac. override)
16 Evap. control system
0 - not Orig. equipped
1 - Funct. properly
3 - vacuum Discon.
(Garb, line)
4 - Mech. Discon.
(tank line)
5 - Incorr. routed hcse
7 - missing item
9 - Malfunct. item (air
cleaner unsealed)
17 Air
0
nl
*4
7
18 '-Air pump belt (if Aspi
'code -0")
0 - not Orig. equippe
1 - Funct. properly
7 - missing item
8 - misadjusted item
(loose)
19 Aspirated air injectic
system
0 - not Crig. equippe
(if conventional
system or none)
1 - Funct. properly
4 - Mech. disconnect
7 - missing its
9 - malfunctioniJ
20 EGR control valve
0 - not Orig. equippe
1 - Funct. properly
3 - vacuum disconnect
7 - missing item
9 - Malfunct. item
21 EGR sensor
0 - not Orig. equippe:
1 - Funct. properly
3 - vacuum disconnect
5 - incorrect hcse
routing
7 - missing item
9 - Malfunct. item
22 Vacuun spark retard
0 - not Orig. equippec
1 - Funct. properly
2 - Elect, disconnect
3 - vacuum disconnect
not Orig. equipped \
Funct. properly j
Mech. disconnect (other
than belt removal)
missing item
-------
FUEL SAMPLE COLLECTION AND LABELING PROCEDURES
A fuel sample shall be taken from each vehicle requiring unleaded fuel.
These samples shall be collected in A ounce glass bottles with a hand
fuel pump. Once the sample is drawn, the fuel shall be replaced with
an equivalent amount of unleaded fuel if the driver requests and the
pump shall be flushed with unleaded fuel.
Each bottle shall be identified with a stick-on label that has the
vehicle identifying survey number on it. The vehicle identifying survey
number is the first entry on data forms described in Attachment A.
Prior to shipment from the field, a sanple tag with the same identifying
number shall be attached to each bottle. The bottles 'will be packaged,
labeled, and shipped to the National Enforcement Investigation Center's
Chemistry Branch in Denver, Colorado, according the shippers require-
ments. The Contractor shall use screw-on caps on all sample bottles,
having either teflon or polyethylene cap liners. The Contractor shall
assure that all sample bottles are capped securely to prevent any leakage
and/or contamination.
6-81
-------
INSTRUCTIONS FOR USING PLUHBTISMO TEST PAPER
1. Use of surgical glove on the hand vhicb will be handling the
plumbtesmo paper.
2. Clean a portion of the inside of the tailpipe large" enough for
the test paper by wiping it out with a P»per level or cloth which
must be discarded after each positive test result. This is
usually accessary in ord«r to vipe avay soot deposits which night
mask the color change.
3. Moisten the PLUMBTESMO with distilled water (be careful not to
wash away chemicals) and insnediately* press firmly for approxi-
mately ten seconds against the surface to be tested. If the
tailpipe is hot you may vish to clamp the test paper in the
tailpipe using a clean clamp or wooden clothes pia (clothes pin
should be discarded after positive test result) .
Contamination of the test paper shall not be permitted. If a
person has recently handled a test paper with a positive reaction,
some lead or reactive chemical nay have beea transferred to his/her
fingers. Handling a subsequent clean test paper with the saoe
glove may cause contamination. Therefore, any glove oust be
discarded after it has contacted paper where there is a positive
reaction. If test paper falls on the ground before use and
subsequently gives a positive reaction vh«n used oa, an unleaded
vehicle, the test shall be repeated because lead deposits are
known to exist in soilnear highways.
4. After removing the test paper vait approximately thirty seconds
to make a determination. A color change to red or pink splotches
indicates the presence of lead.
*Kote: The Plumbtesmo paper mast be applied during the time
that the paper is yellow for the reaction to take
place. After approxiaately 15 seconds the yellow
color disappears and the paper is no longer
effective. Excess water also interferes with the
reaction.
6-82
-------
EMISSION'S SAMPLING. OF HC AND CO
Vehicles are tested in as-received condition with the engine at normal
operating temperature. With engine idling aod transmission in neutral,
the sample probe is inserted into the tailpipe. Exhaust concentration*
are recorded after stabilized readings Are obtained or at the end of
30 seconds, whichever occurs first. The process is repeated as
necessary for multiple exhaust pipes. However, multiple readings are
not necessary for exhaust originating froa a cocoon point. Results
from multiple exhaust pipes are to be numerically averaged. Results
are then recorded on the form for the vehicle being sampled.
6-83
-------
FI£LD QUALITY CONTROL/ASSURANCE
Prior to testing each day, the analyzers shall be zeroed and spanned.
Ambient air is acceptable as a zero gas; an electrical span check is
acceptable. Equipment shall be calibrated in accordance, with the
manufacturers instructions.
Three known calibration gases (Horiba) shall be used. These gases
shall be a mixture of CO and HC in nitrogen and' shall be used to span,
check and calibrate the instrument at lease three times daily. These
calibration gases shall be certified by the nanufacturer and have
accuracy traceable within ±2Z to NBS standards. • Their approximate
concentration shall be:
8Z CO
1560 ppn HC (Hexane equivalent)
42 CO
827 ppn HC (sexane equivalent)
1.6Z CO
320 pp'm HC (Hexane equivalent)
The span check results must be vithin 102 of the nominal standards or
the affected data raay be considered suspect and ineligible for inclusion
in the survey. The span check results and the concentrations of the
standard gases shall be sent to the Project Officer along with clear,
readable copies of the raw data sheets by the Contractor within one
week after each survey site is completed.
Plurabtesrao test paper shall not be more than 12 months old and one piece
froa each box shall be tested within a week before use on a leaded
vehicle or by sooe other raeans to assure that the paper is still
sensitive to lead. Leaded, as well as unleaded vehicles, shall be
tested to r^ake sure the paper is reacting properly (positive reaction
on leaded vehicles) in the field.
Inspectors shall be .required to demonstrate their ability to conduct
the required tasks by inspecting ten vehicles as specified by the
Project Officer prior to the survey.
6-84
-------
APPENDIX E
INSTRUCTIONS AND FORMS FOR AUDIT ACTIVITIES
l. Instructions and Form for Sticker Surveys
2. Instructions and Form for Analyzer Audits of Centralized
- Facilities
3. Instructions and Form for Reviewing Vehicle Records
4. Instructions and Form for Reviewing Station Audit
Records, Centralized
5. Instructions and Form for Reviewing Station Audit
Records, Decentralized
6. Form for Listing Data Summaries Reviewed
7. Form for Review of Licensing/Suspension Records
8. Form for Review of Consumer Inquiries and Complaints
9. Form for Review of Enforcement Records
10. Form for Listing Other Records Reviewed
11. Instructions and Form for Observing Audits of
Centralized Inspections Stations
12. Instructions and Form for Observing Audits of
Decentralized Inspection Stations
13. Instructions and Form for Observing Centralized
Inspections
14. Instructions and Form for Observing Waiver Processing
15. Form for Recording Observation of Other Procedures
16. Form for Observing Spot Checks Using Unmarked Cars
17. Form for General Interview with Air Agency Official
18. Form for Interview with I/M Program Manager
19. Form for Interview Regarding Quality Control
20. Form for Interview Regarding Repair Waivers
21. Form for Interview Regarding Data Analysis
22. Form for Interview Regarding Mechanics Training
23. Form for Exit Interview
24. Form for Interviews with Other Knowledgeable Persons
6-85
-------
APPENDIX E
INSTRUCTIONS AND FORMS FOR AUDIT ACTIVITIES
1. Instructions and Form foe Sticker Surveys
Instructions
Sticker surveys are one means to identify the level of
noncompliance among vehicle owners where windshield or other
stickers are used for enforcement. Sticker surveys are
conducted by visually inspecting the stickers on parked
vehicles to determine compliance. Sticker surveys can be
conducted anywhere there are parked vehicles; however, city
streets, large shopping center parking lots, and other areas
open to public parking are usually more accessible.
The data to collect in the sticker survey is very simple. It
includes the number of vehicles observed that have valid
stickers, the number with expired stickers, the number of
subject vehicles with no stickers, and the number of exempt
vehicles. The latter group would include in-State vehicles
registered outside the I/M area, vehicles displaying exempt
stickers, and vehicles otherwise identified as exempt.
Out-of-State vehicles would also be exempt, of course, but
the simplest way to deal with them is to disregard them
entirely, i.e., do not count them in the number of total
vehicles observed or in the number of exempt vehicles. If a
large percentage (greater than 10%) of exempt vehicles is
encountered, it would be appropriate to categorize them into
the above classes of exempt vehicles in order to assess the
relative effects of each on the I/M program. For instance,
if a large percentage of vehicles are observed that are
in-State, non-I/M vehicles, a conclusion from the survey
might be that the program boundaries or vehicle coverage
should be adjusted to include more vehicles.
In order to prepare for the survey, the auditors must
obviously familiarize themselves with the stickers in use and
how to recognize valid, expired, exempt and any other
stickers in. use. Also they must know how to identify subject
vehicles from exempt vehicles, if means other than examining
the stickers must be used to do so.
Although many survey forms are possible for sticker surveys a
simple tally sheet is easy to use and allows efficient use of
personnel time both in conducting the survey and in
tabulating results. A suggested form follows.
6-86
-------
Sticker Survey Form
DATE TIME
LOCATION NAME OF OBSERVER
VALID EXPIRED NONE EXEMPT
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
oooooooooooooo ooooo ooooo ooooo
Notes/Comments
6-87
-------
2. Instructions and Form for Analyzer Audits of Centralized
Facilities
Instructions
The objective of analyzer audits is to determine whether
accurate readings are being obtained in normal testing (i.e.,
through the probe). Therefore, analyzer audits involve
introducing span gases of known concentration into the
analyzer through the probe in order to simulate an actual I/M
test. Before conducting the analyzer audit, the auditors
should familiarize themselves with the design of the
analyzers and how to go about checking calibration through
the probe. It may be necessary to consult the analyzer
manufacturer on the proper procedures to follow.
Equipment needed:
°. Span gases - Size A (approximately 200 cubic feet)
cylinder of low range span gas (nominally 1.6% CO, 700
ppm propane, balance N2) and of high range span gas
(nominally 8% CO, 3000 ppm propane, balance M2) ; all
span gases must be traceable ±1% to N"BS standards; gas
analysis must be performed by EPA or using EPA-approved
protocol; other size cylinders may be used as long as
an adequate supply of span gas is available; in some
cases, the gas mixture may need to contain CO2 in
order to prevent analyzer lockouts from functioning; in
a few cases, the use of hexane in place of propane as
the HC component may be desirable if the State uses
hexane; regarding span gas component concentrations,
care must be taken to avoid concentrations which are at
or very near break points in the analyzer scale ranges,
i.e., 2% and 5% for CO and 400-500 ppm and 1000 ppm for
HC (as hexane).
0 Cylinder gauges and flow regulator.
0 Hardware to flow gas through the probe, commonly
referred to as a "tailpipe simulator."
0 Calculator; balloons; hand tools.
Audit procedure:
1. Attach gauges/regulator to span gas cylinder.
2. Make sure analyzer is warmed up and ready to test
vehicles.
6-88
-------
3. Identify and record analyzer make, model, identification
number, and propane equivalency factor (PEF).
4. Make sure the analyzer is zeroed and electrically spanned.
5. Put the analyzer into test mode and sample room air to
determine the extent of HC hangup. This will be
indicated by a positive HC reading while testing room
air. If the HC hangup is above 20 ppm, allow the
analyzer to purge itself by sampling room air until the
HC reading is below 20 ppm. Record the final HC hangup
value.
6. Recheck zero and electric span.
7. Put analyzer into proper mode to sample through the
probe. Some analyzers sample through the probe
continuously. Others only sample through the probe while
in "test" mode and may have either no flow or back flow
during non-test modes.
The auditor must know the proper mode in which to audit
the analyzer.
8. Once the analyzer is sampling through the probe, insert
the probe into the tailpipe simulator. Since the gas
cylinder valve is off, a vacuum should be drawn,
resulting in a "no flow" response from the analyzer.
This may be in the form of a light coming on or going off
or it might be a flow meter registering a zero flow. if
a "no flow" signal is not received, this indicates that
major sampling line leaks are present that must be
repaired before proceeding. Record the presence/absence
of leaks.
If no leaks, continue.
9. Open gas cylinder valve, and adjust the flow of span gas
using the regulator until the balloon on the tailpipe
simulator is upright but not expanding. This condition
indicates that the span gas flow is proper.
10. As soon as analyzer readings stabilize (i.e., obtain
maximum values), record them and immediately turn, off the
gas cylinder valve. This should cause the analyzer to
return to a "no flow" response, thus indicating that no
leaks were introduced during the audit that would bias
the results.
11. Remove the probe from tailpipe simulator.
6-89
-------
12. Compare the CO analyzer readings directly to the span gas
CO concentration. The reading should be within -7% to
-t-5% (or + 0.1% CO, whichever is greater) of the cylinder
concentration.
13. Adjust the HC analyzer readings by subtracting the HC
hangup value recorded in step 5. Adjust the propane span
gas concentration by multiplying the propane span gas
concentration by the analyzer's PEF value. Now compare
the adjusted analyzer reading to the adjusted span gas
value. The adjusted reading should be within -7% to +5%
(or + 15 ppm HC, whichever is greater) of the adjusted
gas value.
14. Repeat this procedure for both the low and high range
span gases.
All analyzers in the centralized facility must be audited.
If lane traffic is particularly high, the EPA auditors may
choose to observe State or contractor personnel performing
the audit instead of doing it themselves, in order to
minimize any disruption of the lane's activity.
The report for the analyzer audits should summarize the
number of analyzers which were audited, the number passing
all checks, and the number failing to meet tolerances and
why. The report should also indicate what action was taken
by program officials for problem analyzers, i.e., no action,
taken out of service, or repaired on the spot. If the
latter, the repaired analyzers should be rechecked during the
audit to verify their accuracy after repair.
6-90
-------
ANALYZER AUDIT FORK
Facility Name/Address
Person Ctontacted at Facility (name, title)
Date
EPA Auditor (s)
Other Persons Involved (name, affiliation)
Span Gas Cylinders Used (ID numbers)
(See Procedures on Back)
1
Analyzer
Number
2
Make/Hjdel
3
PEF
4
1C Hangup
5
Leak?
6
1C Reading
7
CD Reading
8 9
HC Gas CO Gas
10
Adjusted
1C Reading
(6 - 4)
11
Adjusted
1C Gas
(8 X 3)
12
% Diff.
(7 vs 9)
13
« Diff.
(10 vs 11)
14
CD
P/F
15
HC
P/F
Notes/Conrents
-------
6-92
-------
Analyzer Audit Procedure
ANALYZERS MUST BE WARMED UP AND READY FOR TESTING
1. Record analyzer number, make, model and PEF.
2. Check/adjust zero and electric span.
3. Check the hangup; purge until less than 20 ppm; record
final HC hangup value.
4. Recheck zero and electric span.
5. Insert probe into tailpipe simulator for initial leak
check; if passed, record "ok" and proceed.
6. Flow span gas through probe; record readings as soon as
stabilized (i.e., obtain maximum values); immediately
close gas cylinder valve.
7. Verify final leak check; remove probe.
8. Compare CO reading to span gas CO concentration. If
within -7% to +5% (or +0.1% CO, whichever is greater), CO
channel passes. Record CO P/F.
9. Adjust HC reading for HC hangup and adjust propane span gas
concentration using PEF. Compare the two adjusted values.
If within -7% to +5% (or + 15 ppm HC, whichever is
greater), HC channel passes. Record HC P/F.
6-93
-------
3. Instructions and Form for Reviewing Vehicle Records
Instructions
The primary purpose of reviewing vehicle records is to
determine whether program data are being collected properly,
i.e., are the forms being correct1-/ completed. In addition,
reviewing vehicle records can ^..tnetimes give the auditor
indications (but not conclusions) of whether proper
procedures are being followed for inspections, waivers,
repairs and other activities. Suspicions raised by reviewing
vehicle records should be followed-up during other audit
activities.
All types of vehicle records should be reviewed, except in
centralized programs with computer-printed forms where it is
not necessary to review inspection forms for passing
vehicles. In all other cases, inspection records must be
reviewed. Additionally, retest records, waiver records, and
repair records or receipts must be reviewed for a sample of
vehicles. All forms must be reviewed for completeness and
legibility.
Recording Form
Person completing form:
Others accompanying:
Date:
Station, Location:
Inspection period covered by forms reviewed:
Number of inspection records reviewed:
Number of retest records reviewed:
Number of waiver records reviewed:
Number of repair records or receipts reviewed:
6-94
-------
Inspection Records:
1. Are forms filled in completely and legibly?
2. Are the proper cutpoincs being used?
3. Are correct pass/fail decisions being made based on the
reported readings?
4. Are all parts of the test being performed as indicated
by forms?
5. Is the test data reasonable? Are there any suspicious
patterns, such as repetitious entries? Are some
readings suspiciously high or low?
6. What is the failure rate for the sample reviewed? How
does it compare to recent reported program results?
7. What is the waiver rate for the sample? How does it
compare to recent program operations?
8. Does -the sample data indicate inconsistencies among
inspection stations? Among inspectors within the same
station?
Retest Records:
1. Are the forms filled in completely and legibly?
2. What is the retest failure rate?
3. Are there inconsistencies among stations or inspectors?
Waiver Records:
l. Are the forms filled in completely and legibly?
2. Were required repairs (e.g., low emissions tune-up)
performed?
3. Was the repair cost ceiling met?
4. Are there inconsistencies among stations or inspectors?
5. Were other waiver criteria met, such as, the
anti-tamper ing or warranty provisions? '
6-95
-------
6. Were the repairs performed (or at least claimed)
appropriate, based on the initial emissions readings?
Repair Records or Receipts:
1. Are the forms filled in completely, legibly, and
clearly?
2. Are the repairs appropriate, based on the initial
emission readings?
3. Are the repairs effective in obtaining emissions
reductions?
4. Are mechanics adjusting to cutpoints or well below?
5. Are repair costs reasonable?
Notes/Comments
5-96
-------
4. Instructions and Form for Reviewing Station Audit Records,
Centralized
Instructions
The purpose of reviewing station audit records is to
determine whether audits are being conducted and documented
according to established procedures. Audit procedures will
vary among programs. In some cases, the audit may consist
only of analyzer checks, while in other cases, it may involve
record checks, sticker/certificate accountability checks,
observation of inspections, and other activities.
Recording Form
Person completing form:
Others accompanying:
Date:
State, location:
Time period covered by audit records reviewed:
What is the audit frequency?
Do the audit records indicate any problems with analyzers?
What was corrective action?
Are records checks performed during audits? What records?
Results?
Are stickers/certificates accounted for during the audit?
How?
Are inspections observed as part of the audit? How many?
What other activities are performed during audits?
Does a station official sign the audit form and keep a copy?
Notes/Comments
6-97
-------
5. Instructions and Form for Reviewing Station Audit Records,
Decentralized
Instructions
One purpose of reviewing audit records in a decentralized
program is to assure audit completeness and adherence to
procedures. Another purpose is to determine whether audits
are being conducted and documented according to established
procedures. In most cases, audits of decentralized stations
will include analyzer checks, records checks, sticker/
certificate accountability checks, and sometimes observations
of inspections or other procedures. Of particular importance
is the assurance that inadequate performance by licensed
stations is routinely identified and corrected. Audit
records and statistics must be reviewed in sufficient detail
to allow a determination of whether stations are being
audited at the proper interval. In addition, the records of
three full days of auditing by each State/local field
investigator must be reviewed in detail in order to evaluate
the quality of the audits.
Recording Form
Person completing form:
Others accompanying:
Date:
Time period covered by review:
Name of State/local auditor:
Number of stations audited:
What is the audit frequency?
Do the audit records indicate any problems with analyzers?
What was corrective action?
Are records checks performed during audits? What records?
Results?
Are stickers/certificates accounted for during the audit?
How?
Are inspections observed as part of the audit? How many?
What other activities are performed during audits?
6-98
-------
Does a station official sign the audit form and keep a copy?
Was audit form filled out completely in. each case?
Number of stations passing audit:
Number of stations failing audits, categorized by. reason:
What were the penalties associated with audit failures?
Notes/Comments
6-99
-------
6. Form for Listing Data Summaries Reviewed
Person completing form:
Others accompanying:
Date:
State/local program:
Which of the following data summaries were reviewed? Give
frequency of report.
Inspection activity - tt inspections, H pass, and It fail.
Inspection activity by station.
Inspection activity by model year, outpoint group,' vehicle
weight class, or other breakdown.
Retest activity as above.
Waiver activity - 8 waivers; breakdown by station or repair
facility.
Repair cost summaries.
Summaries of Station Audits - tt audit's, # pass, ft fail by
reason, # penalties by type, 8 and results of investigations
with unmarked cars.
Others (list)
Notes/Comments
6-100
-------
7. Form for Review of Licensing/Suspension Records
Person completing form:
Others accompanying:
Date:
Total number of licensed stations: tt fleet stations; M
non-fleet stations.
Total number of licensed/certified inspectors:
Number of station infractions identified per month (average
for last year):
Number of station infractions leading to suspensions during
the last year:
Average length of station suspension:
Number of inspector infractions leading to suspensions during
the last year:
Average length of inspector suspension:
Identify other types of penalties and frequency of use,
including oral warning, written warning, revocation of
licenses, fines, notices of violation, etc.
Are penalties imposed in a manner consistent with the
seriousness of the infractions involved?
Notes/Comments
6-101
-------
8. Form for Review of Consumer Inquiries and Complaints
Person, completing form:
Others accompanying:
Date:
Time period covered by review:
Number of consumer inquiries:
Number of consumer complaints:
How are repair complaints investigated?
Number of referee tests in last year:
Number of referee tests resolved in favor of the vehicle
owner:
Are problem stations identified through complaint/referee
process? What is follow-up?
Did resolution of inquiries or complaints involve
notification to owners of emission warranties?
Notes/Comments
6-102
-------
9. Form for Review of Enforcement Records
Person completing form:
Others accompanying:
Date:
Number of vehicles subject to inspection:
How derived?
/
Number of vehicles inspected (initial inspections only) in
the last year:
In sticker programs, results of recent sticker surveys:
In sticker programs, number of citations and average fines
for sticker violations over the last year.
In data-link systems, number of noncomplying vehicles at each
step of the enforcement process.
In data link systems, number and amounts of fines imposed.
Indicate type and source of records reviewed and any other
pertinent details.
Notes/Comments
6-103
-------
10 . Form for Listing Other Records Reviewed
Person completing form:
Others accompanying:
Date:
Other records reviewed (list):
Notes/Comments
6-104
-------
11. Instructions and Form for Observing Audits of
Centralized Inspection Stations
Instructions
At least one audit by a State/local investigator must be
observed. The purpose of the audit observation is to
determine whether actual audits are consistent with
State/local requirements and based on good engineering and
management practices. The EPA auditor(s) should observe the
State/local personnel as they are performing their audits
with as little interference as possible. The EPA auditors
must pay particular attention to procedures followed-and keep
notes accordingly. As part of the analyzer audit
observation, the EPA auditor should ask the State/local
auditor to use EPA's span gas successively with the
State/local span gas in order to verify the accuracy of the
State/local span gas.
Recording Form
Date:
Person completing form:
Other accompanying:
State/local auditor(s):
Station/location:
Analyzer Audit: Describe procedure.
00 Type and concentration of span gas(es).
00 Procedures used for gas span and leak checks.
00 Other quality control checks performed, including
sample line integrity checks, HC hangup checks, etc.
00 Did State/local span gas compare favorably with
EPA's span gas?
Records Review: Describe any records reviewed and procedures
used.
Inspections Observations: Did the State/local auditor
observe inspections? How many?
Were stickers/certificates checked?
Was an audit report/form completed by the State/local auditor?
6-105
-------
How were any analyzer problems handled? Inspector problems?
Record problems?
How long did the audit take?
Time of arrival:
Time of departure:
Notes/Comments
6-106
-------
12. Instructions and Form for Observing Audits of
Decentralized Inspection Stations
Instructions
EPA auditors must accompany three State/local auditors on
their regular audit visits. A total of at least 10 to 20
stations should be audited. The EPA auditors should observe
how the State/local auditors conduct the audits with as
little interference as possible. The EPA auditors must pay
attention to whether established procedures are followed and
keep notes accordingly. Of particular importance is
observing whether State/local auditors identify problems and
how they handle them. As part of the analyzer audit
observation, the EPA auditors should ask the State/local
auditors to use EPA's span gas successively with the
State/local audit gas in order to verify the accuracy of the
State/local gas. This comparison only needs to be performed
at one station for each State/local auditor unless the
stations' gases are used to check analyzers. In this case, a
span gas comparison should be made during each station audit.
Recording Form
Date:
Person completing form:
Others accompanying:
State/local auditor(s)-:
Stat ion/location:
Time of arrival:
Time of departure:
Describe procedures used to check analyzer(s).
00 Make/model; digital or analog?
00 Were all analyzers checked?
00 Type and concentration of span gas(es).
00 Procedures used for gas span and leak checks.
00 Where tampering and/or misfueling checks are
required, were fuel filler inlet gauges examined and
the activity of lead-sensitive test paper confirmed?
00 Results of EPA span gas comparison.
6-107
-------
Describe records review.
00 What records were reviewed? Inspection records?
Calibration records?
00 How were they reviewed?
Describe sticker/certificate accountability checks.
Were inspections observed? How many?
Did auditor verify that the station had up-to-date rules and
reference manuals?
How did the State/local auditor respond to questions from
station personnel?
If any analyzer problems were identified, describe them and
how the State/local auditor handled them.
Same for inspection problems, records problems, and/or
sticker/certificate problems.
Was an audit report/form completed by the State/local
auditor? Was it accurate and complete?
Did the State/local auditor's conduct during the audit
reflect an appropriate knowledge of program rules and
procedures and of auditing procedures? Would additional
analyzer or records checks improve the State/local audit?
Notes/Comments
6-108
-------
13. Instructions and Form for Observing Centralized
Inspections
Instructions
The purpose of observing inspections in centralized stations
is to determine whether they are being conducted according to
procedure descriptions obtained earlier in the I/M audit. As
procedures will vary, the items to watch for will vary. The
form on the next page may need to be modified to reflect
differing requirements in specific I/M programs.
Most of the form has the format of a questionnaire, and
general yes/no answers are appropriate. If a significant
deviation from established procedures is observed repeatedly,
the answers should be expanded in the comments section to
indicate the frequency of occurrence, accompanying
circumstances, etc.
Recording Form
Date:
Person completing form:
Others accompanying:
Station, Location:
Time of arrival:
Time of departure:
Station manager on duty:
Number of lanes:
Number open during visit:
Number of cars waiting in line to begin inspection (all
lanes combined) at time of arrival/departure:
Number of inspections observed:
Number of these which were retests:
Number of inspectors observed:
Was the analyzer in each open lane operating?
Did all vehicles receive the emissions test?
Did all vehicles receive the tampering check?
Was the test probe properly inserted during every test and
kept in for the proper amount of time?
Were procedures (if any) against testing vehicles with
obvious exhaust leaks followed?
6-109
-------
Did vehicles receive proper preconditioning?
Were special tests (e.g., restart test for Fords) performed
when required?
Were tampering/misfueling checks performed correctly?
Were analyzer readings correctly recorded manually, if
necessary?
Were cutpoints selected and applied correctly? Were any
failing vehicles passed, or vice versa?
Did inspectors insist on correct documentation (registra-
tion card, inspection reminder, other) from owners?
If required, were old stickers removed?
Were new stickers placed in the required place on
windshields or other par:; of the vehicle? If not, what
was done with them?
Did inspectors check for completeness of any required
repair information?
Did inspectors give correct materials to owners, including
test printouts, certificates of compliance, failure
brochures, warranty information, etc.?
Was inspector interaction with public appropriate?
Did inspectors handle analyzers and sampling system
correctly?
Was any low flow indicator ignored?
Were sticker/certificate accountability procedures
followed?
Notes/Comments
6-110
-------
14. Instructions and Form for Observing Waiver Processing
Instructions
The purpose of observing a limited number of waivers being
processed is to verify that the larger number of waiver
records reviewed separately have given a true impression of
the waiver system. The EPA auditor should not interact with
the vehicle owner or waiver officer during their exchange.
The auditor should observe the waiver officer and listen to
the conversation, and should review all the documents
presented by the owner to the officer.
Recording Form
Date:
Person completing form:
Others accompanying:
Facility, Location:
Time of arrival:
Time of departure:
Number of waiver counters open:
Number of owners waiting for service at time of
arrival/departure:
Number of waiver transactions observed:
Number of waiver officers observed:
Did waiver officers always insist on required receipts and
invoices?
Did waiver officers check the documentation provided by
owners to verify that all waiver requirements were met?
00 Actual expenditure
Actual expenditure on emissions repairs/
diagnostics
Actual expenditure on repairs appropriate for
type of failure
Performance of specific required repairs
Achievement of a minimum percentage reduction
Estimated cost of remaining necessary repairs
o o
o o
o o
o o
Did waiver officers verify that the vehicles receiving
waivers were free of tampering?
Were any waiver requests rejected? Why?
6-111
-------
Were any waiver requests granted which arguably should
have been rejected under program rules?
Did waiver officers give any repair advice?
Notes/Comments
6-112
-------
15. Form for Recording Observation of Other Procedures
Person completing form:
Date:
Time of observation:
Location:
Procedure observed:
Notes/Comments
6-113
-------
16. Form for Observing Spot Checks Using Unmarked Cars
(Complete before and after, not during the observation.)
Person completing form:
Date:
Time of observation:
Name(s) of State/local auditor(s):
Number of station audits observed:
Description of vehicle used:
Was this audit for emissions only or emissions and safety?
How were stations selected for audit?
Was vehicle set to pass or fail?
How was this verified before delivering car to the station?
If set to fail, how was this done?
How was the vehicle documentation (registration, expired
sticker, etc.) arranged so as not to arouse suspicion?
What did the auditor say when he/she presented the vehicle
for inspection?
What was the inspection result? Was the inspection done
correctly?
Did the auditor observe the inspection itself? If not, did
the auditor pass up an opportunity to observe it?
Were there any obvious deviarions from correct procedures by
either the inspector or che Stare/local auditor?
If the vehicle failed, was a repair authorized?
What repairs were performed?
What was the repair charge?
6-114
-------
How did the auditor document the results?
What action, if any, was recommended by the State/local
auditor after the spot check?
Notes/Comments
6-115
-------
17. Form for General Interview with Air Agency Official
EPA representatives present:
State/local representatives present:
Locat ion:
Date:
Time :
Planned Topics for Discussion (Expand or Reduce in Advance if
Appropriate)
1) Purpose of audit and this interview.
2) Special emphasis areas identified by EPA or State/local
air agency.
3) Review of audit preparation activities and findings.
4) Portions of questionnaire not yet complete; documents not
received by EPA.
5) Schedule, locations, and personnel for on-site visit
activities.
6) Special issues in relationships with other agencies (I/M
operating agencies, enforcement agencies, courts).
7) Plans for involving other State/local agencies in on-site
activities,
8) Any State/local requests needing follow-up by EPA.
Notes/Comments
6-116
-------
18. Form for Interview with I/M Program Manager
EPA representatives present:
State/local representatives present:
Location:
Date:
Time:
Planned Topics for Discussion (Expand or Reduce in Advance if
Appropriate)
1) Purpose of audit and this interview; role of this agency
in implementing SIP.
2) Special emphasis areas identified by EPA, State/local air
agency, or this agency.
3) Review audit preparation activities and findings.
4) Portions of questionnaire not yet completed; documents not
received by EPA.
5) Schedule, locations, and personnel for on-site visit
activities.
6) Any requests needing follow-up by EPA.
Notes/Comments
6-117
-------
19. Form for Interview Regarding Quality Control
EPA representatives present:
State/local representatives present:
Location:
Date:
Time :
Planned Topics £or Discussion (Expand in Advance if
Appropriate)
1) Confirmation of unclear questionnaire responses.
2) Discussion of possible irregularities or uncertainties in
quality control observed to date in site visit.
3) Clarification of EPA quality control requirements or
recommendations, if needed.
4) Conformance with 207(b) requirements.
Notes/Comments
6-118
-------
20. Form for Interview Regarding Repair Waivers
EPA representatives present:
State/local representatives present:
Location:
Date:
Time:
Planned Topics for Discussion (Expand in Advance i£
Appropriate)
I) Confirmation of unclear questionnaire responses.
2) Discussion of possible irregularities or uncertainties in
repair waiver system observations to date in site visit.
3) Discussion of use of warranties to avoid need for repair
waivers.
Notes/Comments
6-119
-------
21. Form for Interview Regarding Data Analysis
EPA representatives present:
State/local representatives present:
Location:
Date:
Time:
Planned Topics for Discussion (Expand in Advance if
Appropriate)
1) Confirmation of unclear questionnaire responses.
2) Review of each type of data summary or report to confirm
meaning of each, entry and how it was developed.
Motes/Comments
6-120 •
-------
22. Form for Interview Regarding Mechanics Training
EPA representatives present:
State/local representatives present:
Location:
Date:
Time:
Planned Topics for Discussion (Expand in Advance if
Appropriate)
1) Confirmation of unclear questionnaire responses.
2) Perceived areas of mechanic shortcomings.
3) Practicality of possible changes in the mechanics training
system.
4) Local needs for EPA assistance.
Notes/Comments
6-121
-------
23. Form for Exit Interview
EPA representatives present:
State/local representatives present
Location:
Date:
Time :
Planned Topics for Discussion
1) Review of actual site visit activities versus planned.
2) State/local feedback on site visit experience.
3) EPA reaction to what was observed during site visit.
(Optional. If discussed, list points discussed.)
O Reporting plans.
5) State/local requests requiring EPA follow-up.
6) Additional activities to be taken by EPA as part of or as
a follow-up to the audit (e.g., tampering and/or idle test
survey).
7) Additional materials needed by EPA from the State/local
agencies to complete the audit.
Notes/Comments
6-122
-------
24 . Form for Interviews with Other Knowledgeable Persons
EPA Representative(s):
Person interviewed:
Organization/occupation:
Relationship to inspection program:
Time with this relationship:
Date of interview:
In person/by telephone:
Location:
Time:
Planned Topics
1) Do you believe from your experience that repairs are
being performed properly, so that nearly all cars pass
reinspection and so that owners do not pay for
unnecessary repairs?
2) Do many vehicle owners get tune-ups before bringing their
cars for inspection, in hopes.of passing on the first try?
3) Do many licensed stations fix cars before doing the
official test?
4) (Centralized) Do most service facilities doing emission
repairs have analyzers? Do they keep them well
adjusted? How?
5) Is there unmet demand for mechanic training? Why: not
offered, too expensive, bad schedule, bad locations?
6) Do many owners or mechanics readjust cars after passing
- . inspection because they believe they have to for better
driveability or fuel economy?
7) Are stickers/certificates available outside normal
channels?
8) What do you see as important strengths of this inspection
program?
9) What would you try to improve?
10) Are there widespread problems with owners getting
legitimate warranty repairs?
11) Are replacement parts available?
12) Is it easy to get an appointment for an inspection and/or
repair?
12) Is waiting time at centralized facilities too long?
13) Is tampering or misfueling a particular problem?
Notes/Comments
6-123
-------
APPENDIX F
CHECKLIST FOR COMPLETING THE I/M AUDIT
6-124
-------
APPENDIX F
CHECKLIST FOR COMPLETING THE I/M AUDIT
I. Advance Preparation
A. Documentation. Assembly
Review applicable portions of SIP
Enabling legislation
Program rules and regulations
Other technical or procedural information
Review other SIP related information
EPA rulemakings
EPA Technical Support.Documents
Review other program documents
Operating contracts
Procedures manuals for testing and/or
quality control
Analyzer specifications
Quality assurance plan
Review available reports on program operations
Periodic reports published by I/M agency
Reports on previous audits
Reports on special surveys or projects
Data summaries obtained from I/M agency
Review recent correspondence
B. Program Questionnaire
Review program design "
Vehicle coverage
Cutpoints
Test procedures - Emissions and/or
tamper ing/misfuel ing
Analyzer specifications
Analyzer maintenance and calibration
Station and inspector licensing
Record keeping at time of inspection
Audit/surveillance activities
Challenge mechanism
Repair waivers
6-125
-------
Review program design (continued)
Enforcement mechanism
Mechanics training and other interface
Consumer issues
Self assessment through data analysis
Future plans
Review operating experiences (as available
information allows)
Operating statistics
Quality control statistics
Data analyses
Consumer protection
Repair waivers
Enforcement
Mechanics training and other interface
Self assessment through data analysis
C. Notice to Program Officials and Other Affected Parties
. Send formal notice to State/local agencies 60
days in advance of site visit
Send completed questionnaire to State/local
agencies for review; request additions and
clarifications, as necessary, and return before
audit visit
Notify all concerned EPA offices of audit
Review State/local responses on returned
questionnaire; distribute to concerned EPA
offices
II. On-Site Audit Visit
A. Initial interviews
Air planning agency officials
I/M operating agency officials
B. Review of program records
Inspection records
Enforcement records
Waiver records
Audit/surveillance records
Repair records
Complaint files (optional)
Inspection station visits
Observation of State/local audits
Analyzer checks
Observation of waiver processing
Interviews of station personnel
Record checks
Observation of inspections
6-126
-------
D. Special "surveys and interviews (may be scheduled
separately)
Enforcement surveys
Idle test surveys
Tampering/misfueling surveys
Interviews of non-program representatives with
special knowledge/experiences
a
E. Exit interview
Review of audit activities
State/local feedback on audit
Discussion of preliminary audit findings
(optional)
Plans for audit report
Requests for additional materials needed by EPA
Follow-up activities by EPA
III. Audit Report
Develop draft report
Coordinate review of draft within EPA
Coordinate review of draft with .State/local
agencies
Finalize audit report; final audit report shall
contain:
Background description of program
Review of audit activities
Discussion of program strengths
Discussion of program weaknesses (if any),
including both deficiencies which must be
addressed and areas with minor weaknesses
where program improvements are not required
but are recommended
EPA recommendations for correcting problem
areas, especially deficiencies
Description of any follow-up activities
including an estimated timeframe for a
follow-up audit, if needed
Discussion of State/local commitments
subsequent to the audit to resolve any
identified problems
State/local comments on draft report shall
• be appended to the final report
Completed audit questionnaire shall be
appended to the final report
6-127
-------
TECHNICAL REPORT DATA
(Please read Instructions on the reverse before completing)
1. REPORT NO.
EPA-450/2-84-008
2.
3. RECIPIENT'S ACCESSION-NO.
4. TITLE AND SUBTITLE
National Air Audit System Guidance Manual for FY'85
5. REPORT DATE
December 1984
6. PERFORMING ORGANIZATION CODE
7. AUTHOR(S)
8. PERFORMING ORGANIZATION REPORT NO
9. PERFORMING ORGANIZATION NAME AND ADDRESS
Office of Air Quality Planning and Standards
U.S. Environmental Protection Agency
Research Triangle Park, North Carolina 27711
10. PROGRAM ELEMENT NO.
11. CONTRACT/GRANT NO.
68-02-3892
12. SPONSORING AGENCY NAME AND ADDRESS
13. TYPE OF REPORT AND PERIOD COVERED
DAA for Air Quality Planning and Standards
Office of Air and Radiation
U.S. Environmental Protection Agency
Research Triangle Park, North Carolina 27711
14. SPONSORING AGENCY CODE
EPA/200/04
15. SUPPLEMENTARY NOTES
16. ABSTRACT
The purpose of developing national air audit system guidelines is to establish
standardized criteria for EPA Regions to follow when auditing State air program
activities. This document, prepared jointly by the State and Territorial' Ai'r
Pollution Program Administrators (STAPPA), the Association of Local Air Pollution
Control Officials (ALAPCO), and the Environmental Protection Agency, provides
national air audit guidelines for air quality planning and SIP activities, new
source review, compliance assurance, air monitoring, and vehicle inspection and
maintenance programs.
17.
KEY WORDS AND DOCUMENT ANALYSIS
DESCRIPTORS
b.IDENTIFIERS/OPEN ENDED TERMS
c. COSATI Field/Group
Air pollution
Air audit
Air quality planning
New source review
Compliance assurance
Air monitoring
Inspection and maintenance
Air Pollution Control
13B
13. DISTRIBUTION STATEMENT
Unlimited
19. SECURITY CLASS (This Report)
Unclassified
21. NO. OF PAGES
398
20. SECURITY CLASS (This page)
Unclassified
22. PRICE
EPA Form 2220-1 (9-73)
------- |