United States
Environmental Protection
Agency
Office of Air Quality
Planning and Standards
Research Triangle Park NC 27711
EPA-450/2-85-008
December 1985
Air
National Air Audit System
Guidance Manual for
FY1986-FY1987
-------
EPA-450/2-85-008
National Air Audit
System Guidance Manual
for FY 1986 - FY 1987
Control Programs Development Division
U.S. ENVIRONMENTAL PROTECTION AGENCY
Office of Air and Radiation
Office of Air Quality Planning and Standards
Research Triangle Park, North Carolina 27711
December 1985
-------
This report has been reviewed by the Control Programs Development Division
of the Office of Air Quality Planning and Standards, EPA, and approved for
publication. Mention of trade names or commercial products is not intended
to constitute endorsement or recommendation for use. Copies of this report
are available through the Library Services Office (MD-35), U.S. Environmental
Protection Agency, Research Triangle Park, North Carolina 27711; or, for a
fee, from the National Technical Information Service, 5285 Port Royal Road,
Springfield, Virginia 22161.
-------
EPA 450/2-85-008
National Air Audit System Guidance Manual
for FY 1986-FY 1987
Contents
Chapter Page
1 Introduction 1-1
2 Air Quality Planning and SIP Activities 2-1
3 New Source Review 3-1
4 Compliance Assurance 4-1
5 Air Monitoring 5-1
6 Vehicle Inspection Programs 6-1
-------
1. INTRODUCTION
This audit manual was developed through the joint efforts of the State
and Territorial Air Pollution Program Administrators (STAPPA), and the Associ-
ation of Local Air Pollution Control Officials (ALAPCO), and the Environmental
Protection Agency (EPA). The project was directed by a work group composed
of members from the three organizations. The audit guidelines and question-
naires in this manual were written and approved by five subcommittees of this
work group. The program areas selected by the work group for development of
audit guidelines were: (a) air quality planning and SIP activity, (b) new
source review, (c) compliance assurance, (d) air monitoring, and (e) vehicle
inspection and maintenance (I/M).
The subcommittees were chaired by State agency personnel with EPA staff
serving as expediters. All State agencies, local agencies, EPA Regional
Offices, and EPA Headquarters were provided an opportunity to comment on a
draft version of these audit guidelines. This manual, from concept to finished
product, is the result of a cooperative effort between State agencies, local
agencies, and EPA. While it necessarily reflects a compromise between the
competing interests of depth of analysis, breadth of review, and resources to
accomplish the audit, it does represent the "real world" ideas of people who
actually implement the duties.
The National Air Audit System (NAAS) was first implemented in 1984.
This FY 1986-1987 audit manual has been based on last year's manual, but has
been revised to include new emphases and to upgrade the quality of the audit
effort over last year. As with last year's manual, this manual has undergone
review by STAPPA, ALAPCO, and EPA Regional Offices and Headquarters.
Last year, EPA set forth overall policy for the NAAS, based on
agreement with STAPPA/ALAPCO and the Regional Offices. The portions of
that policy that will continue to apply this year, together with new
emphases, follow:
0 Program Coverage - Coverage would consist of the same five topics as
last year (air quality planning and SIP activities, new source review,
air monitoring, compliance assurance, and vehicle inspection/main-
tenance)
0 Audit Teams - EPA Regional Offices will select the composition of
the audit teams. A crossover team approach (State/local and outside
Regional Office representatives) is possible for FY 1986-1987 if
Regions and affected States so desire. All audit team members should
have sufficient knowledge of the programs they audit; the audit is
not intended as an on-the-job training program for inexperienced
personnel. Also, as a minimum, middle or senior Regional management
should personally participate in the exit interview with the agency
director.
° Audit Coverage - As was the case last year, each of the four
following audit topics should be covered at each State agency:
1-1
-------
(a) air quality planning and SIP activities; (b) new source
review; (c) compliance assurance; and (d) air monitoring. For
local agencies, only those topics for which the specific implemen-
ting authority rests with the local agency should be audited.
Only those agencies with I/M programs that have been on-going for
one year or more should be considered for I/M audits.
On-site Visits and Pre-preparation - Regions should continue to
conduct on-site audit visits, but should maximize use of telephone
interviews and "on-hand" information to prepare for the visit and
to complete significant portions of the questionnaire. Question-
naires should be exchanged and results studied and evaluated
each on-site visit.
° Schedule - To even out audit workloads on the State and local
agencies and the EPA auditors, STAPPA/ALAPCO and EPA agreed that
the NAAS effort will be spread over two years. Each EPA Region
will be responsible for ensuring that all State agencies and
selected local agencies are audited within the two year period.
It is recommended that approximately half the agencies are audited
the first year and the remaining agencies the second. Regions
may begin their FY 1986-1987 audit as early as they wish, so long
as the audited agency is agreeable and the previous item ("On-
site Visits and Pre-preparation") is accomplished prior to the on-
site visit. Under EPA's Strategic Planning and Management System
(SPMS), the Regional Office must forward a final audit report to
OAQPS within 180 days after the audit is completed.
0 Corrective Actions - Actions to implement the needed improvements
identified in the audits will be initiated through existing
mechanisms, i.e., 105 grants, State/EPA agreements, etc.
° Questionnaires - All Regions must use the uniform questionnaires
contained in this manual. Additional questions may be appended
only with the advance consent (at least 30 days) of the audited
agency. Such additional questionnaires and/or information requests
should be clearly identified as being independent of the NAAS. This
constraint, however, is not intended to restrict a Regional
Office from inquiring about deficiencies that surface during the
a ud i t.
0 National Report - At the end of the two year audit cycle, the
OAQPS will prepare a national report based upon the results of
all Regional audits. The national report will not rank agencies
or focus on deficiencies of specific agencies. Before they are
finally issued, drafts of the national report will be reviewed by
EPA Regions and representatives of STAPPA and ALAPCO.
0 Replacement for Other Audits - The NAAS has replaced Regional
Office audit activities carried out previous to initiation of the
NAAS. It does not, however, replace all portions of the 105
grant evaluations that are specified in the grant regulations.
1-2
-------
0 Other Oversight - The NAAS is intended to eliminate the need for
much of the "item by item" Regional Office oversight on certain
State and local agency programs. The NAAS, however, will not be
a substitute for the necessary flow of communications between
State/local agencies and EPA Regional Offices.
GOALS AND OBJECTIVES OF THE NATIONAL AIR AUDIT SYSTEM
The purpose of developing a national audit manual is to establish
standardized criteria for the EPA Regions' audit of State and local air
program activities. The primary goals of this program are to determine
the obstacles (if any) which are preventing the State or local air
pollution control agencies from being as effective as possible in their
air quality management efforts and to provide EPA with quantitative
information on how to define more effective and meaningful national
programs. States are playing a larger role than ever in the planning and
implementation of complex, and often controversial, air pollution control
strategies. EPA oversight of these and related activities is necessary
for ensuring national consistency and for assisting the States in resolving
identified problems.
The EPA and States can also use these audit results to ensure that
available resources are being focused toward identified needs (e.g.,
attainment and maintenance of standards, adoption of regulations,
implementation of regulations and technical analyses to support control
strategy development).
The EPA also hopes to share the results of these audits in a manner
that permits the "cross-fertilization" of innovative approaches and
systems across States and Regions. Only through this national exchange
can we hope to benefit from the invaluable experiences gained to date by
control agencies in carrying out the requirements of the Clean Air Act.
This audit guideline outlines a program which EPA, State, and local
air pollution control agencies can jointly use to--
0 Meet statutory requirements;
0 Assist in developing at least a minimally acceptable level of
program quality;
0 Allow an accounting to be made to the Congress and the public of
the achievements and needs of air pollution control programs;
0 Enable EPA, States, and local agencies to agree on needed technical
support and areas where program improvements (including regulatory
reform) should be made; this includes improvements to both EPA
and State/local programs;
0 Maximize and effectively manage available resources within the
State and local agencies and EPA, resulting in expeditious
attainment and maintenance of ambient air quality standards as
soon as possible; and
1-3
-------
0 Promote a better understanding of the problems facing air pollution
control agencies, thereby fostering mutual respect among EPA,
State, and local agency staff.
State, local, and EPA Regional Offices, working together, may identify
items in addition to those of the national program that are worthy of
further audit attention. In identifying these, the EPA Regional Office
and the State/local agency should understand in advance what the reasons
are and what the objectives and expected result(s) of this expanded review
will be. Also, the NAAS is not intended to preclude EPA Regions from
dealing, on a case-by-case basis, with significant deficiencies which are
identified during the course of the audit.
The EPA, State, and local agencies should keep in mind that the
audit is intended to improve the overall quality of air pollution control
programs. This intent of improving overall performance needs to be
clearly understood. The standards of performance outlined by these
guidelines are not so rigid that they eliminate the flexibility afforded
by the Clean Air Act. Also, these guidelines should not be construed to
establish performance standards which must absolutely be achieved in
practice. Moreover, while participating agencies will use the audit to
point out where opportunities exist for State or local improvements, it
is not expected that the audit will address every problem. The EPA,
however, will continuously search for and disseminate information about
better ways of consistently, effectively, and efficiently implementing a
comprehensive air pollution control program. This includes possible
reforms of EPA's requirements where feedback from the audits suggests
that certain requirements detract from program effectiveness.
AUDIT PROTOCOL
Each Regional Office must tailor the structure of the audit according
to the particular characteristics of the State and local agencies in the
Region and its own operating procedures. Certain elements and procedural
steps, however, appear necessary or useful in FY 1986-1987 based on
previous experience. These are discussed below.
Advance Preparation
The EPA should send a letter to the control agency well in advance
of the audit. The letter should confirm the date and time of the audit
and describe what resources the State is expected to provide, such as
office space and staff time. This letter should also identify the name
and title of each EPA individual who will participate in the audit.
With the exception of file audits and similar questionnaires, the EPA
Region will provide the control agency with the nationally prepared
questionnaire. These should be sent to the State or local agency 6 weeks
in advance of the audit and, thus, will allow the agency to better prepare
for the audit. The State or local agency should fill out the specified
parts of the questionnaires and return a copy of the completed questionnaire
to the Regional Office 2 weeks before the on-site visit. The Regional
Office subject experts should review the completed questionnaire and use
it to prepare the audit team for the on-site visit. Returning the
completed questionnaire to the Regional Office before the visit should
1-4
-------
serve to minimize wasted effort and time reviewing the questionnaire
during the on-site visit and to prepare the audit team for discussions
that focus on any problems uncovered in the questionnaire. Because of
time limitations, however, it may not be possible in all cases to return
the completed questionnaire to the Regional Office before the on-site
visit; in these cases, the State or local agency will have to make the
completed questionnaires available to the audit team during the on-site
visit.
The chapter for each of the audit topics presents the specific
protocol for that audit topic. This includes procedures such as advance
tailoring of the questionnaires for the air quality planning and SIP
activities audit, selection of files for new source review audits, and
instructions for use of the individual questionnaires.
On-site Visit
The primary purposes of the on-site visit are to—
0 engage in a broad discussion with agency staff to gain insight
into any recent changes in the structure of the organization,
discuss specific problem areas of the agency, and become acquainted
with the staff in order to better open up channels of communication;
0 discuss and clarify answers to the questionnaire and complete
any questions not answered by the audited agency;
0 review on-site documents that are too cumbersome to transmit such
as permits, modeling runs, and supporting files; and
0 audit by observing the agency's daily operations of programs for
air monitoring, compliance assurance, new source review, planning
and SIP activities, and (where appropriate) vehicle inspection
and maintenance.
Typically, the on-site audit is conducted in four phases:
0 The EPA auditors for all programs meet with the State agency
director and top staff to discuss the goals of the audit and to
"break the ice." This meeting usually sets a cooperative tone
for the visit.
0 The EPA auditors conduct a discussion of the questionnaire with
the person(s) in charge of each of the activities to be audited.
0 The EPA auditors will review appropriate files; this will usually
be necessary for each of the five audit areas to varying extents.
0 The exit interview is held as a wrap-up session to inform agency
management of the preliminary results of the audit. This promotes
harmony between EPA and the State by giving immediate feedback of
the results in a face-to-face meeting between the people actually
performing the audit and those responsible for the programs being
audited. The EPA middle or upper management from the Regional
1-5
-------
Office should participate in at least this portion of the on-site
visit.
The time which the audit team spends to complete the various phases
of the on-site visit should usually not exceed 3 days. This general
rule, however, will be difficult to adhere to in certain instances, such
as when satellite facilities of the agency must also be visited. In any
event, the duration of the on-site visit should be mutually agreed upon
in order not to create an undue burden to the agency being audited.
AUDIT REPORTS AND USE OF AUDIT DATA
Each State or local agency audit report should contain the findings
for each of the four or five audited areas. The audit report must include
a copy of the completed questionnaire(s) (except file audit or similar
questionnaires—see specific instructions in appropriate chapters of this
manual) for each of the audit topics to enable national compilation of
audit results. Since the questionnaires will be included, the audit
report should not merely reiterate the answers on the questionnaires, but
present the Regional Office's overall findings. The Region should give
the State or local agency an opportunity to comment on a draft of the
report before it is released outside of EPA. This allows misunderstandings
and errors to be discovered before the report is made final.
The audit report should contain an executive summary that provides
the Regional Office's overall assessment of the audited agency's program
after reviewing all the questions as a whole.
Major deficiencies identified during the audit should also be high-
lighted in the executive summary. This enables the Region to detail all
the findings of the audit without causing the reader to confuse minor
points with major problems. It also identifies to the audited agency
those deficiencies considered most serious.
Where an agency disagrees with the conclusions of the audit, it
should provide to EPA written comments outlining its perspective. These
will be incorporated as an appendix into the final report. The report
should also highlight outstanding and/or innovative program procedures
that are identified.
The audit would be of limited use without some mechanism for rectifying
identified deficiencies. Therefore, it is important that the report recom-
mend measures or steps to treat the causes determined to be responsible for
these inadequacies. Lead agencies responsible for implementing these recom-
mendations and anticipated resource requirements should also be considered.
Each question should be answered on the questionnaire itself;
attachments should be avoided unless a question specifically requests
them. If attachments are requested and included, the attachments should
be forwarded to OAQPS along with the questionnaire.
A recommended format for preparation of the audit report is given in
Table 1-1. Following a standardized format will not only enable reviewers
to easily find material in the text, but will also facilitate compilation
of information into the national report.
1-6
-------
A national report compiling the findings of the audits conducted by
the EPA Regions will be prepared by EPA Headquarters. This analysis will
be based on the reports prepared by the EPA Regions discussed above. It
will not rank agencies or focus on specific deficiencies in individual
programs. While it will address areas of conflict between EPA guidance
and action of implementation experience, it will not be a forum for
addressing unresolved issues between audited agencies and states.
The NAAS initiative is designed as a partnership effort to help EPA
and State and local agencies each do their respective jobs better. It is
our hope that it can become the foundation which all involved can use to
make solid progress in protecting and enhancing the quality of our Nation's
air.
1-7
-------
Table 1-1
RECOMMENDED FORMAT FOR AUDIT REPORTS
Introduction--Purpose of audit, for benefit of potential layman readers;
identify persons on EPA audit team and persons interviewed in audit visit.
Executive Summary—Outl ine major findings, major deficiencies, and major
recommendations. As the name implies, it is a summary designed for the
chief executive--the director--of the audited agency. The summary should
cover all audit topics:
Air Quality Planning and SIP Activities
New Source Review
Compliance Assurance
Air Monitoring
Vehicle Inspection and Maintenance (where audit is conducted)
Air Quality Planning and SIP Activities—Follow recommended format
in Chapter II of Audit Manual
New Source Review—Follow recommended format in Chapter III of Audit Manual
Compliance Assurance—Follow recommended format in Chapter IV of Audit
Manual
Air Monitoring—Follow recommended format in Chapter V of Audit Manual
Vehicle Inspection and Maintenance—Follow recommended format in Chapter VI
of Audit Manual
Appendices
Completed Air Quality Planning and SIP Activities Questionnaire
Completed New Source Review Questionnaires (except for permit
file questionnaire)
Completed Compliance Assurance Questionnaire
Completed Air Monitoring Questionnaire
Completed Vehicle Inspection and Maintenance Questionnaire (where
audit is conducted)
Audited Agency's comments on draft report (for final report)
1-8
-------
Chapter 2
AIR QUALITY PLANNING AND SIP ACTIVITIES GUIDELINES
FY 1986-1987
Section Title Page
Introduction 2-2
A Air Quality Evaluation 2-6
A.I Air Qual ity Reports 2-6
A.2 Section 107 Attainment Status Designations 2-7
A.3 New Violations 2-11
B Emission Inventories 2-12
B.I Use of Criteria Pollutant Emission Inventory 2-12
B.2 VOC Inventories in 03 Nonattainment Areas 2-16
B.3 CO Inventories in CO Nonattainment Areas 2-19
B.4 Highway Vehicle Inventories in 63 and CO
Nonattai nment Areas 2-20
C Modeling 2-23
C.I Training and Experience 2-23
C.2 Model Availability 2-24
C.3 Alternative Modeling Techniques 2-26
C.4 Modeling Analyses 2-28
D SIP Evaluation and Implementation 2-30
D.I Timeliness of Regulatory Development 2-30
D.2 Timeliness of Studies 2-32
D.3 Transportation Control Measures 2-34
D.4 Source Activity 2-35
D.5 Generic Bubble Rules 2-36
2-1
-------
2. AIR QUALITY PLANNING AND SIP ACTIVITY
INTRODUCTION
The FY 1986-1987 audit guidance for air quality planning and SIP
activity has been revised from the guidance used in FY 1985 although the
major program areas covered are the same as in previous years. Those are
air quality evaluation, emission inventories, modeling, and SIP evaluation
and implementation.
Many of the comments received on the November draft guidance expressed
concern that the Air Quality Planning and SIP Activity guidance for the third
consecutive year was more of a survey questionnaire than an audit question-
naire. These comments and others have been addressed in the final version
of the questionnaire. More Regional Office verifications and file review
activities have been added to this chapter. One example of this is the
review of emission trades in those States that have a generic bubble rule
and completion of a file review form for each bubble audited.
Audit criteria are provided below for each of the program areas to
be audited. Each topic is prefaced by a brief discussion of what activities
it encompasses and what we generally hope to accomplish through the audit.
PROTOCOL
Procedurally, the Regional Offices will, upon receiving this
questionnaire from OAQPS, add to it the items listed in Table 1 before
sending it to the State and local agencies. These agencies will then
complete the questionnaire and return it to the Regional Office at least
2 weeks before the on-site visit. The agencies should pay particular
attention to the items listed in Table 1. Regional EPA staff would then
review the State/local agency responses prior to the on-site audit.
Regional Office auditors should discuss all questionnaire items with the
appropriate agency personnel during the audit.
In compiling the FY 1985 national report, it was noted that a number
of the questionnaires for the Air Quality Planning and SIP Activities
chapter were incomplete. Whether or not the agencies return the
questionnaire with all of the questions answered, it is expected that the
Regional Office will verify the answers provided and complete all parts
of the questionnaire that are left blank. It should be remembered that
this is an EPA audit of the agencies' programs; therefore, the answers in
the questionnaire must reflect the findings of the auditors, even if the
agencies are not in complete agreement with these answers. The differences
are to be discussed with the agencies during the audit and between the
issuance of the draft report and the final report. If the differences
2-2
-------
Table 1
List of Items in the Air Quality Planning and SIP Activity Chapter That
Must be Completed by the Regional Office Before the Questionnaires are Sent
to the Agencies
Section
C.4.a. Modeling Analyses
D.I
D.2
Table--Regulatory
Development
Table—Studies
Regional Office Requirements
Fill in columns A, B, and C
Provide agencies with a list of SIP
revisions or strategies due
Provide agencies with a list of
studies due or overdue
List of Items to be Completed by the Agency
Section
A.2.f Table A-l Section 107
Attainment Status
Designation
C.4.b Modeling Analyses
D.I.a Tab 1 e--Regulatory
Development
D.2 Table—Studies
D.3 Transportation Control
Measures
D.5.f Generic Bubble Rules
Agency Requirements
Fill in the table
Fill in columns A, B, and C
Complete the appropriate columns
Complete the appropriate columns
Complete the form
List the agency approved bubbles
2-3
-------
cannot be resolved, the agencies' comments are to be attached to the
final report. The Regional Offices are expected to ensure the quality
of the reports and see that all the questions are answered in the
questionnaire.
The draft report from the Regional Office should consist of an overall
executive summary and highlights of each part (air quality evaluation,
emissions inventories, modeling and SIP evaluation and implementation).
The questionnaire would then follow this narrative audit summary.
AIR QUALITY EVALUATION
States are continuously gathering air quality data to satisfy various
statutory requirements and air management needs. The adequacy of these
ambient measurements is addressed in a separate audit area dealing with air
monitoring requirements. This audit topic is concerned with the State
capabilities to perform air quality evaluations. This includes the
ability to systematically consider available air quality data for the
purpose of disseminating information to the public, assessing section 107
redesignations, and analyzing new violations. The specific areas that
should be evaluated in this audit category are discussed in the questionnaire
that follows this section.
EMISSION INVENTORIES
The emission inventory provides information concerning source
emissions and defines the location, magnitude, frequency, duration, and
relative contribution of these emissions. An inventory is useful in
designing air sampling networks, predicting ambient air quality, designing
control strategies, and interpreting changes in monitored air quality
data. Plans for attaining and maintaining NAAQS1 are dependent on a
complete and accurate emission inventory. The FY 1986-1987 guidance has
focused on the major nonattainment problem of ozone by directing questions
toward the completeness and quality of the agencies VOC emission inventories.
In the implementation of a nationwide program of air quality
management, consistent methods of inventory compilation are essential.
An adequate emissions inventory must be accurate, complete and up-to-
date, and provide for consistency in planning between metropolitan areas,
States, and Regions.
MODELING
Air quality models are being used more extensively in the conduct
of day-to-day activities in the planning and SIP program area. These
activities include such things as attainment demonstrations, major source
compliance determinations, new source review, evaluations of "bubbles,"
and assessing attainment status. Most State agencies should have the EPA
reference models on-line that are available for use in these and other
types of applications. The modeling audit is intended to gather information
regarding the agency's demonstrated expertise and capability to perform
necessary air quality modeling analyses consistent with accepted EPA procedures.
2-4
-------
In accomplishing this objective, this year's audit reviews the various
kinds of modeling applications performed or evaluated at the State/local
program. Because the Region will have already reviewed certain site-specific
modeling analyses which the State/local agency has submitted (such as
bubbles, new source permits, etc.), the questionnaire asks the Regional
Office to list the results of these evaluations.
SIP EVALUATION AND IMPLEMENTATION
An evaluation of SIP development and implementation activities is
necessary to assess whether State plans for attainment and maintenance of
standards remain responsive to identified needs and are being reasonably
carried out. Certain key elements of specific SIP strategies have been
singled out for evaluation. Focusing the NAAS review on these areas is
not intended to preclude the auditor from evaluating other aspects of
major SIP activity in an air program. Nor does it imply that all other
SIP responsibilities are considered less important. Rather, it represents
an attempt to isolate a manageable number of SIP-related efforts for
national review and comparison.
2-5
-------
A. Air Quality Evaluation.
A.I. Air Quality Reports
State/Local Agency Response:
a. Does your agency regularly make reports available to the public
on air qual ity data?
Yes No If no, go to question A.2.
b. If "yes," does the report— (please answer each question)
Yes No
1. indicate whether data shown has been quality
assured according to EPA guidelines?
2. cover all criteria pollutants?
3. cover all monitors within the agency's jurisdiction?
4. describe the completeness of the data collected
during the reporting period?
5. indicate comparisons to NAAQS on an appropriate
basis (i.e., expected exceedances for ozone and
highest 2nd high for CO, etc.)?
6. summarize historical air quality data by year
since the start of the monitoring program in
that area?
7. present maps of the nonattainment areas and maps
depicting the locations of the ambient monitors?
8. describe each of the pollutants?
9. briefly describe each of the sampling techniques?
How often is the report published?
1. Monthly
2. Quarterly
3. Semi annually
4. Annual ly
5. Other (describe)
2-6
-------
d. What is the time period between the end of the data collection
period and publication of the data?
1. 0 to 2 months
2. 3 to 6 months
3. 7 to 12 months
4. >12 months
EPA Regional Office Response: The Regional Office should obtain a copy
of the qir quality report to verify the answers in a, b, c, and d.
A.2. Section 107 Attainment Status Designation
State/Local Agency Response:
a. Within the last full CY did your agency review section 107 primary
or secondary attainment status designations?
Yes No (briefly explain)
b. List below any section 107 reviews completed that did not result in a
request for redesignation? Please note that Table A-l addresses any
reviews that did result in a redesignation request.
If nonattainment
Current check whether
County Name(s) Pollutant Designation primary or secondary
2-7
-------
Indicate below the methods used to review attainment status:
(Check all that apply)
Yes No
1. Staff notification of violations to agency management
2. Consideration of air quality data for the past
2 to 3 years
3. Consideration of modeled exceedances
4. Consideration of control and emission changes
(determination of whether changes in emissions were
caused by control or by changes in operating conditions)
5. Investigation into cause of violations
6. Other (describe)
d. Are there any attainment or unclassified areas under a "call for SIP
revision" because the air monitoring data showed the area to be
nonattai nment?
No Yes If yes, list area(s) and pollutant below.
e. Will the agency submit a redesignation request for the area(s) listed
in d above. Yes No
2-8
-------
f. Please complete Table A-l indicating the counties and pollutant for
each redesignation that was initiated and/or completed in the
previous calendar year (1985 or 1986).
TABLE A-l. SECTION 107 ATTAINMENT STATUS DESIGNATION
Attainment
to nonattainment
Nonattainment to
attainment or
unclassifiable
Primary nonattai-
ment to secondary
nonattainment
Secondary non-
attainment to
primary non-
attainment
Unclassifiable
to attainment
Unclassifiable
to nonattainment
(e.g., PM10,
S02, & NOx)
County Names
[Indicate by (P)
if Part of a County]
Pollutant
Redesigna-
tion
Initiated
Yes/No
Redesignation
Completed &
Submitted to
EPA
Yes/No
For 03 or CO an "attainment" designation also includes "unclassifiable."
2-9
-------
EPA Regional Office Response (confirmation or comment)
g. On how many of the above "completed and submitted redesignations"
in Table A-l has EPA published final rulemaking?
Indicate by county and pollutant.
1. To nonattai nment: primary
secondary
2. From nonattainment: primary
secondary
3. From unclassifiable to attainment: primary
secondary
h. For those cases where final rulemaking has not been taken, please
describe current status and associated problems or issues:
2-10
-------
A.3. New Violations
The purpose of these questions is to learn how many new violations
of the NAAQS occured last year and how the agencies handled these violations.
State/Local Agency Response:
a. During the last CY (1985 or 1986) indicate all existing air
monitoring sites by pollutant that revealed new! violations to
an NAAQS. See below for the list of action codes.
Site Pollutant Action Code*
1.
2.
3.
4.
5.
b. Indicate violations at a monitoring site which was newly established
during the last CY (1985 or 1986).
Site Pollutant Action Code*
1.
2.
3.
4.
5.
*Codes Type of Action
A Microscopic laboratory examination
B Additional monitoring
C Data ignored: a documented exceptional event (RO should check to
see if the event met EPA's definition of "exceptional event."
D Enforcement action based on existing SIP regulation
E Source-specific, local, county or areawide SIP revision
F Verification by modeling studies
G Change in an individual source permit
H Other (describe)
Use as many codes as are applicable. Please note that if no action
code is entered it will be assumed that no action was taken by the
agency. Regional Office should confirm that the violation was not
investigated.
^Violations at a monitoring site which had been violation free for
the previous 3 years.
2-11
-------
B. Emission Inventories
B.I. Uses of Criteria Pollutant Emission Inventories
Emission inventories are used in a number of applications by air pollution control
agencies, including the demonstration of reasonable further progress (RFP) in
03 nonattainment areas. The following questions deal with the uses of emission
inventories by your agency.
State/local agency response
a. RFP Toward Attainment of 0^ Standards^-
The Clean Air Act requires SIP's to provide for maintenance of emission inven-
tories to ensure RFP. In the past year, did your agency actually track and
compare changes in the VOC emission inventories with projected changes in
emissions given in the respective 63 curves in the SIP's? Check yes or no for
each nonattainment area listed below. If no, insert the letter code best
representing the reason RFP is not tracked.
Areas where 03 RFP should be tracked:
(Regional Office to provide list)
1.
Yes
No
Reason
(use codes)
2.
3.
4.
5.
CODE
A.
B.
C.
D.
E.
Alternative tracking mechanisms are used (e.g., air quality data), not
directly involving emission inventories
RFP tracking is not considered a priority task
Insufficient resources available to track RFP by maintaining an up-to-date
emission inventory
Insufficient guidance available on how to do RFP tracking
Other (specify)
1-This question applies only to those areas specified by the Regional Office
as 03 extension areas and areas where EPA has called for SIP revisions. (EPA
Regional Offices must provide a list of these areas.) Note that RFP tracking
means compiling a realistic estimate of an individual year's emissions and
comparing this to the appropriate year in the SIP RFP curve. Merely compiling
air quality data trends as an alternative to compiling emissions data is not
accepted as RFP tracking.
2-12
-------
b. RFP Report: If "yes," did your agency, in the past year:
1. Prepare a report on RFP? yes no
2. Submit the report to EPA? yes no
3. Make the report available for public comment? yes
no
For EPA Regional Office response:
4. Has the RO received the above reports?
yes
no
5. If "yes," has the RO commented or otherwise responded to the
State or local agency on the report? yes no
6. Are you assured that any progress indicated is the result of
real emission reductions rather than the result of changes in
methodologies, emission factors, etc.? yes no
c. RFP Emissions Inventory Update: For each of the areas where 03 RFP
should have been tracked in the past year, what approximate percent
of the VOC emissions in the inventory was updated for the major,
minor, and mobile source categories that year? The areas are the
same as those listed by the Regional Office in question a.
Use one of the following percent ranges in the table.
0-19%, 20-39%, 40-59%, 60-79%, 80-100%
Areas where 03
RFP should be
tracked. See
question a.
1.
2.
3.
4.
5.
Stationary Sources
Major
% RO*
Minor
Regulated
% RO*
Unregul ated
% RO*
Mobile
Sources
% RO*
*The Regional Office auditor should ask the agency for documentation on the
extent of updating the RFP inventory and initial in this column if
documentation appears adequate.
2-13
-------
d. Other Uses of Emission Inventories
Indicate below other uses that were made of your agency's criteria
pollutant emission inventories in the past year, not necessarily just in
nonattainment areas. (Check "x" where appropriate.)
1. Used for developing and
evaluating areawide
control strategies
2. Used as input to dispersion
and other air qua!ity
model s
3. Used to project possible
areas of high pollutant
concentrations to help
place ambient monitors
4. Used for source permits or
inspections, including for
assessing permit, operating,
and inspection fees
5. Used for responding to
information requests
6. Used indirectly for
general program planning
7. Other uses (specify)
PM SO
VOC CO
Pb
e. Quality Assurance
Quality (or validity) assurance for emission inventories involves checks of the
procedures, emission factors, calculations, etc., that were used during compil-
ation as well as checks for missing sources and edit checks for reasonableness.
Specify below the statement that best describes the quality assurance
measures that are conducted on your State's emission inventories.
1. Formal, rigorous, regular checks are implemented
2. Less formal, spot checks are made, on an irregular basis
3. No quality assurance measures are implemented
2-14
-------
f. What should EPA do to help you make your criteria pollutant inventories
more comprehensive, accurate and current? (Check "x" where appropriate.)
Very Current Data and No Strong
Important Useful Guidance Adequate Opinion
1. Provide better guidance on...
-Point sources
-Area sources
-Highway vehicle
-Locating sources
-Questionnaire design
-Quality assurance
-Data handling
-Reflecting SIP regs
in projection inventory
-Other(specify)
2. Improve emission factors
in AP-42
3. Provide computerized
systems having better
data handling capabilities
4. Other(specify)
EPA Regional Office Response (Confirmation or Comment)
2-15
-------
B.2. VOC Inventories in 03 Nonattainment Areas
Current guidance requires that VOC inventories in 03 nonattainment areas be
adjusted in various ways to reflect reactive emissions occurring during the 63
season. The following questions address the adjustments made in your agency's
VOC inventory and apply to all components of the inventory, i.e., point, area
and highway vehicle sources. (NOTE: if your state contains no nonattainment
areas for 03, check here [ ] and go to B.3.)
State/local agency response
a. EPA guidance specifies that methane, ethane, methylene chloride,
methyl chloroform, trifluoromethane and 6 chlorofluorocarbons should be
excluded from 03 SIP inventories as nonreactive. Indicate below your agency's
exclusion of nonreactive VOC compounds from its 03 SIP inventory.
(Check "x" where appropriate.)
1. No VOC compounds have been excluded as nonreactive (if checked,
go to question b.)
Check the compounds that are excluded from the 03 SIP as nonreactive.
2. Methane 3. Ethane
4. Methyl ene chloride 5. Methyl chloroform
6. Trifluoromethane 7. Chlorofluorocarbons CFC-11, CFC-12,
CFC-22, CFC-113, CFC-114, CFC-115
8. Others - specify compounds
9. The agency excludes the following compounds based on vapor pressure
cutpoints:
b. What technical basis does your agency use to identify and quantify
nonreactive VOC? (Check "x" where appropriate.)
1. Nonreactive VOC not excluded, so question not applicable
2. Use EPA's VOC Species Data Manual
3. Use MOBILES option to generate nonmethane VOC emission factors for
highway vehicles
4. Use general species profiles from the literature
5. Sources are asked to list nonreactive compounds in their VOC emissions
6. Other (specify)
2-16
-------
c. Higher summertime temperatures can have a marked impact on the levels
of evaporative VOC from several important source categories. Likewise, lower
summertime volatilities of gasolines, as reflected by lower Reid Vapor Pressures
(RVP) can also have an impact. Have your VOC totals been adjusted for conditions
representative of the 63 season?
1. Yes No
If yes, check the appropriate statement(s) below:
2. Higher 03 season temperatures have been considered in
generating highway vehicle emission factors
3. Higher 03 season temperatures have been considered in estimating
evaporative losses from petroleum product (including gasoline) storage
and handling.
4. Lower summertime RVP's have been considered in estimating evaporative
losses from gasoline storage and handling
d. A number of source categories have recently been identified as being
potentially significant VOC emitters that have not traditionally been included
in VOC inventories, especially those relating to fugitive and/or waste treat-
ment processes. Have the following sources been included in your agency's VOC
inventory? (Specify "yes" or "no," or "N/A" if no such sources are located in
your area.)
1. POTW's (Publicly Owned Treatment Works, i.e., sewage treatment plants)
2. TSDF's (Treatment, Storage and Disposal Facilities for hazardous wastes,
including landfills, surface impoundments, waste piles, storage and
treatment tanks, hazardous waste incinerators, and injection wells)
3. Fugitive leaks from valves, pump seals, flanges, compressors, sampling
lines, etc., in organic chemical manufacturing facilities (esp. SOCMI)
e. EPA has recommended an emission factor of 6.3 Ib/capita-year to account
for miscellaneous commercial/consumer solvent in urban areas. Check below
which one response best describes your agency's handling of this source category.
1. EPA per capita factor used both in base year and projected attainment year
inventories
2. EPA factor used in base year but a lower factor used in attainment year
inventories, projecting lower commercial/consumer solvent use in future
3. Non-EPA factors used in both base year and attainment year inventories
4. This source category not considered in VOC inventory
2-17
-------
f. In question B.l.f, we asked how EPA could help you on your criteria
pollutant inventory. Indicate below where your agency specifically feels
better information or guidance is needed to improve its VOC inventory. (Check
"x" where appropriate.)
Current Data
Very Or Guidance No Strong
Important Useful Adequate Opinion
1. Excluding nonreactive VOC
2. 03 season adjustment of
VOC totals
3. Emission factors for
sewage treatment plants
and hazardous waste
treatment, storage, and
disposal facilities
4. Updated commercial/
consumer solvent factors
5. Other(specify)
EPA Regional Office Response (Confirmation or Comment)
2-18
-------
B.3. CO Emission Inventories in CO Nonattainment Areas
If your State contains no CO nonattainment areas, check here [ ] and go to B.4.
State/Local Agency Response
a. Indicate which response(s) below describe the geographic coverage and
focus of your agency's CO inventory. (Check "x" where appropriate.)
1. Major emphasis is on maintaining a CO inventory for highway vehicle
sources for certain traffic areas such as Central Business Districts,
intersections, or specific nonattainment areas
2. Areawide or countywide CO inventory is maintained, covering major CO
point sources, area sources and highway vehicles
3. CO inventory is not currently maintained
b. Are woodstoves included in your CO emission inventory? Yes No
c. Is the highway vehicle inventory or transportation/traffic data used
to locate potential CO hot spots?
Yes No
EPA Regional Office Response (Confirmation or Comment)
2-19
-------
B.4. Highway Vehicle Inventories in 03 and CO Nonattainment Areas
Highway vehicle emissions are often compiled by the air pollution control agency
acting in concert with the local planning agency or transportation department.
In some instances, the local MPO or DOT will compile the inventory independently
as the lead responsible agency. In general, highway vehicle emissions are cal-
culated by applying mobile source emission factors to transportation data such
as vehicle miles traveled (VMT), trip ends, etc. Mobile source emission factors
are available for various vehicle types and conditions from an EPA emission
factor model entitled MOBILES (or from earlier versions). Important conditions
affecting emissions are vehicle age and mix, speed, temperature, and cold start
operation.
If your State contains no 03 or CO nonattainment areas, check here [ ] and
go to B.5.
State/local agency response
The State or local agency should answer the following questions even if a
transportation or planning agency is responsible for the highway vehicle
inventory.
a. Which agency maintains the highway vehicle emission inventory
for the 03 and/or CO nonattainment areas? (Check "x" where appropriate.)
1. Air pollution agency (State or local)
2. Local planning organization (MPO, COG, RPC, etc.)
3. State or local transportation department (DOT)
4. Other (specify
5. None is maintained
6. Unsure
b. If an agency other than the air agency maintains the highway vehicle
inventory, indicate what difficulties (if any) result. (Check "x"
where appropriate.)
1. No significant difficulties are evident
2. Scheduling and coordination of activities are negatively affected
3. The air agency loses control of the design and format of the
inventory
4. The responsible agency has not been adequately funded to be
responsive
5. Additional technical guidance is needed for effective communi-
cation of program needs to another agency
6. Other (specify)
2-20
-------
c. Conversely, indicate what benefits (if any) accrue from having another
agency responsible for the highway vehicle inventory. (Check "x" where
appropriate.)
1. No significant benefits result
2. Less resource drain on the air agency
3. The air agency doesn't have to develop transportation planning
expertise
4. A better product results
5. Other (specify)
d. Which emission factor model (MOBILE 1, 2, 2.5, or 3) was used to
generate the highway vehicle emission factors for the most recently
developed or maintained inventory? (Indicate number or "U" if unsure.)
e. Were the highway vehicle emission factors in the model tailored to
your area to account for the following parameters? (Indicate "Yes," "No" or if
unsure, specify "U.")
1. Vehicle mix
2. Vehicle age
3. Speed
4. Ozone season temperature
5. Cold/hot start operating modes
f. Were data from the local transportation planning process used to
compile the most recently-developed or maintained highway vehicle inventory?
(e.g., VMT, street locations, traffic volumes, growth patterns, etc.)
Yes No Unsure
g. If not, were gross areawide estimates of VMT or gasoline sales used
to compute emissions? Yes No Unsure
h. An important component of travel sometimes overlooked in highway
vehicle inventories is VMT associated with minor roads and connnectors, often
called "local" or "off network" travel. Was local travel included in
your most recently-developed or maintained highway vehicle inventory?
Yes No Unsure
2-21
-------
i. The results of last year's audit indicated that significantly fewer
highway vehicle inventories contained NOX emissions than VOC emissions.
Indicate if and why this is so for your agency. (Check "x" where appropriate.)
1. Not so. Our highway vehicle inventory contains both NOX and VOC.
2. NOX inventory not perceived as needed because NOX reductions are
not required for 03 control
3. NOX inventory perceived as needed for 03 but it was not included
because of resource limitations
4. Other (specify)
j. Last year's audit asked each agency to specify the base year of the
highway vehicle inventory in the SIP, which gave a limited idea of how well
these inventories have been maintained to the present. What is the latest year
of record for which your agency's highway vehicle inventory has been updated?
When was this done?
1. 19 (latest year of record)
2. 19 (year when latest update was performed)
EPA Regional Office Response (Should confirm the answers in B.4)
2-22
-------
C. Modeling
C.I. Experience and Training
The ability of an agency to effectively deal with modeling problems depends
upon the personnel resources available to the agency. Competent and experienced
personnel are essential to the successful application of dispersion models.
The need for specialists is critical when more sophisticated models are used or
the area being studied has complicated meteorological or topographic features.
Please summarize your agency's staff levels and modeling experience.
State/Local Agency Response:
a. Please complete the following table indicating the number of persons in
each of the training and experience categories:
Experience
Training 0-2 yrs 2-5 yrs 5-10 yrs >10 yrs
1. Meteorologist
2. Engineer/Scientist
with modeling training
3. Engineer/Scientist
without modeling training
4. Other educational background
with modeling training
5. Other educational background
without modeling training
EPA/Regional Office Response (confirmation or comment):
2-23
-------
C.2. Model Availability
The ability of an agency to effectively deal with modeling problems that
may arise also depends on the facilities (hardware and software) available
to the staff to perform or confirm modeling analyses. Please summarize
below your agency's accessibility, expertise and usage of computer-based
air quality models.
State/Local Agency Response:
a. To which air quality models does your staff have access? Also indicate
whether your staff is capable of running the model and the approximate
number of applications during the last fiscal year.
1. Circle UNAMAP version number
2. Specific Models
Other
Don't know
EPA Guide-
1 ine
Model s(i)
Not in
Guideline
Yet(2)
Other
Models(3)
Access
(yes or no)
In-house
Expertise to Use
(yes or no)
Yearly Usage
(Estimated Number
of Applications)
APRAC-1A
AQDM
COM
RAM
CRSTER
TCM
TEM
HIWAY
ISCLT
ISCST
MPTER
CDMQC
EKMA
AIRSHED
PAL
PLUVUE
Screening PT MAX
Techniques(4)pi DIS
PTMTP
PTPLU
VALLEY
COMPLEX
2-24
-------
b. Is access generally by:
1. Telephone line to State/local agency mainframe computer?
2. Telephone line to private or subscription computer?
3. In-house dedicated computer?
4. Telephone line to EPA computer?
c. Does your staff have the capability to modify software for the
above models? Yes No
If yes, which models have been modified?
Where modified guideline models have been used, State/local agency
should answer appropriate part of question C.3.
Footnotes:
(1) Models recommended in the "Guideline on Air Quality Models"
(1978)
(2) Models recommended in the "Regional Workshops on Air Quality
Modeling: A Summary Report" (1981)
(3) In addition to the examples given, list the nonguideline models
available to you and indicate whether any have been used on a
case-by-case basis. Include long range transport models,
photochemical models, complex terrain models and any other models
for situations for where EPA has not provided guidance.
(4) In addition to the EPA screening techniques listed, indicate
the accessibility and usage of any other screening techniques
available to you.
EPA/Regional Office Response (confirmation or comment):
2-25
-------
C.3. Alternative Modeling Techniques
EPA modeling guidance recommends specific models and data bases to be
used in regulatory modeling. However, the guidance also indicates that
an alternative model or data base may be used in an individual case if
it can be demonstrated that the alternative technique is more appropriate
than the recommended technique. Describe the number of, and circumstances
related to, modeling analyses where it was necessary to use alternative
techniques from those specifically recommended in EPA guidance.
State/Local Agency Response:
a. In approximately what number of the modeling analyses performed by your
agency in the last fiscal year was it necessary to use techniques not
specifically recommended in EPA guidance?
1. times out of modeling analyses performed.
b. If an alternative model was used, indicate the reason(s) for its usage,
using the list below: (alternative data bases are covered in part C of
this question)
Number of Reason(s)
Cases for Use*
1. Use of nonguideline model
2. Modification of guideline model
3. Use of nonrecommended option in
a guideline model
4. Use of guideline model outside
its stated limitation
5. Other (describe )
*Reasons for use: (List one or more of the following codes as applicable
in the space above)
CODES
A. The alternative technique was judged, for technical reasons,
to be more appropriate for the situation.
B. Lack of access to the guideline model recommended for the situation.
C. The alternative model was judged, through a performance evaluation,
to be more appropriate for the situation.
D. No EPA guidance applies to the situation.
E. Other (specify below).
2-26
-------
c. If there are cases where the selection/usage of data bases for
models are different from those recommended in EPA guidance, please
indicate the number and circumstances surrounding each case.
Number Reason (brief Statement)
of Cases
1. Use of less than 5 years
of off-site or less than
1 year of on-site
meteorological data.
2. Use of techniques other
than those contained in
the "Guideline on Ai r
Quality Models" for
determining background
3. Use of techniques other
than those contained in
EPA policy on treatment
of calms
4. Use of techniques other
than those contained in
the EPA policy on design
of receptor network
EPA Regional Office Response (confirmation or comment):
2-27
-------
C.4. Modeling Analyses
a. State and local agencies will normally conduct and submit to EPA modeling
analyses to support certain actions. EPA will review these analyses case-
by-case and approve or disapprove them in the Federal Register.
TO BE COMPLETED BY THE EPA REGIONAL OFFICE:
Please indicate in column A approximately how many modeling analyses
described above were submitted to EPA during the last fiscal year in each of
the program areas listed below. In column B, indicate the number of analyses
where EPA has required the State/local agency to revise the analysis. In column
C, use the code letters provided below to indicate the technical areas contributing
to the answer given in column B.
A B C
# of analysis # of analysis Contributing
reviewed requiring revision factors*
1. Bubble (emission
trades)
2. Section 107
redesignations
3. New source review
(including PSD)
4. Nonattainment area
SIP analyses
5. Lead SIP's
6. Other SIP modeling
*Indicate by code(s) which of the following technical areas were either
inadequate or deviated from EPA Guidance.
CODES
A. Use of inappropriate guideline model
B. Use of nonguideline model without a performance evaluation to
demonstrate acceptability of the model
C. Urban/rural dispersion coefficients
D. Emission inventory and operating design parameters
E. Meteorological data base
F. Receptor network design
G. Complex terrain considerations
H. Downwash consideration
I. Comparison with acceptable air quality levels
J. Technical documentation of modeling analysis
K. Other
State/Local Agency Response (confirmation or comment)
2-28
-------
b. In those instances where the modeling analysis supporting a particular
action is performed by industry and/or other governmental entity, the
agency will review and approve or disapprove the modeling analyses.
TO BE COMPLETED BY THE STATE/LOCAL AGENCY:
Indicate in column A approximately how many modeling analyses described
above were reviewed during the last fiscal year in each of the indicated
program areas. In column B, indicate the number of analyses where the agency
required the responsible party to revise the analysis. In column C, use the code
letters provided below to describe the technical area contributing to the answer
given in column B.
A B C
I of analysis I of analysis Contributing
reviewed requiring revision factors*
1. Bubble (emission
trades)
2. Section 107
redesignations
3. New source review
(including PSD)
4. Nonattainment area
SIP analyses
5. Lead SIP's
6. Other SIP modeling
*Indicate by code(s) which of the following technical areas were either
inadequate or deviated from EPA Guidance.
CODES
A. Use of inappropriate guideline model
B. Use of nonguideline model without a performance evaluation to
demonstrate acceptability of the model
C. Urban/rural dispersion coefficients
D. Emission inventory and operating design parameters
E. Meteorological data base
F. Receptor network design
G. Complex terrain considerations
H. Downwash consideration
I. Comparison with acceptable air quality levels
J. Technical documentation of modeling analysis
K. Other
EPA Regional Office Response (confirmation or comment):
2-29
-------
D. SIP Evaluation and Implementation
D.I. Timeliness of Regulatory Development
The purpose of this portion of the audit is to determine whether
strategies related to SIP's or section lll(d) plans are being de-
veloped on time, and if not, to determine if corrective actions are
needed. Note: this question does not apply to State-initiated SIP's
that are not required under the Clean Air Act.
State/Local Agency Response:
a. On Table D-l, list the dates when your agency submitted or plans to
submit revisions related to SIP's or section lll(d) plans. Complete
both parts (Part D plans or plan elements and non-Part D plans or
plan elements. The EPA Regional Office must provide a list of
submittals expected in each part of Table D-l.
Where plans or elements of plans have not been submitted or have not
been completed, list a projected submittal or completion date. If
your agency has no activities to track and none are listed in Table
D-l, skip this question and go to D.2.
For any delays in submission, indicate in Table D-l the following
codes for the primary causes of the delays:
Code Cause of Delay
A Unclear or unassigned responsibilities
B No formal tracking system
C Schedule was overly optimistic
D Change in guidance or scope of work from original plan
E Executive or legislative oversight
F Insufficient resources
G Schedule contingent on other action that did not occur
H Other (specify)
2-30
-------
TABLE D-l. TIMELINESS OF REGULATORY DEVELOPMENT
Required Activity (geographic area,
pollutant, description of plan,
plan element, or submittal).
List is to be provided by the
Regional Office.
Original Due
Date
Activity Completed
by Agency
Submitted
to EPA;
Date of
Submission
Not submitted
to EPA;
Projected
Submission
Date
Activity Not Completed
by Agency
On
Schedul e;
Projected
Compl etion
Date
Not on
Schedule;
Projected
Compl etion
Date
Reason
for
Delay;
Use Code
from
previous
page
Part D Plans or Plan Elements
1.
2.
ro
co
5.
Non-Part D Plans or Plan Elements
1.
2.
3.
EPA Regional Office Response (confirmation or comment; also, explain why any completed plan has not
been submitted.
-------
D.2. Timeliness of Studies
The purpose of this question is to determine whether additional studies
(such as nontraditional TSP or CO hot-spot modeling studies) approved as
part of the SIP are being carried out on time and the reason for any
delays.
State/Local Agency Response:
a. Has your agency carried out additional studies approved as part of SIP
control strategies? Please complete Table D-2 by listing the dates your
agency submitted or plans to submit the studies. The EPA Regional Office
must provide a list of submittals expected in Table D-2. Where a study
has been delayed, indicate in Table D-2 the primary cause of the
delay, using the applicable codes listed below the table.
Regional Office Response (confirmation or comment):
2-32
-------
TABLE 0-2. TIMELINESS OF STUDIES
Studies Due or Overdue; (geo-
graphic area, pollutant, descrip-
tion of study) .
List to be provided by the
Regional Office
Original Due
Date
Study Completed by
Agency
Submitted
to EPA;
Date of
Submission
Not submitted
to EPA;
Projected
Submission
Date
Study Not Completed
by Agency
On
Schedule;
Projected
Completion
Date
Not on
Schedule;
Projected
Compl etion
Date
Reason
for
Delay;
Use Codes
listed >
below.
1..
2.
ro
i
co
co
5.
Code Cause of Delay
A Unclear or unassigned responsibilities
B No formal tracking system
C Schedule was overly optimistic
D Change in guidance or scope of work from original plan
E Executive or legislative oversight
F Insufficient resources
6 Schedule contingent on other action that did not occur
H Other (specify)
-------
ro
CO
0.3. Transportation Control Measures
The purpose of this set of questions is to determine the degree to which transportation control
measures have been implemented. Complete one page for each affected metropolitan area.
State/Local Agency Response:
a. Metropolitan area:
b.
Indicate below:
-- the three transportation control measure categories
with the greatest emission reduction credits assumed in the SIP;
-- the agency responsible for implementing the measure; and
— the extent to which the measure has been fully implemented and
the basis for determining the extent of implementation.
List of TCM Categories
(to be agreed to in advance
by the Regional Office)!
Emission Reductions
in SIP (tons/year)
VOC
CO
Implementing Agency
Percent of
Implemention
Basis
1.
2.
3.
4.
Regional Office Response (confirmation or comment; the Regional Office should attempt to
review available documentation on the TCM's listed and verify the agency's basis for
the determination of extent of implementation):
agreement with the Regional Office, the audited agency may list more than three TCM's,
particularly where rankings of emission reductions for CO and VOC are different.
-------
D.4. Source Activity
In preparing the SIP, the agency projected growth of source activity
into the future, generally to the attainment date. The purposes of this
set of questions are to determine (a) if verification checks on these
projections are made; (b) how close the projections came to reality;
and, (c) if significant differences are found, whether any action is
contemplated to account for those differences.
Complete one sheet for each metropolitan area (a) that received an
attainment date extension to 1987 for the CO or 63 standard or
(b) where EPA called for a SIP revision for CO or 03.
State/Local Agency Response:
Metropolitan Area:
a. What are the growth projections (average percent per year) assumed in the
latest SIP revision for the following indicators of activity compared
to actual growth that occurred (from latest data available)?
SIP Projection Actual Growth Difference
(% per year) (% per year) (% per year)
1. population
2. employment
3. vehicle miles traveled
4. major stationary source
emissions (if a composite
growth rate assumed in SIP)
b. Indicate how recently these projections have been reevaluated by the
agency using more current information, whether significant differences
were found, and if so, what action is contemplated.
Category Time of Significant Action
Reevaluation Differences Contemplated*
(# of years Found
ago) (yes or no)
1. population
2. employment
3. vehicle miles traveled
4. major stationary source
emissions (if a composite
growth rate assumed in SIP)
* e.g., undertake study, revise SIP, undetermined, etc.
2-35
-------
D.5 Generic Bubble Rules
EPA must know which emission limits apply to each source in a SIP for
two reasons: (1) to enable proper enforcement under section 113 of the CAA
if necessary, and (2) for compilation of the SIP's as required under section
110(h) of the CAA. EPA therefore must be informed of each bubble approved
under a generic bubble rule to avoid confusion and/or a conflict that might
ensue if EPA were to enforce the original emission limits contained in the
EPA approved SIP.
a. Does the agency have a generic emission trading (bubble) rule?
Yes Identify the rule
No The remainder of this question may be ignored.
b. Was the generic rule sent to EPA for approval? Yes No
c. Date rule was sent to EPA .
d. Has the rule been approved by EPA?
Yes No If yes, ŁR_ approval date
How many individual bubbles have been approved under the generic bubble
rule?
1. Prior to the last calendar year .
2. During the last calendar year .
f. The agency is to supply a list of the bubbles approved during the
last two years to the EPA Regional Office in the following format, at
least two weeks before the onsite visit is scheduled.
List of Approved Bubbles
Company Geographic Area Pollutants
The audit team is expected to conduct a file review of at least 3
bubbles from the list and complete the attached file review check list for
each one audited.
2-36
-------
EMISSION TRADING FILE REVIEW CHECKLIST
(Fill out a separate form for each bubble reviewed)
1. a. Name of Company_
b. City/State
c. Date of application
d. Date of agency's approval
e. Date emission reduction credits were created
f. Pollutant
2. Was a permit or other enforceable document issued? Yes_
3.
No
Points of Emission
Totals
* Actual Emissions
Before
Trade
After
Trade
*Allowable Emissions
Before
Trade
After
Trade
* Specify units, e.g., T/yr
(use additional sheets if needed)
4. a. Did the bubble result in banked emissions? Yes No
b. If yes, what was the amount of banked emissions
T/yr.
5. a. Is there documentation to show the banked emissions are consistent
with the SIP demonstration? Yes No
b. If yes, how
6. Will the applicable net baseline emissions increase? Yes No
If yes, explain
7. Is there documentation to show that the generic rule procedures, approved
by EPA, were followed? Yes No If no, explain
2-37
-------
8. Is there documentation to show that the new emission limits allowed
under the bubble will not interfere with attainment and maintenance
of the NAAQS? Yes No If no, explain
9. Has the PSD baseline date been triggered? Yes No
If yes, when?
10. Is there documentation to show that the new emission limits do not
violate the PSD increments? Yes No If no, explain
11. Did the file contain information that the agency followed EPA
modeling guidance in approving the bubble? Yes No
12. Was the bubble subject to the agency's notice and comment
procedures before approval? Yes No . If no, explain
13. Was an information copy of the bubble application sent to EPA?
Yes No
14. Was a copy of the permit sent to EPA? Yes No
2-38
-------
Chapter 3
New Source Review Audit Guidelines
FY 1986-1987
Section Title Page
1.0 Introduction 3-1
2.0 FY 1986-1987 NSR Audit Procedures 3-1
3.0 Completing the Audit Report 3-4
4.0 Description of the Permit File Questionnaire 3-9
4.1 Source Information 3-9
4.2 Public Participation and Notification 3-9
4.3 Applicability Determinations 3-10
4.3.1 Definition of Source 3-10
4.3.2 Fugitive Emissions 3-11
4.3.3 Potential to Emit 3-11
4.3.4 Emission Netting 3-12
4.3.5 Emission Limits 3-13
4.4 Control Technology 3-14
4.4.1 NSR/PSD Sources 3-14
4.4.2 Non-NSR/PSD Sources 3-15
4.5 Air Quality Monitoring Data (PSD) 3-16
4.6 PSD Increment Analysis 3-16
4.7 NAAQS Protection 3-18
4.8 Emission Offset Requirements 3-18
F1g. 1. Major NSR Audit Topics and Associated Questions 3-6
Table 1. Reference Tables for Use With FY 1986-1987 NSR Audit
Questionnaires 3-20
Form 1. NSR Permit Summary Questionnaire 3-22
Form 2. NSR Audit Summary Questionnaire 3-24
Form 3. Permit File Questionnaire for Major Sources Subject
to PSD or Part D 3-27
Form 4. Permit File Questionnaire for Major Sources Not Subject
to PSD or Part 0 3-46
3-i
-------
3. NEW SOURCE REVIEU
1.0 INTRODUCTION
The procedures for carrying out the FY 1986-1987 NSR audit will remain
largely the same as they were for FY 1985. That is, the same four questionnaires
will be used, and the onsite audit will continue to focus on the examination
of current permit files. Perhaps the most significant change is the overall
switch to a two-year audit cycle. As a result, individual NSR programs will
be audited every two years. Some changes have been made with respect to
the audit forms, primarily for the purpose of clarification, but also in
some cases to modify or expand the type of information that will need to be
collected. Specific changes are included in a later section which describes
each of the questionnaires. The four questionnaires are:
1. NSR Permit Summary Questionnaire (Form 1)
2. NSR Audit Summary Questionnaire (Form 2)
3. Permit File Questionnaire for Major Sources Subject to PSD or
Part D (Offsets) (Form 3)
4. Permit File Questionnaire for Sources not Subject to PSD or Part D
(Offsets) (Form 4)
Some of the audit subjects covered in this section continue to involve,
in whole or in part, issues that could be affected by proposed EPA rulemaking
or ongoing litigation [e.g., CMA agreement rulemaking proposed on August 25,
1983 (48 FR 38742)]. These particular items are potentially impacted by
regulatory amendment. Should changes to the affected requirements be
promulgated, EPA will issue revised guidance as to how the audit should
handle them. Until such time that the existing Federal requirements and
the State rules developed pursuant to these 40 CFR Part 51 provisions can
be changed, this guideline will assume that all rules will continue to be
implemented under the EPA requirements presently in effect.
2.0 FY 1986-1987 NSR AUDIT PROCEDURES
Approximately 30 days before the scheduled onsite audit, the Regional
Office should send a copy of Form 1 to the appropriate agency. Audited
agencies should be asked to complete the questionnaire before the onsite
audit so that it can be returned to the audit team before or during their
visit. A set of instructions for completing Form 1 should accompany the
questionnaire when it is sent to an agency. The instructions provide important
information which will help to ensure that the agency responses will be made
in a reasonably consistent format.
Tne file review is a very important aspect of the NSR audit process.
For the new source review audit, permit files should be selected on the
basis of permit action type, source type and size, source location, public
concern, and other factors geared to ensuring review of a variety of permitting
actions and decisions by the agency. Criteria to consider include:
3-1
-------
0 Review both large (major) and small (minor) sources;
8 Review both new plants and plant modifications;
0 Review a PSD source for which preconstruction monitoring data
requirements apply;
0 Review a PSD source near a Class I area;
0 Review sources that avoided PSD or Part D review because of
restrictions on their operation or capacity;
e Review some sources in nonattainment and sanctioned areas,
if applicable;
8 Review some of the most common source types in that State
(for example, boilers and asphalt plants), but also review
a variety of other source types; and
8 Review a controversial permit, a permit of high public
interest, or one that would be of particular interest for
reasons other than those described above.
By combining several of these factors in one permitting action, it
may be possible to satisfy the criteria above with only a relatively few
permits. Generally, however, in order to obtain a reasonable sampling of
permits, the auditor should randomly select at least five PSD/Part D and
ten other permits issued since the last audit. If this random selection
does not seem to represent the variety of criteria indicated above, the
auditor should note this and specifically select additional permits for
review which do reflect the missing criteria.
For the 1986-87 NSR audit, the auditor is NOT required to fill out the
file audit questionnaires for every permit file selected and examined.
Instead, after selecting the permit files that will be examined, in accordance
with the criteria described above, audit questionnaires should be filled
out for a minimum of ONE PSD, ONE Part D (offsets) and THREE other (non-PSD/
Part D) permits. The following procedures should be used:
1. For completion of the PSD/Part D permit questionnaire (Form 3)--
Select the most recent PSD permit and the most recent Part D (offset)
permit and fill out the major source questionnaire, to the extent applicable,
for each one. In case the agency did not issue at least one of each of
these types of permits, select and review at least two of the other type,
preferably the most recent new plant and the most recent modified plant.
2. For completion of the non-PSD/Part D questionnaire (Form 4)--
Beginning with the most recently issued permits, select at least three
permit files that represent a variety of permit review situations as described
in the criteria above. A file should be rejected only if it too closely
resembles a file already selected for evaluation using the questionnaire,
except that the auditor should try to include at least one permit where
3-2
-------
major review was avoided through restrictions to the source's operation or
capacity.
3. All remaining major and minor source permits should be examined as time
allows using the appropriate questionnaire as a guide to ensure that the
applicable audit topics are adequately addressed. The auditor may complete
the questionnaire for any or all of the remaining permit files, but he or
she is not required to do so. It is understood that permit file review
time may vary greatly. Auditors are encouraged to review as many of the
selected files as possible, but, should time run short, they are advised to
conduct only a limited review of these remaining files, concentrating on
problems identified by the completed questionnaires to determine whether
the problem is common to several permits or only to an isolated case. Of
course, if time is available, auditors should conduct a more detailed
review of the remaining permits (as well as others not originally selected)
to see if any additional observations can be made.
In conducting the onsite audit, auditors are advised to strike a
reasonable balance between examining the selected permit files and maintaining
meaningful dialogue with the appropriate agency personnel"! Feedback from
the FY 1985 audit indicated that dialogue with agency personnel tended to be
minimal because of the high priority given to the examination of permit files
(including completion of the file audit questionnaires) by the FY 85 NAAS
guidance. The lack of dialogue was generally cited as a problem by the
auditors and the audited agencies alike. An example of how such dialogue
could be achieved is to have an agency representative—preferably the
permitting engineer most familiar with the selected permit file—present
during the examination of one or more permit files.*
The presence of the agency representative would enable him or her to
observe how the file audit is actually conducted. More importantly, such
individual would be able to help expedite the file search by indicating at
the appropriate times during the file audit where (and whether) specific
information is to be found. At the same time, he or she is likely to
become more appreciative of the need for carefully organized and well
documented files as specific information is being sought from the file to
complete the appropriate questionnaire. It is also important that an
opportunity be afforded for the auditor to explain applicable new source
review requirements and procedures where a lack of understanding may be
apparent. In all, both the auditor(s) and the agency representatiye(s)
stand to benefit from dialogue occurring in conjunction with the file
audit.
Auditors will find it useful to take with them a calculator and a
few basic documents:
(1) Copies of 40 CFR 51.18 and 51.24,
*An alternative approach might be to meet with some of the permitting engineers
after having completed several file audits to explain how the file audit
was conducted and to explain how certain conclusions were reached in responding
to the file audit questionnaires. During this time, the auditor may also
want to ask certain questions of the enginners with regard to any unanswered
or unclear issues.
3-3
-------
(2) AP-42, for checking whether all emission units were included, and
(3) A copy of the reference sheet included in this guidance. (Table 1.)
3.0 COMPLETING THE AUDIT REPORT
At the close of each audit, the auditor(s) should have (a) approximately
5 completed NSR questionnaires—2 for major sources and 3 for minor (or not
subject to PSD or offsets) sources; (b) notes (or additional questionnaires)
on approximately 10 additional permit files; and (c) a subjective impression
of the audited agency. This information must now be organized and reported
so that it can be incorporated into the State or local agency audit report.**
The NSR audit report should be written in a narrative form, organized
as closely as possible according to the NSR audit topics and specific areas
of concern (questions) under each topic. Figure 1 identifies the seven NSR
audit topics and associated audit questions. These topics and questions
were selected for auditing by the new source review audit committee comprised
of State, local, and EPA representatives. The audit topics and questions,
originally defined for the FY 1984 audit, serve as the basis for development
of the permit file questionnaires. For FY 1986-87, the questions are to be
answered primarily on the basis of information found in the audited files,
but discussions with agency personnel will also be useful.
The information that will be available to each auditor consists of
both background information (such as the number of sources proposing to
locate within 100 km of a Class I area) and evaluative information (such
as whether an agency is incorrectly using actual emissions, rather than
potential to emit, to determine applicability to PSD or offsets). In
preparing the NSR audit narrative, the auditor should try to use both types
of information to formulate the findings and recommendations.
For each problem identified from the permit files, the following
considerations should be taken into account and discussed in the narrative
where possible:
a. The number of cases where the problem was identified versus the
total number of permit files examined for that particular problem.
b. Whether the problem is a new one or is a carryover of a situation
that had been identified by previous audits.
c. The likely reasons for the occurrence of the problem, e.g., inadequate
procedures, failure to adhere to existing procedures, insufficient training
or resources, lack of EPA guidance, etc.
The auditor's subjective judgment can affect conclusions in two ways:
by deciding how serious the problem itself is, and by deciding whether
there are special circumstances which affect the seriousness of the problem.
It is important to make certain that perceived problems are discussed with
the audited agency in order to more clearly understand the true nature of
the problem or to determine whether a problem actually exists. Where a
problem is suspected, the auditor's assessment and the basis for that
assessment should be clearly documented in the narrative report.
3-4
-------
If there are fewer than five permits in a data base, it may be difficult
to draw conclusions on how widespread a problem is. However, subjective
impressions often offer valuable insight and are encouraged in these situations.
Recommendations should reflect the potential seriousness of the problem.
The factors outlined above should also be considered in developing recommendations
for resolving each identified problem. Potential solutions to problems
found during the audit should be discussed with the audited agency during
the exit interview. Tentative solutions to some problems may be negotiated
at the time and described in the audit report.
The NSR audit reports (narrative plus appropriate number of copies of
Forms 1 through 4) should be incorporated into the State or local agency
audit report and forwarded to EPA (CPDD) in accordance with existing guidance.
It is important that all of the completed forms that are submitted as part
of the audit report be neat and legible. If originals are not sent to
Headquarters, then the proper care should be taken to ensure that all copied
material can be easily read.
3-5
-------
Figure 1. MAJOR NSR AUDIT TOPICS AND ASSOCIATED AUDIT QUESTIONS
I. PUBLIC PARTICIPATION REQUIREMENTS
1. For which new or modified sources was the public afforded an
opportunity to comment on proposed permits?
2. Do the public notices routinely provide adequate information?
3. Were other State and local air pollution control agencies and
other officials whose jurisdictions might be affected by the proposed new
or modified source notified of the proposed action?
II. APPLICABILITY DETERMINATIONS
1. Does the agency apply the proper source definition(s) and exemption
provisions?
2. Does the agency typically use the best available emission projections
and federally enforceable restrictionsC*] in defining a new source's (or
unit's) "potential to emit"?
3. Does the agency routinely use an existing source's "potential to
emit" to determine major source status for proposed modifications?
4. Does the agency use as its netting baseline actual emissions
expressed in TPY?[*]
5. Verify that the agency does not allow for "double counting" of
emission decreases used for netting purposes.
6. Does the agency adequately address fugitive emissions[*] in
calculating the "potential to emit" and the "net emission increase"?
7. Does the agency properly apply the §107 area designations when
determining what type of preconstruction review will be required of major
construction?
8. Verify that the agency does not approve major construction
projects in designated nonattainment areas under an EPA-imposed construction
moratorium.
III. CONTROL TECHNOLOGY
1. Does the review agency check the applicants' selection of the
appropriate control technology?
2. Does the BACT analysis consider each regulated pollutant emitted
in significant amounts?
3. Does the review agency require the consideration of more than
one control alternative? To what extent are economic, energy, and non-air
environmental impacts considered in the BACT analysis?
3-6
-------
4. What tendency is there for the agency's BACT/LAER determinations
to conform exactly to minimum EPA requirements?
5. Does the agency adequately review non-NSR/PSD sources for applicability
to the NSPS and NESHAP requirements?
IV. AIR QUALITY MONITORING DATA--PSD
1. Does the agency follow the correct procedures to exempt applicants
from the preconstruction monitoring requirements?
2. Does the agency adequately ensure that existing data meets Federal
criteria for representative air quality data when applicants are not required
to conduct new monitoring?
3. Do the source monitoring data adhere to PSD quality assurance
requirements?
V. AMBIENT AIR QUALITY IMPACT
a. PSD Increment Consumption
1. Does the agency adequately consider the baseline concentration
and emission changes which affect increment consumption?
2. Are long- and short-term PSD increments being given adequate
consideration as part of the ambient impact analysis?
3. Does the agency make an adequate assessment of new sources and
modifications on the Class I area increments?
b. NAAQS Protection
1. What emission baseline does the agency require to be used to
evaluate the impact on the NAAQS of new and modified sources?
2. Does the agency routinely evaluate the ambient impact of minor
source construction?
3. Does the agency's ambient impact analysis provide adequate protection
against the development of "hot spots"?
c. Dispersion Models
1. Does the agency use adequate models and model options to carry out
the ambient impact analyses?
2. Does the agency perform an independent, internal review of the
modeling analyses contained in the permit application?
VI. EMISSION OFFSET REQUIREMENTS
1. Does the agency require that all offsets be Federally enforceable?
3-7
-------
2. Does the agency routinely ensure that the emission offsets are
not otherwise needed to show RFP or attainment?
3. Does the agency require that the emission baseline for offsets
be expressed in the same manner as for RFP?
4. Does the agency's offset requirement cover other emission
increases since the last offset review?
5. Does the agency require that offsets occur on or before the time
of new source operation?
6. Does the agency allow offsets resulting form early source shutdowns
or production curtailments?[*]
VII. PERMIT SPECIFICITY AND CLARITY
1. Does the agency identify all emission units and their allowable
emissions in the final permit(s)?
2. Are the allowable emission rates stated or referenced in the
permit conditions?
3. Are the compliance test methods stated or referenced in the
permit terms and conditions?
4. If a source's calculated potential to emit is based on less than
full design capacity and continuous, year-round operation, are all limiting
restrictions clearly identified in the permit?
*This audit question could be affected by proposed EPA rulemaking or by
ongoing litigation [e.g., CMA agreement rulemaking proposed on August 25,
1983 (48 FR 38742)]. Should changes to the affected requirements be
promulgated, EPA will issue revised guidance as to how the audit should
handle them. Until such time that the existing Federal requirements and
the State rules developed pursuant to these 40 CFR Part 51 provisions can
be changed, this guideline will assume that all rules will continue to be
impemented under the requirements presently in effect.
3-8
-------
4.0 DESCRIPTION OF THE PERMIT FILE QUESTIONNAIRES
The auditor will gather data primarily from selected permit files.
Depending on the permit file selected, the auditor will use either Form 3
or Form 4 in accordance with the procedures described in the previous
section. The choice of questionnaires should be based on the type of
preconstruction review that the reviewing agency actually carried out in
each case. Form 3 is designed to be used to evaluate permit files for
which the reviewing agency considered the proposed source subject to PSD
or Part D (nonattainment area/offset) requirements.
Form 4 was designed to evaluate permit files where the reviewing
agency determined that the proposed source was not subject to PSD or Part D
(offset) requirements. This would include cases where a major source
underwent a modification involving insignificant emission increases, as
well as sources that were allowed to avoid PSD or Part D review by restricting
their potential to emit. This questionnaire includes questions that will
help to determine whether the agency followed the correct procedures in
subjecting a source to a non-PSD/Part D source review rather than a PSD/Part D
source review.
4.1 Source Information--(Section I)
The basic data needed to identify the permit file reviewed are requested
in Section I of both questionnaires (Forms 3 and 4). The questions pertain
to the overall source—not the particular configuration of emission units
which may be the subject of the current permit review. Thus, "Source
Category" refers to one of the 28 listed PSD sources or any other category
which best describes the overall source.
"Location" refers to a geographical identifier that will help the
auditor to identify the specific source under review. The identifier is
primarily for the auditor's benefit and may be expressed as a complete
address, or simply in terms of the city or county of location.
"Region" refers to the two-digit Arabic number, such as 01 or 10;
and "State" is the appropriate two-letter code, such as AL or AZ.
"Type of Review" refers to the status of the proposed source action
relative to the overall source. Thus, the addition of a new boiler to an
existing source would be a modification rather than a new source. (Note
that the status of individual emission units should be designated in
Section III.)
4.2 Public Participation and Notification--(Section II)
Public participation requirements for review of new and modified
sources are set forth under 40 CFR 51.18(h) and 51.24(q). These requirements
call for the issuance of a public notice which informs the public of a
pending permit action and of the opportunity for public comment or hearing
prior to final agency action on a source application.
3-9
-------
Previous audit results indicate that some agencies require public
notification for all permits issued, but many agencies do not. This year's
audit seeks further information as to what specific sources the public was
notified of, and how adequate the notification was.
Both questionnaires ask for the same information that was requested
last year. Because of concerns pertaining to the usefulness of public
notices versus the costs of providing such notices, the FY 86-87 audit
continues to ask what it costs to issue a public notice. The answer should
indicate the amount charged by the newspaper or other media to publish the
notice. If this information is not available in the file (e.g., a copy of
the receipt), the auditor may wish to determine an approximate cost from
the audited agency, but it is not recommended that too much time be spent
trying to obtain this cost.
The public notice should inform the public of the availability for
their inspection of the application submitted by the source, the estimated
impact of the source on ambient air quality, and the agency's proposed
action to approve or disapprove the permit. Instructions for submitting
comments, as well as the opportunity for a public hearing, should also be
addressed. The auditor should verify that notices issued by the agency
adequately inform the public of the permit being considered and of their
opportunities to provide input to the final determination.
In addition to providing adequate notice to the public in general,
certain parties are to receive specific notification of proposed permit
actions where those parties would be directly affected by the proposed
source. The auditor should verify that the agency has, and uses, a mechanism
for notifying the appropriate government officials when the proposed
source may affect their jurisdiction. The auditor should particularly
note, in the case of PSD sources, whether and at what point in the process
the Federal Land Manager (FLM) is notified of any pending agency action
on a source locating within 100 km of a Federal Class I area. In addition,
the auditor should identify, for information gathering purposes, any
other criteria used to trigger notification of the FLM.
4.3 Applicability Determinations--(Section III)
State and local governments are expected to regulate not only PSD and
Part D sources but also construction of other air pollution sources. The
agencies are, however, particularly expected to strive for the level of
consistency needed to satisfy the minimum Federal requirements for subjecting
new and modified PSD and Part D sources to preconstruction review.
4.3.1 Definition of Source
The auditor should verify, through the review of selected permit
files, that the appropriate levels and detail of review are being made.
The listing of emission units provides a basis for determining the answers
to several questions, so it should be as complete as possible.
Agencies must use, as a minimum, the appropriate Federal definitions
of "source" to make applicability determinations. The number of definitions
used by any particular agency will depend upon the specific Federal
3-10
-------
preconstruction review requirements being implemented by the audited agency
under an approved SIP or delegated authority. The auditor should be familiar
with the following situations:
For PSD, the agency should use a reasonable grouping of emission
units as one stationary source, classified according to its primary activity,
i.e., same two-digit SIC code. The industrial grouping will determine
the applicable emission threshold (100/250 TPY) governing major source
status, and therefore whether PSD applies.
For nonattainment areas, including areas where the construction ban
(40 CFR 52.24) is in effect, one of several definitions of source may apply.
The possibilities include the plantwide definition, as described for PSD
above, the dual definition which considers a "source" to be both the plant
and each of its individual pieces of process equipment, or another definition
based on previous EPA requirements preceding the Alabama Power court decision.
The auditor must know which definition is actually being used by the agency
in order to determine that it is being correctly applied.
For NSPS and NESHAPS, the applicable "source" is defined by various
subparts of 40 CFR Parts 60 and 61, respectively. The auditor should verify
that the NSPS/NESHAP applicability determinations are made independently of
the PSD or Part D (offset) determinations. This is particularly important
where the PSD or Part D requirements do not apply, e.g, "minor" sources,
major sources which have de minimis net emission increases for the pollutant
of concern, or sources where exemptions from the PSD or Part D requirements
are otherwise granted by the agency.
4.3.2 Fugitive Emissions
Fugitive emissions, to the extent they are quantifiable and emitted by
any of the listed source categories, should be included in the emission
calculations for determining whether a source is major and subject to PSD
or Part D review. For the auditors' convenience, the listed source categories
have been included in the Reference Table (see Table 1) which is to be used
with the FY 85 audit questionnaire. For other source categories, i.e.,
those not listed, the source must first be evaluated as to whether it is
major without using fugitive emissions. However, fugitive emissions should
be included in the ambient impact analysis and other review requirements
whether the source is major or minor. The auditor should verify that the
emission factors used to calculate fugitive emissions are documented and
reviewed by the agency independently from any use of such factors by the
applicant.
4.3.3 Potential to Emit
The status (PSD/Part D or non-PSD/Part D) of new or modified sources
must be determined on the basis of the source's potential to emit. "Potential
to emit" is a source's maximum capacity to emit a pollutant under its
physical and operational design. In order for any physical or operational
limitations to be considered as part of the source's design (to restrict
the maximum capacity of the source), the limitations must be made an enforceable
part of the permit. Moreover, the limitations must be Federally enforceable,
which requires that the permit condition(s) be identified in the construction
permit or an operating permit that has been specifically incorporated in
the approved SIP.
3-11
-------
The auditor must determine whether the audited agency correctly applies
the concept of "potential to emit" when making applicability determinations.
Both questionnaires ask questions concerning the use of acceptable, well-
documented emission factors as well as the use of special limitations to
define a new source's potential to emit. The auditor should determine
whether restrictions to a source's potential to emit are properly applied,
particularly when they are used to allow the source to avoid PSD or Part D
review.
For modified sources, it is important to note that major source status
in terms of potential emissions of the existing source must be taken into
account. This involves the existing source's maximum capacity, which may
take into account all control equipment and operating restrictions that are
Federally enforceable. Previous audits have shown that there may be a
tendency on the part of some air pollution control agencies to overlook the
potential to emit of the existing source, particularly when actual emissions
are significantly less than the applicable major source cutoff size. The
auditor, by completing Form 4, should be able to determine whether any
problems exist with this aspect of the audited agency's applicability
procedures.
4.3.4 Emission Netting
For modifications to existing sources, once the major or minor status
of the existing source has been affirmed, the applicability review of
proposed modifications should be based on the net change in actual emissions
on a tons-per-year basis. For example, emission changes occurring from
retiring equipment or other methods of emission reduction generally will
be credited on the basis of the difference in the emission unit's actual
emissions before and after the reduction. Actual emission estimates
generally should be based on either: (1) reasonable engineering assumptions
regarding actual emission levels and representative facility operation
over a two-year period, or (2) permitted allowable emissions determined on
a site-specific, case-by-case basis so as to be representative of actual
source emissions. Where an emission unit has not begun normal operations,
the potential to emit of the unit should be used.
Any net change in actual emissions that would result in a significant
emission increase at an existing major stationary source must generally be
reviewed as a major modification. However, for this to be true in a nonattain-
ment area, the existing source must also have the potential to emit in
major amounts the nonattainment pollutant(s) for which a significant net
increase would occur. For proposed new major sources subject to PSD, PSD
review applies to all criteria and noncriteria pollutants that would be
emitted in significant amounts.
For the auditors' convenience, the EPA-defined significant emission
rates for criteria and noncriteria pollutants regulated under the Clean Air
Act have been included in the Reference Table (Table 1) attached for use
with the file audit questionnaires. The auditor should check all applic-
ability determinations carefully with respect to significant emissions.
Some agencies do not appear to use the EPA significance values to trigger
review of major modifications. Instead, they may be using some uniform
cutoff point that tends to be more restrictive than the required significance
values for some pollutants but less restrictive for other pollutants.
3-12
-------
The worksheet provided in Section III.D. of both questionnaires should
be used to determine the net change in emissions. It should be noted
that EPA policy requires the emission changes resulting from the proposed
modification itself to be significant before considering other contemporaneous
emission increases and decreases that may have occurred before the proposed
modification. If the proposed modification does not result in a significant
emission increase, then a major modification is said not to occur regardless
of how previous contemporaneous emission changes would alter the net
emission change. State and local agencies may implement a more stringent
policy if they wish to do so. Where this is the case, the auditor should
note such policy and evaluate the permit in accordance with the more
stringent policy.
Adequate safeguards should be taken by the agency to prevent the use
of contemporaneous decreases in actual emissions if the decreases are not
creditable. The auditor must know how "contemporaneous" is defined by
each audited agency. Contemporaneous emission decreases should be surplus
and should not credited more than once. No decrease previously relied on
by a PSD source can be considered again in determining the net change of
a current or future modification. For nonattainment areas, any required
emission reduction that has occurred or is scheduled to occur pursuant to
the attainment date contained in and required by the SIP control strategy
cannot be counted for netting purposes.
Finally, in nonattainment situations, no reduction relied on previously
to meet the reasonable further progress requirement of Part D of the Clean
Air Act can be used for calculating emissions. If necessary, the auditor
should inquire about the agency's policy and procedure for preventing
double counting, but documentation 1n the file which specifically states
that the decrease was not relied on or counted elsewhere is preferable
and should be encouraged. Should documentation not be readily available,
the auditor should so indicate.
4.3.5 Emission Limits
Agencies may vary in the number of permits that they issue to a source
having more than one emission unit. No Federal requirements exist to
govern the number of permits which may apply to any source. What is important,
however, is that each emission unit is identified clearly, along with its
allowable emission rates, or design, equipment, work practice or operational
standards, as may be appropriate to address each pollutant emitted. "Appro-
priate" often means having more than one limit for each pollutant. For
example, there may be limits for the same pollutant to ensure compliance with
(a) an NSPS (e.g., ID/million btu, rolling 30-day average), (b) 3-hour, 8-hour,
or 24-hour NAAQS or PSD increments (e.g., Ib/hour), (c) a restriction on
capacity or operating hours (e.g., Ib/day), and (d) an applicability determin-
ation or an annual NAAQS (e.g., ton/year). As a minimum, the permit should
contain sufficient emission limits to ensure adequate control of all regulated
pollutants which the source has the potential to emit in significant amounts.
3-13
-------
It is particularly important, when an agency issues one permit to a
large complex, that each emission unit is identified separately, along
with its allowable emission rate, as opposed to a single composite emission
rate for each pollutant. The auditor should verify that, for each permit
issued, there is separate and clear identification of the affected emission
units and their corresponding allowable emissions.
In addition to identifying the allowable emissions, equipment or other
standard for each separate emission units, it is important that such
limitations be addressed adequately in conditions on the permit(s) for a
new or modified source. The auditor should examine the adequacy of the
conditions in terms of their clarity and enforceability. The auditor
should pay close attention to the use of clear and precise averaging periods
over which the various pollutant emissions are to be measured. [Note: In
many cases, averaging periods may be a part of the required test method and
may not be specifically stated on the permit. In such cases, auditors should
discuss this with the audited agency and verify that the agency regulations
do require proper averaging periods by reference. Also, some agencies may
incorporate by reference the test method as well as the averaging period.]
Finally, the emission rates must be consistent with acceptable measurement
procedures; otherwise, compliance will be difficult if not impossible to
ascertain and the conditions would be unenforceable.
Test methods used to determine compliance of the source with its
allowable emission rates should be clearly defined or referenced as condi-
tions to the final permit. These compliance tests should be specific to
the individual emission units to which they apply. The auditor should
verify the documentation of the compliance test methods and their adequacy
for covering each applicable emission unit for which allowable emission
rates are defined. Where test methods are not specified in the permit, the
auditor should determine whether the SIP specifications are otherwise
applicable and sufficient. (This is likely to involve a discussion with
Agency personnel.)
4.4 Control Technology--(Section IV.)
4.4.1 NSR/PSD Sources (Form 3).
The primary objective for the auditor is to determine whether good,
well-supported, BACT/LAER determinations are being made. Secondary objectives
are to measure the frequency of BACT/LAER determinations set equal to
existing new source performance standards, and to determine the amount of
legitimate attention being given by review agencies to the requirement for
the application of LAER on new and modified major sources constructing in
nonattainment areas.
Pollutants regulated under the Clean Air Act are subject to a BACT
analysis if they would be emitted in significant amounts by a source whose
construction is subject to PSD. A pollutant subject to regulation under the
Clean Air Act generally has had a standard of performance under §111 or 112
and/or NAAQS promulgated for it. The analysis for the subject source
should address both fugitive and nonfugitive emissions. The auditor should
verify that the BACT analysis considers all significant emission increases
3-14
-------
rather than being restricted to criteria pollutants or major emission
changes.
In selecting BACT, the applicant generally should be required to
consider more than one control strategy, unless it can be demonstrated that
the single proposed strategy clearly represents the highest degree of contin-
uous emission reduction available. In all cases, the control strategies
considered should be technically feasible and should address the economic,
energy and environmental impacts of the particular alternative. Quantifiable
impacts should be identified. The auditor should verify that adequate
alternative control strategies are included where appropriate.
In each case, the BACT analysis submitted by the applicant must be
reviewed independently by the permit agency. In particular, candidate
control equipment should be assessed to ensure that reasonable performance
claims, including consideration of continuing compliance, are being made.
Atypically high control efficiencies should be examined for their reason-
ableness, particularly where they would result in emission rates that
would enable the applicant to avoid a certain requirement or to meet ambient
constraints. Where the alternative representing the most stringent emission
reductions is not selected, the permit agency should review carefully the
alternatives to ascertain that the most appropriate one was selected. The
agency should routinely check to see whether any technically feasible
alternatives were not considered, and why. The auditor should verify that
the agency performs an adequate independent review of the BACT analysis
submitted by the applicant.
For each permit reviewed which was subject to BACT or LAER, the auditor
should note the regulatory baseline assumed by the review agency. In how
many instances do the agency's BACT/LAER determinations conform exactly to
existing SIP, NSPS, or NESHAP requirements? The auditor should verify that
adequate documentation is provided for those determinations which simply
meet the minimum requirements. For cases where LAER determinations conform
exactly to NSPS, the auditor should examine the reasons why LAER was not
determined to be a more stringent limitation.
4.4.2 Non-NSR/PSD Sources (Form 4).
For sources that are not subject to PSD or Part D control technology
requirements, i.e., BACT or LAER, it is still Important for the reviewing
agency to address a number of control technology considerations. Methods
of reducing emission should be checked for the reasonableness of the
performance claims associated with them. This is especially true when the
applicant intends to avoid major review by demonstrating that emission
levels will fall below the major source threshold levels.
Sources that do not qualify for major review under PSD or Part D may
still be subject to NSPS or NESHAP requirements for certain pollutants.
Auditors should determine whether each source was adequately reviewed for
such applicability regardless of its major source status.
3-15
-------
4.5 Air Quality Monitoring Data (PSD)--(Sectlon V. Form 3 only)
Every PSD source with the potential to emit significant amounts of a
particular criteria pollutant, where both the existing air quality and the
estimated impact of the source or modification are significant, must meet
the requirements for preconstruction air quality monitoring data, unless
exempted under provisions for temporary emissions or compliance with the
Offset Policy. In the latter case, which applies only to VOC emissions, if
the source satisfies all conditions of the emission e, postapproval monitoring
may be provided in lieu of providing preconstruction data [40 CFR 51.24(m)(l)(v)].
Only PSD sources are required to submit such data. For PSD sources not
required to submit ambient data, the applicable exemption should be clearly
stated in the preliminary determination.
The requirement for ambient air quality monitoring data may be met in
one of two ways. First, the permitting agency may require that the PSD
applicant establish a monitoring network designed to collect the appropriate
air quality data. Second, if existing air quality data is representative
of the air quality in the area where the source would have an impact, then
such representative data may be provided in the place of monitoring data
collected by the applicant.
The Ambient Monitoring Guidelines for PSD contain minimum quality
assurance requirements that must be met by the applicant when monitoring
must be performed. The detailed criteria for quality assurance generally
should not be audited by the new source review auditors. Instead, the
Regional ambient monitoring staff is usually better able to audit the quality
assurance procedures. It is important that the two groups discuss in
advance the division of responsibility of audited areas, to avoid overlap
or omissions. The new source review auditor should determine: (1) whether
a monitoring plan was submitted by the source and evaluated by the permitting
agency; (2) whether a quality assurance plan was submitted by the applicant;
and (3) whether the permitting agency evaluated the data for compliance
with 40 CFR 58 Appendix B.
Use of representative data is restricted by the criteria described in
EPA's "Ambient Monitoring Giffde~lines for Prevention of Significant Deteri-
oration (PSD),11 EPA-450/4-80-012, Revised February 1981. Generally, only
new sources in remote areas may use existing data gathered at sites greater
than 10 km away. For all sources in flat terrain, monitors within 10 km
are acceptable. For complex terrain, the guidelines are very difficult to
meet, and new data are almost always required. In addition to the monitor
location criteria, there are also restrictions concerning data currentness
and quality. The auditor should be familiar with the guidelines concerning
representative data and verify that the audited agency is following them.
4.6 PSD Increment Analysis--(Section VI. Form 3 only)
Before a permit is granted, the permit agency must determine that no
national ambient air quality standards will be violated. In the special
case of a PSD permit, the agency must further verify that no allowable
PSD increment will be exceeded by the source under review. In all cases,
the ambient impact analysis must be reviewed carefully by the permit agency
responsible for managing the ambient air quality. The auditor should
3-16
-------
determine the adequacy of the ambient air analysis performed as part of the
preconstruction review.
Allowable PSD Increments exist only for S02 and TSP at the present
time. There are a number of Important considerations that the permit
agency must routinely take Into account In order to ensure that the maximum
allowable increments are not exceeded. The permit agency must give the
proper attention to such things as the baseline concentration, baseline
area and baseline date(s); the appropriate emission changes for increment
consumption purposes; long and short-term increment averaging periods; and
special Class I area impacts.
The baseline concentration generally reflects actual emissions occurring
at the time of receipt of the first complete PSD application in the §107
attainment or unclassifiable area. This ambient concentration is adjusted
to include projected emissions of major sources commencing construction
before January 6, 1975, but not in operation as of the baseline date, and
to exclude the impacts of actual emission changes resulting from construction
at a major stationary source commencing after January 6, 1975.
Changes in emissions contributing to the baseline concentration from
any source subsequent to the baseline date and from any major source construc-
tion commenced after January 6, 1975, can either consume or expand the PSD
increment. Where actual emissions cannot be used, e.g., the source has not
yet begun to operate or sufficient operating data is not available, then
allowable emissions must be used. The auditor should verify that the
agency considers the appropriate emission changes relative to the baseline
concentration.
The date of receipt of the first complete application for a major new
source or major modification subject to PSD becomes the baseline date, and
the area in which the baseline is triggered is known as the baseline^ area.
The analysis can become somewhat complicated when the baseline area for the
proposed source includes more than one Section 107 attainment or unclassified
area, particularly if the baseline date has already been triggered in some,
but not all, of the Section 107 areas within that baseline area. Auditors
should be familiar with the PSD increment analysis process to help alleviate
some of the potential confusion that could occur during the review of the
PSD permit files. A good presentation of the increment analysis is contained
in the EPA PSD Workshop Manual (EPA-450/2-80-081, October 1980).
Both TSP and S02 have long and short-term averaging periods for which
PSD increments have been established. These maximum allowable increases are
not to be exceeded more than once per year for other than an annual averaging
period. The auditor should verify that each PSD application considers all
appropriate averaging periods with complete documentation in the permit file.
For sources proposing to locate near a Class I area, an increment
analysis may be required under conditions that would not trigger an analysis
in any other locations. Any emissions from a proposed source should be
considered significant when the source would locate within 10 km of the Class I
area and cause an ambient impact equal to or greater than 1 ug/m3 (24-hour
average). Generally, sources locating within 100 km of a Class I area should
be screened to determine their impact on the Class I area.
3-17
-------
4.7 NAAQS Protection—(Section V. Form 4; Section VII. Form 3)
States may differ as to the emission baseline used to protect the
NAAQS. In some cases, the allowable emissions (or some other "representative"
emission estimates) from all major sources are used for modeling air quality.
In other cases, the modeled allowable emissions from the proposed source or
modification are added to the background air quality which 1s based solely
on monitoring data. The auditor should Identify the emission baseline
required by the agency and gain an understanding of the specific approach
utilized to estimate the Impact of a new or modified source. This Information
will be used to assess current practices and for consideration of future
policy development.
In evaluating the NAAQS, the ambient Impact analysis should determine
the maximum long-term and short-term Impacts of the proposed new source
or modification. However, maximum ambient Impact may actually occur at
other locations when the Impacts of other sources and background data are
taken Into account. Hot spots may also occur where growth resulting from
minor sources or sources otherwise exempted from detailed permit review are
not subjected to a rigorous ambient analysis. The auditor should verify
that the agency performs a detailed analysis of a source's maximum ambient
Impact beyond those areas of maximum Impact of the source alone.
EPA has recommended the use of a number of models for specific types
of applications and has stated Its preference for certain new models for
analyzing the Impact of sources on ambient air quality. However, utilization
of any particular model should be consistent with the design and Intent
of the model Itself. Some models are very specific as to terrain and
applicability. The auditor should verify that Impact analyses are being
performed with the appropriate models, and that the permit agency conducts
Its own Independent review of the source's analysis (Including the replication
of modeling results when appropriate) to ensure conformance to accepted
procedures. EPA guidance 1s provided 1n "Guideline on A1r Quality Models,"
EPA-450/278-027, April 1978. This report 1s currently undergoing revision.
Additional guidance 1s also provided 1n "Regional Workshops on A1r Quality
Modeling: Summary Report," OAQPS, April 1981 and "Guideline for Use of
City-Specific EKMA 1n Preparing Ozone SIP's," EPA-450/4-80-027, March 1981.
EPA's "Guideline on A1r Quality Models" Includes, among other things,
guidance on the selection of air quality dispersion models.
4.8 Emission Offset Requ1rements--(Sect1on VIII. Form 3 only)
Part D of the Clean A1r Act Intends that certain stringent requirements
be met by major sources approved for construction 1n nonattalnment areas.
One such requirement calls for the proposed source or modification to get
emission reductions (offsets) from existing sources 1n the area such that
there will be reasonable further progress toward attainment of the applicable
NAAQS. The specific audit objectives are: (1) to assure that reviewing
agencies are requiring, where appropriate, adequate emission offsets as a
condition to authorizing major construction in designated nonattalnment
areas; and (2) to assure that emission offsets are being obtained in a
manner consistent with RFP.
All emission reductions used to offset proposed new emissions must
be made enforceable. This is true whether the offsets are obtained from
3-18
-------
another source owned by the applicant or from a source not under common
ownership. In either case the offsets should be fully agreed upon and
documented, preferably within the permit of the source from which the
offset Is obtained. In addition, Federal enforceablllty requires that
an external offset be made a part of the applicable SIP. This would require
a specific SIP revision if the offset is not made part of a permit issued
pursuant to the State's construction permit requirements approved pursuant
to 40 CFR 51.18 or 51.24. Conditions to State or local operating permits
are not always considered to be part of the applicable SIP(s). The auditor
should verify that all offsets are documented by means of well-defined
emission limits pertaining to the emission offset.
The proposed emissions offset cannot be otherwise needed to show RFP
toward attaining the NAAQS. To use the same emission offset for two
different purposes would result in "double counting" those emissions with
the net result being subsequent deterioration of air quality. The auditor
should seek assurance from the agency that compliance with annual RFP
increments is independent of the offsets being obtained from proposed new
or modified sources. In addition, the permit file should be checked to
determine whether any documentation is provided to address this issue. All
findings sould be recorded in Form 3.
In order for the system for getting offsets to be consistent with the
demonstration of reasonable further progress, both should be expressed in
the same emission terms, i.e., actual or allowable emissions. Section
173(1)(A) of the Clean Air Act sets the emission offset baseline as the
"allowable" emissions of the source, but also requires that the offsets must
be sufficient to represent RFP. Consequently, where the RFP demonstration
is based on an inventory of actual emissions, EPA requires that offsets to be
attained by a proposed new or modified source also be based on actual
emissions. Form 3 requires that the auditor determine whether there is
consistency in the emission baseline for offsets and the RFP demonstration.
In order to comply with the Act requirement that emission offsets
must be sufficient to represent RFP, any increases in area and minor source
growth not considered in the approved RFP demonstration must be covered by
offsets required of the proposed new or modified source. Failure to account
for these emission increases would result in air quality deterioration
just as 1n the case of "double counting." The auditor should verify that
area and minor source growth considerations are made in order to establish
the offset level, particularly when more than one year has passed since the
last offset.
Section 173(1)(A) of the Clean Air Act requires that offsets be obtained
and in effect "by the time the [new or modified] source is to commence
operation." No specific guidance is available to identify when a source
has officially "commenced" operation. Some agencies may allow a shakedown
period similar to the shakedown provision allowed for net emission increases
in 40 CFR 51.18(j)(1)(vii)(f). The auditor should focus primarily on
whether offsets were sought to be in effect in a timely manner, which may
include, for replacement facilities, a shakedown period not to exceed 180
days. The auditor should also determine whether the effective date for the
offsets is documented in the permit file(s).
3-19
-------
TABLE 1
REFERENCE TABLES
FOR USE WITH FY 1986-1987 NSR AUDIT QUESTIONNAIRES
I. Questionnaire abbreviations:
0 CBD = cannot be determined from Information available In permit file
* NA = not applicable
0 PSD = prevention of significant deterioration
* Part D = nonattainment area provisions applying to sources which emit
a nonattainment pollutant and locate within that nonattainment area.
II. Pollutant Criteria
Use this
Pollutant Abbreviation
C
R
I
T
E
R
I
A
R
E
G
U
L
A
T
E
D
Carbon monoxide CO
Nitrogen oxides NOX
Sulfur dioxide SCfe
Particulate matter TSP or PM
Ozone (as volatile VOC
organic compounds)
Lead PB
Asbestos AS
Beryllium BE
Mercury HG
Vinyl chloride VC
Fluorides FL
Su If uric acid mist SAM
Hydrogen sulfide H2S
Total reduced sulfur TRS
Reduced sulfur compounts RSC
Significant
Emission
Levels, TPY
100
40
40
25
40
0.6
0.007
0.0004
0.1
1.0
3
7
10
10
10
Significant Air Quality
Concentrations (for Monitoring
Determinations), ug/m3
575, 8-hr avg
14, annual avg
13, 24-hr avg
10, 24-hr avg
(100 TPY of VOC)
0.1, 3-month avg
No monitoring required
0.001, 24-hr avg
0.25, 24-hr avg
15, 24-hr avg
0.25, 24-hr avg
No monitoring required
0.2, 1-hr avg
No monitoring required
No monitoring required
NOTE: For each regulated pollutant, _any emission rate is significant
that causes an air impact of 1 ug/m3 (24-hr) or greater in any Class I area
located within 10 km of the source.
III. The following source categories are major if X100 TPY, including fugitive
emissions. (One exception exists; see note for last source category in list.)
Coal cleaning plants (with thermal dryers)
Kraft pulp mills
Portland cement plants
Primary zinc smelters
Iron and steel mills
Primary aluminum ore reduction plants
Pimary copper smelters
Municipal incinerators > 250 TPY
Hydrofluoric, sulfuric, or nitric acid plants
Petroleum refineries
3-20
-------
Lime Plants
Phosphate rock processing plants
Coke oven batteries
Sulfur recovery plants
Carbon black plants (furnace process)
Primary lead smelters
Fuel conversion plants
Sintering plants
Secondary metal production plants
Chemical process plants
Fossil-fuel boilers (or combination thereof)
totaling > 250 million BTU/hr heat input
Petroleum storage & transfer units with total
storage capacity > 300,000 bbls
Taconite ore processing plants
Glass fiber processing plants
Charcoal production
Fossil fuel-fired steam electric plants
> 250 million Btu/hr heat input
Any other NSPS or NESHAP source as of
August 7, 1980 [Note: for PSD, major
source status based on emissions >250 TPY.]
3-21
-------
Form 1
NSR PERMIT SUMMARY
QUESTIONNAIRE
I. GENERAL INFORMATION AUDIT PERIOD: / to /
fto.Yr. "Mo.Tf.
REGION:
STATE:
|~| State Q Local Agency
AGENCY NAME:
Please answer the questions below for the specified audit period based
on the number of construction permits that you (the above-named State or
local agency) issued to sources (major and minor) in your jurisdiction.
Use the accompanying list of instructions to formulate your responses.
II. PERMIT SUMMARY
1. PSD and Part D (Offset) Construction "Permits"
a. Prevention of significant deterioration (_> 100 or 250 TPY)
b. Part D major sources in nonattainment areas (_> 100 TPY)
c. Combination (i.e., PSD and Part D)
d. TOTAL (a + b + c)
2. Other Source Construction "Permits"
a. Non-PSD permits (_>100 TPY) in attainment/unclassified areas
b. Minor sources (<100 TPY)
i. Minor sources undergoing ambient impact analysis
ii. Sources avoiding major source review via restrictions
not otherwise required but imposed to lower source's
potential to emit.
c. TOTAL (a + b)
III. PRECONSTRUCTION MONITORING FOR PSD
No. of PSD sources subject to preconstruction monitoring requirements.
No. of PSD sources actually required to collect data via monitoring.
No. of PSD sources allowed to use existing representative data.
3-22
-------
INSTRUCTIONS FOR FORM 1
INSTRUCTIONS
I. GENERAL INFORMATION
" This section should be filled out by the EPA Regional Office
before forwarding the questionnaire to the audited agency. The audit
period represents the 12-month (approximate) period from the time of the
last audit.
II. PERMIT SUMMARY
" Major source permits: enter "N/A" if you do not have
program authority; "0" if you have authority, but no permits of a particular
type were issued.
0 "Permit" should be defined in terms of the entire source or project
for which a particular construction approval (for a new source or modification)
was requested. Consequently, one application should generally be regarded
as a "permit" regardless of the number of agency permits (for individual
emission units) actually issued. For cases where an application would
qualify for two permit groupings (e.g., major source review for both PSD
and offsets), the permit should be listed under 11.I.e.
u All "permit" numbers reported should pertain to new construction
(which may involve a completely new plant or a modificatin to an existing
one) or a new method of operation for which a permit analysis was required.
Permit extensions, minor revisions, etc., should not be included. If the
exact number of "permits" is not known, please provide your most reasonable
estimate and place an "(E)" after the value provided.
" If EPA performs the application review and issues a PSD permit
(i.e., the State or local agency does not have either a SIP approved PSD
program or delegated authority), do not include such "permit" in line l.a.
However, if a permit is required by the State or local agency in addition
to EPA's PSD review, then that permit should be included in line 11.2.a.
Also, use line II.2.a. to account for permits issued to major (>_ 100 tpy)
new or modified sources which are not subject to PSD because their emissions
are less than the 250 tpy cutoff for unlisted PSD sources.
" Sources may avoid major source review by agreeing to limitations
which would restrict their potential emissions to an amount below the 10U
or 250 tpy threshold. This is often accomplished by limiting the source's
hours of operation, fuel use, or operating rate via Federally enforceable
permit conditions. If this occurs, it should be noted in line Il.Z.b.ii.
III. PRECONSTRUCTION MONITORING FOR PSD
For sources subject to PSD monitoring, indicate number of sources for
which (a) ambient monitoring was required, or (b) the use of existing data
was allowed. For cases where a PSD source is required to monitor for one
or more pollutants, but is also allowed to use existing representative data
for another pollutant, the source should be counted once for each event.
Therefore, sources may be double counted under III.l.a. and b.
3-23
-------
FORM 2
NSR AUDIT SUMMARY QUESTIONNAIRE
I. GENERAL INFORMATION: AUDIT PERIOD: / / to /
flo.Yr. flo.YF.
REGION:
STATE: |~| State Agency |~| Local Agency
II. NUMBER OF PERMITS AUDITED
Indicate the number of permit files audited (including those for which a
questionnaire was not completed) for each of the following types
of permits:
a. PSD only b. Part D only c. PSD/Part D d. all other
III. TIME REQUIRED TO AUDIT PERMITS
Indicate the amount of time, in hours, spent auditing the total number of
permits specified above, as well as the range in time needed for auditing
individual permits for which a questionnaire was completed. (Times should
be stated to the nearest half hour.)
a. Hours for total audit of files.
b. Hours for maximum single file audit.
c. Hours for minimum single file audit.
IV. PERMIT SELECTION
a. The audited agency [ ] was [ ] was not told prior to the audit
which permits would be examined.
b. The audited agency [ ] did [ ] did not participate in the selection
of permits which were audited.
Comments -
3-24
-------
Form 2
V. CONDITION OF PERMIT FILES
For the two categories specified below, mark the response that best
describes the condition of the audited agency's permit files:
a. Organization - [ ] Information in each file well organized.
[ ] Information available but not well organized;
did/did not (circle one) significantly lengthen
the time required to audit files.
[ ] Information not contained in a central file,
but maintained in separate files; did/did not
(circle one) have opportunity to examine aTT~
pertinent information.
Comments -
b. Documentation - [ ] Al1 files reviewed contained necessary documentation.
[ ] Some ( %) files reviewed contained necessary
documentation
[ ] Files reviewed typically lacked necessary documentation.
Comments -
VI. SIGNIFICANT PROBLEMS
a. List the five (or fewer) most significant problems found as a result
of the NSR audit. Start with the most significant problem and continue
listing in descending order. [Each problem listed should be supported
by discussion contained in the audit narrative.]
i.
11.
iii.
iv.
v.
3-25
-------
Form 2
b. For each problem identified on previous page, select the reason(s)
which you believe may contribute to the particular problem:
(1) (ID (111) (iv) (v)
0 Inadequate agency procedures [] [] [] [] []
0 Failure of agency to follow its own [] [] [] [] []
procedures
* Inadequate agency rules/regulations [] [] [] [] []
0 Inadequate agency resources/organization [] [] [] [] []
0 Need for EPA policy or guidance [] [] [] [] []
0 Other: Specify [ ] [ ] [ ] [ ] [ ]
Comments -
VII. PROGRAM IMPROVEMENTS
Briefly describe below program improvements that have occurred since the
last NSR audit. These should also be discussed in full detail in the
narrative report. The improvements generally should relate to specific
audit findings identified during previous audits.
3-26
-------
FORM 3
PERMIT FILE QUESTIONNAIRE
FOR MAJOR SOURCES SUBJECT TO PSD OR PART D (OFFSETS)
LNOTE: Unless otherwise indicated place an "X" in the box beside each statement or
response which applies. Many of the questions will allow more than one response.]
SECTION I. SOURCE INFORMATION
A.I.Company/Source Name:
2.Size/Source Category):
3.Location:
c
1
. Date Complete
Appl i cation rec'd:
1 1 1 1 1 1
mo day yr
E
D. Date Permit to
Construct Issued:
1 1 1 1 1 1 1
mo day yr
. This permit was reviewed for (list
pollutants):
1. L J PSD for:
2. L J Offsets for:
1. Region J [
2. State
I I 1
3. Agency
a. L J State
b. L J Local:
4. Auditor _
5. Permit #
6. Type of Review:
a. L J New Major Source
b. L J Major Modification
F.I. [ J This source is located within 10 km of a Class I area.
2. L J This source is located within 1UO km of a Class I area.
3. [ J Source is in an attainment area and significantly impacts a designated
nonattainment area or any area where a NAAQS violation exists.
4. [ J None of the above.
SECTION II. PUBLIC PARTICIPATION REOTUIREMENTS
A. Public Notice:
1. was published in a newspaper (approx. cost:
2. provided opportunity for public hearing . . .
provided opportunity for written comment . .
described agency's preliminary determination
included estimated ambient impact
3.
4.
5.
6.
indicated addt'l info, available for inspection
1.
2.
3.
4.
The following other affected government agencies were
notified:
other agencies and officials within the State . . .
other States
Federal Land Manager
EPA .'
YES
rr
L J
L j
L J
L J
L J
L J
L J
NO
IT]
L J
L J
L J
[ J
L J
L J
L J
CBD
rr
L J
L j
L J
[ J
L J
YES NO CBD
L J
L J
L J L J L J
L ] [ ] [ J
3-27
-------
C. Documentation for parts A and B consists of:
1. copy of notice/correspondence in file
2. indication on processing checklist
3. no documentation provided
4. other, explain:
Public Notice
a.[
a.[
a.[
Notification of
Other Agencies
b.[ ]
Section III. APPLICABILITY DETERMINATIONS
A. Definition of Source.
l.a. [ ] The new source or modification for which this permit application
was made was considered by the reviewing agency to consist of the
following new and modified pollutant-emitting activities associated
with the same industrial grouping, located on contiguous or adjacent
sites, and under common control or ownership (if more than 5, list
only 5 largest):
New Modif.
Emission Unit/Size
Pollutant
C ] [ ]
2.
b. [ ] CBD
Were any new or modified pollutant-emitting activities (other than fugitive
emissions) omitted which should have been included:
[ ] NO.
Emission
Unit/Size
[ ] CBO.
Emission
Pollutant TPY
[ ] YES, as follows:
Reason given by agency for not
New Modif. considering as part of source
C ] [ ]
3. If Section III.A.2.c. above was checked, did this new source or modification
escape any PSD or Part D analyses for significant emissions as a result
of the omission of the listed activities? [ ] Yes [ ] No
3-28
-------
B. Fugitive emissions.
1. This source or modification:
a. [ ] Does not have any quantifiable fugitive emissions. (GO TO Section III.C.)
b. [ ] Has too little documentation in the file to determine whether
quantifiable fugitive emissions were included in the emission estimates.
c. Has quantifiable fugitive emissions which were:
i. [ ] Included in determining whether the source/modification was major
for the following pollutants: ,
ii. [ ] Not included in determining whether the source was major for the following
pollutants:
because the source is neither one of the 28 PSD source categories
nor regulated under Section 111 or 112 of the Act.
iii. [ ] Not included in determining whether the source was major for the following
pollutants: , although they should have been.
C. Potential to Emit (PJE). The determination of whether a source is major
should be based on PTE which, because most sources do not operate 8,760 hours
per year at 100% capacity, can differ greatly from actual emissions.
1. The emissions of this source or modification were determined:
a. Using emission rates based on emission factors which were:
i. [ ] Well established (e.g., AP-42) or well documented in the file.
ii. [ ] Not well established and lacking adequate documentation for the
following units and pollutants:
b. [ ] CBD
c. [ ] Using another method. Explain:
2. The emissions of this source were determined on the basis of:
a. [ ] CBD; GO TO C.3.
b. [ ] Maximum capacity to emit at full physical and operation design;
GO TO C.3.
c. [ ] Limited capacity based on control equipment, or physical, operational
or emission limitations, not otherwise required (e.g., NSPS), as
follows:
3-29
-------
If used, was limitation identified on:
Cgnstr. Permit Operating Permit
Limitation Pollutants Ye? TjcT Yes NŁ IW
i. [ ] Control equipment* _ [ ] [ ] [ ] [ ] [ ]
11. [ ] Emission limit* _ [ ] [ ] [ ] [ ] [ ]
111. [ ] Operating hours _ [ ] [ ] [ ] [ ] [ ]
1v. [ ] Operating rate _ [ ] [ ] [ ] [ ] [ ]
v. [ ] Fuel/material _ [ ] [ ] [ ] [ ] [ ]
restriction
*Not otherwise required, i.e., main purpose of limitation is to reduce potential
emissions.
If Section III.C.2.C. above is marked, please describe the limitations:
3. Do the calculated emissions correctly represent the source's potential
emissions?
a. [ ] YES
b. [ ] CBD
Explain:
c. NO, because:
i. [ ] calculations based on control equipment, or physical, operational
or emission limitations that are not Federally enforceable
ii. [ ] emission factors not acceptable
111. [ ] did not adequately address fugitive emissions
iv. [ ] did not address all pollutant-emitting activities (other than
fugitives)
Explain:
v. [ ] other. Explain:
3-30
-------
D. Emission Netting.
1. Check the appropriate box for this permit action:
a. [ ] New source; emission netting not applicable. 60 TO Section III.E.
b. [ ] Major modification
2. The determination of significant emissions can be a complex process when
emission netting occurs. The following work sheet should be used to
determine whether the proper procedure was followed:
Proposed Emission
Changes. TPY
Pollutant (+) (-) Net
a. TSP
b. S02
c. NOX
d. 03 (VOC)
e. CO
f.
g.
h.
Other creditable,
contemporaneous
Overall
emissions changes, TPY Net Change,
(+) (-) TPY
3. Did agency correctly identify whether emissions were significant?
[ ] Yes [ ] CBD [ ] No; Explain
4. Did analysis consider all pollutants for which a net increase in emissions
occurred?
C ] Yes
[ ] No, failed to address one or more pollutants. Explain:
[ ] CBD; Explain
3-31
-------
***COMPLETE THE FOLLOWING APPLICABLE STATEMENTS.***
5.a. Emission netting was based on actual emissions.
[ ] CBD; GO TO Question 7. [ ] No; GO TO Question 6. [ ] Yes
b. Indicate whether the calculation of actual emissions properly considered
the following criteria:
Yes No CBD
1. Representative of normal unit operation [ ] [ ] [ ]
11. Based on a two-year average [ ] [ ] [ ]
111. Expressed 1n TRY [ ] [ ] [ ]
6. Emission netting was based on another approach.
a. [ ] No.
b. [ ] Yes, and approach was acceptable. Explain:
c. [ ] Yes, but approach was Incorrect. Explain:
7. Emission decreases were considered.
a. [ ] No; GO TO E.
b. [ ] Yes, and the decreases:
Were Were not CBD N/A
[ ] [ ] [ ] a. Contemporaneous with the proposed modifica-
tion.
[ ] [ ] [ ] b. Previously relied on to determine a net
emission change.
[ ] [ ] [ ] [ ] c. Previously counted as part of the SIP attain-
ment strategy (Part D source only).
[] [] [][]d. Previously relied on to meet the "reasonable
further progress" requirement of Part D (Part
D source only).
[ ] [ ] [ ] e. Made Federally enforceable as permit condi-
tions.
3-32
-------
E. Emission limits.
1. Did the agency identify the appropriate allowable emission rates with respect
to each emission unit?
L J Yes L J No; explain:
2. The number of limitations actually appearing or referenced in the construction
permi t i s .
3. How many of the limitations: Number
Yes TJo
a. Include clear and concise averaging periods
compatible with appropriate requirements
(e.g., NSPS, short-term NAAQS)?
b. Are compatible with acceptable measurement
techniques?
c. Consist of design, equipment, work practice,
or operational standards?
d. Appear Federally enforceable?
e. Include stated or referenced compliance test
methods?
3-33
-------
SECTION IV.BACT/LAER DETERMINATIONS ([ J If not applicable, mark here, THEN
GO TO SECTION V.)
A. BACT Analysis.
Other (11s
TSP S02 VOC NOx CO
1. Pollutant emitted in significant amounts. [][][][][] [ ] [ ]
2. BACT/LAER analysis made (indicate w/"B" [][][][][] C ] [ ]
or "L")
3. BACT/LAER was specified in permit. [][][][][] [ ] C ]
YES NO CBD
4. a. Did the application address more than one
control option for BACT? [ ] [ ]
b. If NO, was the selected BACT clearly acceptable? [ ] [ ] [ ]
Comments:
c. If YES, did each option address the economic,
and environmental impacts associated with that option? [ ] [ ]
5. Does the file contain documentation to show that
the reviewing agency verified the applicant's [ ] [ ]
calculations and assumptions for BACT/LAER?
B. BACT/LAER Stringency. (Use the appropriate symbol(s) below to answer this part.)
1. Is the source (or modification) one for which NSPS or NESHAP has been
established?
[ ] No; GO TO Section V. [ ] Yes.
2. The Agency's BACT/LAER determination BACT LAER
compared to NSPS/NESHAP is:
1. TSP [ ] [ ]
"A" -- more stringent 2. S02 [ ] [ ]
"B" — equal 3. HC [ ] [ ]
"C" — less stringent 4. NOx [ ] C ]
"D" -- did not address, 5. CO [ ] [ ]
but should have 6. [ ] [ ]
"NA" — not applicable 7. [ ] [ ]
8. [ ] [ ]
3-34
-------
SECTION V.AIR QUALITY MONITORING DATA -- PSD ([ ] If not subject to PSD, mark
here. THEN GO TO Section VI)
A. Air Quality Monitoring Worksheet:
For each "yes" in (a), complete the
(a) Are following:
Pollutant
1.
2.
3.
4.
5.
6.
7.
TSP
SO 2
CO
NOX
VOC/03
potential (b) Are modeled
emissions concentrations (c) Is existing air
significant? significant? quality significant?
YES NO YES NO CBO YES NO C6D
[ ] [
C 1 [
C
C
[
[
[
C
c :
c
c
[ ] C ]
C ] [ ]
C ]
C ]
C ]
[ ] [ ]
c ] c :
c ] c :
c ] c :
c ] c :
c ] c ]
] [ ] [ ]
] C 1
] C 1
i
[
[ j
[
c :
c j
B. Applicability.
Was source required to address PSD air quality monitoring data requirements
(either source monitoring or use of existing data)?
a. [ ] Yes, required to address air quality monitoring data requirements
for at least one pollutant.
b. [ ] No, existing air quality for all pollutants was determined to be de
minimis for ^11_ pollutants (GO TO SECTION VI).
c. [ ] No, proposed ambient concentration increases for all pollutants were
demonstrated to be de minimis (GO TO SECTIOTTTI).
d. [ ] No, for the following reason(s):
(GO TO SECTION VI)
3-35
-------
C. Ambient Monitoring.
1. Was ambient monitoring required of applicant?
[ ] No; GO TO D, below [ ] Yes
2. Did the applicant submit a monitoring plan, Including quality
assurance (QA) procedures?
a.
b.
c.
] YES, for
] NO, for GO TO Question 4
] CBD, for GO TO Question 4
3. Is the monitoring plan 1n the permit file?
a. [ ] YES, for
b. [ ] NO, for
4. For how long did the monitors collect air quality data?
a. [ ] 12 months or more for:
b. [ ] 4 to 12 months for:
c. [ ] less than 4 months for:
If less than 12 months of data were submitted, summarize explanation:
D. Representative Data
1. Was the use of existing data allowed?
[ ] No, GO TO Section VI. [ ] Yes
2. Is the basis for allowing the use of existing data documented 1n the
permit file?
a. [ ] YES, for
>. [
b. [ ] NO, for
3.a. Did the agency's determination of "representative" adequately consider:
YES for: NO for: CBD for:
1. Location of existing monitors [ ] [ ] [ ]
11. Quality of the existing air
quality data [] [] []
111. Currentness of existing air
quality data [] [] []
b. If "NO" for any pollutant, please explain:
3-36
-------
SECTION VI. PSD INCREMENT ANALYSIS
A. Modeling Analysis
Note: It Is important that an auditor knowledgeable in modeling techniques and
required procedures participate in this portion of the audit.
CLASS I
TSP S02
1. Was a PSD increment analysis performed?
[ ] NO, 60 TO VI.D. [ ] YES, as follows [ ] [ ]
2. How was the analysis performed?
[ ] By the applicant with adequate review (including
replication of results, if appropriate) by the
agency for [
[ ] By the applicant, without adequate agency for . . . [
[ ] By the reviewing agency for [
[ ] Not applicable for [
3. Identify the dispersion model(s) used
to perform the increment analysis:
Model Used Pollutant/Area Classification
(Identify one model per line)
a.
b.
c.
[ ] S02 for [ ] Class I [ ] Class II
[ ] TSP for [ ] Class I [ ] Class II
[ ] S02 for [ ] Class I [ ] Class II
[ ] TSP for [ ] Class I [ ] Class II
[ ] S02 for [ ] Class I [ ] Class II
[ ] TSP for [ ] Class I [ ] Class II
CLASS II
TSP S02
C ] C ]
] C ]
[ ]
C ]
] C ]
C ] C ]
C ] C ]
C ] C ]
[ ] [ ]
Averaging Times
3hr 24hr Annual
[ ] [ ]
[ ]
[ ] [ ]
[ ]
[ ] C ]
[ ]
[ J
C ]
C ]
C ]
d. [ ] CBD
4. Did the agency select appropriate model(s)?
[ ] a. Yes, and documentation supports use of each
model as being appropriate.
[ ] b. Yes, model was appropriate, but inadequate docu-
mentation was available to explain its selection.
[ ] c. Cannot be determined. Documentation not provided
to justify model selection.
[ ] d. No, documentation failed to address appropriate
considerations. Explain:
FOR MODEL (see A.3.
above):
3.a 3.b 3.c
3-37
-------
5. Did the agency exercise the appropriate model options (urban/rural,
receptor network design, wind speed profiles, building wake effects,
final/gradual plume rise, etc.)?
FOR MODEL
3.a 3.b 3.c
[ ] a. Yes, and documentation supports use of options [ ] [ ] [ ]
as being appropriate.
[ ] b. Yes, options were appropriate, but inadequate [ ] [ ] [ ]
documentation was available to explain its
selection.
[ ] c. Cannot be determined. Documentation not provided [ ] [ ] [ ]
to justify options selection.
[ ] d. No. Explain: [ ] [ ] [ ]
6. Did analysis consider the appropriate meteorological data?
[ ] 1. Yes, five consecutive years of the most recent representative
sequential hourly National Weather Service data.
[ ] 2. Yes, one year of NWS data (if 5 not available) + use of highest modeled
results.
[ ] 3. Yes, five years of on site data subjected to quality assurance procedures.
[ ] 4. Yes, at least one year of hourly sequential on site data, including
worst-case conditions and subjected to quality assurance procedures.
[ ] 5. Yes, screening data were used to obtain conservative results.
[ ] 6. No. Explain:
[ ] 7. CBD
B. Baseline Area.
The baseline area for either TSP or $02 (or both) is defined as one or more
designated attainment/unclassified (§107) areas and will include the §107 area
the source will locate in, plus any other §107 areas where the pollutant
impact exceeds 1 ug/m3 annual average (1 ug/m3, 24-hour average, for Class I areas)
1. Were §107 areas properly applied? TSP SO?
[ ] a. Yes, the baseline area consists of all portions of the
designated attainment/unclassified areas as listed in [ ] [ ]
40 CFR 81 Subpart C.
[ ] b. No, the baseline area consists of only portions of the
designated attainment/unclassified areas. [ ] [ ]
Explain:
[ ] c. Cannot be determined from available information. [ ] [ ]
3-38
-------
2. Did the baseline area include any other areas besides the area in which
the source would construct?
[ ] a. No, file documentation demonstrated no significant impact
beyond area where source would locate.
[ ] b. No, but documentation was not provided to indicate whether
other areas should have been included.
[ ] c. Yes, for some of the [ ] TSP [ ] S02 attainment/unclassified areas
in baseline area.
[ ] d. CBD.
3. Did the source trigger the baseline date?
[ ] a. Yes, for entire baseline area.
[ ] b. -
Yes, for some of the [ ] TSP [ ] S02 attainment/unclassified areas
[ ] c. No, baseline date(s) for all areas within baseline area
previously triggered.
[ ] d. CBD.
C. Increment Consumption
1. Did the analysis include, where appropriate (or explain why not):
TSP SO?
a. emissions from major sources commencing construction
after 1/6/75 in determining increment consumed (PSD
Workshop Manual, Pt I, Sec. C.2)?
(i) Yes, for [ ] [ ]
(ii) No, for [ ] [ ]
(iii) No prior major source emissions consumed
increment, for [] []
(iv) CBD, for [ ] [ ]
b. emissions from minor sources occurring after the
applicable baseline date(s) within the impact area
in determining increment consumed?
(i) Yes, for [ ] [ ]
(ii) No. Explain: [ ] [ ]
(iii) Not applicable. Source triggered baseline date. [ ] [ ]
(iv) CBD, for [ ] [ ]
2. What impact concentrations were used for the short-term increments?
Concentration JJsed
Pollutant
TSP
SO 2
Highest
H
Highest of the
2nd highest
H
Other (explain)
[ ]
L J
3-39
-------
D. Is there any reason to believe that an increment analysis should
have been performed but was not?
C ] No.
Increment
[ ] Yes, as follows: [ ] TSP [ ] Class I [ ] Class II
[ ] S02 [] Class I [ ] Class II
Explain each "yes":
3-40
-------
SECTION VII. NAAQS PROTECTION
1. Was a NAAQS analysis performed?
[ ] NO, 60 TO Question 8
[ ] YES, as follows:
Pollutant Identify model
a. [ ] TSP [ ]24hr:
b. [ ] S02 [ ]3hr: [ ]24hr:
c. [ ] NOX
d. [ ] CO [ ]lhr: [ ]8 hr:
e. [ ] 03 [ ]lhr:
f. [ ] Pb
used In the blanks below.
[ ]annual:
[ ]annual:
[ ]annual:
[ ]3mos:
2. For the pollutants checked above, how was the analysis performed?
FOR THESE POLLUTANTS:
[ ] a. by the applicant with adequate review
(Including replication of results, If [ ] all [ ]:.
appropriate) by the agency.
[ ] b. by the applicant without adequate review
(Including replication of results, If [ ] all [ ]:
appropriate) by the agency.
Explain:
[ ] c. by the agency. [ ] all [ ]:
3. Did the applicant/agency use the appropriate model(s) to complete the analysis?
FOR THESE MODELS:
[ ] a. YES, and documentation supports use
of the model(s). [ ] all [ ]:
[ ] b. YES, but Inadequate documentation was
available to explain Its use. [ ] all [ ]:
[ ] c. CANNOT BE DETERMINED, documentation not
provided to justify model selection. [ ] all [ ]:
[ ] d. NO, documentation failed to address
appropriate considerations. [ ] all [ ]:
Explain:
3-41
-------
4. Did the applicant/agency use the appropriate model options (urban/rural,
receptor network design, wind speed profiles, building wake effects, final/
gradual plume rise)?
FOR THESE MODELS:
[ ] a. YES, and documentation supports use of
the option(s). [ ] all [ ]: _
[ ] b. YES, but inadequate documentation was [ ] all [ ]: _
available to explain their use.
[ ] c. CANNOT BE DETERMINED, documentation not [ ] all [ ]: _
provided to justify model option selection.
[ ] d. NO, documentation failed to address [ ] all [ ]: _
appropriate considerations.
Explain: _
5. Did the analysis consider appropriate meteorological data?
[ ] a. YES, five consecutive years of the most recent representative
sequential hourly National Weather Service data.
[ ] b. YES, five years of on site data subjected to quality assurance procedures.
[ ] c. YES, at least one year of hourly sequential on site data, including
worst-case conditions and subjected to quality assurance procedures.
[ ] d. Yes, one year of NWS data if 5 not available + use of highest
modeled results.
[ ] e. YES, screening data were used to obtain conservative results.
[ ] 6. NO. Explain: _ _
f
[ ] g.
6. Is there sufficient information in the file to verify that emissions
from the following stationary sources (including sources with permits,
but not yet in operation) were adequately considered when appropriate?
YES NO NA
a. Existing major stationary sources.
Explain: _ [ ] [ ] [ ]
b. Existing minor and area stationary sources.
Explain: _ [ ] [ ] [ ]
7. Did the analysis provide adequate consideration
of multi-source pollutant interactions?
[ ] a. YES, analysis adequately defined points of maximum impact determined
from consideration of all sources in the vicinity rather than the
maximum impact of the proposed source alone.
[ ] b. NO, analysis ignored significant emissions from other sources in
the vicinity.
Explain: __
[ ] c. CBD.
3-42
-------
8. Is there any reason to believe that one or more NAAQS analyses should
have been performed but were not?
[ ] CBD.
[ ] NO.
[ ] YES; explain.
3-43
-------
VIII. EMISSION OFFSET REQUIREMENTS
A. Emission offsets:
[ ] 1. were not applicable to this source. (GO TO SECTION IX.)
[ ] 2. were applied to the following pollutants:
[ ] 3. should have been applied to
the following pollutants:
[ ] 4. use cannot be determined from information in the file. Explain:
B. For the six questions below, identify the applicable pollutants, then
use: "Y" for yes, "N" for no, "NA" for not applicable, or "CBO" in
spaces below. (Note: You may need to consult the file for the source(s)
from which the offsets are being obtained to be able to respond to the
following questions.)
Information obtained from:
(Check appropriate box, below)
Offset
TMS source Other
Pollutant ( ) ( ) ( ) permit permit (Explain);
1. Emission offsets
obtained by this source
are expressed in the same
terms (i.e. actual or
allowable) as are those
emissions used in the [ ] [ ] [ ]
RFP demonstration.
2. Mi nor/area source
growth was taken into account
in determining the amount
of emission offsets [ ] [ ] [ ]_
needed.
3. Offsets are surplus,
i.e., would not interfere [ ] [ ] [ ]
with RFP.
4. Offsets are [][][]
Federally enforceable.
(Continued on next page.)
3-44
-------
Information obtained from:
(Check appropriate box, below)
Offset
This source Other
Pollutant ( ) ( ) ( ) permit permit (Explain):
5. Offsets were required
to occur on or before the
dates of the start-up [ ] [ ] [ ]
of the new or modified
source.
6. Offsets were not
utilized from early source
shutdowns or production [ ] [ ] [ ]
curtailments, except for
replacements.
C. Comments:
SECTION IX. COMMENTS. NOTES
3-45
-------
FORM 4
PERMIT FILE QUESTIONNAIRE
FOR SOURCES NOT SUBJECT TO PSD OR PART D (OFFSETS)
[NOTE: Unless otherwise indicated, place an "X" in the box beside each statement or
response which applies. Many of the questions will allow more than one response.J
SECTION I. SOURCE INFORMATION
A.I. Company/Source Name:
2. Size/Source Category^
3. Location:
B. 1. Region I I I
1
C. Date Complete I D. Date Permit to
Application rec'd:| Construct Issued:
1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
mo day yr I mo day yr
1
E. Source Location
1. L J Attainment/unclassified for
all criteria pollutants
2. L J Nonattainment for:
2.
3.
4.
5.
6.
State I | I
Agency
a. L J State
b. L J Local:
Auditor
Permit f
Type of Review:
a. L J New Source
b. L J Modification
3. [ J Nonattainment area subject to a construction ban for:
4. L J Within 10 km of a Class I area
SECTION II. PUBLIC PARTICIPATION REQUIREMENTS
A. Public Notice:
YES NO CBD
1. [ J was not issued because exempted by agency rules. 60 TO II.B.
2. L J was not issued, but agency rules do not exempt. GO TO II.B.
3. was published in a newspaper (approximate cost $ ) ....[][][ J
4. provided opportunity for public hearing LJLJLJ
5. provided opportunity for written comment [][J[J
6. described agency's preliminary determination LJLJLJ
7. included estimated ambient impact LJLJLJ
8. indicated availability of additional information for public
inspection [J[][J
9. resulted in comments ("0" if notice produced no comments)
B. The following other affected government agencies were notified:
1. other agencies and officials within the State LJLJLJ
2. other States LJLJLJ
3. Federal Land Manager LJLJLJ
4. EPA ' [ J [ ] [ J
3-46
-------
C. Documentation for Section II parts A and B consists of: Notification of
Public Notice Other Agencies
1. copy of notice/correspondence in file a.[ ] b.[ ]
2. indication on processing checklist* a.[ ] b.[ ]
3. no documentation provided a.[ ] b.[ ]
4. other, explain:
* i.e., no copies, but some official indication in file that notice was provided
SECTION III. APPLICABILITY DETERMINATIONS
A. Definition of Source. The source or modification for which this permit
application was made was considered by the reviewing agency to consist of:
l.a.[ ] The following new and modified pollutant-emitting activities associated
with the same industrial grouping, located on contiguous or adjacent
sites, and under common control or ownership (if more than 5, list
only 5 largest):
Emission TPY/
New Modif. Unit/Size Pollutant
[ ] [ ] i.
C ] E ] 11.
[ ] [ ] iii.
C ] C ] iv.
[ ] [ ] v.
b. [ ] CBD
2. Were any new and modified pollutant-emitting activities (other than fugitive
emissions) omitted which should have been included:
[ ] No [ ] CBD [ ] Yes, as follows:
Emission Unit Emissions Reason given by agency for not
or Activity/Size Pollutant"TFY New Modif. considering as part of source
i. [ ] [ ]
ii. [ ] [ ]
iii. MM
iv. [ ] [ ]
3. If Section III.A.2.C. above was marked, did this new source or modification
escape PSD or Part D (major source locating in a nonattainment area) review
as a result of the omission of the activities?
[ ] Yes [ ] No [ ] CBD
3-47
-------
B. Fugitive Emissions.
1. This source or modification:
a. [ ] Does not have any quantifiable fugitive emissions. (60 TO Section III.C
b. [ ] Has too little documentation in the file to determine whether
quantifiable fugitive emissions occur or were considered.
c. Has quantifiable fugitive emissions which were:
i. [ ] Included in determining whether the source/modification was major
for the following pollutants:
ii. [ ] Not included in determining whether the source was major for the
following pollutants: because the
source is neither one of the 28 PSD source categories nor regulated
under Sections 111 or 112 of the Act.
iii. [ ] Not included in determining whether the source was major for the
following pollutants: , although
they should have been.
2. Did this source escape PSD or Part D review as a result of the omission
of fugitive emissions?
[ ] Yes [ ] No [ ] CBD
C. Potential to Emit (PTE). Determination of whether a source is major should
be based on PTE rather than actual emissions.
1. The emissions of this source or modification were determined:
a. Using emission rates based on emission factors which were:
i. [ ] Well established (e.g., AP-42) or well documented in the file.
ii. [ ] Not well established and lacking adequate documentation for the
following units and pollutants:
b. [ ] CBD
c. [ ] Using another method. Explain:
2. The emissions of this source were determined on the basis of:
a. [ ] CBD, Go to C.3.
b. [ ] Maximum capacity to emit at full physical and operational design;
GO TO C.3.
c. [ ] Limited capacity based on control equipment, physical, operational
or emission limitations, not otherwise required (e.g., NSPS),
as follows:
3-48
-------
If used, was limitation identified on:
Preconst. Permit Operating Permit
Limitation Pollutants Yes WT~ Yes NŁ NA^
i. [ J Control equipment* [ J [ J [ J L J [ J
ii. L J Emission limit* L J L J L J L J L J
iii. [ J Operating hours [ J [ J [ J [ J [ J
iv. L J Operating rate L J L J L J L J L J
v. L J Fuel/material L J L J L J L J L J
restriction
vi. L J L J L J L J L J L J
*Not otherwise required (i.e., main purpose of limitation is to reduce
potential emissions).
If Section III.C.2.C. above is marked, please describe the limitations:
3. Do the calculated emissions correctly represent the source's potential
emissions?
a. L J YES
b. [ J CBD
Explain:
c. No, because:
i. L J calculations based on control equipment, physical, operational
or emission limitations that are not Federally enforceable
ii. L J emission factors not acceptable
iii. [ J did not adequately address fugitive emissions
iv. L J did not address all pollutant-emitting activities (other than
fugitives)
Explain:
v. L J other. Explain:
3-49
-------
D. Emission Netting.
1. Check the appropriate box for this permit action:
a. [ ] New source; emission netting not applicable. 60 TO Section III.E.
b. [ ] Source was not required by agency to determine any net change in
emissions, but should have been. EXPLAIN:
GO TO Section III.E.
c. [ ] Source review included a determination of a net change in emissions.
2. The determination of significant emissions can be a complex process when
emission netting occurs. The following worksheet should be used to
determine whether the proper procedure was followed:
Other creditable,
Proposed emission contemporaneous Overall net
changes, TPY emissions changes, TPY change,
Pollutant (+) (-} Net"(+) (-)TPY
a. TSP
b. S02
c. NOX ___
d. 03(VOC) ___
e. CO _ __
f . :
h.
3. Did agency correctly identify whether emissions were significant?
[ ] Yes [ ] CBD [ ] No, explain
4. Did analysis consider all pollutants for which a net increase in emissions
occurred?
[ ] Yes.
[ ] No, failed to address one or more pollutants. Explain:
[ ] CBD. Explain:
3-50
-------
***COMPLETE THE FOLLOWING APPLICABLE STATEMENTS.***
5.a. Emission netting was based on actual emissions.
[ ] CBD; 60 TO Question 7. [ ] No; GO TO Question 6. [ ] Yes
b. Indicate whether the calculation of actual emissions properly considered
the following criteria:
Yes No CBD
1. Representative of normal unit operation [ ] [ ] [ ]
11. Based on a two-year average [ ] [ ] [ ]
111. Expressed In TRY [ ] [ ] [ ]
6. Emission netting was based on another approach.
a. [ ] No.
b. [ ] Yes, and approach was acceptable. Explain:
c. [ ] Yes, but approach was Incorrect. Explain:
7. Emission decreases were considered.
a. [ ] No; GO TO E.
b. [ ] Yes, and the decreases:
Were Uere not CBD N/A
[ ] [ ] [ ] a. Contemporaneous with the proposed modifica-
tion.
[ ] [ ] [ ] b. Previously relied on to determine a net
emission change.
[] [] [][]c. Previously counted as part of the SIP attain-
ment strategy (Part D source only).
[] [] [][]d. Previously relied on to meet the "reasonable
further progress" requirement of Part D (Part
D source only).
[ ] [ ] [ ] e. Made Federally enforceable as permit condi-
tions.
3-51
-------
E. Definition of Major. This source was NOT subjected to PSD or Part D provisions
because:
1. Potential emissions from the new or existing source were less than
(check appropriate box):
a. [ ] 100 tpy for Part D and/or PSD (28 listed PSD source categories)
pollutants; or
b. [ ] 100 tpy for Part D and/or 250 tpy for PSD (non-11sted PSD source
categories) pollutants.
2. [ ] Emission increases resulting from modification were not significant.
3. [ ] Source eligible for exemption; describe:
4. Review agency erred and source should have been subject to:
a. [ ] PSD review; explain: .
b. [ ] Part D review: explain:
5. [ ] CBD
F. Emission limits.
1. Did the agency Identify the appropriate allowable emission rates with
respect to each emission unit 1n the construction permit?
[ ] Yes. [ ] No; Explain:
2. The number of limitations actually appearing 1n the preconstructlon
perm1t(s) 1s .
3. How many of the limitations:
Number
Yes No CBD Total
a. Include clear and concise averaging periods
compatible with appropriate requirements
(e.g., NSPS, short-term NAAQS)?
b. Are consistent with acceptable measurement
techniques?
c. Consist of design, equipment, work practice,
or operational standards?
d. Appear Federally enforceable?
e. Include stated or referenced compliance test
methods?
3-52
-------
G. Applicability Summary. Is there any reason to believe that this application
should have been subject to PSD or Part D provisions?
1. [ ] NO, GO TO Section IV.
2. [ ] YES. Explain:
SECTION IV.CONTROL TECHNOLOGY
1. Does the file contain documentation to show that the reviewing agency
verified the applicant's calculations and assumptions pertaining to the selected
control technology?
[ ] Yes [ ] No
Comments:
2. Does file documentation show that the reviewing agency ascertained compliance
of estimated emissions with applicable SIP limits?
[ ] Yes [ ] No
Comments:
3. Is the source subject to NSPS or NESHAP requirements?
[ ] Yes [ ] No, GO TO Section V.
4. Was the source identified by the agency as being subject to:
a. NSPS: [ ] Yes, for .
[ ] No, because no NSPS apply.
[ ] No, but should have been. Explain:
b. NESHAP: [ ] Yes, for:
[ ] No, because no NESHAP apply.
[ ] No, but should have been. Explain:
3-53
-------
SECTION V. AMBIENT AIR QUALITY ANALYSIS (NAAQS Protection)
1. Was an ambient Impact analysis performed?
[ ] NO, GO TO Question 8.
[ ] YES, as follows:
Pollutant Identify model used 1n the appropriate blanks below.
a. [ ] TSP [ ]24hr: [ ]annual:
b. [ ] S02 [ ]3hr: [ ]24hr: [ ]annual:
c. [ ] NOx [ jannual:
d. [ ] CO [ ]lhr: [ ]8hr:
e. [ ] 03 [ ]lhr:
f. [ ] Pb [ ]3mos:
2. For the pollutants checked above, how was the analysis performed:
FOR THESE POLLUTANTS:
[ ] a. by the applicant with adequate review
(Including replication of results, 1f [ ] all [ ]:
appropriate) by the agency.
[ ] b. by the applicant without adequate review
by the agency [ ] all [ ]:
Explain:
[ ] c. by the agency. [ ] all [ ]:
3. Did the applicant/agency use the appropriate model(s) to complete the analysis?
FOR THESE MODELS:
[ ] a. YES, and documentation supports use
of the model(s). [ ] all [ ]:
[ ] b. YES, but Inadequate documentation was
available to explain Its use. [ ] all [ ]:
[ ] c. CBD, documentation not
provided to justify model selection. [ ] all [ ]:
[ ] d. NO, documentation failed to address [ ] all [ ]:
appropriate considerations.
Explain:
3-54
-------
4. Did the agency exercise the appropriate model options (urban/rural, receptor
network design, wind speed profiles, building wake effects, final/gradual
plume rise, etc.)?
FOR THESE MODELS:
[ ] a. Yes, and doumentation supports use of each option [ ] all [ ]:
as being appropriate.
[ ] b. Yes, each option was appropriate, but inadequate [ ] all [ ]:
documentation was available to explain its selection.
[ ] c. Cannot be determined. Documentation not provided [ ] all [ ]:
to justify option selection.
[ ] d. No, documentation failed to address appropriate [ ] all [ ]:
considerations. Explain:
5. Did the analysis consider appropriate meteorological data?
[ ] a. YES, five consecutive years of the most recent representative
sequential hourly National Weather Service data.
[ ] b. YES, one year of MWS data (if 5 not available) + use of highest modeled
results.
[ ] c. YES, five years of on site data subjected to quality assurance procedures.
[ ] d. YES, at least one year of hourly sequential on site data, including
worst-case conditions and subjected to quality assurance procedures.
NO; Explain:
CBD
[ ] e.
[ ] f-
6. Is there sufficient information in the file to verify that emissions
from the following stationary sources (including sources with permits,
but not yet in operation) were adequately considered when appropriate?
YES NO NA
a. existing major stationary sources.
If no, explain: [ ] [ ] [ ]
b. existing minor and area stationary sources.
If no, explain: [ ] [ ] [ ]
7. Did the analysis provide adequate consideration of multi-source pollutant
interactions?
a. [ ] YES, analysis adequately defined points of maximum impact determined
from consideration of all sources in the vicinity rather than the maximum
impact of the proposed source alone.
b. [ ] NO, analysis ignored significant emissions from other sources in
the vicinity.
Explain:
c. [ ] Cannot be determined from information available in file.
3-55
-------
8. Is there any reason to believe that the proposed project should have
been subjected to an ambient Impact analysis that was not performed?
a. [ ] YES, source was within 10 km of a Class I area but Us
ambient Impact was not considered.
b. [ ] YES; Explain:
c. [ ] NO.
d. [ ] CBD.
SECTION VI. ADDITIONAL REVIEW
Was this source subject to any of the following additional reviews?
Yes No
[ ] [ ] BACT
[ ] [ ] LAER
[ ] [ ] PSD/Increment analysis
Comments:
3-56
-------
Chapter 4
Compliance Assurance Audit Guidelines
FY 1986-1987
A. INTRODUCTION
The major parts of the compliance assurance element in the FY 1986-1987
audit period will be the same as FY 1985: periodic review and assessment
of source data, file reviews, and overview inspections. The only change
will be a greater emphasis on volatile organic compound (VOC) sources (in
States with ozone nonattainment areas) and on the national emission
standard for hazardous air pollutants (NESHAP) program [including demolition
and renovation (D&R)] in the file review and overview inspection portions of
the audits.
The questions which follow were developed for use by all ten Regions
to ensure consistency in the National Air Audit System (NAAS) effort, and
provide an accurate basis for national comparison of State compliance
programs. All questions must be answered and procedures followed for
each audit.
The time period to be covered by the audits is the most recent twelve
months preceeding the on-site visit, with the exception of Compliance
Data System (CDS) data. For CDS data, the most recent fiscal year should
be used (i.e., FY 1985 or FY 1986).
B. PERIODIC REVIEW AND ASSESSMENT OF SOURCE DATA
To assess the adequacy of State compliance programs in meeting Clean
Air Act requirements, the EPA Regional offices continually review source
compliance status and inspection information submitted by the States for
the SIP, NSPS, and NESHAPs programs. This information is contained in
the CDS. In preparation for each audit, the Regions are to obtain CDS
retrievals for operating Class A State implementation plan (SIP), new
source performance standards (NSPS), and NESHAP sources on inspection
frequency, compliance rates, and enforcement activity. (These retrievals
should cover the most recent fiscal year.) This data must then be analyzed
by answering the following questions to show the status of the State's
compliance program:
1) What percentage of sources in the State received the required
inspections as specified in in the section 105 grant agreement? Prepare
an inspection summary for each State as follows:
4-1
-------
Progress in Meeting Inspection Commitments
Class Al SIP NSPS* NESHAP*
Total Sources
required in
the grant to
be inspected
Number Inspected
in Proceeding
Twelve months**
Percentage
*Assumes program delegated to State
**If time period crosses parts of two fiscal years
and the grant requirements are different, then
this must be accounted for.
2) What is the compliance status breakdown of sources in each air
program? Prepare compliance chart for each State as follows:
Compliance Status of Sources
In Meeting In
Program Compliance Schedule Violation Unknown Total
Class A SIP
Class Al SIP*
NSPS
NESHAP
*Subset of Class A SIP
3) How many operating NSPS sources have not had their required 180-
day start-up test? Prepare summary chart for NSPS sources operating with
and without required performance test as follows:
NSPS Sources Operating With and Without
Performance Test
Number of Number Operating Number Operating
Operating NSPS With Test >180 Days Without Test
4-2
-------
4) Based on CDS, what percentage of Class Al SIP, NSPS, and NESHAP
long-term complying sources (defined as being in compliance two consecutive
quarters or more) have recently (within the past year) had State compliance
inspections? Specify results for each program separately including the
numbers used to derive the percentages.
5) Regarding "Timely and Appropriate" (T&A) response to violators:
For the current fiscal year, what are the numbers of violators
in each category II (a)-(d) of the "timely and appropriate"
guidance?
What procedures have been established for reporting of data by
the State? Have these procedures then been followed by the
State? Be specific.
Give specifics of State actions and results in cases where EPA
deferred NOV issuance beyond day 120.
Has the State always satisfied penalty requirements where applicable
under the "Timely and Appropriate" guidance?
Overall, has the State followed all of the "Timely and Appropriate"
procedures it agreed to?
6) How many long-term violators (shown in CDS as in violation two
consecutive quarters or more) have not been subject to enforcement activity
or additional surveillance activity? Provide a source by source listing
of the State's Class A, NSPS, and NESHAP long-term violators including
type and date of most recent surveillance or enforcement activity.
Following the investigation into each of these questions, findings
are to be outlined for each State program under the heading of "Pre-visit
Assessment of State Compliance Program." These findings should be clear
and concise statements on what CDS reflects about the program and how
"timely and appropriate" is being implemented. Conclusions should then be
drawn on the condition of the State's compliance program, and summarized
in paragraph form.
This pre-visit assessment should be sent to the State prior to the
audit. The Region should include in this transmittal questions it wants
to follow-up on and any other air compliance related items requiring
discussion during the audit - such as findings from the overview inspections.
4-3
-------
C. FILE REVIEW
An effective State and local compliance program must have a well
documented file on each source. This file should be available for use by
management and field personnel. The structure and location of files are
optional as long as any needed data can be supplied upon request. The
files should contain information supporting the compliance status of each
source.
The audit team should review a representative sample of files from
the three air programs (SIP, NSPS, NESHAP) in each State or local agency.
In States with ozone nonattainment areas, the sample should concentrate
on VOC sources. In most cases, each State audit file review should
consist of 15-20 files. Selection of sources for file review should be
based on time in violation, NSPS sources with CEM requirements, recently
reported compliance changes, citizen or congressional inquiries, problems
surfaced in the CDS pre-visit program analysis, personal knowledge of the
source or, as noted earlier, VOC sources in ozone nonattainment. areas or
NESHAP sources.
For each file reviewed, the following 17 questions must be answered.
The purpose of these seventeen questions is to gather the information
necessary to answer the three file review summary questions for the audit
report.1
1. Can the reviewer, from information available in the file, determine
the programs to which the source is subject? If not, why? The various
programs are SIP, PSD, NSPS and NESHAP including D&R.
2. From the information available in the file, can the source's
compliance status be determined for all regulations to which it is subject?
3. Does the file contain documentation supporting the source's
compliance status?^ (As a minimum, the file should contain: (a) documen-
tation that the source was inspected and that the regulated emission
points and pollutants were evaluated, and (b) a determination of the
compliance status of the source and documentation of the basis for that
determination.)
file review checklist, which was a refined version of these 17
questions, has been deleted from the FY 1986 - FY 1987 audit guidance.
The Regions may continue to use the checklist but it is no longer an
official component of the guidance.
2For the purposes of this report, the term "source" is synonymous with
facility and consists of one or more emission points or processes. For
the NESHAP's D&R program, it means site.
4-4
-------
4. Are all major emission points identified (i.e., in an inspection
report, operating permit, etc) and each point's compliance status indicated?
5. Does the file identify which emission points are subject to NSR,
NSPS, PSD, and NESHAP requirements? If yes, are regulated continuous
emission monitoring (CEM) requirements or permit conditions shown to be
in compliance and documented? Are required start-up performance tests
included? (In the case of NESHAP D&R sources, fugitive emissions should
be identified.)
6. Does the file identify special reporting requirements to which a
source may be subject (i.e., excess emission reports from a malfunction
or CEM requirements) and are any such reports found in the file?
7. Does the file include technical reviews, source tests, CEM performance
specification tests, permit applications, correspondence to and from the
company, and other supporting documentation? (plus any reference to any
samples taken from NESHAP D&R sources)
8. What methods of compliance documentation are used (e.g., source
test, CEM, fuel sampling and analysis, inspection, certification, engineering
analysis, asbestos sampling and analysis etc.)?
9. Was the method used to ascertain compliance the most appropriate
one for the type of source being documented? Is the method prescribed by
NSPS, NESHAP, or SIP? If not, explain.
10. If the documentation includes an inspection, does the inspection
report contain control equipment parameters observed during the inspection
(pressure drops, flow rates, voltages, opacities)? Were observed control
equipment operating parameters or CEM emission levels compared to permit
conditions, design parameters, or baseline observations? Were plant
operating parameters recorded?
11. If documentation is by a stack test, were visible emission
observations or CEM emissions levels and operating parameters recorded
during the test? Was there a quality assurance procedure used with a
stack test? Who conducted, observed, and reviewed the test?
12. Are enforcement actions contained in the file? (For NESHAP's D&R
sources this would include a "notice of deficiency" or the equivalent).
13. Are compliance actions taken in a timely manner? Do any take
longer than 30 days from the time of violation detection? If yes, how
long? (NESHAP D&R violations require immediate action).
14. Is there documentation to support the enforcement action?
4-5
-------
15. Is there documentation to show follow-up to the enforcement
action (reinspection, letter, etc.)?
16. Are citizens' complaints documented in the file and followed
up?
17. What action does the Agency take with respect to excess emission
reports?
The review team should summarize their findings following the file
review by answering the following three questions and including the
responses in the NAAS report.
1. Do all files reflect a resonable profile of the source (meaning
that the files contain inspection reports, stack test reports, CEM data,
enforcement actions, etc). If not, explain.
2. Do all files contain adequate written documentation to support
the compliance status reported to EPA? If not, explain.
3. Are violations documented and pursued to return the source to
compliance expeditiously? Explain.
D. OVERVIEW INSPECTION
To provide quality assurance for compliance data in State or local
files furnished to EPA, and to promote effective working relationships
between EPA and State or local agencies, EPA should continue the overview
inspection program begun in FY 1984. It is envisioned that the Regions
will continue to inspect 2-3 percent of the Class A, NSPS, and NESHAP
sources in the CDS inventory each fiscal year. The FY 1986 - FY 1987
overview portion of the audit should focus on the overview inspections
performed since the last audit. As with file reviews, the overview
inspections should include a representative portion of VOC sources with
special emphasis on those source types most environmentally significant
or troublesome based on impact on ozone nonattainment areas and NESHAP
sources including D&R.
EPA should notify the State and local agencies of its intent at
least 30 days before each inspection is to take place to encourage their
participation (this may not be possible for NESHAP D&R sources but as
much advance notification as possible should be given). Each inspection
should be an independent verification of the source's compliance status
at minimum, and should review the State and local inspector's procedures
for determining compliance if the inspection is jointly performed.
4-6
-------
To promote uniformity the following questions must be answered for
the overview inspection effort:
1) How were sources selected by the Region for the overview inspections?
2) How many inspections were performed?
3) What exactly did each inspection consist of? Specify inspection
procedures used as well as the degree and extent of involvement
of State personnel.
4) Uhat was their purpose (that is, to independently verify State
reported compliance, to observe State inspection practices, or
some combination of these)? Other purposes?
5) What were the results of each inspection? Answer should relate
to purpose stated in item 4.
6) Discuss the important points overall of the overview inspection
findings. Give recommendations for resolution of any problems
discovered during the effort.
D. COMPLIANCE ASSURANCE REPORT FORMAT
For each audit performed, the Region must prepare a compliance
assurance report that includes a complete summary of all audit activities
and answers to the three questions on page six and the six questions on
page seven. In addition, the report should include the conclusions
reached on the pre-visit assessment (see page three). The main body of
the report should follow the questionnaire exactly in each of the three
areas (Pre-Visit Program Assessment, File Review, and Overview Inspections).
In addition, each report must include an overall summary of findings for
each State program including positive and negative points, and recommendations
for resolution. This summary of findings should be at the beginning of
the compliance assurance report and/or contained in the reports executive
summary. Each report should be reviewed by the audited agency to help
eliminate misconceptions or misunderstandings and to ensure factual
accuracy before it is finalized.
4-7
-------
Chapter 5
Air Monitoring Audit Guidelines
FY 1986-1987
Section Title Page
11.1 Introduction 1
11.2 Regulatory Authority to Perform a System Audit 3
11.3 Preliminary Assessment and Systems Audit Planning 9
11.4 Guidelines for Conducting Systems Audits of
State and Local Agencies 15
11.5 Criteria for the Evaluation of State and Local
Agency Performance 26
11.6 Systems Audit Questionnaire - Short Form 33
11.7 Systems Audit Questionnaire - Long Form 54
5-i
-------
Summary of Changes to the
National Air Audit System Guidance for Air Monitoring
The FY 1986-1987 audit guidance is essentially the same as the FY-85
guidance provided for in the document EPA 450/2-84-008, with only a few
minor modifications. The principal change is the adoption of a biennial
audit schedule for auditing States and/or independently funded local
agencies, rather than an annual audit. Accordingly, the audit instructions
have been modified to reflect a biennial audit schedule. It is assumed
that approximately 50 percent of the agencies will be audited each year
of the biennium by the Regional Offices, resulting in an audit of all
agencies during the two year cycle. As in the past, the EPA's Control
Programs Development Division, Office of Air Quality Planning and
Standards, will develop a schedule for submission of regionally prepared
audit reports. The only change to the air monitoring questionnaires is
the incorporation of two questions from the short form questionnaire
(B4b and C4a) into Sections B and C of the long form questionnaire.
5-ii
-------
5. AIR MONITORING
This material is identical to:
SECTION 2.0.11
SYSTEM AUDITS CRITERIA
AND PROCEDURES FOR
AMBIENT AIR MONITORING PROGRAMS
of the
QUALITY ASSURANCE HANDBOOK FOR
AIR POLLUTION MEASUREMENT SYSTEMS,
VOLUME II. EPA-600/4-77-027a
5-iii
-------
Section No. 2.0.11
April 30, 1985
Page 1
11.0 SYSTEMS AUDIT CRITERIA AND PROCEDURES FOR AMBIENT AIR MONITORING
PROGRAMS
11.1 Introduction
11.1.1 General - A systems audit is an on-site review and inspection of a
state or local agency's ambient air monitoring program to assess its
compl iance with established regulations governing the collection, analysis,
validation, and reporting of ambient air quality data. A systems audit of
each state or autonomous agency within an EPA Region is performed biennially by
a member of the Regional Quality Assurance (QA) staff.
The purpose of the guidance included here is to provide the regulatory
background and appropriate technical criteria which form the basis for the air
program evaluation by the Regional Audit Team. To promote national uniformity
in the evaluation of state and local agency monitoring programs and agencies'
performance, all EPA Regional Offices are required to use at least the short
form questionnaire (Section 11.6), corrective action implementation request
(CAIR) (Section 11.4.2), and the systems audit reporting format (Section
11.4.4) for each audit. Use of sections of the long form questionnaire is left
to the discretion of the Regional QA Coordinator, with the concurrence of
the State or local agency. The short form questionnaire is essentially the
same as the monitoring audit questionnaire used in FY-84. No substantive
changes have been made; however, the questionnaire has been reorganized to
improve the information received and facilitate its completion. In addition,
requests for resubmission of data already possessed by EPA have been deleted.
The scope of a systems audit is of major concern to both EPA Regions and
the agency to be evaluated. A systems audit as defined in the context of this
document is seen to include an appraisal of the following program areas:
network management, field operations, laboratory operations, data management,
quality assurance and reporting. The guidance provided concerning topics for
discussion during an on-site interview have been organized around these key
program areas (Section 11.5). The depth of coverage within these areas may be
increased or decreased by using one or more sections of the long-form
questionnaire (Section 11.7) in conjunction with the short-form questionnaire
(Section 11.6). Besides the on-site interviews, the evaluation should include
-------
Section No. 2.0.11
April 30, 1985
Page 2
the review of some representative ambient air monitoring sites and the
monitoring data processing procedure from field acquisition through reporting
into the Storage And Retrieval Of Air Data (SAROAD) computer system.
The systems audit results should present a clear, complete and accurate
picture of the agency's acquisition of ambient air monitoring data.
11.1.2 Road Map jto Using this Section - This section contains guidance
sufficient information for operating a systems audit of an agency responsible
for operating ambient air monitoring sites, as part of the State and Local Air
Monitoring Stations (SLAMS) network, and to report the results in a uniform
manner. The following topics are covered in the subsections below:
o A brief sketch of the regulatory requirements which dictate that systems
audits be performed, indicating the regulatory uses to which the audit
results may be put (Section 11.2);
o A discussion of
1) the requirements on the agency operating the SLAMS network;
2) program facets to be evaluated by the audit; and
3) additional criteria to assist in determining the required extent of
the forthcoming audit; (Section 11.3)
o A recommended audit protocol for use by the Regional Audit Team, followed by
a detailed discussion of audit results reporting (Section 11.4);
o Criteria for the evaluation of State and local agency performance including
suggested topics for discussion during the on-site interviews (Section
11.5);
o A short-form questionnaire, based on the National Air Monitoring Audit
Questionnaire prepared by the STAPPA/ALAPCO Ad Hoc Air Monitoring Audit
Committee. (10-20-83) (Section 11.6);
o A long-form questionnaire, organized around the six key program areas to be
evaluted (Section 11.7); and
o A Bibliography of APA guideline documents, which provides additional
technical background for the different program areas under audit (Section
11.8).
The guidance provided in this section is addressed primarily to EPA Regional
-------
Section No. 2.0.11
April 30, 1985
Page 3
QA Coordinators and members of the Regional audit teams to guide them in
developing and implementing an effective and nationally uniform yearly audit
program. However, the criteria presented can also prove useful to agencies
under audit to provide them with descriptions of the program areas to be
evaluated.
Clarification of certain sections, special agency circumstances, and
regulation or guideline changes may require additional discussion or
information. For these reasons, a list of contact names and telephone numbers
is given in Table 11-1.
11.2 Regulatory Authority to Perform a Systems Audit
11.2.1 General Regulatory Authority - The authority to perform systems audits
is derived from the Code of Federal Regulation (Title 40). Specifically:
40 CFR Part 35, which discusses agency grants and grant conditions, and 40 CFR
Part 58, which deals specifically with the installation, operation and quality
assurance of the SLAMS/NAMS networks.
The regulations contained in 40 CFR Part 35 mandate the performance of
yearly evaluation of agency 105 grant programs by the Regional Administrators
or their designees. Pertinent regulatory citations are summarized in Table
11-2. All citations are quoted directly from the regulations and are intended
as an indication of the context within which systems audits are performed and
the impact that audit results may have on a given agency. Even though this is
the regulatory authority to conduct such audits, for the SLAMS network, the
specific authority is derived from 40 CFR Part 58. Three specific citations
from 40 CFR Part 58 are also quoted in Table 11-2.
In addition to the regulations presented in Table 11-2, a further
requirement is imposed on reporting organizations submitting data summary
reports to the National Aerometric Data Bank (NADB) through the SAROAD
computer system. SAROAD acceptance criteria call for at least 75 % data
completeness, which has been accepted as a data quality objective for state
and local agencies' monitoring operations. The Regional QA Coordinator may
wish to use this requirement together with information obtained by accessing
the SAROAD NA285 or NA288 Computer Programs, discussed in section 11.3. The
percent data completeness may be effectively used as an indicator of whether a
-------
Section No. 2.0.11
April 30, 1985
Page 4
Assistance Area
TABLE 11-1. LIST OF KEY CONTACTS AND TELEPHONE NUMBERS
Telephone
Office/Laboratory
Laboratory
Areas and NPAP
General QA
Guidance
Monitoring
Objectives/Siting
PARS System
SAROAD
System /NADP
Name
William J. Mitchell
William F. Barnard
Stanley SI eva
Gardner Evans
Jake Summers
Number
(919) 541-2769
FTS 629-2769
(919) 541-2205
FTS 629-2205
(919) 541-5651
(919) 541-3887
(919) 541-5694
EPA Location
EMSL/QAD/PEB
EMSL/QAD/PEB
OAQPS/MDAD/MRB
EMSL/MAD/DRB
OAQPS/MDAD/NADB
NPAP = National Performance Audit Program
PARS = Precision and Accuracy Reporting System
NADB = National Aerometric Data Bank
-------
Section No. 2.0.11
April 30, 1985
Page 5
TABLE 11-2. SUMMARY OF REGULATORY AUTHORITY TO CONDUCT SYSTEM AUDITS
A. Highlights of 40 CFR 35
Section Number
and Description Text
35.510-2
Grant Amount
"In determining the amount of support for a control
agency, the Regional Administrator will consider
A. The functions duties and obligation assigned to
the agency by an applicable implementation
plan,
B. the feasibility of the program in view of the resources
to be made available to achieve or maintain EPA
priorities and goals
C. the probable or estimated total cost of the program in
relation to its expected accomplishments
D. the extent of the actual or potential pollution problem
E. the population served within the agency's jurisdiction
F. the financial need, and,
H. the evaluation of the agency's performance."
35.510-3
Reduction in
Grant Amount
"If the Regional Administrator's annual performance
evaluation reveals that the grantee will fail or has failed
to achieve the expected outputs described in his approved
program, the grant amount shall be reduced "
35.520
Criteria for
(Grant) Award
"No grant may be awarded to any interstate or inter-
municipal air pollution control agency unless the applicant
provides assurance satisfactory to the Regional
Administrator that the agency provides for adequate
representation of appropriate State, interstate, local and
(when appropriate) international interests in the air
quality control region, and further that the agency has the
capability _qf developing and implementing &_ comprehensive air
quality ^Tan for the air quality control region."
-------
Section No. 2.0.11
April 30, 1985
Page 6
TABLE 11-2. SUMMARY OF REGULATORY AUTHORITY TO CONDUCT SYSTEM AUDITS
A. Highlights of 40 CFR 35 (Cont'd)
==============================================================================
Section Number
and Description Text
35 .520 No grant may be awarded unless the Regional Administrator
has determined that (1) the agency has the capability or will
Criteria for develop the capability, to achieve the objectives and outputs
(Grant) Award described in its EPA-approved program, and (2) the agency
has considered and incorporated as appropriate the
recommendations~bT the latest EPA~perfonnance~eva1uation J_n
its program."
35.530 In addition to any other requirements herein, each air
pollution control grant shall be subject to the following
Grant Conditions conditions:
A. Direct cost expenditures for the purchase of
B. The sum of non-Federal recurrent expenditures....
C. The grantee shall provide such information as the Regional
Administrator may from time to time require to carry out
his functions. Such information may contain, but is not
limited to: Air quality data, emission inventory data,
data describing progress toward compliance with regulations
by specific sources, data on variances granted, quality
assurance information related Jto data collection and
analysis~and similar regulatory motions, source reduction
plans and procedures, real time air quality and control
activities, other data related to air pollution emergency
episodes, and similar regulatory actions.
35.538-1 "Agency evaluation should be continuous throughout the
budget period. It is EPA policy to limit EPA evaluation
Agency to that which is necessary for responsible management of
Evaluation regional and national efforts to control air pollution. The
Regional Administrator shall conduct an agency performance
evaluation annually in accordance with 35.410.
-------
Section No. 2.0.11
April 30, 1985
Page 7
TABLE 11-2. SUMMARY OF REGULATORY AUTHORITY TO CONDUCT SYSTEM AUDITS
A. Highlights of 40 CFR 35 (Cont'd)
Section Number
and Description Text
35.410
Evaluation of
Agency
Performance
"A performance evaluation shall be conducted at least
annually by the Regional Administrator and the grantee
to provide a basis for measuring progress toward achievement
of the approved objectives and outputs described in the work
program. The evaluation shall be consistent with the
requirements of 35.538 for air pollution control agencies....
-------
Section No. 2.0.11
April 30, 1985
Page 8
TABLE 11-2. SUMMARY OF REGULATORY AUTHORITY TO CONDUCT SYSTEM AUDITS
B. Highlights of 40 CFR 58
Section Number
and Description Text
58.20 "By January 1, 1980 the State shall adopt and submit to
the Administrator a revision to the plan which will:
Air Quality
Surveillance A. Provide for the
Plan
Content (SLAMS) B. Provide for meeting the requirements of Appendices A, C, D,
and E, to this part
C. Provide for the operation of
D. Provide for the review of the air quality surveillance
system vn jin annual basis to determine if the system
meets the monitoring objectives defined in Appendix D to
this part. Such review must "
58.23
Monitoring
Network
Completion
58.34
NAMS Network
Completion
"By January 1, 1983:
A. Each station in the SLAMS network must be in operation,
be sited in accordance with the criteria in Appendix E to
this part, and be located a described on the station's
SAROAD site identification form, and
B. The quality assurance requirements of appendix A to this
part must be fully implemented."
"By January 1, 1981:
A. Each NAMS must be in operation
B. The quality assurance requirements of Appendix A_ Jto this
part musf~b"e fully implemented for "all NAMS."""
Appendix A "Agencies operating all or a portion of a SLAMS network are
Section 2.4 required to participate in EPA's national performance audit
program and Jto permit ^n annual EPA systems audit of their
National ambient air monitoring program....for additional iWormati on
Performance and about these programs. Agencies should contact Either tfie
Systems Audit
appropriate EPA Regiona^l Quality ControTCoordinator
-------
Section No. 2.0.11
April 30, 1985
Page 9
rigorous systems audit, using the long form questionnaire, might be needed or
not.
11.2.2 Specific Regulatory Guidance - The specific regulatory requirements of
an EPA-acceptable quality assurance program are to be found in Appendix A to
40 CFR Part 58. Section 2.2 of Appendix A details the operations for which an
agency must have written procedures. The exact format and organization of
such procedures is not indicated, however. Thus, many approaches to
appropriate documentation have been suggested by EPA, local agencies and other
groups.
One approach adopted by many EPA Regional Offices is the organization of
the required material into the framework recommended by the EPA Quality
Assurance Management Staff in the document titled "Interim Guidelines for the
Preparation of Quality Assurance Project Plans" (QAMS 005/80, December 1980).
The sixteen (16) elements described in the guideline document provide the
framework for organizing the required Air Program operational procedures,
integrating quality assurance activities and documenting overall program
operations. This approach is consistent with the required eleven items of
40 CFR Part 58, Appendix A. Table 11-3 illustrates this consistency and
demonstrates how each required program element will be evaluated in the
context of the program areas used in the organization of the long-form
questionnaire.
11.3 Preliminary Assessment and Systems Audit Planning
In performing a systems audit of a given agency, the Regional QA
Coordinator is seeking a complete and accurate picture of that agency's
current ambient air monitoring operations. Past experience has shown that
four (4) person-days should be allowed for an agency operating 10-20 sites
within close geographical proximity. The exact number of people and the time
alloted to conduct the audit are dependent on the magnitude and complexity of
the agency and on the EPA Regional Office resources. During the alloted time
frame, the Regional QA Audit Team should perform those inspections and
interviews recommended in Section 11.4. This includes on-site interviews with
key program personnel, evaluations of some ambient air monitoring sites
operated by the agency, and scrutiny of data processing procedures.
-------
Section No. 2.0.11
April 30, 1985
Page 10
TABLE 11-3. SPECIFIC REGULATORY REQUIREMENTS TO BE EVALUATED IN A SYSTEMS AUDIT
REQUIREMENT
(40 CFR 58, Appendix A)
(1) Selection of Methods and Analyzers
(1) Selection of Methods, Analyzers
(11) Documentation of Quality Control
Information
(2) Installation of Equipment
(3) Calibration
(7) Calibration and Zero/Span Checks
for Multiple Range Analyzers
Only applicable if other than
automated analyzers are used and
analyses are being performed on
filters - e.g., NO- or lead and
TSP
(10) Recording and Val idating Data
PERTINENT SECTION
OF CAMS DOCUMENT
005/80
Project
Description
Organization &
Responsibility
QA Objectives
Sampl ing
Procedures
Sample Custody
Calibration
Procedures and
Frequency
Analytical
Procedures
Data Reduction,
Validation and
Reporting
PERTINENT SECTION
OF QUESTIONNAIRE
(11.7)
Planning
PI anning
Planning
PI anning
Field Operations
Field/Lab Operation
Field/Lab Operation
Lab Operations
Data Management
-------
Section No. 2.0.11
April 30, 1985
Page 11
TABLE 11-3. SPECIFIC REGULATORY REQUIREMENTS TO BE EVALUATED
IN A SYSTEMS AUDIT (cont'd)
REQUIREMENT
(40 CFR 58, Appendix A)
(4) Zero/span checks and adjustments
of automated analyzers
(5) Control Checks and their frequency
(6) Control Limits for Zero/ Span
(7) Calibration and Zero/Span for
Multiple Range Analyzers
(9) Quality control checks for air
pollution episode monitoring
Appendix A - Sections 2.0, 3.0
and 4.0
(8) Preventive and Remedial
Maintenance
Appendix A - Section 4.0
(10) Recording and Validating Data
(4) Zero/Span checks and adjectments
of automated analyzers
(6) Control Limits and Corrective
Actions
(11) Documentation of Quality Control
Information
(10) Data Recording and Validation
PERTINENT SECTION
OF OAMS DOCUMENT
005/80
Internal Quality
Control Checks
Performance and
Systems Audits
Preventive
Maintenance
Specific Routine
Procedures used
to Assess Data
Precision,
Accuracy and
Compl eteness
Corrective
Action
Quality
Assurance
Reports to
Management
PERTINENT SECTION
OF QUESTIONNAIRE
(11.7)
Field/Lab Operations
QA/QC
QA/QC
Field/Lab Operations
QA/QC
Data Management
Field/Lab Operations
Reporting
-------
Section No. 2.0.11
April 30, 1985
Page 12
11.3.1 Frequency of Audits - The EPA Regional Office has the regulatory
authority to evaluate agency performance annually. Regional Offices are
urged to use the short-form questionnaire (Section 11.6), the CAIR (Fig.
11-4), and the audit reporting format (Section 11.4.4.). Utilizing the above
to provide OAQPS with this audit information will establish a uniform basis
for audit reporting throughout the country. For many well-established
agencies, an extensive systems audit and rigorous inspection may not be
necessary every year. The determination of the extent of the systems audit
and its rigor is left completely to EPA Regional Office discretion.
Therefore, the option is provided here that extensive inspections and
evaluations may be accomplished using the short-form questionnaire (Section
11.6), and appropriate section(s) of the long-form questionnaire (Section
11.7). It is suggested that a complete systems audit using the long-form
questionnaire be performed at least once every three years. Biennial reports
must still, however, include the short form, CAIR, and the report completed
according to Section 11.4.4
The primary screening tools to aid the EPA Regional QA Audit Team in
determining which type of audit to conduct and its required extent are:
A. National Performance Audit Program (NPAP) Data—which provide detailed
information on the ability of participants to certify transfer standards
and/or calibrate monitoring instrumentation. Audit Data summaries provide
a relative performance ranking for each participating agency when compared
to the other participants for a particular pollutant. These data could be
used as a preliminary assessment of laboratory operations at the different
local agencies.
B. Precision and Accuracy Reporting System (PARS) Data--which provide detailed
information on precision and accuracy checks for each local agency and each
pollutant, on a quarterly basis. These data summaries could be used to
identify out-of-control conditions at different local agencies, for certain
pollutants.
C. National Aerometric Data Bank (NADB) NA285 Data Summaries--which provide a
numerical count of monitors meeting and those not meeting specifications on
-------
Section No. 2.0.11
April 30, 1985
Page 13
monitoring data completeness on a quarterly basis, together with an
associated summary of precision and accuracy probability limits. An
additional program, NA288, will provide data summaries indicating the
percent of data by site and or by state for each pollutant.
11.3.2 Selection of Monitoring Sites for Evaluation - It is suggested that
approximately five percent (5%) of the sites of each local agency included in
the reporting organization be inspected during a systems audit. Many
reporting organizations contain a large number of monitoring agencies, while
in other cases, a monitoring agency is its own reporting organization. For
smaller local agencies, no fewer than two (2) sites should be inspected. To
insure that the selected sites represent a fair cross-section of agency
operations, one half of the sites to be evaluated should be selected by the
agency itself, while the other half should be selected by the Regional QA
Audit Team.
The audit team should use both the Precision and Accuracy Reporting
System (PARS) and the SAROAD computer databases in deciding on specific sites
to be evaluated. High flexibility exists in the outputs obtainable from the
NADB NA285 computer program; data completeness can be assessed by pollutant,
site, agency, time period and season. These data summaries would assist the
Regional audit team in spotting potentially persistent operational problems in
need of more complete on-site evaluation. At least one site showing poor data
completeness, as defined by SAROAD, must be included in those selected to be
evaluated.
If the reporting organization under audit operates many sites and/or its
structure is complicated and perhaps inhomogeneous, then an additional number
of sites above the initial 5% level should be inspected so that a fair and
accurate picture of the state and local agency's ability to conduct field
monitoring activities can be obtained. At the completion of the site
evaluations, the Regional audit team is expected to have established the
adequacy of the operating procedures, the flow of data from the sites and to
be able to provide support to conclusions about the homogeneity of the
reporting organization.
-------
Section No. 2.0.11
April 30, 1985
Page 14
11.3.3 Data Audits - With the implementation by many agencies of automated
data acquisition systems, the data management function has, for the most part,
become increasingly complex. Therefore, a complete systems audit must include
a review of the data processing and reporting procedures starting at the
acquisition stage and terminating at the point of data entry into the SAROAD
computer system. The process of auditing the data processing trail will be
dependent on size and organizational characteristics of the reporting
organization, the volume of data processed, and the data acquisition system's
characteristics. The details of performing a data processing audit are left,
therefore, to Regional and reporting organization personnel working together
to establish a data processing audit trail appropriate for a given agency.
Besides establishing and documenting processing trails, data processing
audits procedure must involve a certain amount of manual recomputation of raw
data. The preliminary guidance provided here, for the number of data to be
manually recalculated, should be considered a minimum enabling only the
detection of gross data mishandling:
(a) For continuous monitoring of criteria pollutants, the Regional QA
Coordinator should choose two 24-hour periods from the high
and low seasons for that particular pollutant per local agency
per year. (In most cases the seasons of choice will be winter
and summer). The pollutant and time interval choices are left
to the Regional auditor's discretion.
(b) For manual monitoring, four 24-hour periods per local agency
per year should be recomputed.
The Regional QA Coordinator should choose the periods for the data
processing audit while planning the systems audit and inspecting the
completeness records provided by the NADB NA285 system. The recommended
acceptance limits for the differences between the data input into SAROAD and
that recalculated during the on-site phase of the systems audit, are given in
Table 11-4.
-------
Section No. 2.0.11
April 30, 1985
Page 15
TABLE 11-4. ACCEPTANCE CRITERIA FOR DATA AUDITS
Data Acquisition Measurement Tolerance
Mode Pollutants Range (ppm)(a) Limits
Automatic Data
Retri eval
Stripe hart
Records
Manual
Reduction
$02, 03, N02
CO
S02, 03, N02
CO
TSP
Pb
0-0.5, or 0-1.0
0-20, or 0-50
0-0.5, or 0-1.0
0-20, or 0-50
+3 ppb
+0.3 ppm
+20 ppb
+1 ppm
+2 g/m3 (b)
+0.1 g/m3
(a) Appropriate scaling should be used for higher measurement ranges.
(b) Specified at 760 mm Hg and 25oC.
Systems audits conducted on large reporting organizations (e.g. four
local agencies) require recomputation of eight 24-hour periods for each of the
criteria pollutants monitored continuously. This results from two 24-hour
periods being recomputed for each local agency, for each pollutant monitored,
during a given year. For manual methods, sixteen 24-hour periods are
recomputed, consisting of four periods per local agency, per year.
11.4 Guidelines for Conducting Systems Audits of State and Local Agencies
A systems audit should consist of three separate phases:
o Pre-Audit Activities
o On-Site Audit Activities
o Post-Audit Activities
Summary activity flow diagrams have been included as Figures 11-1, 11-2
and 11-3, respectively. The reader may find it useful to refer to these
diagrams while reading this protocol.
11.4.1 Pre-Audit Activities - At the beginning of each fiscal year, the
Regional QA Coordinator or a designated member of the Regional QA Audit Team,
should establish a tentative schedule for on-site systems audits of the
agencies within their region.
-------
Section No. 2.0.11
April 30, 1985
Page 16
DEVELOP AUDIT SCHEDULE
CONTACT REPORTING ORGANIZATIONS
TO SET TENTATIVE DATES
REVISE SCHEDULE AS NECESSARY
CONTACT REPORTING ORGANIZATION TO
DISCUSS AUDIT PROCEDURE
INITIATE TRAVEL PLANS
FIRM DATES FOR ON-SITE VISITS
SEND QUESTIONNAIRE AND REQUEST
PRELIMINARY SUPPORT MATERIAL
REVIEW MATERIAL DISCUSS WITH
REPORTING ORGANIZATION QA OFFICER
FINALIZE TRAVEL PLANS WITH INFORMATION
PROVIDED BY REPORTING ORGANIZATION
DEVELOP CHECKLIST OF POINTS
FOR DISCUSSION
CONTACT AGENCY TO SET SPECIFIC
INTERVIEW AND SITE INSPECTION TIMES
TRAVEL ON-SITE
Figure 11-1. PRE-AUDIT ACTIVITIES
-------
Section No. 2.0.11
April 30, 1985
Page 17
AUDIT TEAMINITIAL INTERVIEW OF REPORTING ORGANIZATION DIRECTOR
INTERVIEW WITH KEY PERSONNEL
AUDIT GROUP 1
AUDIT GROUP 2
INTERVIEW PLANNING MANAGER
INTERVIEW LABORA1
fORY DIRECTOR
INTERV]
OPERATIC
[EW FIELD
)NS MANAGER
VISIT SITES (AGENCY SELECTED)
VISIT LABORATORY
WITNESS OPERATIONS
VISIT SITES (REGION SELECTED)
REVIEW SAMPLE RECEIVING AND
CUSTODY
~AW~Cl
VISIT AUDIT AND CALIBRATION
FACILITY
SELECT PORTION OF DATA
INITIATE AUDIT TRAIL
SELECT PORTION OF DATA
INITIATE AUDIT TRAIL
AUDIT TRAIL
IY OPERATIONS
:NT FUNCTION
—
MEET TO
DISCUSS
FINDINGS
—
ESTABLISH TRAIL THROUGH FIELD
OPERATIONS TO DATA MANAGEMENT
'FINALIZE AUDIT TRAILS AND COMPLETE DATA AUDIT
PREPARE AUDIT RESULTS SUMMARY OF
(a) overall operations (b) data audit findings
(c) laboratory operations (d) field operations
INITIATE REQUESTS FOR CORRECTIVE ACTION IMPLEMENTATION REQUESTS (CAIR)
DISCUSS FINDINGS WITH KEY PERSONNEL QA OFFICER |
EXIT INTERVIEW WITH REPORTING ORGANIZATION DIRECTOR TO OBTAIN
SIGNATURES ON CAIR
FIGURE 11-2.
ACTIVITIES
ON-SITE AUD
T COMPLETE
-------
Section No. 2.0.11
April 30, 1985
Page 18
TRAVEL BACK TO REGIONAL HEADQUARTERS
AUDIT TEAM WORKS TOGETHER TO PREPARE REPORT
INTERNAL REVIEW AT REGIONAL HEADQUARTERS
INCORPORATE COMMENTS AND REVISE DOCUMENTS
ISSUE COPIES TO REPORTING ORGANIZATION DIRECTOR
FOR DISTRIBUTION AND WRITTEN COMMENT
INCORPORATE WRITTEN COMMENTS RECEIVED
FROM REPORTING ORGANIZATION
SUBMIT FINAL DRAFT REPORT FOR
INTERNAL REGIONAL REVIEW
REVISE REPORT AND INCORPORATE COMMENTS
AS NECESSARY
PREPARE FINAL COPIES
DISTRIBUTE TO REPORTING ORGANIZATION
DIRECTOR, OAQPS AND REGION
Figure 11-3. POST-AUDIT ACTIVITIES
-------
Section No. 2.0.11
April 30, 1985
Page 19
Six (6) weeks prior to the audit, the Regional QA Coordinator should
contact the Quality Assurance Officer (QAO) of the reporting organization to
be audited to coordinate specific dates and schedules for the on-site audit
visit. During this initial contact, the Regional QA Coordinator should
arrange a tentative schedule for meetings with key personnel as well as for
inspection of selected ambient air quality monitoring and measurement
operations. At the same time, a schedule should be set for the exit interview
used to debrief the agency Director or his designee, on the systems audit
outcome. As a part of this scheduling, the Regional QA Coordinator should
indicate any special requirements such as access to specific areas or
activities. The Regional QA Coordinator should inform the agency QAO that he
will receive a questionnaire, precision and accuracy data, and completeness
data from NADB programs NA273 and NA288 which is to be reviewed or completed.
He should emphasize that the completed questionnaire is to be returned to the
EPA Region within one (I) month of receipt. The additional information called
for within the questionnaire is considered as a minimum, and both the Region
and the agency under audit should feel free to include additional information.
The Regional Audit Team may use this initial contact or subsequent
conversations to obtain appropriate travel information, pertinent data on
monitoring sites to be visited, and assistance in coordinating meeting times.
Once the completed questionnaire has been received, it should be reviewed
and compared with the criteria and information discussed in Section 11.2 and
with those documents and regulations included by reference in Section 11.5.
The Regional QA Audit Team should also use the PARS and NADB NA273 and NA288
to augment the documentation received from the reporting organization under
audit. This preliminary evaluation will be instrumental in selecting the
sites to be evaluated and in the decision on the extent of the monitoring site
data audit. The Regional Audit Team should then prepare a checklist detailing
specific points for discussion with agency personnel.
The Region Audit Team could be made of several members to offer a wide
variety of backgrounds and expertise. This team may then divide into groups
once on-site, so that both audit coverage and time utilization can be
-------
Section No. 2.0.11
April 30, 1985
Page 20
optimized. A possible division may be that one group assess the support
laboratory and headquarters operations while another evaluates sites and
subsequently assesses audit and calibration information. The team leader
should reconfirm the proposed audit schedule with the reporting organization
immediately prior to travelling to the site.
11.4.2 On-Site Activities - The Regional QA Audit Team should meet initially
with the agency's Director or his designee to discuss the scope, duration, and
activities involved with the audit. This should be followed by a meeting with
key personnel identified from the completed questionnaire, or indicated by the
agency QAO. Key personnel to be interviewed during the audit are those
individuals with responsibilities for: planning, field operations, laboratory
operations, QA/QC, data management, and reporting. At the conclusion of these
introductory meetings, the Regional Audit Team may begin work as two or more
independent groups. A suggested auditing method is outlined in Figure 11.2.
To increase uniformity of site inspections, it is suggested that a site
checklist be developed and used.
The importance of the data processing systems audit cannot be overstated.
Thus, sufficient time and effort should be devoted to this activity so that
the audit team has a clear understanding and complete documentation of data
flow. Its importance stems from the need to have documentation on the quality
of ambient air monitoring data for all the criteria pollutants for which the
agency has monitoring requirements. The data processing systems audit will
serve as an effective framework for organizing the extensive amount of
information gathered during the audit of laboratory, field monitoring, and
support functions within the agency.
The entire audit team should prepare a brief written summary of findings
organized into the following areas: planning, field operations, laboratory
operations, quality assurance/quality control, data management, and reporting.
Problems with specific areas should be discussed and an attempt made to rank
them in order of their potential impact on data quality. For the more serious
of these problems, Corrective Action Implementation Request (CAIR) forms
should be initiated. An example form is provided in Figure 11-4. The forms
have been designed such that one is filled out for each major deficiency noted
that requires formal corrective action.
-------
Section No. 2.0.11
April 30, 1985
Page 21
CORRECTIVE ACTION IMPLEMENTATION REQUEST (CAIR)
Reporting Organization
State or Local Agency
Deficiency Noted:
Agreed-upon Corrective Action:
Schedule for Corrective Action Implementation;
Signed
Di rector
QA Officer
Audit Team Member
Date
Date
Date
:===================
Corrective Action Implementation Report:
Signed Director Date
Signed QA Officer Date
Figure 11-4. Example of a CAIR Form
-------
Section No. 2.0.11
April 30, 1985
Page 22
The format, content, and intended use of CAIRs is fully discussed in
Section 11.4.5 of this document. Briefly, they are request forms for specific
corrective actions. They are initiated by the Regional QA Audit Team and
signed upon mutual agreement by the agency's Director or his designee during
the exit interview.
The audit is now completed by having the Regional Audit Team members meet
once again with key personnel, the QAO and finally with the agency's Director
or his designee to present their findings. This is also the opportunity for
the agency to present their disagreements. The audit team should simply state
the audit results including an indication of the potential data quality
impact. During these meetings the audit team should also discuss the systems
audit reporting schedule and notify agency personnel that they will be given a
chance to comment in writing, within a certain time period, on the prepared
audit report in advance of any formal distribution.
11.4.3 Post-Audit Activities - The major post-audit activity is the
preparation of the Systems Audit Report. The report format is presented in
Section 11.4.4.
To prepare the report, the audit team should meet and compare
observations with collected documents and results of interviews and
discussions with key personnel. Expected QA Project Plan implementation is
compared with observed accomplishments and deficiencies and the audit findings
are reviewed in detail. Within thirty (30) calendar days of the completion of
the field work, the audit report should be prepared and submitted.
The Systems Audit Report is submitted to the audited agency together with
a letter thanking agency personnel for their assistance, time and cooperation.
It is suggested that the body of the letter be used to reiterate the fact that
the audit report is being provided for review and written comment. The letter
should also indicate that, should no written comments be received by the
Regional QA Coordinator within thirty (30) calendar days from the report date,
it will be assumed acceptable to the agency in its current form, and will be
formally distributed without further changes.
-------
Section No. 2.0.11
April 30, 1985
Page 23
If the agency has written comments or questions concerning the audit
report, the Regional Audit Team should review and incorporate them as
appropriate, and subsequently prepare and resubmit a report in final form
within thirty (30) days of receipt of the written comment. Copies of this
report should be sent to the agency Director or his designee for his internal
distribution. The transmittal letter for the ammended report should indicate
official distribution and again draw attention to the agreed-upon schedule for
Corrective Action Implementation.
11.4.4 Audit Reporting - The Systems Audit Report format discussed in this
section has been prepared to be consistent with guidance offered by the
STAPPA/ALAPCO Ad Hoc Air Monitoring Audit Committee. The format is considered
as acceptable for annual systems audit reports submitted to the OAQPS.
Regional Audit Team members shall use this framework as a starting point and
include additional material, comments, and information provided by the agency
during the audit to present an accurate and complete picture of its operations
and performance evaluation.
At a minimum, the systems audit report should include the following six
sections:
Executive Summary -- summarizes the overall performance of the agency's
monitoring program. It should highlight problem areas needing additional
attention and should describe any significant conclusions and/or broad
recommendations.
Introduction -- describes the purpose and scope of the audit and
identifies both the Regional Audit Team members, key agency personnel, and
other section or area leaders who were interviewed. It should also indicate
the agency's facilities and monitoring sites which were visited and inspected,
together with the dates and times of the on-site audit visit. Acknowledgement
of the cooperation and assistance of the Director and the QAO should also be
considered for inclusion.
Audit Results — presents sufficient technical detail to allow a complete
understanding of the agency operations. The information obtained during the
audit should be organized using the recommended subjects and the specific
instructions given below. It will be noted that the report format follows the
four-area organization of the short-form questionnaire.
-------
Section No. 2.0.11
April 30, 1985
Page 24
A* Network Design and Siting
1) Network Size — Provide an overview of the network size and the
number 57local agencies responsible to the state for network
operation.
2) Network Design and Siting—Describe any deficiencies in network
design or probe siting discovered during the audit. Indicate what
corrective actions are planned to deal with these deficiencies.
3) Network Audit — Briefly discuss the conclusions of the last network
annual audit and outline any planned network revision resulting from
that audit.
4) Non-criteria Pollutants— Briefly discuss the agency's monitoring
and quality assurance activities related to non-criteria pollutants.
B. Resources and Facilities
1) Instruments and Methods—Describe any instrument non-conformance
with the requirements of 40 CFR 50, 51, 53, and 58. Briefly
summarize agency needs for instrument replacement over and above
non-conforming instruments.
2) Staff and Facilities—Comment on staff training, adequacy of
facilities andavailability of NBS-traceable standard materials and
equipment necessary for the agency to properly conduct the bi-weekly
precision checks and quarterly accuracy audits required under 40 CFR
Part 58, Appendix A.
3) Laboratory Facilities — Discuss any deficiencies of laboratory
procedures, staffing and facilities to conduct the tests and analyses
needed to implement the SLAMS/NAMS monitoring the Quality Assurance
plans.
C. Data and Data Management
1) Data Processing and Submittal — Comment on the adequacy of the
agency's staff andfacilities to process and submit SAROAD air
quality data as specified in 40 CFR 58.35 and the reporting
requirements of 40 CFR 58, Appendices A and F. Include an indication
of the timeliness of data submission by indicating the fraction of
data which are submitted more than forty-five (45) days late.
2) Data Review — A brief discussion of the agency's performance in
meeting the 75% criteria for data completeness. Additionally,
discuss any remedial actions necessary to improve data reporting.
-------
Section No. 2.0.11
April 30, 1985
Page 25
3) Data Correction — Discuss the adequacy and documentation of
corrections and/or deletions made to preliminary ambient air data,
and their consistency with both the agency's QA Manual and Standard
Operating Procedures, and any revised protocols.
4) Annual Report — Comment on the completeness, adequacy and
timeliness of submission of the SLAMS Annual Report which is required
under 40 CFR 58.26.
D. Quality Assurance/Qua!ity Control
1) Status of Quality Assurance Manual—Discuss the status of the
Agency's Quality Assurance Plan. Include an indication of its
approval status, the approval status of recent changes and a general
discussion of the consistency, determined during the systems audit,
between the Agency Standard Operating Procedures and the Quality
Assurance Plan.
2) Audit Participation— Indicate frequency of participation in an
audit program. Include as necessary, the agency's participation in
the National Performance Audit Program (NPAP) as required by 40 CFR
Part 58. Comment on audit results and any corrective actions taken.
3) Accuracy and Precision—As a goal, the 95 percent probability
limits^for precision fall pollutants) and TSP accuracy should be less
than +15 percent. At 95 percent probability limits, the accuracy for
all other pollutants should be less than +20 percent. Using a short
narrative and a summary table, compare the reporting organization's
performance against these goals over the last two years. Explain any
deviations.
Discussion — includes a narrative of the way in which the audit results
above are being interpreted. It should clearly identify the derivation of
audit results which affect both data quality and overall agency operations,
and should outline the basis in regulations and guideline documents for the
specific, mutually-agreed upon, corrective action recommendations.
Conclusions and Recommendations -- should center around the overall
performance of the agency's monitoring program. Major problem areas should be
highlighted. The salient facts of mutually agreed upon corrective action
agreements should be included in this section. An equally important aspect to
be considered in the conclusion is a determination of the homogeneity of the
agency's reporting organizations and the appropriateness of pooling the
Precision and Accuracy data within the reporting organizations. The checklist
-------
Section No. 2.0.11
April 30, 1985
Page 26
in Figure 11-5 should be included and submitted with the supporting
documentation.
Appendix of Supporting Documentation -- contains a clean and legible copy
of the completed short-form questionnaire and any Corrective Action
Implementation Request Forms (CAIR). Additional documentation may be included
if it contributes significantly to a clearer understanding of audit results.
11.4.5 Follow-up and Corrective Action Requirements - An effective corrective
action procedure for use by the Regional QA Audit Team follows. As a means of
requesting corrective actions identified during the on-site audit, the auditor
completes one copy of the form, shown in Figure 11-4, for each major
deficiency noted. These CAIR forms are presented to, and discussed with, the
agency's Director or his designee, and its QAO during the exit interview.
Once Agreement has been reached, both the auditor and the Director sign the
form. The original is given to the agency Director or his designee and a copy
is retained by the auditor. A photocopy of the completed CAIR is included in
the audit report. It is taken to be the responsibility of the agency to
comply with agreed-upon corrective action requests in the specified time
frame.
11.5 Criteria for the Evaluation of State and Local Agency Performance
This section is designed to assist the Regional Audit Team in
interpretation of the completed questionnaire received back from the agency
prior to the on-site interviews. It also provides the necessary guidance for
topics to be further developed during the on-site interviews.
This section is organized such that the specific topics to be covered and
the appropriate technical guidance are keyed to the major subject areas of the
long-form questionnaire (Section 11.7). The left-hand side of the page
itemizes the discussion topics and the right-hand side provides citations to
specific regulations and guideline documents which establish the technical
background necessary for the evaluation of agency performance. A more
complete bibliography of EPA guideline documents is presented in 11.8.
-------
Section No. 2.0.11
April 30, 1985
Page 27
REPORTING ORGANIZATION HOMOGENEITY CHECKLIST
Yes No
1. Field operations, for all local agencies, conducted
by a common team of field operators?
2. Common calibration facilities are used for all
local agencies?
3. Precision checks performed by common staff for
all local agencies?
4. Accuracy checks performed by common staff for
all local agencies?
5. Data handling follows uniform procedures for
all local agencies?
6. Central data processing facilities used for
all reporting?
7. Traceability of all standards established by
one central support laboratory?
8. One central analytical laboratory handles all
analyses for manual methods?
Figure 11-5. Example of Reporting Organization
Homogeneity Checklist
-------
11.5.1 Planning -
Topics for Discussion
o General information on
reporting organization and
status of Air Program, QA
Plan and availability of SOPs
o Conformance of network design
with regulation, and
completeness of network
documentation
o Organization staffing and
adequacy of educational
background and training of
personnel
key
o Adequacy of current facilities
and proposed modifications
11.5.2 Field Operations -
Topics for Discussion
o Routine operational practices
for SLAMS network, and
conformance with regulations
o Types of analyzers and samplers
used for SLAMS network
Section No. 2.0.11
April 30, 1985
Page 28
Background Documents
o State Implementation Plan
o U.S. EPA CAMS 005/80
o Previous Systems Audit
report
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods,
Section 2.0.1.
o 40 CFR 58 Appendices D
and E
o OAQPS Siting Documents
(available by pollutant)
o QA Handbook for Air
Pollution Measurement
Systems, Vol. I -
Principles, Section 1.4
Vol. II - Ambient Air
Specific Methods,Section
2.0.5
Background Documents
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Section 2.0.9
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II
o 40 CFR 50 plus appendices
A through G (potentially
K for PM10)
o 40 CFR 58 Appendix C -
Requirements for SLAMS
analyzers
-------
Section No. 2.0.11
April 30, 1985
Page 29
Topics for Discussion
o Adequacy of field procedures,
standards used and field
documentation employed for
SLAMS network
o Frequency of zero/span checks,
calibrations and credibility
of calibration equipment used
o Traceability of monitoring and
calibration standards
o Preventive maintenance system
including spare parts, tools
and service contracts for major
equipment
o Record keeping to include
inspection of some site log
books and chain-of-custody
procedures
o Data acquisition and handling
system establishing a data
audit trail from the site to
the central data processing
facility
11.5.3. Laboratory Operations -
Topics for Discussion
o Routine operational practices
for manual methods used in
SLAMS network to include
quality of chemical and
storage times.
Background Documents
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II
o Instruction Manuals for
Designated analyzers
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.9
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Section 2.0.7
o 40 CFR 58 Appendix A
Section 2.3
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Section 2.0.6
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II - Ambient
Air Specific Methods
Sections 2.0.3 and 2.0.9
Background Documents
o 4-0 CFR 50 Appendices A -B,
and QA Handbook, Vol. II
-------
Section No. 2.0.11
April 30, 1985
Page 30
Topics for Discussion
o List of analytical methods
used for criteria pollutants
and adherence to reference
method protocols
o Additional analyses performed
to satisfy regional, state
or local requirements
Laboratory quality control
including the regular usage
of duplicates, blanks, spikes
and multi-point calibrations
o Participation in EPA NPAP and
method for inclusion of audit
materials in analytical run
o Documentation and traceability
of laboratory measurements
such as weighing, humidity and
temperature determinations
o Preventive maintenance in the
laboratory to include service
contracts on major pieces of
instrumentation
o Laboratory record keeping and
chain-of-custody procedures
to include inspection of
logbooks used
Background Documents
o 40 CFR 58 Appendix C; "List
of Designated Reference
and Equivalent Methods"
o Refer to locally available
protocols for analysis of
aldehydes, sulfate,
nitrate, pollens, hydro-
carbons, or other toxic
air contaminants
0 U.S. EPA APTD-1132
"Quality Control Practices
in Processing Air
Pollution Samples"
o 40 CFR 58 Appendix C; "List
of Designated Reference
and Equivalent Methods"
o 40 CFR 58 Appendix A
Section 2.4
o QA Handbook for Ai r
Pollution Measurement
Systems, Vol. II,
Section 2.0.10
o 40 CFR 58 Appendix C; "List
of Designated Reference
and Equivalent Methods"
o 40 CFR 58 Appendix C; "List
of Designated Reference
and Equivalent Methods"
o 40 CFR 58 Appendix C; "List
of Designated Reference
and Equivalent Methods"
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Section 2.0.6
-------
Section No. 2.0.11
April 30, 1985
Page 31
Topics for Discussion
o Adequacy of Laboratory
facilities, Health and Safety
practices and disposal of
wastes
o Data acquisition, handling
and manipulation system
establishing data flow in
the laboratory, data back-up
system and data reduction
steps.
o Data validation procedures,
establishing an audit trail
for the laboratory to the
central data processing
facility
11.5.4. Data Management -
Topics for Discussion
o Data flow from field and
laboratory activities to
central data processing
facility
o Extent of computerization of
data management system and
verification of media changes,
transcriptions and manual
data entry
o Software used for processing
and its documentation; to
include functional description
of software, test cases and
configuration control for
subsequent revisions
o System back-up and recovery
capabilities
Background Documents
o Handbook for Analytical
Quality Control in Water
and Wastewater Laboratories
o QA Handbook for Air
Pollution Measurement
Systems, Yol. II,
Sections 2.0,3 and 2.0.9
o Annual Book of ASTM
Standards, Part 41, 1978.
Standard Recommended
Practice for Dealing with
Outlying Observations
{E 178-75)
Background Documents
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Section 2.0.3
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Section 2.0.9
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Sections 2.0.3 and 2.0.9
-------
Section No. 2.0.11
April 30, 1985
Page 32
Topics for Discussion
o Data screening, flagging and
valIdatlon
o Data correction procedures
and key personnel allowed to
correct ambient air data
o Reports generated for in-house
distribution and for submittal
to EPA
o Responsibility for preparing
data for entry into the SAROAD
and PARS systems and for
responsibility for its final
validation prior to submission
11.5.5 QA/QC Program -
Topics for Discussion
o Status of QA Program and its
implementation
o Documentation of audit
procedures, integrity of
audit devices and acceptance
criteria for audit results
o Participation in the National
Performance Audit Program
For what pollutants and
ranking of resul ts
o Additional internal audits
such as document reviews or
data processing audits
Background Documents
o Val idation of Air
Monitoring Data, EPA-
600/4-80-030
o Screening Procedures for
Ambient Air Quality Data,
EPA-450/2-78-037
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Section 2.0.9
o Aeros Manual Series,
Vol. II, Aeros User's
Manual, EPA-450/2-76-029
Background Documents
o 40 CFR 58 Appendix A
and QAMS 005/80
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Sections 2.0.11 and 2.0.12
o 40 CFR 58 Appendix A
o QA Handbook for Air
Pollution Measurement
Systems, Vol. II,
Section 2.0.10
-------
Section No. 2.0.11
April 30, 1985
Page 33
Topics for Discussion Background Documents
o Procedure and implementation
of corrective action
o Frequency of performance and o 40 CFR 58 Appendix A
concentration levels for
precision checks for each
criteria pollutant
11.5.6. Reporting -
Topics for Discussion Background Documents
o Preparation of precision and o PARS User's Manual
accuracy summaries for the (in preparation)
PARS system o 40 CFR 58 Appendix A
o Other internal reports used
to track performance and
corrective action
implementation
o Summary air data reports o 40 CFR 58 Appendices
required by regulations F and G
o Completeness, legibility and o 40 CFR 58 Appendix A
validity of P & A data on
Form 1
11.6 Systems Audit Questionnaire (Short-Form)
The short-form questionnaire has been designed specifically for use in
annually reviewing state and local agencies air monitoring programs. If the
Regional QA Coordinator decides that a more rigorous systems audit and site
inspections are necessary, he can utilize appropriate section(s) of the
Long-Form Questionnaire (Section 11.7). This questionnaire has been designed
around the format recommended by STAPPA/ALAPCO in the National Ambient Air
Monitoring Questionnaire and is organized around four (4) major topics
consistent with the reporting format outlined in Section 11.4.4. They are:
A. Network Design and Siting
B. Resources and Facilities
C. Data Management, and
D. Quality Assurance and Quality Control
-------
Section No. 2.0.11
April 30, 1985
Page 34
NATIONAL AIR MONITORING SYSTEMS AUDIT
QUESTIONNAIRE
(SHORT FORM)
Agency
Address
Telephone Number (Area Code) Number
Reporting Period (beginning-ending dates) _
Organization Director
Air Program Supervisor
Data Management Supervisor
Quality Assurance Officer
Questionnaire Completed
(date)
On-Site Visit
Date: Audit Team Members
Affiliation of Audit Team
SF-1
-------
Section No. 2.0.11
April 30, 1985
Page 35
SHORT FORM QUESTIONNAIRE
TABLE OF CONTENTS
PAGE NO.
A. NETWORK DESIGN AND SITING
1. Network Size SF-3
2. Network Design and Siting SF-5
3. Network Review SF-6
4. Non-Criteria Pollutants SF-7
B. RESOURCES AND FACILITIES
1. Instruments and Methods SF-8
2. Staff and Facilities SF-9
3. Laboratory Operations and Facilities SF-10
4. Standards and Traceability SF-11
C. DATA AND DATA MANAGEMENT
1. Timeliness of Data SF-13
2. Data Review SF-14
3. Data Correction SF-15
4. Annual Report SF-16
D. QUALITY ASSURANCE/QUALITY CONTROL
1. Status of Quality Assurance Program SF-17
2. Audit Participation SF-18
3. Precision and Accuracy Goals SF-19
SF-2
-------
Section No. 2.0.11
April 30, 1985
Page 36
A. NETWORK DESIGN SITING
1. NETWORK SIZE
(a) Complete the table below for each of the criteria pollutants monitored as
part of your air monitoring network. Include only those sites that are
presently operating and those which are temporarily inoperative (off line
less than 30 days). Do not include additional monitors which are
collocated or index sites.
Number of Monitors
502N02CO03 TS~PPb~
NAMS
SLAMS
(excluding NAMS)
SPM
TOTAL
(b) SLAMS Network Description
1. What is the date of the most current official SLAMS Network
Description?
2. Where is it available for public inspection?
SF-3
-------
Section No. 2.0.11
April 30, 1985
Page 37
3. Does it include for each site the following?
YES NO
SAROAD Site ID#
Location
Sampling and Analysis Method
Operative Schedule
Monitoring Objective and Scale
of Representativeness
Any Proposed Changes
(c) For each of the criteria pollutants, how many modifications (SLAMS
including NAMS) have been made since the last systems audit? (List
the total SLAMS and NAMS)
Date of last systems audit
Number _of Monitors
Pollutant Added Deleted Relocated
Sulfur Dioxide
Nitrogen Dioxide
Carbon Monoxide
Ozone
Total Suspended
Particulates
Lead
(d) Briefly discuss changes to the Air Monitoring Network planned for the
next audit period. (Equipment is discussed in Part B).
SF-4
-------
Section No. 2.0.11
April 30, 1985
Page 38
2. NETWORK DESIGN AND SITING
Indicate by SAROAD Number any non-conformance with the requirements of 40
CFR 58, Appendices D and E.
Site ID
Monitor (SAROAD) Reason for Non-Conformance
S02
03
CO
N02
TSP
Pb
SF-5
-------
Section No. 2.0.11
April 30, 1985
Page 39
3. NETWORK REVIEW
Please provide the following information on your previous internal Network
Review required by 40 CFR 58.20d.
Review performed on: Date
Performed by:
Location and Title of Review Document:
Briefly discuss all problems uncovered by this review.
SF-6
-------
Section No. 2.0.11
April 30, 1985
Page 40
4. NON-CRITERIA POLLUTANTS
Does your agency monitor and/or analyze for non-criteria and/or toxic air
pollutants? Yes No
If yes, please complete the form below.
Monitoring SOP Available
Pollutant Method/Instrument Yes/No
SF-7
-------
Section No. 2.0.11
April 30, 1985
Page 41
B. RESOURCES AND FACILITIES
1. INSTRUMENTS AND METHODS
(a) Please complete the table below to indicate which analyzers do not
conform with the requirements of 40 CFR 53 for NAMS, SLAMS, or SIP
related SPM's.
Site Comment on
Pollutant Number Make/Model Identification Variances
CO
SO 2
N02
03
TSP
Pb
(b) Please comment briefly on your currently identified equipment needs,
SF-8
-------
Section No. 2.0.11
April 30, 1985
Page 42
2. STAFF AND FACILITIES
(a) Please indicate the number of people available to each of the following
program areas:
Comment on Need for
Program Area Number Additional Personnel
Network Design
and Siting
Resources and
Facilities
Data and Data
Management
QA/QC
(b) Comment on your agency's needs for additional physical space
(laboratory, office, storage, etc.)
SF-9
-------
Section No. 2.0.11
April 30, 1985
Page 43
3. LABORATORY OPERATION AND FACILITIES
(a) Is the docunentation of Laboratory Standard Operating Procedures
complete? Yes No
Please complete the table below.
Analysis Date of Last Revision
TSP
Pb
S04
N03
S02
(bubblers)
N02
Others (list by pollutant)
(b) Is sufficient instrumentation available to conduct your laboratory
analyses? Yes No
If no, please indicate instrumentation needs in the table below.
Instrument New or Year of
Needed Analysis Replacement Acquisition
SF-10
-------
Section No. 2.0.11
April 30, 1985
Page 44
4. STANDARDS AND TRACEABILITY
(a) Please complete the table for your agency's laboratory standards.
PrimarySecondaryRecertification
Parameter Standard Standard Date
CO
N02
S02
03
Weights
Temperature
Moisture
Barometric
Pressure
Flow
Lead
Sulfate
Ni trate
Other (specify)
SF-11
-------
Section No. 2.0.11
April 30, 1985
Page 45
(b) Please complete the table below for your agency's site standards (up to
7% of the sites, not to exceed 20 sites).
Primary Secondary Recertifi cation
Parameter Standard Standard Date
CO
N02
S02
03
SF-12
-------
Section No. 2.0.11
April 30, 1985
Page 46
C. DATA AND DATA MANAGEMENT
1. TIMELINESS OF DATA
For the current calendar year or portion thereof which ended at least 135
calendar days prior to the receipt of this questionnaire, please provide
the following percentages for required data submitted.
% Submitted on Time*
Monitoring 302 1X5 03 NOZ TSP Pb
Qtr.
1
(Jan. 1-March 31)
2
(Apr. 1-June 30)
3
(July 1-Sept. 30)
4
(Oct. 1-Oec. 31)
* "On-Time" = within 135 calendar days after the end of the quarter in
which the data were collected.
SF-13
-------
Section No. 2.0.11
April 30, 1985
Page 47
2. DATA REVIEW
What fraction of the SLAMS sites (by pollutant) reported less than 75% of
the data (adjusted for seasonal monitoring and site start-ups and
terminations)?
Calendar Year
Pollutant
Ozone
Percent of Sites
<75% Data Recovery
1st
Quarter
2nd
Quarter
3rd
Quarter
4th
Quarter
Nitrogen Dioxide
Sulfur Dioxide
Carbon Monoxide
Total Suspended
Particul ates
Lead
SF-14
-------
Section No. 2.0.11
April 30, 1985
Page 48
3. DATA CORRECTION
(a) Are changes to submitted data documented in a permanent file?
Yes No
If no, why not?
(b) Are changes performed according to a documented Standard Operating
Procedure or your Agency Quality Assurance Project Plan?
Yes No
If not according to the QA Project Plan, please attach a copy of your
current Standard Operating Procedure.
(c) Who has signature authority for approving corrections?
(name) (Program Function)
SF-15
-------
Section No. 2.0.11
April 30, 1985
Page 49
4. ANNUAL REPORT
(a) Please provide the dates annual reports have been submitted in the last
two years.
(b) Does the agency's annual report (as required in 40 CFR CFR 58.26)
include the following?
YES NO
1. Data summary required in Appendix F.
2. Annual precision and accuracy information
described in Section 5.2 of Appendix A.
3. Location, date, pollution source and duration
of all episodes reaching the significant harm
levels.
4. Certification by a senior officer in the State
or his designee.
(c) Describe any deficiencies which cause the answer to part (b) of this
question to be No.
SF-16
-------
Section No. 2.0.11
April 30, 1985
Page 50
D. QUALITY ASSURANCE/QUALITY CONTROL
1. STATUS OF QUALITY ASSURANCE PROGRAM
(a) Does the agency have an EPA-approved quality assurance program plan?*
Yes No
If yes, have changes to the plan been approved by the EPA?
Yes No
PI ease provide:
Date of Original Approval
Date of Last Revision
Date of Latest Approval
(b) Do you have any revisions to your QA Program Plan still pending?
Yes No
If answer is No, give a brief summary of the deficiencies.
SF-17
-------
Section No. 2.0.11
April 30, 1985
Page 51
2. AUDIT PARTICIPATION
(a) Date last systems audit was conducted?
By whom?
(b) Does the agency participate in the National Performance Audit
Program (NPAP) as required under 40 CFR 58 Appendix A?*
Yes No
(c) Please complete the table below.
Parameter Audited Date of Last NPAP Audit
S02 (Continuous)
CO
Pb
ReF Device
S02 (bubbler)
N02 (bubbler)
If No, give a brief summary of deficiencies.
SF-18
-------
Section No. 2.0.11
April 30, 1985
Page 52
3. PRECISION AND ACCURACY GOALS
As a goal, the 95 percent probability limits for precision (all pollutants)
and TSP accuracy should be less than +15 percent. At 95 percent probability
limits, the accuracy for all other pollutants should be less than +20 percent.
Using a short narrative and a summary table, compare tTfe reporting
organization's performance against these goals over the last year. Explain any
deviations.
Precision and accuracy are based on reporting organizations; therefore,
this question concerns those reporting organizations that are the responsibility
of the agency. A copy of a computer printout has been provided which contains
the precision and accuracy data submitted to EMSL for each of the agency's
reporting organizations. The printout, containing at least the last four
completed calendar quarters of precision and accuracy data, was obtained using
the NADB program NA273. This data should be verified using agency records. If
found in error, please initiate corrections. Based on the data provided or
corrections thereto, complete the table in part "a" below indicating the number
of reporting organizations meeting the goals stated above for each pollutant by
quarter.
SF-19
-------
Section No. 2.0.11
April 30, 1985
Page 53
(a) Precision Goals
Pollutant
# of Reporting
Organization
Precision
Qtr/Yr Qtr/Yr Qtr/Yr Qtr/Yr
03
N02
S02
CO
TSP
Pb
(b) Accuracy Goals
Pollutant
# of Reporting
Organization
Precision
Qtr/Yr Qtr/Yr Qtr/Yr Qtr/Yr
03
N02
S02
CO
TSP
Pb
(c) To the extent possible, describe problems preventing the meeting of
precision and accuracy goals.
SF-20
-------
Section No. 2.0.11
April 30, 1985
Page 54
11.7 Systems Audit Questionnaire (Long-Form)
The long-form systems audit questionnaire which follows is intended to provide a
complete picture of agency ambient air monitoring operations and quality
assurance implementation. The following instructions might prove helpful in
completing this survey questionnaire.
1. For ease in completing the questionnaire, it is not necessary to type.
Filling it out legibly in blank ink is acceptable.
2. Feel free to elaborate on any point or question in the form. Use additional
pages as necessary to give a complete response.
3. Uhen necessary, include copies of documents which will aid in understanding
your response.
4. Please pay careful attention in completing the questionnaire. The
information supplied will have a direct bearing on the conclusions drawn and
recommendations made concerning the evaluation of your organization's
program.
5. The Regional Quality Assurance Coordinator or a member of his staff may be
contacted for assistance in completing the questionnaire.
-------
Section No. 2.0.11
April 30, 1985
Page 55
SYSTEMS AUDIT QUESTIONNAIRE (LONG FORM)
GENERAL INFORMATION
Questionnaire completion date
On-site systems audit date
Reporting period
Agency name and address
Mailing address (if different from above)
Telephone nunber (FTS)
Commercial ( )
Agency Director
Agency QA Officer
Reporting organizations making up this agency
Systems audit conducted by
Affiliation of audit team
Key Personnel: Completed Questionnaire Interviewed
PI anning ___^
Field Operations
Laboratory Operations
QA/QC ~~~~ ~~~~~
Data Management
Reporting
Persons Present during exit interview
LF-1
-------
Section No. 2.0.11
April 30, 1985
Page 56
LONG FORM QUESTIONNAIRE
TABLE OF CONTENTS
NETWORK MANAGEMENT
1. General
2. Network Design and Siting
3. Organization, Staffing and Training
4. Facilities
FIELD OPERATIONS
1. Routine Operations
2. Quality Control
3. Preventive Maintenance
4. Record Keeping
5. Data Aquisition and Handling
LABORATORY OPERATIONS
1. Routine Operations
2. Quality Control
3. Preventive Maintenance
4. Record Keeping
5. Data Aquisition and Handling
6. Specific Pollutants
TSP
Lead
DATA AND DATA MANAGEMENT
1. Data Handling
2. Software Documentation
3. Data Validation and Correction
4. Data Processing
5. Internal Reporting
6. External Reporting
QUALITY ASSURANCE/QUALITY CONTROL
1. Status of Quality Assurance Program
2. Audits and Audit System Traceability
3. National Performance Audit Program (NPAP)
and Additional Audits
4. Documentation and Data Processing Review
5. Corrective Action System
6. Audit Result Acceptance Criteria
PAGE NO,
LF-3
LF-8
LF-12
LF-15
LF-16
LF-20
LF-25
LF-27
LF-28
LF-30
LF-32
LF-37
LF-38
LF-40
LF-42
LF-45
LF-48
LF-51
LF-54
LF-57
LF-63
LF-65
LF-68
LF-69
LF-73
LF-75
LF-77
LF-79
LF-2
-------
Section No. 2.0.11
April 30, 1985
Page 57
A. NETWORK MANAGEMENT
1. GENERAL
(a) Provide an organization chart clearly showing the agency's structure and
its reporting organizations. (Attach sheet(s) as necessary.)
(b) What is the basis for the current structure of the agency's reporting
organizations?
Yes No
Field operations for all local agencies, conducted
by a common team of field operators?
Common calibration facilities are used for all
local agencies?
Precision checks performed by common staff for
all local agencies?
Accuracy checks performed by common staff for
all local agencies?
Data Handling follows uniform procedures for
all local agencies?
Central data processing facilities used for all
reporting?
Traceability of all standards established by
one central support laboratory?
One central analytical laboratory handles all
analyses for manual methods?
(c) Does the agency feel that the data for the reporting organizations it
contains can be pooled?
Yes No Please comment on either answer
(d) Briefly describe any changes which will be made within the agency's
monitoring program the next calendar year.
LF-3
-------
Section No. 2.0.11
April 30, 1985
Page 58
(e) Complete the table below for each of the criteria pollutants monitored
as part of your air monitoring network.
Number of Monitors
NOZC(5 03 TSP
NAMS
SLAMS
(excluding NAMS)
SPM
TOTAL
(f) What is the date of the most current official SLAMS Network
Description?
I. Where is it available for public inspection?
II. Does it include for each site the following?
YES NO
SAROAD Site ID#
Location
Sampling and Analysis Method
Operative Schedule
Monitoring Objective and Scale
of Representativeness
Any Proposed Changes
LF-4
-------
Section No. 2.0.11
April 30, 1985
Page 59
(g) For each of the criteria pollutants, how many modifications (SLAMS
including NAMS) have been made since the last systems audit? (List
the total SLAMS and NAMS)
Date of last systems audit
Number of Monitors
Pollutant Added Deleted Relocated
Sulfur Dioxide
Nitrogen Dioxide
Carbon Monoxide
Ozone
Total Suspended
Particulates
Lead
(h) Briefly discuss changes to the Air Monitoring Network planned for the
next audit period. (Discuss equipment needs in Section B.S.g)
LF-5
-------
Section No. 2.0.11
April 30, 1985
Page 60
(i) Does an overall SLAMS/NAMS Monitoring Plan exist?
Yes No
(j) Has the agency prepared and implemented Standard Operating Procedures
for all facets of agency operation? Yes No
If no, list subject of any missing SOPs
(k) Do the Standard Operating Procedures adequately address at least the
eleven (11) item quality control program required by Appendix A to 40
CFR 58? Yes No Comment
(1) Clearly identify by section number and/or document title, major changes
made to documents since the last on-site review.
Title/Section # Pollutant(s) Affected
LF-6
-------
Section No. 2.0.11
April 30, 1985
Page 61
(m) Does the agency have an implemented plan for operations during emergency
episodes? Yes No Indicate latest revision, approval date and
current location of this plan.
Document Title
Revision Date
Approved
(n) During episodes, are communications sufficient so that regulatory actions
are based on real-time data?
Yes No
(o) Identify the section of the emergency episode plan where quality control
procedures can be found.
LF-7
-------
Section No. 2.0.11
April 30, 1985
Page 62
2- NETWORK DESIGN AND SITING
(a) Indicate by SAROAD Number any non-conformance with the
requirements of 40 CFR 58, Appendices D and E.
Site ID
Monitor (SAROAD) Reason for Non-Conformance
S02
03
CO
N02
TSP
Pb
(b) Please provide the following Information on your previous Network Review
required by 40 CFR 58.20d.
Review performed on: Date
Performed by:
Location and Title of Review Document:
Briefly discuss all problems uncovered by this review.
LF-8
-------
Section No. 2.0.11
April 30, 1985
Page 63
(c) Have MAMS Hard Copy Information Reports (NHCIRs) been prepared and
submitted for all monitoring sites within the network?
Yes No
(d) Does each site have the required information including:
YES NO
SAROAD identification number?
Photographs/si ides to the four cardinal compass
points?
Startup and shutdown dates?
Documentation of instrumentation?
Reasons for periods of missing data?
(e) Who has custody of the current network documentation?
(Name) (Title)
(f) Does the current level of monitoring effort, site placement,
instrumentation, etc., meet requirements imposed by current grant
conditions? Yes No Comment
(g) How often is the network design and siting reviewed?
Date of last review
LF-9
-------
Section No. 2,0.11
April 30, 1985
Page 64
(h) Please provide a summary of the monitoring activities conducted as the
SLAMs/NAMS network by the agency as follows:
I. Monitoring is seasonal for (indicate pollutant and month of high
and low concentrations).
Month(s)
High Low
Pollutant Concentration Concentration Collocated
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
II. Monitoring is year-round for (indicate pollutant)
Pollutant Collocated
Y/N
Y/N
Y/N
Y/N
Y/N
LF-10
-------
Section No. 2.0.11
April 30, 1985
Page 65
(i) Does the number of collocated monitoring sites meet the requirements of
40 CFR 58 Appendix A?
Yes No Comment
(j) Does your agency monitor and/or analyze for non-criteria air and/or toxic
air pollutants? Yes No
If yes, please complete the form below.
Monitoring SOP Available
Pollutant Method/Instrument Yes/No
LF-11
-------
Section No. 2.0.11
April 30, 1985
Page 66
3. ORGANIZATION. STAFFING AND TRAINING
(a) Please indicate the key individuals responsible for the following:
Agency Director
SLAMS Network Manager
Quality Assurance Officer
Field Operations Supervisor
Laboratory Supervisor
Data Management Supervisor
SLAMS Reporting Supervisor
(b) Please indicate the number of people available to each of the following
program areas:
Program Area Nunber
Comment on Need for
Additional Personnel
Network Design
and Siting
Resources and
Facilities
Data and Data
Management
QA/QC
LF-12
-------
Section No. 2.0.11
April 30, 1985
Page 67
(c) Does the agency have an established training program?
Yes No
I. Where is this documented?
(rev date)
II. Does it make use of seminars, courses, EPA sponsored college level
courses? Yes No
III. Indicate below the three (3) most recent training events and
identify the personnel participating in them?
Event Dates Participant(s)
LF-13
-------
Section No. 2.0.11
April 30, 1985
Page 68
(d) Does the agency subscribe to recognized publications? Please provide
list of periodicals. Are periodicals available to all personnel?
Periodical Title Distribution
LF-14
-------
Section No. 2.0.11
April 30, 1985
Page 69
4. FACILITIES
(a) Identify the principal facilities where the work is performed which is
related to the SLAMs/NAMS network? (Do not include monitoring
sites but do include any work which is performed by contract or other
arrangements).
Facility Location Main SLAMS/NAMS Function
(b) Please review the entries on the above table. Are there any areas of
facilities which you believe should be upgraded? Please identify by
location.
(c) Are there any significant changes which are likely to be implemented to
agency facilities before the next systems audit? Comment on your
agency's needs for additional physical space (laboratory, office,
storage, etc.)
Facility Function Proposed Change - Date
LF-15
-------
Section No. 2.0.11
April 30, 1985
Page 70
B. FIELD OPERATIONS
1. ROUTINE OPERATIONS
(a) Is the documentation of Monitoring Standard Operating Procedures complete?
Yes No
Please complete the table below.
Pollutant
Monitored Date of Last Revision
TSP
Pb
S02
(continuous)
N02
S02
(bubblers)
N02
03
CO
Others (list by pollutant)
(b) Are such procedures available to all field operations personnel?
Yes No Comment
(c) Are standard operating procedures prepared and available to field
personnel which detail operations during episode monitoring?
Yes No Comment
LF-16
-------
Section No. 2.0.11
April 30, 1985
Page 71
(d) For what does each reporting organization within the agency monitor?
Provide the list requested below.
Reporting Organization # of Sites Pollutants
(e) On the average, how often are most of your sites visited by a field
operator? per
(f) Is this visit frequency consistent for all reporting organizations within
your agency? Yes No
If no, document exceptions
(g) On the average, how many sites does a single site operator have
responsibility for?
(h) How many of the sites of your SLAMS/NAMS network are equipped with
manifold(s) #
I. Briefly describe most common manifold type.
II. Are manifolds cleaned periodically? Yes No
If yes, how often? per
LF-17
-------
Section No. 2.0.11
April 30, 1985
Page 72
III. If the manifold is cleaned, what is used?
IV. Are manifold!s) equipped with a blower? Yes No
V. Is there sufficient air flow through the manifold at all times?
Yes _ No
Approximate air flow is ^ .
{flow units)
VI. Is there a conditioning period for the manifold after cleaning?
Briefly comment on the length of time the conditioning is performed.
(i) What material is used for instrument lines?
(j) Has the agency obtained necessary waiver provisions to operate equipment
which does not meet the effective reference and equivalency requirements?
Yes No
Comment on Agency use of approved/non-approved instrumentation.
LF-18
-------
Section No. 2.0.11
April 30, 1985
Page 73
(k) Please complete the table below to indicate which analyzers do not conform
with the requirements of 40 CFR 53 for NAMS, SLAMS, or SIP related SPM's.
Si te Comment on
Pollutant Number Make/Model Identification Variances
CO
SO 2
N02
03
TSP
(1) Please comment briefly and prioritize your currently identified
instrument needs.
LF-19
-------
Section No. 2.0.11
April 30, 1985
Page 74
2. QUALITY CONTROL
(a) Are field calibration procedures included in the documented Standard
Operating Procedures? Yes No
Comment on location (site, lab, office) of such procedures
(b) Are multipoint calibrations performed? Indicate both the frequency and
pollutant.
Reporting Organization Pollutant Frequency
(c) Are calibrations performed in keeping with the guidance offered in
Section 2.0.9 Vol. II of the Quality Assurance Handbook for Air Pollution
Measurement Systems? Yes No
If no, why not?
(d) Are calibration procedures consistent with the operational requirements
of Appendices to 40 CFR 50 or to analyzer operation/instruction manuals?
Yes No
If no, briefly explain deviations
LF-20
-------
Section No. 2.0.11
April 30, 1985
Page 75
(e) Have changes been made to calibration methods based on manufacturer's
suggestions for a particular instrument? Yes No
Are these also documented? Yes No
(f) Do standard materials used for calibrations meet the requirements of the
appendices to 40 CFR 50 (EPA reference methods) and Appendix A to 40 CFR
58 (traceability of materials to NBS-SRMs or CRMs)? Yes No
Comment on deviations
(g) Are all flow-measurement devices checked and certified?
Yes No Comment
(h) What are the authoritative standards used for each type of flow
measurement? Please list them in the table below, indicate the frequency
of calibration standards to maintain field material/device credibility.
Flow Devices Primary Standard Frequency of Calibration
(i) Where do field operations personnel obtain gaseous standards?
Are those standards certified by: YES NO
The agency laboratory?
EPA/EMSL/RTP standards laboratory?
A laboratory separate from this agency but
part of the same reporting organization?
The vendor?
NBS?
LF-21
-------
Section No. 2.0.11
April 30, 1985
Page 76
(j) Does the documentation include expiration date of certification?
Yes No
Reference to primary standard used? Yes No
What traceability protocol is used?
Please attach an example of recent documentation of traceability (tag,
1 abel, 1 og sheet).
(k) Is calibration equipment maintained at each site? Yes No
For what pollutants?
(1) How is the functional integrity of this equipment documented?
(m) Please complete the table below for your agency's site standards (up to 7%
of the sites, not to exceed 20 sites).
Primary Secondary RecertiTi cation
Parameter Standard Standard Date
CO
N02
S02
03
LF-22
-------
Section No. 2.0.11
April 30, 1985
Page 77
(n) Are level 1 zero and span (z/s) calibrations (or calibration checks) made
for all continuous monitoring equipment and flow checks made for TSP
samplers? Yes No
Please complete table below:
Span Cone.
Pollutant (ppm) Frequency
I. Continuous analyzers
Flow Rate Frequency
II. TSP Samplers
(o) Does the agency have acceptance criteria for zero/span checks?
Yes No Comment
I. Are these criteria known to the field operations personnel?
Yes No
II. Are they documented in standard operating procedures?
Yes No
If not, indicate document and section where they can be found.
LF-23
-------
Section No. 2.0.11
April 30, 1985
Page 78
III. Do the documents discussed in (ii) above indicate when zero/span
adjustments should and should not be made? Yes No
Indicate an example
IV. Are zero and span check control charts maintained? Yes No
(p) In keeping with 40 CFR 58 regulations, are any necessary zero and span
adjustments made after precision checks? Yes No
If no, comment on why not
(q) Are precision check control charts maintained? Yes No
(r) Who has the responsibility for performing zero/span checks?
(s) Are precision checks routinely performed within concentration ranges and
with a frequency which meet or exceed the requirements of 40 CFR 58,
Appendix A? Yes No
Please comment on any discrepancies.
(t) Please identify person(s) with the responsibility for performance of
precision checks on continuous analyzers?
Person(s)
Ti tl e
LF-24
-------
Section No. 2.0.11
April 30, 1985
Page 79
3. PREVENTIVE MAINTENANCE
(a) Has the field operator been given any special training in performing
preventive maintenance? Briefly comment on background and/or courses
(b) Is this training routinely reinforced? Yes No
If no, why not?
(c) If preventive maintenance i s MINOR, it is performed at (check one or
more): field site , headquarters facilities , equipment is sent
to manufacturer .
(d) If preventive maintenance is MAJOR, it is performed at (check one or
more): field site , headquarters facilities , equipment is sent
to manufacturer .
(e) Does the agency have service contracts or agreements in place with
instrument manufacturers? Indicate below or attach additional pages to
show which instrumentation is covered.
(f) Comment briefly on the adequacy and availability of the supply of spare
parts, tools and manuals available to the field operator to perform any
necessary maintenance activities. Do you feel that this is adequate to
prevent any significant data loss?
LF-25
-------
Section No. 2.0.11
April 30, 1985
Page 80
(g) Is the agency currently experiencing any recurring problem with equipment
or manufacture^ s) ? If so, please identify the equipment and/or
manufacturer, and comment on steps taken to remedy the problem.
LF-26
-------
Section No. 2.0.11
April 30, 1985
Page 81
4. RECORDKEEPING
(a) Is a log book(s) maintained at each site to document site visits,
preventive maintenance and resolution of site operational problems and
corrective actions taken? Yes No Other uses
(b) Is the logbook maintained currently and reviewed periodically?
Yes No Frequency of Review
(c) Once entries are made and all pages filled, is the logbook sent to the
laboratory for archiving? Yes No
If no, is it stored at other location (specify)
(d) What other records are used? YES NO
Zero/span record?
Gas usage log?
Maintenance log?
Log of precision checks?
Control charts?
A record of audits?
Please describe the use and storage of these documents.
(e) Are calibration records or at least calibration constants available to
field operators? Yes No Please attach an example field
calibration record sheet to this questionnaire.
LF-27
-------
Section No. 2.0.11
April 30, 1985
Page 82
5. DATA ACQUISITION AND HANDLING
(a) With the exception of TSP, are instrunent outputs (that is data) recorded
to (a) stripcharts, (b) magnetic tape acquisition system (c) digitized
and telemetered directly to agency headquarters? Please complete the
table below for each of the reporting organizations, or agencies within
the overall R.O.
Data Acquisition Media
Reporting Organization Pollutants (a, b, c or combination)
(b) Is there stripchart backup for all continuous analyzers? Yes No
(c) Where is the flow of high-volume samplers recorded at the site?
For samplers with flow controllers? Log sheet , Dixon chart ,
Other (specify)
On High-volume samplers without flow controllers? Log sheet ,
Dixon chart , Other (specify)
(d) What kind of recovery capabilities for data acquisition equipment are
available to the field operator after power outages, storms, etc?
Briefly describe below.
LF-28
-------
Section No. 2.0.11
April 30, 1985
Page 83
(e) Using a summary flow diagram, indicate below all data handling steps
performed at the air monitoring site. Identify the format, frequency and
contents of data submittals to the data processing section. Clearly
indicate points at which flow path differs for different criteria
pollutants. Be sure to include all calibration, zero/span and precision
check data flow paths. How is the integrity of the data handling system
verified?
LF-29
-------
Section No. 2.0.11
April 30, 1985
Page 84
C. LABORATORY OPERATIONS
1. ROUTINE OPERATIONS
(a) What analytical methods are employed in support of your air monitoring
network?
Analysis Methods
TSP
Pb
S04
N03
S02
(bubblers)
N02
Others (list by pollutant)
(b) Are bubblers used for any criteria pollutants in any agencies?
Yes No _^^^ If yes, attach a table which indicates the number of
sites where bubblers are used, the agency and pollutant(s).
(c) Do any laboratory procedures deviate from the reference, equivalent, or
approved methods? Yes No If yes, are the deviations for lead
analysis , TSP filter conditioning or other— (specify below)?
(d) Have the procedures and/or any changes been approved by EPA? Yes
No Date of Approval
LF-30
-------
Section No. 2.0.11
April 30, 1985
Page 85
(e) Is the documentation of Laboratory Standard Operating Procedures complete?
Yes No . Please complete the table below.
Analysis Methods
TSP
Pb
S04
N03
S02
(bubblers)
N02
Others (list by pollutant)
(f) Is sufficient instrumentation available to conduct your laboratory
analyses? Yes No . If no, please indicate instrumentation
needs in the table below.
Instrument New or Year of
Needed Analysis Replacement Acquisition
LF-31
-------
Section No. 2.0.11
April 30, 1985
Page 86
2. QUALITY CONTROL
(a) Please complete the table for your agency's laboratory standards.
Primary Secondary Recertifi cation
Parameter Standard Standard Date
CO
N02
S02
03
Weights
Temperature
Moisture
Barometric
Pressure
Flow
Lead
Sulfate
Nitrate
VOC
LF-32
-------
Section No. 2.0.11
April 30, 1985
Page 87
(b) Are all chemicals and solutions clearly marked with an indication of shelf
life? Yes No
(c) Are chemicals removed and properly disposed of when shelf life expires?
Yes No
(d) Are only ACS chemicals used by the laboratory? Yes No
(e) Comment on the traceability of chemicals used in the preparation of
calibration standards?
(f) Does the laboratory:
purchase standard solutions such as those for use with lead or other
AA analysis? Yes No
make them themselves? Yes No
if the laboratory staff routinely make their own standard solutions,
are procedures for such available? Yes No Where?
Attach an example.
LF-33
-------
Section No. 2.0.11
April 30, 1985
Page 88
(g) Are all calibration procedures documented? Yes No
Where? ^__^
(title) (revision)
Unless fully documented, attach a brief description of a calibration
procedure.
(h) Are at least one duplicate, one blank, and one standard or spike included
with a given analytical batch? Yes No Identify analyses for
which this is routine operation?
(i) Briefly describe the laboratory's use of data derived from blank analyses?
Do criteria exist which determine acceptable/non-acceptable blank data?
Please complete the table below.
Pollutant Blank Acceptance Criten'a
S02
N02
S04
N03
Pb
TSP
VOC
Other
LF-34
-------
Section No. 2.0.11
April 30, 1985
Page 89
(j) How frequently and at what concentration ranges does the lab perform
duplicate analysis? What constitutes acceptable agreement? Please
complete the table below.
Pollutant Frequency Acceptance Criteria
S02 Bubblers
N02 Bubblers
S04
N03
Pb
TSP
VOC
Other
(k) How does the lab use data from spiked samples? Please indicate what may
be considered acceptable percentage recovery by Analysis? Please
complete the table below.
Pollutant ^ Recovery Acceptance Criten'a
S02 Bubblers
N02 Bubblers
S04
N03
Pb
TSP
VOC
Other
LF-35
-------
Section No. 2.0.11
April 30, 1985
Page 90
(1) Does the laboratory routinely include samples of reference material
obtained from EPA within an analytical batch? Yes No
If yes, indicate frequency, level, and material used.
(m) Are mid-range standards included in analytical batches? Yes No
If yes, are such standards included as a QC check {span check) on
analytical stability? Please indicate the frequency, level and compound
used in the space provided below.
(n) Do criteria exist for "real-time" quality control based on the results
obtained for the mid-range standards discussed above? Yes No
If yes, briefly discuss them below or indicate the document in which they
can be found.
(o) Are appropriate acceptance criteria documented for each type of analysis
conducted? Yes No Are they known to at least the analysts
working with respective instruments?
LF-36
-------
Section No. 2.0.11
April 30, 1985
Page 91
3. PREVENTIVE MAINTENANCE
(a) For laboratory equipment, who has responsibility for major and/or minor
preventive maintenance?
Person Title
(b) Is most maintenance performed:
in the lab? Yes No
in the instrument repair facility? Yes No
at the manufacturer1s facility? Yes No
(c) Is a maintenance log maintained for each major laboratory instrument?
Yes No Comment
(d) Are service contracts in place for the following analytical instruments:
YES NO
Analytical Balance
Atomic Absorption Spectrometer
Ion Chromatograph
Automated Colorimeter
LF-37
-------
Section No. 2.0.11
April 30, 1985
Page 92
4. RECORDKEEPING
(a) Are all samples that are received by the laboratory:
logged-in? Yes No
assigned a unique laboratory sample number? Yes No
routed to the appropriate analytical section? Yes No
Discuss sample routing and special needs for analysis (or attach a copy
of the latest SOP which covers this). Attach a flow chart if possible.
(b) Are logbooks kept for all analytical laboratory instruments?
Yes No
(c) Do these logbooks indicate:
YES NO
analytical batches processed?
quality control data?
calibration data?
results of blanks, spikes and duplicates?
initials of analyst?
LF-38
-------
Section No. 2.0.11
April 30, 1985
Page 93
(d) Is there a logbook which indicates the checks made on:
weights? Yes No
humidity indicators? Yes No
balances? Yes No
thermometer(s)? Yes No
(e) Are logbooks maintained to track the preparation of filters for the field?
Yes No
Are they current? Yes No
Do they indicate proper use of conditioning? Yes No
Weighings? Yes No
Stamping and numbering? Yes No
(f) Are logbooks kept which track filters returning from the field for
analysis? Yes No
(g) How are data records from the laboratory archived?
Where?
Who has the responsibility? Person
Tit! e
How long are records kept? Years
(h) Does a chain-of-custody procedure exist for laboratory samples?
Yes No
(i) Has chain-of-custody been documented and implemented as part of standard
laboratory procedures? Yes No If yes, indicate date, title
and revision number where is can be found.
LF-39
-------
Section No. 2.0.11
April 30, 1985
Page 94
5. DATA ACQUISITION AND HANDLING
(a) Identify those laboratory instruments which make use of computer
interfaces directly to record data? Which ones use stripcharts?
integrators?
(b) Are QC data readily available to the analyst during a given analytical
run? Yes No
(c) For those instruments which are computer interfaced, indicate which are
backed up by stripcharts?
(d) What is the laboratory's capability with regard to data recovery? In
case of problems, can they recapture data or are they dependent on
computer operations? Discuss briefly.
(e) Has a user's manual been prepared for the automated data acquisition
instrumentation? Yes No Comment
Is it in the analyst's or user's possession? Yes No
Is it current? Yes No
LF-40
-------
Section No. 2.0.11
April 30, 1985
Page 95
(f) Please provide below a data flow diagram which establishes, by a short
summary flowchart; transcriptions, validations, and reporting format
changes the data goes through before being released to the data
management group. Attach additional pages as necessary.
LF-41
-------
Section No. 2.0.11
April 30, 1985
Page 96
6. SPECIFIC POLLUTANTS: TSP AND LEAD
TSP
(a) Are filters supplied by EPA used at SLAMS sites? Yes No
Comment
(b) Do filters meet the specifications in the Federal Register 40 CFR 50?
Yes No Comment
(c) Are filters checked for surface alkalinity? Yes No
Indicate frequency
(d) Are filters visually inspected via strong light from a view box for
pinholes and other imperfections? Yes No
If no, comment on way imperfections are determined?
(e) Are filters permanently marked with a serial number? Yes No
Indicate when and how this is accomplished:
(f) Are unexposed filters equilibrated in controlled conditioning environment
which meets or exceeds the requirements of 40 CFR 50? Yes No
If no, why not?
(g) Is the conditioning environment monitored? Yes No
Indicate frequency
Are the monitors properly calibrated? Yes No
Indicate frequency
LF-42
-------
Section No. 2.0.11
April 30, 1985
Page 97
(h) Is the balance checked with Class "S" weights each day it is used?
Yes No If no, indicate frequency of such checks
(i) Is the balance check information placed in QC logbook? Yes No
If no, where is it recorded?
(j) Is the filter weighed to the nearest milligram? Yes No
If not, what mass increment
(k) Are filter serial numbers and tare weights permanently recorded in a
bound notebook? Yes No
If no, indicate where
(1) Are filters packaged for protection while transporting to and from the
monitoring sites? Yes No
(m) How often are filter samples collected? (Indicate average lapse time
(hrs.) between end of sampling and laboratory receipt.)
(n) Are field measurements recorded in logbook or on filter folder?
(o) Are exposed filters reconditioned for at least 24 hrs in the same
conditioning environment as for unexposed filters? Yes No _
If no, why not?
(p) Are exposed filters removed from folders, etc., before conditioning?
Yes No
LF-43
-------
Section No. 2.0.11
April 30, 1985
Page 98
(q) Is the exposed filter weighed to the nearest milligram? Yes No
(r) Are exposed filters archived? Yes No When?
Where?
Indicate retention period
(s) Are blank filters reweighed? Yes No If no, explain why not.
If yes, how frequently?
(t) Are analyses performed on filters? Yes No . Indicate analyses
other than Pb and mass which are routinely performed.
(u) Are sample weights and collection data recorded in a bound laboratory
logbook? Yes No On data forms? Yes No
(v) Are measured air volumes corrected to reference conditions as given in
CFR regulations (Qstd of 760 mm Hg and 25oC) prior to calculating
the Pb concentration? Yes No
If not, indicate conditions routinely employed for both internal and
external reporting
LF-44
-------
Section No. 2.0.11
April 30, 1985
Page 99
LEAD
(a) Is analysis for lead being conducted using atomic absorption spectrometry
with air acetylene flame? Yes No
If not, has the agency received an equivalency designation of their
procedure.
(b) Is either the hot acid or ultrasonic extraction procedure being followed
precisely? Yes No Which?
(c) Is Class A borosilicate glassware used throughout the analysis?
Yes No
(d) Is all glassware scrupulously cleaned with detergent, soaked and rinsed
three times with distilled-deionized water? Yes No
If not, briefly describe or attach procedure.
(e) If extracted samples are stored, are linear polyethlyene bottles used?
Yes No Comment
(f) Are all batches of glass fiber filters tested for background lead content?
Yes No At a rate of 20 to 30 random filters per batch of 500
or greater? Yes No Indicate rate
(g) Are ACS reagent grade HN03 and HC1 used in the analysis? Yes
No If not, indicate grade used
LF-45
-------
Section No. 2.0.11
April 30, 1985
Page 100
(h) Is a calibration curve available having concentrations that cover the
linear absorption range of the atomic absorption instrumentation?
Yes No Briefly describe
(i) Is the stability of the calibration curve checked by alternately
remeasuring every every 10th sample a concentration =1 ug Pb/ml;
=10 ug Pb/ml? Yes No If not, indicate frequency.
(j) Are measured air volumes corrected to reference conditions as given in
CFR regulations (Qstd of 760 mm Hg and 25oC) prior to calculating the
Pb concentration? Yes No If not, indicate conditions
routinely employed for both internal and external reporting.
(k) In either the hot or ultrasonic extraction procedure, is there always a
30-min H20 soaking period to allow HN03 trapped in the filter to
diffuse into the rinse water? Yes No Comment
LF-46
-------
Section No. 2.0.11
April 30, 1985
Page 101
(1) Is a quality control program in effect that includes periodic
quantification of (1) lead in 3/4" x 8" glass fiber filter strips
containing 100-300 ug Pb/strip, and/or (2) a similar strip with 600-1000
ug strip, and (3) blank filter strips with zero Pb content to determine
if the method, as being used, has any bias? Yes No
Comment on lead QC program or attach applicable SOP.
(m) Are blank Pb values subtracted from Pb samples assayed? Yes No
If not, explain why.
LF-47
-------
Section No. 2.0.11
April 30, 1985
Page 102
D. DATA AND DATA MANAGEMENT
1. DATA HANDLING
(a) Is there a procedure, description, or a chart which shows a complete data
sequence from point of acquisition to point of submission of data to EPA?
Yes No
Please provide below a data flow diagram indicating both the data
flow within the reporting organization and the data received from the
various local agencies.
LF-48
-------
Section No. 2.0.11
April 30, 1985
Page 103
(b) Are data handling and data reduction procedures docunented?
For data from continuous analyzers? Yes No
For data from non-continuous methods? Yes No
(c) In what format and medium are data submitted to data processing section?
Please provide separate entry for each reporting organization.
Reporting
Organization Data Medium Format
(d) How often are data received at the processing center from the field sites
and laboratory?
at least once a week?
every 1-2 weeks?
once a month?
(e) Is there documentation accompanying the data regarding any media changes,
transcriptions, and/or flags which have been placed into the data before
data are released to agency internal data processing? Describe.
LF-49
-------
Section No. 2.0.11
April 30, 1985
Page 104
(f) How are the data actually entered to the computer system? Digitization
of stripcharts? Manual or computerized transcriptions? Other?
(g) Is a double-key entry system used for data at the processing center? Are
duplicate card decks prepared? Yes No If no, why not?
(h) Have special data handling procedures been adopted for air pollution
episodes? Yes No If yes, provide brief description.
LF-50
-------
Section No. 2.0.11
April 30, 1985
Page 105
2. SOFTWARE DOCUMENTATION
(a) Does the agency have available a copy of the AEROS Manual?
Yes No Comment
(b) Does the agency have the PARS user's guide available? Yes No
Comment (provide guide #)
(c) Does the Data Management Section have complete software documentation?
Yes No Comment
If yes, indicate the implementation date and latest revision dates for
such documentation.
(d) Do the documentation standard follow the guidance offered by the EPA
Software Docunentation Protocols? Yes No
If no, what protocols are they based on?
LF-51
-------
Section No. 2.0.11
April 30, 1985
Page 106
(e) What is the origin of the software used to process air monitoring data
prior to its release into the SAROAD/NADB database?
I. Purchased? Yes No ; Supplier
Date of latest version
II. Written in-house? Yes No ; Latest version
Date
III. Purchased with modifications in-house? Yes No ;
Latest version Date
IV. Other (specify)
(f) Is a user's manual available to data management personnel for all software
currently in use at the agency for processing SLAMS/NAMS data?
Yes No Comment
(g) Is there a functional description either:
included in the user's manual? Yes No
separate from it and available to the users? Yes No
(h) Are the computer system contents, including ambient air monitoring data,
backed up regularly? Briefly describe, indicating at least the media,
frequency, and backup-media storage location.
LF-52
-------
Section No. 2.0.11
April 30, 1985
Page 107
(1) What is the recovery capability (how much time and data would be lost) in
the event of a significant computer problem?
(j) Are test data available to evaluate the integrity of the software?
Yes No Is it properly documented? Yes No
LF-53
-------
Section No. 2.0.11
April 30, 1985
Page 108
3. DATA VALIDATION AND CORRECTION
(a) Have validation criteria, applicable to all pollutant data processed by
the reporting organization been established and documented? Yes
No
If yes, indicate document where such criteria can be found (title,
revision date).
(b) Does documentation exist on the identification and applicability of flags
(i.e. identification of suspect values) within the data as recorded with
the data in the computer files? Yes No
(c) Do documented data validation criteria employed address limits on and for
the following:
I. Operational parameters, such as flow rate measurements or flow rate
changes.
II. Calibration raw data, calibration validation and calibration
equipment tests.
III. All special checks unique to a measurement system
IV. Tests for outliers in routine data as part of screening process
V. Manual checks such as hand calculation of concentrations and their
comparison with computer-calculated data
LF-54
-------
Section No. 2.0.11
April 30, 1985
Page 109
(d) Are changes to data submitted to NADB documented In a permanent file?
Yes No If no, why not?
(e) Are changes performed according to a documented Standard Operating
Procedure or your Agency Quality Assurance Project Plan? Yes No _
If not according to the QA Project Plan, please attach a copy of your
current Standard Operating Procedure.
(f) Who has signature authority for approving corrections?
(name)(Program Function)
(g) Are data validation summaries prepared at each critical point in the
measurement process or information flow and forwarded with the applicable
block of data to the next level of validation? Yes No
Please indicate the points where such summaries are performed.
(h) What criteria are applied for data to be deleted? Discuss briefly.
LF-55
-------
Section No. 2.0.11
April 30, 1985
Page 110
(i) What criteria are applied to cause data to be reprocessed? Discuss.
(j) Is the group supplying data provided an opportunity to review data and
correct erroneous entries? Yes No If yes, how?
(k) Are correct data resubmitted to the issuing group for cross-checking
prior to release? Yes No
LF-56
-------
Section No. 2.0.11
April 30, 1985
Page 111
4. DATA PROCESSING
(a) Does the agency generate data summary reports? Yes No
Are the data used for in-house distribution and use? Yes No
Publication? Yes No
Other (specify)
(b) Please list at least three (3) reports routinely generated, providing the
information requested below.
Distribution Period Covered
(c) Have special procedures been instituted for pollution index reporting?
Yes No If yes, provide brief description.
(d) Who at the agency has the responsibility for submitting data to SAROAD/
NADB? (name) (title)
Is the data reviewed and approved by an officer of the agency prior to
submittal? Yes No
(name) (title)
LF-57
-------
Section No. 2.0.11
April 30, 1985
Page 112
(e) Are those persons different from the individuals who submit data to PARS?
Yes No If yes, provide name and title of individual
responsible for PARS data submittal.
(name) (title) PARS
Data review and approval (name)
(title)
(f) How often are data submitted to:
SAROAD?
PARS?
(g) How and/or in what form are data submitted?
TO SAROAD?
TO PARS?
(h) Are the recommendations and requirements for data coding and submittal,
in the AEROS User's Manual followed closely for SAROAD? Yes No
Comment on any routine deviations in coding procedures.
(i) Are the recommendations and requirements for data coding and submittal,
in the PARS User's Guide, followed closely? Yes No Comment
on any routine deviations in coding and/or computational procedures.
LF-58
-------
Section No. 2.0.11
April 30, 1985
Page 113
(j) Does the agency routinely request a hard copy printback on submitted data;
from SAROAD/NADB? Yes No
from PARS? Yes No
(k) Are records kept for at least 3 years by the agency in an orderly,
accessible form? Yes No
If yes, does this include raw data , calculation , QC data , and
reports ? If no, please comment.
(1) In what format are data received at the data processing center? (Specify
appropriate pollutant.)
(a) concentration units (b) % chart (c) voltages (d) other;
(m) Do field data include the following documentation?
Site ID? Yes No
Pollutant type? Yes No
Date received at the center? Yes No
Collection data {flow, time, date)? Yes No
Date of Laboratory Analysis (if applicable) Yes No
Operator/Analyst? Yes No
(n) Are the appropriate calibration equations submitted with the data to the
processing center? Yes No If not, explain.
LF-59
-------
Section No. 2.0.11
April 30, 1985
Page 114
(o) Provide a brief description of the procedures and appropriate formulae
used to convert field data to concentrations prior to input into the data
bank.
S02
N02
CO
03
TSP
CH4/THC
LF-60
-------
Section No. 2.0.11
April 30, 1985
Page 115
Pb
Other
(p) Are all concentrations corrected to EPA standard (298oK, 760 mm Hg)
temperature and pressure condition before input to the SAROAD?
Yes No If no, specify conditions used
(q) Are data reduction audits performed on a routine basis? Yes No
If yes,
at what frequency?
are they done by an independent group?
(r) Are there special procedures available for handling and processing
precision, accuracy, calibrations and span checks? Yes No
If no, comment
If yes, provide a brief description:
Span check data
Calibration data
LF-61
-------
Section No. 2.0.11
April 30, 1985
Page 116
Precision data
Accuracy data
(s) Are precision and accuracy data checked each time they are recorded,
calculated or transcribed to ensure that incorrect values are not
submitted to EPA? Yes No Please comment and/or provide a brief
description of checks performed.
(t) Is a final data processing check performed prior to submission of any
data? Yes No
If yes, document procedure briefly
If no, explain
LF-62
-------
Section No. 2.0.11
April 30, 1985
Page 117
5. INTERNAL REPORTING
(a) What reports are prepared and submitted as a result of the audits required
under 40 CFR 58 Appendix A?
Report Title Frequency
(Please include an example audit report and, by attaching a coversheet,
identify the distribution such reports are given within the agency.)
(b) What internal reports are prepared and submitted as a result of precision
checks also required under 40 CFR 58 Appendix A?
Report Title Frequency
(Please include an example of a precision check report and, identify the
distribution such reports receive within the agency.)
(c) Do either the audit or precision reports indicated include a discussion
of corrective actions initiated based on audit or precision results?
Yes No If yes, identify report(s) and section numbers.
LF-63
-------
Section No. 2.0.11
April 30, 1985
Page 118
(d) Does the agency prepare Precision and Accuracy summaries other than
Form 1? Yes No If no, please attach examples of recent
summaries including a recent Form 1.
(e) Who has the responsibility for the calculation and preparation of data
summaries? To whom are such P and A summaries delivered?
Name Title Type of Report Recipient
(f) Identify the individual within the agency who receives the results of the
agency's participation in the NPAP and the internAL distribution of the
results once received.
Principal Contact for NPAP is (name, title)
Di stribution is
(name) (title)
LF-64
-------
Section No. 2.0.11
April 30, 1985
Page 119
6. EXTERNAL REPORTING
(a) For the current calendar year or portion thereof which ended at least 135
calendar days prior to the receipt of this questionnaire, please provide
the following percentages for required data submitted.
% Submitted on Time*
Monitoring
Qtr.
$02 CO 03 N02 TSP Pb
1
(Jan. 1-March 31)
2
(Apr. 1-Oune 30)
3
(July 1-Sept. 30)
4
(Oct. 1-Dec. 31)
* "On-Time" = within 135 calendar days after the end of the quarter in
which the data were collected.
(b) Identify the individual within the agency with the responsibility for
preparing the required 40 CFR 58 Appendix F and G reporting inputs.
Name Ti tl e
(c) Identify the individual within the agency with the responsibility for
reviewing and releasing the data.
Name Ti tl e
LF-65
-------
Section No. 2.0.11
April 30, 1985
Page 120
(d) Does the agency regularly report the Pollutant Standard Index (PSD?
Briefly describe the media, coverage, and frequency of such reporting.
(e) What fraction of the SLAMS sites (by pollutant) reported less than 75* of
the data (adjusted for seasonal monitoring and site start-ups and
terminations)?
FY
Percent of Sites
Pollutant <75% Data Recovery
1st
Quarter
2nd
Quarter
3rd
Quarter
4th
Quarter
Ozone
Nitrogen Dioxide
Sulfur Dioxide
Carbon Monoxide
Total Suspended
Particulates
Lead
LF-66
-------
Section No. 2.0.11
April 30, 1985
Page 121
(f) Does the agency's annual report (as required in 40 CFR CFR 58.26) include
the following?
YES NO
Data summary required in Appendix F.
Annual precision and accuracy information
described in Section 5.2 of Appendix A.
Location, date, pollution source and duration
of all episodes reaching the significant harm
levels.
Certification by a senior officer in the State
or his designee.
(g) Please provide the dates at which the annual reports have been submitted
for the last 2 years.
LF-67
-------
Section No. 2.0.11
April 30, 1985
Page 122
E. QUALITY ASSURANCE/QUALITY CONTROL
1. STATUS OF QUALITY ASSURANCE PROGRAM
(a) Does the agency have an EPA-approved quality assurance program plan?
Yes No
If yes, have changes to the plan been approved by the EPA? Yes No
Please provide:
Date of Original Approval
Date of Last Revision
Date of Latest Approval
(b) Do you have any revisions to your QA Program Plan still pending?
Yes No
(c) Is the QA Plan fully implemented? Yes No Comment:
(d) Are copies of QA Plan or pertinent sections available to agency personnel?
Yes No If no, why not?
(e) Which individuals routinely receive updates to QA Plan?
LF-68
-------
Section No. 2.0.11
April 30, 1985
Page 123
2. AUDITS AND AUDIT SYSTEM TRACEABILITY
(a) Does the agency maintain a separate audit/calibration support facility
laboratory? Yes No
(b) Has the agency documented and implemented specific audit procedures?
Yes No
(c) Have audit procedures been prepared in keeping with the requirements of
Appendix A to 40 CFR 58? Yes No
If no, comment on any EPA approved deviations
(d) Do the procedures meet the specific requirements for independent standards
and the suggestions regarding personnel and equipment? Yes No
Comment:
(e) Are SRM or CRM materials used to routinely certify audit materials?
Yes No
(f) Does the agency routinely use NBS-SRM or CRM materials? Yes No
For audits only? For calibrations only? For both? For
neither, secondary standards are employed .
LF-69
-------
Section No. 2.0.11
April 30, 1985
Page 124
(g) Please complete the following table to summarize auditing method for
CO, N02, 02, 03 analyzers, and High-Volume Samplers.
Audit
Pollutants Audit Method Standard
CO
03
N02
S02
N02
(continuous)
(bubblers)
S02
TSP
LF-70
-------
Section No. 2.0.11
April 30, 1985
Page 125
(h) Are SRM or CRM materials used to establish traceability of calibration
and zero/span check materials provided to field operations personnel?
Yes No
(i) Specifically for gaseous standards, how is the traceability of audit
system standard materials established? Are they:
purchased certified by the vendor?
certified by the QA support laboratory which is part of this agency?
Other? (Please comment briefly below).
(j) Are all agency traceability and standardization methods used documented?
Yes No Indicate document where such method can be found.
LF-71
-------
Section No. 2.0.11
April 30, 1985
Page 126
(k) Do the traceability and standardization methods conform with the guidance
of Section 2.0.7 Vol. II of the Handbook for Air Pollution Measurement
Systems?
For permeation devices? Yes No
For cylinder gases? Yes No
(1) Does the agency have identifiable auditing equipment (specifically
intended for sole use) for audits?
Yes No If yes, provide specific identification
(m) How often is auditing equipment certified for accuracy against standards
and equipment of higher-authority?
(n) As a result of the audit equipment checks performed, have pass/fail
(acceptance criteria) been decided for this equipment? Indicate what
these criteria are with respect to each pollutant. Where are such
criteria documented?
Pollutant Criteria
LF-72
-------
Section No. 2.0.11
April 30, 1985
Page 127
3. NATIONAL PERFORMANCE AUDIT PROGRAM (NPAP) AND ADDITIONAL AUDITS
(a) Identify the individual with primary responsibility for the required
participation in the National Performance Audit Program.
For gaseous materials? (name, title)
For laboratory materials? (name, title)
(b) Does the agency currently have in place any contracts or similar
agreements either with another agency or outside contractor to perform
any of the audits required by 40 CFR 58?
Yes No Comment
If yes, has the agency included QA requirements with this agreement?
Yes No
Is the agency adequately familiar with their QA program?
Yes No
(c) Date last systems audit was conducted?
By whom?
LF-73
-------
Section No. 2.0.11
April 30, 1985
Page 128
(d) Please complete the table below
Parameter Audi tea
Date of Last NPAP
S02 (Continuous)
CO
Pb
ReF Device
S02 (bubbler)
N02 (bubbler)
(e) Does the agency participate in the National Performance Audit Program
(NPAP) as required under 40 CFR 58 Appendix A? Yes No
If no, why not? Summarize below.
LF-74
-------
Section No. 2.0.11
April 30, 1985
Page 129
4. DOCUMENTATION AND DATA PROCESSING REVIEW
(a) Does the agency periodically review its record-keeping activities?
Yes No
Please list below areas routinely covered by this review, the date of the
last review, and changes made a a direct result of the review.
Area/Function Date of Review Changes? Discuss Changes
Y/N
Y/N
Y/N
Y/N
Y/N
(b) Are data audits (specific re-reductions of stripcharts or similar
activities) routinely performed for criteria pollutant data reported by
the agency? Yes No
If no, please explain.
(c) Are procedures for such data audits documented? Yes No
LF-75
-------
Section No. 2.0.11
April 30, 1985
Page 130
(d) Are they consistent with the recommendations of Sections 2.3-2.9 of
Vol. II of the QA Handbook for Air Pollution Measurement Systems?
Yes No If no, why not?
(e) What is the frequency and level (as a percentage of data processed) of
these audits?
Poll. Audit Freq. Period of Data Audited % of Data Rechecked
(f) Identify the Criteria for acceptable/non-acceptable result from a data
processing audit for each pollutant, as appropriate.
Pollutant Acceptance Criteria Data Concentration Level
(g) Are procedures documented and implemented for corrective actions based on
results of data audits which fall outside the established limits?
Yes No
If yes, where are such corrective action procedures documented?
LF-76
-------
Section No. 2.0.11
April 30, 1985
Page 131
5. CORRECTIVE ACTION SYSTEM
(a) Does the agency have a comprehensive Corrective Action program in place
and operational? Yes No
(b) Have the procedures been documented? Yes No As a part of the
agency QA Plan? Yes No As a separate Standard Operating
Procedure? Yes No Briefly describe it or attach a copy.
(c) How is responsibility for implementing corrective actions on the basis of
audits, calibration problems, zero/span checks, etc. assigned?
Briefly discuss.
(d) How does the agency follow-up on implemented corrective actions?
LF-77
-------
Section No. 2.0.11
April 30, 1985
Page 132
(e) Briefly describe two (2) recent examples of the ways in which the above
corrective action system was employed to remove a problem area with
I. Audit Results:
II. Data Management:
LF-78
-------
Section No. 2.0.11
April 30, 1985
Page 133
6. AUDIT RESULT ACCEPTANCE CRITERIA
(a) Has the agency established and has it documented criteria to define
agency-acceptable audit results? Yes No
Please complete the table below with the pollutant, monitor and
acceptance criteria.
Pollutant Audit Result Acceptance Criteria
CO
03
N02
(continuous)
S02
N02
(bubblers)
S02
TSP
(b) Were these audit criteria based on, or derived from, the guidance found
in Vol. II of the QA Handbook for Air Pollution Measurement system,
Section 2.0.12? Yes No
If no, please explain.
If yes, please explain any changes or assumptions made in the derivation.
LF-79
-------
Section No. 2.0.11
April 30, 1985
Page 134
(c) What corrective action may be taken if criteria are exceeded? If
possible, indicate two examples of corrective actions taken within the
period since the previous systems audit which are based directly on
the criteria discussed above?
Corrective Action #1
Corrective Action #2
(d) As a goal, the 95 percent probability limits for precision (all
pollutants) and TSP accuracy should be less than +15 percent. At 95
percent probability limits, the accuracy for all other pollutants should
be less than +20 percent. Using a short narrative and a summary table,
compare the reporting organization's performance against these goals over
the last year. Explain any deviations.
NOTE: Precision and accuracy are based on reporting organizations; therefore
this question concerns the reporting organizations that are the responsibility
of the agency. A copy of a computer printout has been provided which contains
the precision and accuracy data submitted to EMSL for each of the agency's
reporting organizations. The printout, containing at least the last four
completed calendar quarters of precision and accuracy data, was obtained using
the NADB program NA273. This data should be verified using agency records.
If found in error, please initiate corrections. Based on the data
provided or corrections thereto, complete the tables below indicating
the number of reporting organization s meeting the goal stated above
for each pollutant by quarter.
(Report level 2 checks unless otherwise directed by Regional Office.)
LF-80
-------
I. Precision Goals
Section No. 2.0.11
April 30, 1985
Page 135
# of Reporting
Pollutant Organization
Precision
Qtr/Yr Qtr/Yr Qtr/Yr Qtr/Yr
03
N02
S02
CO
TSP
Pb
II. Accuracy Goals
# of Reporting
Pollutant Organization
Precision
Qtr/Yr Qtr/YrQtr/Yr Qtr/Yr
03
N02
S02
CO
TSP
Pb
LF-81
-------
Section No. 2.0.11
April 30, 1985
Page 136
(e) To the extent possible, describe problems preventing the meeting of
precision and accuracy goals.
LF-82
-------
Section No. 2.0.11
April 30, 1985
Page 137
11.8 Bibliography
Guideline documents for the SLAMS Air Program, arranged in descending
chronological order, the most recent ones first.
Reference
EPA-600/4-83-023
June 1983
EPA-600/7-81-010
May 1981
EPA-QAMS-005/80
December 1980
EPA-600/4-80-030
June 1980
EPA-600/4-79-056
September 1979
EPA-600/4-79-057
September 1979
EPA-600/4-79-019
March 1979
EPA-450/4-79-007
February 1979
EPA-600/4-78-047
August 1978
EPA-450/2-78-037
July 1978
EPA-450/3-78-013
April 1978
EPA-450/3-77-018
December 1977
Report Title
Guideline on the Meaning and Use of
Precision and Accuracy Data Required
by 40 CFR Part 58 Appendices A and B
A Procedure for Establishing Traceability
of Gas Mixtures to Certified National
Bureau of Standards SRMs
Interim Guidelines and Specifications
for Preparing Quality Assurance Project
PI ans
Validation of Air Monitoring Data
Transfer Standards for Calibration of Air
Monitoring Analyzers for Ozone
Technical Asssitance Document for the
Calibration of Ambient Ozone Monitors
Handbook for Analytical Quality Control
in Water and Wastewater Laboratories
Guidance for Selecting TSP Episode
Monitoring Methods
Investigation of Flow Rate Calibration
Procedures Associated with the High
Volume Method for Determination of
Suspended Particulates
Screening Procedures for Ambient Air
Quality Data
Site Selection for the Monitoring of
Photochemical Air Pollutants
Selecting Sites for Monitoring Total
Suspended Particulates
-------
Section No. 2.0.11
April 30, 1985
Page 138
Reference Report Title
EPA-600/4-77-027a QA Handbook for Air Pollution Measurement
May 1977 Systems, Vol. II - Ambient Air Specific
Methods
EPA-450/3-77-013 Optimum Site Exposure Criteria for S02
April 1977 Monitoring
EPA-450/2-76-029 Aeros Manual Series, Vol. II - Aeros
December 1976 User's Manual
EPA-450/2-76-005 Aeros Manual Series, Vol. V - Aeros
April 1976 Manual of Codes
EPA-600/9-76-005 QA Handbook for Air Pollution Measurement
March 1976 Systems, Vol. I - Principles
EPA-450/2-76-001 Aeros Manual Series, Vol. I - Aeros
February 1976 Overview
EPA-450/3-75-077 Selecting Sites for Carbon Monoxide
September 1975 Monitoring
APTD-1132 Quality Control Practices in Processing
March 1973 Air Pollution Samples
47 FR 54912, Dec. 6, 1982; Amendments to reference methods for S02,
48 FR 17355, Apr. 22, 1983 TSP and CO in 40 CFR Part 50 Appendices A,
B, and C
Proposed amendments to 40 CFR Part 58 are pending.
Proposed revision (Handbook, Vol. II, Sections 2.0.7 and 2.0.9 are pending,
-------
Chapter 6
Guidelines for Auditing Motor Vehicle
Emissions Inspection Programs
FY 1986 - FY 1987
TABLE OF CONTENTS
Section Title
1.0 INTRODUCTION AND PURPOSE
2.0 AUDIT TOPICS 6-2
2.1 Program Design 6-3
2.2 Program Operations 6-4
2.3 Economy and Efficiency 6-4
2.4 Program Effectiveness 6-5
3.0 AUDIT PROCESS FOR I/M PROGRAMS 6-6
3.1 Advance Preparation 6-7
3.1.1 Documentation Assembly 6-7
3.1.2 Program Questionnaire 6-8
3.1.3 Notice to Program Officials 6-8
3.2 On-Site Audit Visit 6-9
3.2.1 Interviews 6-11
3.2.2 Records Review 6-12
3.2.3 Inspection Station Visits 6-13
3.2.4 Special Surveys 6-14
3.2.5 Exit Interview 6-15
3.3 Audit Report 6-15
3.4 Follow-up Activities 6-15
APPENDICES
A.* Description of I/M Program Elements 6-17
B. Inspection/Maintenance Program Audit Questionnaire 6-18
C.* Description of On-Site Audit Activities 6-45
D.* Procedures Used in QMS Tampering Surveys 6-45
E.* Instructions and Forms for Audit Activities 6-45
F.* Checklist for Completing the I/M Audit 6-45
Incorporated by reference to FY1985 audit guidelines manual
6-1
-------
Guidelines for Auditing Motor Vehicle
Emissions Inspection Programs
FY 1986 - FY 1987
1.0 INTRODUCTION AND PURPOSE
Motor vehicle emissions inspection/maintenance (I/M) programs are currently
required in over 30 States or localities under Part D of the Clean Air
Act. At present, I/M programs are in operation in 48 urban areas in 28
States across the country affecting over 40 million vehicles. With only
a few exceptions, these programs measure the tailpipe emissions concentra-
tions of subject vehicles; most of them also inspect for evidence of
emission control system tampering or misfueling. In order to ensure that
these programs are administered and operated properly and that they
achieve the desired air quality benefits, the responsible State and local
agencies must continually monitor and evaluate their respective programs.
In addition, EPA has a regulatory responsibility to review these programs
in each State for conformance to policy and SIP commitments.
A major part of EPA's efforts to evaluate I/M programs will be accomplished
through periodic program audits (or program evaluations). The primary
purpose of this report is to present I/M auditing guidelines for use by
EPA personnel involved in these in-depth audits. Both emissions inspections
and tampering/misfueling inspection elements are covered by the guidelines.
The auditing guidelines are designed to ensure that EPA's I/M audits in
the various States are both comprehensive and consistent. The secondary,
but equally important, purpose of this report is to provide additional
information to State and local agencies which would be useful in their
internal I/M audits or program evaluations.
The primary purpose of the I/M audit is to allow EPA to ensure that each
State or locality is implementing and enforcing its I/M program in a
manner consistent with its SIP. This objective must be satisfied during
the audit, either during the audit visit or as part of audit follow-up
activities. As a result of the audit, both EPA and the State or local
agencies involved will be able to determine what, if any, program improve-
ments may be required to allow SIP goals and commitments to be met.
A secondary purpose of the I/M audit is to evaluate the overall effectiveness
and efficiency of each I/M program. Because EPA reviews the design and
operation of I/M programs across the country, EPA has the opportunity to
observe the strengths and successes of the various programs. EPA has
historically tried to serve as a clearinghouse for the transfer of I/M
information among States and localities in order to allow each State/local ity
to learn from others' experiences. As a result of the audit, EPA may be
able to suggest administrative or program modifications which would
increase program effectiveness, program efficiency or both.
2.0 AUDIT TOPICS
The topics reviewed during the audit include, at a minimum, all design
and operational aspects of the I/M program which affect the program's
6-2
-------
ability to meet legal requirements of the SIP and the Clean Air Act. The
starting point is to relate the design of the program in the SIP to its
actual operation in order to evaluate whether applicable laws, rules/
regulations, etc. are being administered properly. Evaluation of the
actual program must include:
1. Whether the program is being adequately enforced, i.e., whether
vehicles are being inspected.
2. Whether inspection standards are adequate.
3. Whether vehicles are being inspected properly, i.e., according to
established procedures, using the proper cutpoints, etc.
4. Whether vehicles identified for repair are being repaired and
repaired effectively.
Another important part of the audit is consideration of factors relating
to program economy and efficiency. In many cases, the costs of an I/M
program can be at least partially offset by cost savings associated with
better fuel economy and other side benefits of proper maintenance.
Another cost consideration relates to how efficiently repairs are being
provided for failed vehicles. Time lost by vehicle owners in complying
with program requirements is a cost which should be minimized. Also,
ensuring that warranty coverage is provided and that owners are made
aware of their warranty rights can help to reduce the costs of I/M repairs
to consumers.
Finally, the audit must focus on program effectiveness. Program effectiveness
is a function of both program design and program operations. A certain
degree of program effectiveness is required for each I/M program and is
committed to in the SIP. The I/M audit will be one means for EPA to
ensure that each I/M program achieves its required level of effectiveness.
In some cases, the I/M audit may also identify ways in which increased
effectiveness could be obtained through minor design or operational
changes.
The discussions which follow are intended to explain the need for each
part of the audit process described below in Section 3.0. The discussions
also give some indications of the considerations EPA will use in evaluating
I/M programs.
2.1 Program Design
Program design parameters which should be reviewed during an audit include
inspection test procedures, emission standards (cutpoints), inspection
station licensing requirements, analyzer specifications and maintenance/
calibration requirements, quality control procedures, audit/surveillance
procedures, internal control systems (quality assurance), enforcement
procedures, vehicle coverage considerations, waiver procedures, consumer
assistance and protection, and mechanics training. Each of these elements
has an effect on overall program effectiveness. Also, many of them have
6-3
-------
an effect on the availability of the emissions performance warranty in a
particular State or locality. Detailed descriptions of each I/M design
element are found in Appendix A.
2.2 Program Operations
One of the critical parts of the program audit is to determine whether
the various elements of the I/M program as designed (and approved in the
SIP) are actually being carried out in the program. Field observations
are necessary to determine whether:
1. Vehicles are being tested properly and the results are being
reported correctly.
2. Emission standards are being properly applied.
3. Licensing requirements are being met.
4. Proper analyzers are being used.
5. Analyzers are being calibrated and maintained.
6. Quality control procedures are being followed.
7. Inspection and other records are being kept properly.
8. Needed data analyses are being done and used effectively.
9. Inspection stations are receiving proper surveillance and
supervision.
10. Waivers are being processed properly.
11. Owners of non-complying vehicles are being identified and
prosecuted according to procedures outlined in the SIP.
12. Failed vehicles are being repaired effectively.
13. Consumer assistance and protection provisions are being administered
properly.
14. Mechanics training is being conducted appropriately.
2.3 Economy and Efficiency
There are several aspects of I/M programs which affect their effectiveness
but which also affect program economy and efficiency. Some of these
factors have a subtle but nonetheless real effect on program costs and
effectiveness.
The primary area with economy/efficiency influences is the quality of I/M
repairs. To help ensure proper, cost-effective I/M repairs, I/M programs
6-4
-------
should direct efforts at both vehicle owners and mechanics. In order to
protect themselves as consumers, owners of failed vehicles need to identify
the kinds of repairs they need, what these repairs should cost, the
normal range of emissions levels expected after proper repair, and their
applicable warranty and waiver rights.
The I/M program must also direct efforts at the repair industry in order
to maximize the quality and minimize the costs of I/M repairs. Mechanics
training programs are a common part of many I/M programs. However,
usually only a small fraction of practicing mechanics ever participate in
the training program. Therefore, to the extent possible, efforts should
be directed at those mechanics who do not go through training to give
them diagnostic information and information about who or where to call to
get more assistance or training. State/local agencies can also help to
improve I/M repairs by having a system for monitoring repair costs
and waiver rates at repair facilities.
In some cases, some false ideas about economies or efficiencies in I/M
programs have evolved. For instance, there are many State/local programs
which have not adopted the quality control or other procedures needed to
allow owners to enforce their emission performance warranty rights.
Doing so might mean, in some cases, spending slightly more funds for
administering the I/M program, but it would eliminate much of the consumer
expense for I/M repairs on 1981 and newer vehicles. Therefore, providing
warranty coverage is a cost efficient practice. Other activities which
can have benefits which exceed their costs are:
1. Repair brochures for owners.
2. Mechanics training.
3. Repair information (manuals) for mechanics.
4. Hotlines for consumers and mechanics.
5. Newsletters for mechanics.
While each of these activities can be designed to be an effective tool,
they must be reviewed carefully to make sure that they are serving the
intended purpose.
2.4 Program Effectiveness
Perhaps the most important purpose of the program audit is to assess
overall program effectiveness. This assessment, however, involves a
judgment collectively based on the evaluations of program SIP design and
actual program operations.
The ultimate test of I/M program effectiveness is the air quality benefit
of the program. Unfortunately, the air quality issue is much too complex
to address in the program audit. The program audit must instead deal
with whether the program is being managed effectively to produce the
6-5
-------
desired emissions reductions. In other words, the audit should determine
whether adequate steps are being taken to enforce the program and to
ensure that vehicles are being inspected properly and effectively repaired.
Each SIP commits to a certain level of effectiveness (i.e., emissions
reductions) from the I/M program. A number of SIP design factors affect
the ability of a program to be effective in reducing emissions including:
vehicle class coverage, model year coverage, geographic coverage, failure
rates, frequency of inspections, waivers, cutpoints, and others. Apparent
deviations from the SIP will be investigated and documented during the
audit. Follow-up action by EPA may include a quantification of emissions
reductions if useful to obtaining program improvements or to reconciling
the SIP with actual practices. A few specific items which may lead EPA
to quantify emissions reductions are discussed in Appendix A and include:
high rates of non-compliance, loose cutpoints for 1981 and newer vehicles,
low model year coverage, and high waiver rates.
3.0 AUDIT PROCESS FOR I/M PROGRAMS
The audit process for I/M programs is comprised of four basic elements:
1. Advance preparation.
2. Audit visit.
3. Audit report.
4. Follow-up actions.
The preparation for the audit allows the auditors to familiarize themselves
with the design and operations of the program under review and to identify
those particular aspects of the program which may need special emphasis
during the audit visit. Proper preparation will allow the auditors to
establish priorities for various audit activities in order to make effective
and efficient use of the limited time available during the audit visit.
It will also reduce the State/local resources needed to cooperate with
EPA in the audit and the disruption of the I/M program itself.
The audit visit is for the purpose of verifying and documenting whether
the program is being properly administered, operated, and enforced, i.e.,
according to established laws, rules and regulations, and procedural
requirements in the SIP. Additionally, the audit visit will allow a
better evaluation of program effectiveness and efficiency.
The audit report must properly and objectively reflect the findings of
the audit, both positive and negative. The audit report should specifically
identify:
1. The strengths of the program.
2. Any identified SIP deficiencies for which program improvements must
be implemented.
6-6
-------
3. EPA recommendations for correction of these deficiencies.
4. Any aspects of the program which need further study; this would
include aspects with potential problems or areas where EPA can
suggest minor modifications which would improve program effectiveness
or efficiency.
5. EPA efforts to coordinate the audit report with the State/local
agencies; this would include a discussion of any improvements or
commitments made by the State/local agencies as a result of the
audit; official comments of the State/local agencies on the draft
audit report should be appended to the audit report.
EPA staff will plan follow-up activities as necessary to encourage and
assist State or local implementation of the improvements discussed in the
audit report. In cases where deficiencies must be corrected, follow-up
audit visits may be necessary after corrective actions have been implemented.
3.1 Advance Preparation
To ensure a successful audit, it is extremely important that the audit
staff adequately prepare themselves for the audit. The auditors assigned
to perform the audit must collectively possess as much knowledge as
possible about the operations of t/M programs in general and about the
specific details of the program under review. The goal of audit preparation
is to learn the basic design of the I/M program, to identify as much as
possible about the operating characteristics of the program, and to
determine as much as possible the potential strengths and weaknesses of
the program.
3.1.1 Documentation Assembly
The first step in preparing for the audit visit is to consolidate documentation
which is immediately available in EPA Regional and headquarters files and
to obtain any missing documentation. This would obviously include the
applicable SIP and related documentation. Each State's SIP must include
a complete description of its I/M program, the program's enabling legislation,
and the I/M rules and regulations. In some cases, the SIP may identify
planned program changes, such as phasing in tighter cutpoints in subsequent
years of the program, and/or I/M program enhancements as "extraordinary"
measures. In many cases, but not necessarily always, SIP I/M documentation
will include procedures manuals for testing and/or quality control,
discussions of data analyses and/or other quality assurance procedures,
and mechanics training programs. Auditors should be careful not to
overrely on SIP materials, however, for these materials can sometimes be
incomplete or outdated. The SIP may not contain operating contracts,
equipment specifications, or procedures manuals which may need to be
obtained separately.
Other important sources of information in preparing for an audit visit
are periodic operating reports on I/M programs, public awareness and
information printings, EPA documentation of previous audits or investigations,
and routine correspondence. Operating data on I/M programs are often
formally reported on a quarterly, semi-annual, or annual basis. These
6-7
-------
data can be helpful in determining whether the program is operating
properly and effectively. For instance, information on the number of
vehicles inspected during the period can be indicative of an enforcement
problem. Also, information on failure rates and waiver rates is useful
and can be indicative of program effectiveness. In addition to formal
reports to agency heads, governors, or legislatures, agencies which
operate I/M programs may be able to provide copies of routine data summaries
prepared for in-house use.
Routine correspondence to and from program officials, citizens, and other
interested parties should be reviewed to determine what, if any, issues
may have already been identified or addressed. It is particularly important
that auditors be aware of any special sensitivities in a specific State
or locality revealed in previous correspondence.
3.1.2 Program Questionnaire
To facilitate audit preparation, an I/M program questionnaire has been
developed. This document contains questions covering the relevant aspects
of an I/M program. Completing the questionnaire will require EPA staff,
with State or local assistance as necessary, to learn and document the
most important design features and operating experiences of the I/M
program in question. In addition, for the first round of in-depth audits
the questionnaire will document some seemingly less important design
features. It will be useful for EPA to have these features documented to
avoid misunderstandings during the audit and for future reference, and
they can be documented at this point with minimal additional effort.
The questionnaire is in two sections, one dealing with design and intended
operating aspects and the other with actual recent operating experience.
EPA Regional Office staff and EPA headquarters staff will collaborate on
completing the questionnaire to the best of their ability based on available
documentation or personal knowledge, referencing the source of information
or documentation for each question if it is not obvious. (If agreeable
to the Region and the State/local agency, the State/local agency may
complete this step.) The Regional Office will then send the questionnaire
to the relevant State or local agencies and ask them to correct and
explain any errors on EPA's part and to provide answers where EPA staff
were unable to do so, if the answers are reasonably available to the
agencies themselves. The agencies may be asked to describe how answers
could be obtained for questions that cannot be readily answered. These
questions will be discussed during the site visit, or earlier by telephone.
It is recommended that the Regional Office send the questionnaire to the
State/local agencies at least 90 days prior to the site visit in order to
allow ample time for review and coordination.
3.1.3 Notice to Program Officials
After audit preparation has progressed to the point where an audit visit
can be scheduled, the EPA Regional Office should send a formal written
notice of the audit to the appropriate State and/or local program officials.
The written notice should give the State/local officials ample lead time
6-8
-------
(about 60 days should be sufficient in most cases) to prepare for the
audit visits, and the audit visits should be scheduled at a mutually
convenient time. The formal notice should also specify, whenever possible,
those individuals who will comprise EPA's audit team and what State/local
organizations should be represented. Finally, the formal notice should
identify any special issues raised during the advance preparation.
In all cases, a copy of the completed program questionnaire will be sent
to the State/local program officials for their official review. The
results of their review could be discussed either prior to or during the
audit visit, whichever is felt to be the most constructive. In some
cases, EPA may request to have certain information not covered by the
questionnaire submitted for review prior to the audit visit in order to
complete audit preparations. These cases would probably be unusual;
however, if they occur, the State/local agency obviously must be given
adequate time to submit the needed information. Any unanswered questions
will be subjects for discussion during the on-site visit.
When agencies in addition to the air planning agency must be included in
the audit, the Regional Office will determine how to secure their involvement
and cooperation, and will notify each of the upcoming site visit. In
addition, the Regional Office will provide notice to the Office of Mobile
Sources of scheduled audit visits at least 60 days in advance.
3.2 On-Site Audit Visit
The purpose of the audit visit is to verify information already available
and to gather new information as needed to satisfy the objectives of the
audit. One objective should always be to identify those areas where EPA
can provide assistance to strengthen I/M programs. Sometimes such assistance
may involve specific aid to a particular State, and at other times it may
involve more general assistance aimed at resolving an overall technical
issue.
The audit visit should be adequately planned to ensure that all needed
activities are conducted within the time constraints involved. Generally,
a two- or three-person EPA audit team should be able to complete the
on-site visit in three or four days, at times working independently,
depending on the size and complexity of the program. These days need not
be consecutive, and the Regional Office may find it desirable to separate
special surveys, records review, inspection station visits, and interviews
with officials.
The audit must be planned and coordinated with State/local agencies in
order to minimize the level of intrusion and disruption of normal program
activities. The level of State/local resources which will need to be
devoted to the I/M audit will vary according to the type and size of the
program. The greatest demand on State/local staff time would be for
initial and exit meetings, each of which may be several hours long and
may involve several State/local personnel. Many audit activities would
involve observations of State/local personnel performing their regular
6-9
-------
functions, such as auditing stations or processing waivers. Therefore,
the audit would not greatly increase the demands on these persons' time.
Except for very large and complex I/M programs, it is expected that the
audit will require no more than about two State/local staff persons' time
for three to four days.
In States with I/M programs operating in multiple urbanized areas, it
will usually be necessary to visit each I/M area in addition to the city
in which program officials and records are located. This is because each
urbanized area will usually have a separate manager, potentially different
enforcement agency practices, and potentially different repair industry
competence and enthusiasm for the I/M program. However, visits to the
other areas can usually be limited to one day, can be separated in time
from the main on-site visit, and may utilize different EPA personnel. It
may also be possible to utilize contractor or other third-party support
for the other areas, if acceptable to State or local officials.
Because a written record of the auditors' work should be retained in the
form of working papers, it is requisite that each auditor keep detailed
notes as to persons contacted or interviewed and information received,
its source, and when received. Sample forms for such notes are included
in Appendix E. These samples are intended to remind the user of the
important information to be recorded. Regionally prepared substitutes
may be used.
Auditors should be alert to situations which do not appear to be in
keeping with program procedures or regulations in the SIP; such situations
should be fully investigated during the audit visit. Auditors should
also be alert to situations that could be indicative of fraud, abuse, or
illegal acts; these situations may be more appropriately reported to
higher level program officials rather than being investigated during the
audit, except to identify the effect that such acts have on the program.
Finally, auditors should be alert to signs that their visit may have
resulted in observed behavior not typical of normal program operations.
Where practical, auditors should select specific facilities and personnel
for observation without advance notification to the State or local agencies.
An important part of the audit visit is an evaluation of the internal
control system applicable to the program, organization(s), and activities
under review. Each State or locality operating an I/M program should
have an established methodology and capability for evaluating program
operations in order to assess program results. Through this system of
administrative controls, the State or locality should continuously or
periodically compare actual program operations to intended design. The
emphasis in this system should be to identify those activities where
deviations are occurring and then to take corrective action. Internal
audits are usually an integral part of the internal control system.
In the following sections, specific activities of the I/M audit visit are
listed and discussed. A more detailed discussion of these activities is
found in Appendix C. The purpose of each activity is to collect information
related to one or more of the audit topics listed in Section 2.0 and
6-10
-------
discussed in detail in Appendix A. In some instances, the audit topics
may have been sufficiently clarified during audit preparation. However,
whenever time allows, it is advisable to discuss each of these areas with
program officials during the audit visit. Even if no new information is
gained, the discussions will at least enable the auditors to verify that
the information obtained during audit preparation is accurate.
3.2.1 Interviews
The following State/local officials or their designees must be interviewed
during the audit visit: the air planning agency officials with responsibility
for mobile sources, the I/M manager (whether employed by the air agency
or another agency), and operations personnel with close knowledge of
current practices and experiences in the areas of enforcement, quality
control, repair waivers, data analysis, and mechanic training. The I/M
manager may be able to discuss all of these areas, or additional staff
may need to be included.
The agendas for these interviews should be based on the questionnaire and
the findings of any on-site activities which have proceeded the interview.
The interviews should seek clarification of program design, intended
procedures, and observed procedures. They are not occasions for EPA
assessments of deficiencies or strengths. Such assessments, if the
Regional Office wishes to make them during the site visit, should occur
during the exit interview with agency officials. Such assessments may be
postponed to later correspondence or meetings, but must be part of the
formal audit report. The exit interview should also be used to offer EPA
assistance on any matter of interest to the I/M officials.
The starting point for the audit visit should be an initial meeting(s)
with State and/or local officials involved in air quality planning for
mobile sources. In addition to discussions of program design and organi-
zation, this meeting should focus on any problem areas in the program
which State/ local officials have already identified and their plans for
resolving them. Also any planned program modifications, improvements,
and expansions should be discussed.
Interviews with the I/M manager and other program operations personnel
should focus on operational aspects of the program including: compliance
rates; failure rates; waiver rates; repair costs; mechanic training
efforts; complaints (types, how resolved, etc.); internal control efforts
(surveillance, results of station audits, data analyses, investigations
using unmarked vehicles, etc.); and enforcement aspects (procedures,
results, etc.). Again known problems and planned resolutions should be
discussed.
In addition to interviewing agency officials with direct involvement in
the I/M program, it may be desirable to interview non-program individuals
with knowledge of the I/M program and differing perspectives on its
operation. The objective is not to solicit complaints nor to promote the
I/M program to these public sectors. Rather it is to compensate for the
necessary brevity of EPA's own audit and to be sensitive to suggestions
6-11
-------
that EPA's audit was too brief to detect actual problems of importance.
Useful interviews can be held with contractor representatives, auto club
officials, service industry association officials, auto dealers association
officials, consumer agency officials, instructors of mechanics, and
similar persons. These people can be interviewed by telephone at a more
convenient time than the site visit, either before or after the visit.
3.2.2 Records Review
Another important phase of the audit visit is the review of records
relevant to the I/M program. These records include enforcement records,
inspection records, waiver records, surveillance records, and complaint
files.
Inspection records should be reviewed to determine what data are collected
and how the data are used. When inspection records are kept manually, it
is important to review a sample of the records for completeness, legibility,
accuracy of inspection standards, reasonableness of test scores, and
accuracy of the pass/fail decision. Where retests are involved, the
emission reductions obtained from repairs would also be of interest.
In reviewing waiver records, it is important to determine:
1. How waivers are processed to ensure compliance with waiver criteria.
2. How many waiver applications are being received, approved, and
denied.
3. The extent to which waivers are denied because of inappropriateness
of repairs.
4. The extent to which waiver transactions can be tracked by repair
facility.
Surveillance records should be reviewed to determine the data collected
during each type of surveillance activity and how these data are used.
Review of these records should also allow a determination of the frequency
of various surveillance activities and how often problems are identified
and resolved during audits and other surveillance activities. Reviewing
cases where stations or inspectors have been suspended for infractions is
also advisable.
Complaint files can be reviewed to determine the number and kinds of
complaints being received. However, reviewing complaints should be given
a low priority, unless observations during the audit indicate that there
are problems, such as improper inspections or inadequate repairs, which
could be further evaluated by reviewing complaints. In such cases, it is
advisable to also review a sample of complaint reports to determine how
they were investigated and resolved.
6-12
-------
3.2.3 Inspection Station Visits
During the audit visit, all types of inspection stations and other licensed
facilities should be visited, including regular inspection stations,
fleet stations, referee or waiver stations, licensed or certified repair
facilities, reinspection stations, or any other type of station conducting
initial inspections or retests. Depending on the type of station, any or
all of the following activities may be part of the visit:
1. Audit or observe an audit of analyzers, using audit span gas.
2. Observe inspections.
3. Observe waiver processing.
4. Check records.
5. Interview station personnel.
In centralized systems, in order to prevent bottlenecks in the lane, it
is often preferable to observe how analyzer audits are performed by
State/local personnel rather than EPA's auditors actually doing the
analyzer audits. In centralized facilities, it is always possible to
observe inspections. In decentralized programs, analyzers can be checked
by the EPA auditors or the EPA auditors can observe the procedures being
used by the State/ local personnel. If the State/local analyzer audit
procedures do not include a calibration check through the probe, the EPA
auditor should conduct such a test. In decentralized systems, there are
often no vehicles waiting to be inspected, so inspections cannot always
be observed. Decentralized stations are generally required to keep
inspection records and analyzer calibration records; therefore, these
records should be reviewed during the station visits.
The number of station visits in an audit visit will vary according to the
type and size of the I/M program under evaluation. In the case of a
centralized program with only a few (less than five) inspection stations,
there should be sufficient time available during the audit to visit all
or most of the facilities. In larger centralized programs and in decen-
tralized programs, only a fraction of the stations can be visited during
the audit. The exact number will vary in each case, however, the following
guidelines should be used in deciding how many and which stations to
visit.
In centralized programs:
1. Visit at least three stations; more should be visited whenever
time allows, or if inconsistencies are found, or when the program
has an extremely large number of centralized facilities (e.g.,
New Jersey).
2. Choose stations with a high volume of inspections (based on
records review).
6-13
-------
3. Choose stations that represent a reasonable cross section of
types of neighborhoods.
4. Choose at least one station, if possible, that has a past record
of possible quality control problems; i.e., low failure rates,
high level of analyzer audit failures, etc. (based on records
review).
In decentralized programs:
1. Visit at least 10-20 stations, depending on the size of the
program, more if possible or when inconsistencies are found.
2. Choose stations which represent a cross section of types of
neighborhoods.
3. Choose stations which cover several different State/ local agency
field investigators.
4. Choose stations which represent a cross section of different
types of businesses (service stations, independent garages, auto
dealers, chain service centers, etc.).
5. Choose stations which represent a cross section of types/makes of
analyzers.
In decentralized programs, the EPA auditors will accompany at least three
State/local field auditors (unless, of course, fewer than three State/local
auditors are utilized in the program) on their regular audit visits. The
EPA auditors will choose which State/local auditors are accompanied and
must be careful to get the right mix of types of stations. This can be
done by reviewing the planned audit schedules of the State/local auditors.
Whenever they exist, waiver stations, mobile stations, referee stations,
fleet stations, reinspection stations, and roadside pullover teams should
also be visited.
3.2.4 Special Surveys
In some cases there may be a need for special surveys as part of the
audit visit (or at some more convenient time). In areas that use window
or compliance stickers for enforcement purposes, a sticker compliance
survey must be conducted. Such a survey involves visiting a number of
parking lots to observe a sample of vehicles and collect data on sticker
compliance or noncompliance.
Other potentially useful data gathering surveys are emissions testing
surveys and tampering surveys. Such surveys involve collecting emissions
short test data and/or tampering data on a representative group of local
vehicles. Emissions testing surveys can be conducted in shopping center
or other parking lots as part of a voluntary testing program or as part
of a mandatory roadside pullover program. The latter method is far
6-14
-------
preferable because it removes the biases associated with voluntary testing
and ensures the collection of a certain amount of data. For tampering
surveys, the mandatory roadside pullover program is the only survey
method acceptable because of these biases.
3.2.5 Exit Interview
The exit interview is usually the last scheduled activity during the site
visit. The purpose of the exit interview is to serve as a wrap-up session
to inform agency officials of the preliminary findings of the audit. It
is important to note and to stress at the exit interview that the discussions
are tentative due to limitations imposed by time constraints. The limited
time available during the audit visit does not allow all information and
data to be fully evaluated prior to the exit interview. Also, the audit
team often does not have the opportunity to meet to fully discuss the
observations, opinions, and conclusions of individual auditors. The site
visit team usually will not include an EPA manager with authority to give
final endorsement to the group conclusions. Finally, there is insufficient
time to evaluate how the findings regarding individual elements of the
program fit into EPA's overall evaluation of the program. In some cases,
aspects of the program may necessitate computer modeling analyses before
a final evaluation can be made.
Despite these limitations, the exit interview should be used to convey as
much information as possible to program officials. At a minimum, they
should be briefed on the activities that were conducted during the audit
visit and on any follow-up activities that will be needed to supplement
the audit visit. If possible, an estimate of the time required for these
follow-up activities should also be provided. If definite problems are
identified during the audit, they should be discussed during the exit
interview. Discussion should also cover suspected but unverified problems,
and how they will be further evaluated during follow-up activities.
State/local officials should also be given a projection of when the draft
audit report will be submitted to the State for review.
3.3 Audit Report
The EPA Regional Office will prepare an audit report which will document
the findings of the audit, present EPA conclusions, and suggest possible
improvements. The Office of Mobile Sources will be given an opportunity
to review and comment on the Region's draft audit report prior to the
final draft being sent to the audited State and local agencies for review.
All official comments received from the State and local agencies should
be appended to the report. By agreement, the Region and OMS may share
the drafting of the report. There is no fixed format for the report.
3.4 Followup Activities
If design or operating problems which are serious enough to constitute
non-implementation of the SIP or to make the SIP inadequate are identified
in the audit report, the Regional Office is expected to prepare a plan
for giving the State or locality reasonable opportunity and assistance in
6-15
-------
solving those problems. This plan should expect faster State or local
action on more serious problems, and provide more time for deliberations
and alterations for less serious ones. Generally, the Regional Offices
will have the lead in obtaining corrections from the States and localities.
The Office of Mobile Sources will provide technical support and track
progress in each I/M program.
If the audit report identifies improvements which are desirable but not
required under the SIP or the Clean Air Act, the Regional Office will
transmit these suggestions to the State or locality also. Appropriate
emphasis should be given to these suggestions as they are likely to be
the most significant result of the audit in many I/M programs.
The audits should be of sufficient scope and adequately documented and
reported in such a way that the record will adequately support EPA followup
action in the form of a call for SIP revisions or a finding of
non-implementation of the SIP, should that be necessary. In some cases,
follow-up may be necessary to collect data to support such actions.
6-16
-------
APPENDIX A*
DESCRIPTION OF I/M PROGRAM ELEMENTS
*This appendix remains unchanged from the FY 1985 NAAS audit guideline
manual and is, therefore, incorporated by reference (National Air Audit
System Guidance Manual for FY85, EPA-450/2-84-008, December, 1984).
6-17
-------
APPENDIX B
INSPECTION/MAINTENANCE PROGRAM AUDIT QUESTIONNAIRE
Part I: Program Design
Part II: Operating Experiences
6-18
-------
I/M AUDIT QUESTIONNAIRE
Part I: Program Design
1. Vehicle Coverage
2. Outpoints
3. Test Procedures
4. Tampering and Misfueling Inspection Requirements and
Procedures
5. Analyzer Specifications
6. Analyzer Maintenance and Calibration
7. Station Licensing; Inspector Licensing
8. Record Keeping at Time of Inspection
9. Data Handling Subsequent to Inspection
10. Station Audit/Surveillance Activities
11. Challenge Mechanism
12. Repair Waivers
13. Enforcement
14. Mechanics Training and Licensing/Certification
15. Consumer Issues
16. Self Assessment Through Data Analysis
17. Other Data Collection Activities
18. Future PI ans
6-19
-------
Instructions - Part I
(1) EPA staff should complete Part I of the questionnaire to the best of
their ability before the questionnaire is sent to the State/local
agency. Documents and staff contacts should be used to obtain
necessary information prior to the site visit. EPA staff should
clearly mark any questions which do not apply to a particular program
before forwarding. EPA staff should forward the questionnaire to
the State approximately 60 to 90 days in advance of the site visit.
(2) EPA staff should specifically note on the "EPA Comments" page* of
each section those questions which could not be answered or for
which there was conflicting information, and request a State/local
answer. The State/local agency may also be requested to provide
documentation for answers provided earlier by telephone. The
questionnaire should be forwarded to the State/local agency with the
request that it be reviewed, completed as necessary, and returned
to EPA at least 30 days prior to the site visit.
(3) The State/local agency should provide any lengthy answers and/or
explanations on the "State/local Comments" page*. Any brief
State/local answers, corrections or additions on the question/answer
pages should be identified by agency initials in the margin.
(4) Additional comment pages should be inserted as needed.
*The space for some answers to questions and for EPA and State/local
comments has been deleted from the questionnaire as it appears in this
appendix.
6-20
-------
1. Vehicle Coverage
(a) Exemptions
Are exemptions made for:
Age? yes/no
Vehicle weight? yes/no
Fuel type?
Di es el s
Other
yes/no
yes/no
Describe:
Describe:
Fuel Type:
Motorcycles? yes/no
New vehicles? yes/no
(less than 1 yr. old)
Other factors? yes/no Describe:
(b) Subject vehicles
Is the program statewide?
no
If yes, estimate no. of subject vehicles
Statewide:
If no, estimate no. of subject vehicles in each
urbanized area:
Urbanized Area
No. of Subject Vehicles
Explain how this estimate was derived:
(c) Explain how vehicles are assigned weight classes, usage types,
fuel types or other exempting characteristics in the
registration/enforcement process:
6-21
-------
2. Cutpoints
Model Year Vehicle Class CO Cutpoint (%) HC Outpoint (ppm)
Were outpoints changed in the last 12 months? yes/no
If yes, give the date they were changed and the previous cutpoints
Date:
Model Year Vehicle Class CO Cutpoint (%) HC Cutpoint (ppm)
* Where this format is not appropriate, attach a detailed list of
cutpoints, and the exact vehicle categories to which they apply.
3. Test Procedures
Attach an official copy of the current usual or primary test
procedure. Where not completely described in the test procedure,
provide the following information. If appropriate, distinguish
between official requirements for a valid test, contractual
requirements on the testing contractor, non-binding advice to
inspection stations, and mere common practice.
(a) Describe the test modes used (idle, 2500 RPM, loaded) and their
order in the test procedure. Indicate whether a tachometer is
required. For loaded mode, specify speed and horsepower:
(b) Describe any conditions under which modes are not performed:
(c) Describe any preconditioning mode and indicate whether it is
mandatory or optional:
(d) Is there a test for visible emissions? yes/no
If yes, describe:
(e) Is a CO 2 or (CO + CO 2 ) cutpoint used? yes/no
If yes, describe:
(f) Is some other exhaust leak check used? yes/no
If yes, describe:
6-22
-------
(g) Describe provisions for ensuring that:
(1) The vehicle is properly warmed up.
(2) The analyzer is properly warmed up.
(3) There is adequate probe insertion depth.
(4) All vehicle accessories are turned off during testing.
(5) Vehicles with multiple exhausts are tested properly.
(h) Indicate the required frequency for the following analyzer
quality control checks:
Electric zero/span check:
Hydrocarbon hangup check:
(i) Is the usual or primary test procedure consistent with the
207(b) Ford restart test? yes/no Explain:
(j) Describe any special test procedures used, including those for
Ford vehicles; indicate which model year vehicles they apply
to, and when they are used:
(k) Describe how standards for a specific vehicle are selected for
comparison to the vehicle's emission readings. (Does the
inspector select the cutpoints or does the analyzer do it
automatically?)
(1) If vehicle standards are selected according to vehicle type or
weight, explain how the vehicle type or weight is determined
during the test:
(m) Is the pass/fail decision made by.
The inspector?
The analyzer?
If the pass/fail decision is made by the analyzer, what
prevents the inspector from ignoring the machine's decision?
6-23
-------
4. Tamper"!ng/Mi'sfueling Inspections
Does the program have provisions for a tampering/misfueling
inspection? yes/no
Attach an official copy of the tampering/misfueling inspection
procedures. Where not completely described in the inspection
procedures, provide the following information:
(a) Identify the vehicles subject to tampering/misfueling
inspections (including tampering inspections that are a
condition for repair waivers) on the basis of vehicle age,
vehicle type and any other relevant factors:
(b) Describe the established procedures for conducting the
tampering/misfueling inspection:
(c) Describe the equipment, supplies and/or manuals required to
perform tampering/misfuel ing inspections:
(d) Describe the mechanism for updating reference manuals:
(e) List the original equipment emission control components that
are checked:
(f) Does the tampering/misfueling check include testing the
tailpipe with lead-sensitive (Plumbtesmo) test paper? yes/no
(g) Describe the repair requirements for vehicles which fail the
tampering/misfueling inspection:
(h) Grace period: Vehicles failing the tampering/misfueling check
must have the necessary repairs done with days.
(i) Describe the procedures to enforce repair grace periods:
(j) Requirements for replacement parts:
No special requirements
Original equipment manufacturer parts only
Other:
(k) Describe the procedures used for reinspections to assure that
replacement parts are installed and working properly:
(1) Describe the mechanism for resolving issues of:
(1) Whether vehicles are/are not supposed to be equipped with
certain components.
6-24
-------
(2) Whether lead-sensitive test paper failures are due solely
to contaminated fuel purchase.
5. Analyzer Specifications
If available, attach an official copy of the approved analyzer
specifications and/or a list of approved analyzers. Where not
completely covered in the specifications sheet, provide the
following information:
(a) Identify the analyzer specifications required for each type of
facility in the program (i.e., contractor lanes, decentralized
stations, fleet stations, state-run stations, certified repair
facilities, etc.). If a standard specification (e.g., BAR 74,
BAR 80, ETI, BAR 84) is used, only a reference is needed.
(b) In a decentralized program using a very limited number of
approved analyzers, identify the models and the approximate
number of each in service:
6. Analyzer Maintenance and Calibration
(a) Routine analyzer maintenance and calibration is performed by:
Inspector Contractor Program official
(Agency )
(b) Facility-gas span checks are performed:
One or more times per day
Weekly
Monthly
Other Specify:
(c) List the components and concentrations of the facility span
gas(es):
Component Concentration
(d) The accuracy of the facility span gas is:
NBS traceabl e +_ 1% NBS traceabl
Other Specify:
6-25
-------
(e) When ordering facility span gas, does the order specify that
the gas must be named in accordance with EPA recommended
practice for naming I/M calibration gas? yes/no
(f) Facility-gas span checks are done:
Through the sampling probe
Through the port
(g) What tolerance is used as a basis for determining the need for
adjustments? +_ % of span gas value
If a different tolerance is used, describe:
(h) Leak checks: Indicate the type and frequency of leak checks
performed, and the magnitude of detectable leaks:
Probe/port comparison
Frequency: Magnitude detectable: %
Vacuum leakdown
Frequency: Magnitude detectable: %
Low flow indicator
Frequency: Magnitude detectable: %
Other Specify:
Frequency: Magnitude detectable:
Explain how these magnitudes were determined:
(i) Describe the frequency and details of other preventive
maintenance (e.g. less frequent multi-point calibration checks,
filter checks, hose cleaning):
(j) Are records required for all routine maintenance and calibra-
tion? yes/no
Describe:
(k) Analyzers with automatic data collection: Describe frequency
and details of preventive maintenance for data collection
mechani sm:
6-26
-------
(1) Describe any service agreements (with analyzer manufacturers
or other firms) for analyzer maintenance and calibration,
including the frequency of service visits and details of
service:
(m) What happens if an instrument fails any of the checks listed
above?
7. Station Licensing/Inspector Licensing
(a) For all decentralized programs and decentralized portions of
centralized programs, provide a copy of the inspection station
licensing requirements.
(b) Provide a copy of the training and licensing/certification
requirements for inspectors. Is periodic relicensing or
recertification necessary? yes/no
Describe:
(c) Are repair facilities registered, licensed or certified? yes/no
If yes, describe the details of this program:
(d) Describe the records required to be kept at stations, including
inspection records, analyzer maintenance/calibration records,
copies of audit reports or others:
(e) In decentralized stations, does the agency keep a registry of
inspectors and their employing stations? yes/no
8. Record Keeping at time of Inspection
(a) Attach a copy of the inspeciton form.
(b) Indicate what data are collected on the inspection form:
Station identification
Inspector identification
Signature of inspector
Vehicle identification number
Vehicle license plate number
Vehicle make
Vehicle model year
6-27
-------
Vehicle odometer reading
Outpoints used
Emission readings
Pass/fail
Waiver information
207(b) compliance certification
Time and date of inspection
Other specify:
(c) How is the data collected?
Manually
Automatically
Combination Explain:
(d) Are repair data collected for failed vehicles? yes/no
Describe:
(e) Sticker-enforced programs:
Is a sticker number recorded for each vehicle? yes/no
Can a sticker number be traced to a specific vehicle?
yes/no
Explain:
(f) Non-sticker enforced programs:
Is a certificate of inspection serial number recorded
for each vehicle? yes/no
Can a certificate be traced to a specific vehicle?
yes/no
Explain:
6-28
-------
9. Data Handling Subsequent to Inspection
(a) Are inspection data for each station summarized periodically?
yes/no
If yes, how often?
(b) Who prepares the data summaries?
Station personnel
Contractor
Program official
(c) Indicate which of the following data appear in the data summary;
Number of inspections
Number of passes
Number of initial failures
Number of stickers or certificates of compliance issued
Number of waivers issued
Other Specify:
(d) Are the data segregated for 1981 and newer vehicles? yes/no
(e) How are data from all stations consolidated?
(f) What periodic reports are prepared for distribution outside the
I/M agency?
(g) What data does the I/M program formally report to EPA? How
frequently?
(h) Provide copies of data summaries, reports, etc.
10. Station Audit/Surveillance Activities
Attach an official copy of the station audit/surveillance
procedures. Where not completely described in the audit procedures,
provide the following information:
(a) How often are inspection stations audited by program officials?
(Give name of agency and identify which officials (title) will
participate in station audits )
6-29
-------
(b) How many program officials are directly involved in station
audits?
(c) Describe the training provided to auditors:
(d) Describe how internal data analysis is used to assist in
station audits:
(e) Are covert audits performed using unmarked vehicles? yes/no
o Are vehicles pre-set to pass or fail the I/M test?
o How frequently are garages checked with unmarked cars?
(Are garages selected for special audits on the basis of
complaints or other factors?)
Are penalties imposed for infractions? yes/no
Describe:
(f) Checks performed by auditor:
Audit gas span checks
Leak check-type:
Sample line integrity (including probe condition)
Observe quality control checks by station personnel
Check facility calibration gas for correct concentrations
Review station records
Observe inspector testing vehicles
Other (describe):
(g) List the components and concentrations of the audit span
gas(es):
Components Concentrations
(h) The accuracy of the audit span gas is:
NBS traceable +1% NBS traceable +2%
Other Specify:
(i) When ordering audit span gas, does the order specify that the
gas must be named in accordance with the EPA recommended
practice for naming I/M calibration gas? yes/no
6-30
-------
(j) How are stickers/compliance certificates accounted for?
(k) Are records/stickers collected? yes/no
Explain:
(1) In a decentralized program with manually collected data, do
the official procedures instruct the auditor to examine forms
for errors or discrepancies and provide feedback to the
stations? yes/no
(m) Are penalties for infractions specified?
(n) Does the auditor report the result of the audit in writing?
yes/no If yes, provide a sample audit form or report.
(o) Where are copies of the auditor's report filed?
11. Challenge Mechanism
(a) Describe how owners can obtain an unbiased challenge test;
describe the procedures, equipment, etc.
(b) Is there a fee for this test? yes/no amount: $
(c) How are motorists notified of the availability of the
challenge test?
12. Repair Waivers
(a) Describe provisions for repair waivers or other mechanisms to
get a sticker/certificate without passing cutpoints:
(b) How much must a motorist spend on emission-related repairs in
order to receive a waiver?
Can the cost to repair tampered components be included in the
cost limit? yes/no
(c) Are certain repairs required? yes/no
If yes, describe:
(d) Are minimum emission reductions required? yes/no
If yes, describe:
(e) Can tampered vehicles receive waivers? yes/no
If no, describe how tampered vehicles are prevented from
receiving waivers:
6-31
-------
(f) Can vehicles with warranty rights receive waivers? yes/no
(g) Documentation required:
Repair receipts
Repair form (provide copy)
Other Specify:
How is documentation verified and by whom?
(h) Describe any methods used to track waivers by garage or
mechanic:
13. Enforcement
Identify the basic enforcement approach:
Registration-enforced
Sticker-enforced
Data link
Combination Describe:
For Registration-Enforced Programs:
(a) Describe methods used to prevent motorists from avoiding
inspection (by registering in another county, registering as a
commercial vehicle, registering as a higher weight class, etc.)
(b) Describe step by step how the enforcement system works for a
particular vehicle; identify the responsible office; describe
how documentation of passing the test is created and handled;
identify which steps are manual and open to error:
(c) Explain how blank certificates are accounted for to prevent
abuse:
For Sticker-Enforced Programs:
(a) Are exempted vehicles given stickers? yes/no
If yes, how are exemption stickers distributed to the correct
vehicles?
(b) Explain how police can distinguish between exempt, complying
and noncomplying vehicles:
6-32
-------
(c) Explain how stickers are accounted for to prevent abuse:
(d) In the case of a failed vehicle that is not repaired
immediately, describe the type of temporary sticker issued, its
period of validity, and how its expiration is determined:
(e) Can parked vehicles be ticketed by police? yes/no
If yes, on-street only, or in publicly owned lots, or on
private property?
(f) Identify which police agencies can give citations:
(g) What is police policy on I/M enforcement? (Only ticket cars
stopped for other violations? Active enforcement of stickers?)
(h) What is the fine for a sticker violation?
Is there a grace period? yes/no grace period:
Is a court appearance necessary? yes/no
Are these cases handled in traffic court? yes/no
Identify the prosecuting office:
Expl ain:
(i) Does the fine go the State or local treasury?
(j) Are sticker surveys conducted? yes/no
By whom?
How often?
Survey size?
(k) Describe any other procedures used to assess the level of
sticker noncompliance:
(1) Is the total number of citations issued by all police agencies
monitored? yes/no
Data-Link Enforcement:
(a) How are vehicles scheduled for inspection?
(b) How are motorists notified?
(c) Explain how delinquent vehicles are identified, and the length
of time involved:
6-33
-------
(d) Describe the follow-up procedure for dealing with delinquent
vehicles, including the length of time involved:
(e) How are vehicles which have left the program area or been
junked identified and purged from the list?
(f) What are the penalties for non-compliance?
(g) Is a court appearance necessary? yes/no
Is there a grace period? yes/no grace period:
Are these cases handled in traffic court? yes/no
Identify the prosecuting office:
Explain:
(h) Is there a back-up enforcement method (some way to identify
non-complying vehicles on the road)? yes/no Explain:
(i) Is there any way to assess the level of compliance separately
from the data-link tracking system? yes/no
Explain:
14. Mechanics Training and Licensing/Certification
(a) Provide a copy of the course curriculum outline, including
names of the instructor(s), number of classes, course duration
(hours of instruction) and fees.
(b) Provide a copy of the registration, licensing and certification
requirements for mechanics. Is periodic relicensing or
recertification required? yes/no
(c) Describe any follow-up with training participants, including
the use of newsletters, mailing lists, etc:
(d) Explain how mechanics training is promoted:
(e) Is a repair assistance hotline provided? yes/no
Describe:
(f) How are repair manuals made available to mechanics who have not
attended training courses?
15. Consumer Issues
(a) Provide samples of brochures and other materials on public
awareness that are available to motorists.
6-34
-------
(b) Indicate what information is provided to motorists at the time
of inspection:
Types of repairs to be expected for different types
of I/M failures
Expected range of repair costs for different repairs
Lists of certified repair facilities
Where to call to get information/assistance or
to file a complaint
Fuel economy indicators from emission readings
Normal emissions levels of tuned vehicles
Emission performance warranties
(c) Is a consumer assistance hotline used? yes/no
Describe:
(d) Describe any other I/M public awareness efforts:
16. Self Assessment Through Data Analysis
Describe agency efforts to internally monitor program activities
through data analysis, covering what is done in the following areas:
(a) Tracking failure rates, waiver rates, etc. overall and by
inspection station or by inspector.
(b) Tracking performance of scheduled audits.
(c) Tracking audit results to ensure proper station performance.
(d) In sticker/certificate systems, maintaining sticker/certificate
accountability.
(e) Tracking number of inspections to assess compliance rates.
(f) Reviewing average emissions levels before and after repair to
assess effectiveness of repairs.
(g) Tracking types of repairs and/or repair costs to assess
appropriateness of repairs, impact on the public, and adequacy
of existing repair cost limits.
(h) In sticker systems, conducting periodic field sticker surveys
to assess overall level of compliance as well as to identify
any local hotspots of non-compliance.
6-35
-------
17. Other Data Collection Activities
(a) Are roadside pullovers used to check emissions and/or check for
tampering? yes/no
If yes, attach a description including equipment used,
calibration procedures and types of data collected.
(b) Are public opinion surveys performed? yes/no
If yes, attach a description of questions asked, data
collected, etc.
(c) Are repair cost surveys performed? yes/no
(d) Is data collected from motorists complaints? yes/no
Describe:
(e) Describe any other method of collecting data to evaluate the
program:
18. Future Plans
Indicate any plans for changing basic parts of the program.
Cutpoint changes. Describe:
Changes in testing procedures. Describe:
Changes in vehicle coverage, i.e. more model year coverage,
higher weight class coverage, broader geographic coverage in
existing area, etc.
Describe:
Additions of program areas, i.e., new cities.
Describe:
Improvements or reductions in data analyses and/or quality
control.
Describe:
Additions of tampering or emissions checks.
Describe:
Changes in waiver criteria. Describe:
6-36
-------
Changes to accommodate the emissions performance warranty
requirements.
Describe:
Changes in program type, fees, and any other administrative
factors.
Describe:
6-37
-------
I/M AUDIT QUESTIONNAIRE
Part II: Operating Experiences
1. Operating Statistics - Emissions Inspections/Tampering Inspections
2. Quality Control Statistics
3. Data Analyses
4. Consumer Protection
5. Repair Waivers
6. Enforcement
7. Mechanics Training
8. Self Assessment Through Data Analysis
6-38
-------
Instructions - Part II
(1) If EPA has access to suitable, recent reports from the I/M program,
EPA staff should complete Part II of the questionnaire to the extent
possible and make narrative comments on topics of special concern
before forwarding the questionnaire to the State. EPA staff should
clearly mark any questions which do not apply to a particular
program before forwarding. The questionnaire should be forwarded to
the State approximately 60 to 90 days in advance of the site visit.
(2) The questionnaire should be forwarded to the State/local agency with
a request that it be reviewed, completed as necessary, and returned to
EPA at least 30 days prior to the site visit.
(3) While the State should make reasonable efforts to answer as much of
this part of the questionnaire as possible, EPA does not expect the
State to expend extensive time and/or resources in gathering new
data or conducting special data analyses. The intention is to gather
as much information and date about the operating program that is
currently available. In some cases, the State may have already
compiled much of the information in a separate operating report. If
this is the case, the State report can be appropriately referenced in
the questionnaire and attached as an appendix, rather than repeating
the information in the questionnaire.
(4) Additional answer sheets should be inserted as needed.
6-39
-------
1. Operating Statistics - Emissions/Tampering Inspections
(a) Identify:
(1) The number of licensed inspection stations (non-fleet).
(2) The number of fleet inspection stations.
(3) The number of vehicles required to be inspected.
(4) The number of inspectors involved in the program.
(b) Provide the following data for emissions inspections only,
tampering inspections only, and both emissions and tampering
inspections (joint statistics). If data as described are not
available, provide similar data which fit the description as
closely as possible. Where no similar data are available,
describe what transcription and/or processing would be required
to obtain similar data.
(1) Number of vehicles inspected at licensed inspection
stations and at fleet stations, and failure rate (by
month) for all vehicle ages combined over the previous 12
months. (If procedures or cutpoints are different for
different vehicle types (passenger cars, light trucks,
heavy duty trucks) and the data are kept segregated,
provide separate statistics for each type.)
(2) Failure rate by model year (or cutpoint group) and vehicle
type (for previous 12 month period as a whole).
(3) Failure rate as above by separate urbanized areas or other
regional groupings.
(4) Number of first, second and third emissions retests.
(5) If inspections and reinspections may be performed at
either centralized stations or decentralized licensed
stations, identify the number of initial emissions
inspections and reinspections at each type of station.
(c) For previous years, identify for each year:
(1) The number of vehicles inspected.
(2) The failure rate.
(d) For tampering and misfueling inspections, identify:
(1) Failure rates by component inspected for the previous 12
months (by month).
6-40
-------
(2) Failure rates by component inspected for previous years
(by year).
(3) The number of referee cases in the last 12 months (by
month) and in previous years where:
a. A referee was requested to determine whether a
vehicle was originally equipped with certain
components.
b. The determination (of (a)) was in favor of the owner
(i.e., it was determined that the vehicle was not
originally equipped with the component in question).
c. A referee was requested for a lead-sensitive
(Plumbtesmo) test paper failure.
d. The determination (of (c)) was in favor of the owner,
(e) Provide statistics from any tampering surveys.
2. Quality Control Statistics
(a) For the previous 12 months, provide the following data (by
month) for audits of inspection stations. Where audits were
performed at both centralized and decentralized inspection
stations, provide separate data for each:
(1) Number of overt audits performed.
(2) Number of covert audits performed.
(Provide tabular results of all covert audits.)
(3) Number of analyzers checked with span gas.
(4) Number of analyzers failing the span gas checks.
(5) Distribution of errors (i.e., percentage of analyzers
within +5%, +10% (etc.) of bottle value and percentage
within -5%, -10% (etc.) of bottle value).
(6) Number of analyzer leaks identified during audit.
(7) Number of temporary facility or analyzer shutdowns
resulting from audits.
(8) Number of penalties and warnings resulting from audits
(describe nature of penalties and/or warnings).
(b) Provide data on any quality control problems repeatedly
identified through audits and how they were investigated and
resolved. (Include any span gas problems and/or analyzer
problems.)
6-41
-------
3. Data Analysis
(a) For each of the last 12 months, identify:
(1) Mean HC and CO readings of passing vehicles (by model year
(or cutpoint group) and vehicle type).
(2) Mean HC and CO readings of failing vehicles (by model year
(or cutpoint group) and vehicle type) for both initial and
final tests.
(3) Number of stickers or certificates issued (by vehicle
type).
(b) Provide copies of the latest inspection data summaries and
repair cost summaries.
4. Consumer Protection
For the last 12 months identify:
(a) The number and type of motorists' complaints received.
(b) The number and type of consumer inquiries received.
(c) The number of challenge tests provided and how the results
compared to initial tests at program test stations.
5. Repair Waivers
For each of the last 12 months identify:
(a) Waiver rates (by model year (or cutpoint group) and vehicle
type).
(b) Mean HC and CO readings of waived vehicles at initial and final
inspection.
6. Enforcement
Registration Enforcement:
Estimate the percentage of motorists evading the inspection
requirement by registering outside the I/M program area.
Explain how this estimate was derived.
6-42
-------
Sticker Enforcement:
For each of the past 12 months identify:
(a) The number of citations issued, number of fines
imposed and the average fine.
(b) The results of any local sticker surveys.
Data Link Enforcement:
For each of the past 12 months identify:
(a) The number of vehicles at each stage of the process
as of the first of the month.
(b) The number of penalties assessed and the average fine.
(c) The number of cases of failing to be inspected on
time that were resolved without penalties.
(d) The average number of days from missed inspection to
each subsequent step in the process.
7. Mechanic Training
(a) For the previous 12 months identify:
(1) The number and type of mechanics training courses offered
(include a general description of the course content).
(2) The number of course participants.
(3) The number of participants passing the course.
(4) The number and nature of inquiries from mechanics.
(b) For previous years identify:
(1) The number of mechanics trained each year.
(2) The number of mechanics passing the course each year.
(c) Identify:
(1) The total number of licensed or certified mechanics.
(2) The total number of licensed or certified garages.
(d) Describe the size and make-up of the circulation list for
periodic newsletters or other information distribution.
6-43
-------
8. Self-Assessment Through Data Analysis
(a) Describe results of recent internal assessments; provide copies
of data or narrative reports, if possible.
(b) Identify and describe significant problems identified through
the internal control/self assessment system, and how these
problems were resolved.
6-44
-------
APPENDIX C*
DESCRIPTION OF ON-SITE AUDIT ACTIVITIES
APPENDIX D*
PROCEDURES USED IN QMS TAMPERING SURVEY
APPENDIX E*
INSTRUCTIONS AND FORMS FOR AUDIT ACTIVITIES
APPENDIX F*
CHECKLIST FOR COMPLETING THE I/M AUDIT
*These appendices remain unchanged from the FY 1985 NAAS audit guideline
manual and are, therefore, incorporated by reference (National Air Audit
System Guidance Manual for FY85, EPA-450/2-84-008, December, 1984).
6-45
-------
TECHNICAL REPORT DATA
(Please read Instructions on the reverse before completing)
1. REPORT NO.
EPA-450/2-85-008
3. RECIPIENT'S ACCESSION NO.
4. TITLE AND SUBTITLE
National Air Audit System Guidance Manual for
FY 1986 - 1987
5. REPORT DATE
December 1985
6. PERFORMING ORGANIZATION CODE
7. AUTHOR(S)
8. PERFORMING ORGANIZATION REPORT NO.
9. PERFORMING ORGANIZATION NAME AND ADDRESS
Office of Air Quality Planning and Standards
U.S. Environmental Protection Agency
Research Triangle Park, North Carolina 27711
10. PROGRAM ELEMENT NO.
11. CONTRACT/GRANT NO.
68-02-3892
12. SPONSORING AGENCY NAME AND ADDRESS
13. TYPE OF REPORT AND PERIOD COVERED
DDA for Air Quality Planning and Standards
Office of Air and Radiation
U.S. Environmental Protection Agency
Research Triangle Park. North Carolina 27711
14. SPONSORING AGENCY CODE
EPA/200/04
15. SUPPLEMENTARY NOTES
The purpose of developing national air audit system guidelines is to establish
standardized criteria for EPA Regions to follow when auditing State air program
activities. This document, prepared jointly by the State and Territorial Air
Pollution Program Administrators (STAPPA), the Association of Local Air Pollution
Control Officials (ALAPCO), and the Environmental Protection Agency, provides
national air audit guidelines for air quality planning and SIP activities, new
source review, compliance assurance, air monitoring, and vehicle inspection and
maintenance programs.
17.
KEY WORDS AND DOCUMENT ANALYSIS
DESCRIPTORS
b.lDENTIFIERS/OPEN ENDED TERMS C. COS AT I Field/Group
Air pollution
Air audit
Air quality planning
New source review
Compliance assurance
Air monitoring
Inspection and maintenance
Air Pollution Control
13B
18. DISTRIBUTION STATEMENT
Unlimited
19. SECURITY CLASS (This Report)
Unclassified
21. NO. OF PAGES
299
20. SECURITY CLASS (TMspage)
Unclassified
22. PRICE
EPA Form 2220-1 (Rev. 4-77) PREVIOUS EDITION is OBSOLETE
------- |