GUIDELINE SERIES
OAQPS NO. 1.2-005
August 1973
GUIDELINES FOR EVALUATING
STATE AND LOCAL AIR POLLUTION
CONTROL AGENCIES
HU MM
MU MM
My Ml
US ENVIRONMENTAL PROTECTION AGENCY
Office of Air Quality Planning and Standards
Research Triangle Park, North Carolina
-------
i\i\i i
450R73118
GUIDELINES FOR EVALUATING
STATE AND LOCAL AIR POLLUTION
CONTROL AGENCIES
Control Programs Development Division
July 1973
Information from Workshop
Research Triangle Park, North Carolina 27711
December 5-6, 1972
-------
CONTRIBUTORS .
/ '
Evaluation Process Don Smith
Type of Evaluation Dan deRoeck
Evaluation Criteria (Workshop Sessions)
Management Ray Morrison, Stu Roth
Engineering Tom Donaldson, Dan deRoeck, Greg Glahn
Enforcement Tom Williams, Tom Cooper
Technical Services Frank Noonan, Neil Berg, Rick White
v.
WORKSHOP PARTICIPANTS
-------
LIST OF ATTENDEES - EVALUATION OF STATE AND LOCAL
AIR POLLUTION CONTROL PROGRAMS WORKSHOP
December 5-6, 1972
Name
Robert DeSpaln (303) 837--3886"- ~
William O'Neal, Or.(215) 597-9153
Martin BMttain (214)'749-1235
Glen Cass
Rick White (206) 442-1226
Paul Martin (202) 254-7551
Dave Bryant (703) 557-2151
Ray Cooksey (703) 557-2151
Dick Solomon (919) 549-2358
Stuart Roth (212) 264-2518
Alvln Pesackowltz (202) 755-2765
Joseph Montgomery (202) 755-0460
Mary Gibbons (202) 755-0482
Gcnaro Cdstantlno (617) 223-6339
Fred Meado.ws (202) 755-2096
Greg Glahti(401) 526-3034
John Buslk (202) 755-2755
Wayne Blackard (415) 556-7554
Robert Duprcy (202) 755-OOCO
Dick Burns (617) 223-3479 . ^
Organization and Region
A1r & Water - Region VIII
Region III
" Region VI
" Region 17
" Region X
Grants Admin. Division
U.S. Gen. Acct'g. Office
*
' M
EPA Office of Audit
A1r Programs - Region II
Dlv. of Stationary Source Enf.
OAWP, OPMO, PPR
OAWP, OPMO
Program Support .Branch, Region I
Grants Admin. Division
A1r Programs Branch
EO, DSSE
Air & Water D1v. - Region IX
Enforcement
Program Support Branch, Reg. I.
Mailing Address,
Denver ••••—.—.,,.. __„_
Philadelphia
. Dallas
; Atlanta
Seattle
• Washington, D.C.
Falls Church, Va.
Falls Church, Va.
Durham, N.C.
.: New York City
.: Waterside Mall, Wash. D.C.
Waterside Mall, Wash., D.C.
Waterside Mall, Rm. 937D, Wash., D.C.
J.F.K. Federal Bldg., Boston . ••
Waterside Mall
Atlanta, Ga.
Headquarters, Wash., D.C.
San Francisco, Calif.
Washington, D.C.
Boston
Workshop area (Preferred)
-Admin. ..„
Admin.
Adniin.
Enf.
Tech. Services
Admin.
Adir.in.
Tech. Ser.
Enf.
Admin.
Enf.
Enf. ,
Eng.
Admin.
Admin.
Eng.
Enf.
Eng. .
Admin.
-------
Name. „ ••• Organization and Region ' , Mailing Address 'Workshop Area
Tom Cooper (312) 353-6942 A1r & Water, Region V Chicago Eng'.
J. Clescerl (312) 353-6942 ' " -" „ .
Jon Clark Monitoring and Reports Branch ' Mutual Bldg., Durham, N.C.
John Eagles . Emergency Operation Control Center Mutual Bldg., Durham, N.C.
Nell Berg ' -.Program Guidelines-& Information Branch . Research Triangle Park, N.C.
' Lloyd Hedgepeth " • " "
Korean Edmlsten . » " . • •" = ' •
Don Smith ; ' " •
Tom Williams * - -- '' " ." ' ' '' ' " " :
Frank Noonan ' Environmental Impact Statement Review Section Mutual Bldg., Durham, N.C.
PO -. •
Ray Morrison' • Program .Guidelines & Information Branch Research Triangle Park, N.C.
Dan deRoeck : .. "
Tom Donaldson " ' • "
Jean Schueneman. Director, Control.Programs Developneit Division '• Research Trlangle'Park, N.C.
-------
Section 1. The Evaluation Process
Background
/
State and local air pollution control agencies are encouraged
through Federal financial assistance to develop effective programs
for the prevention and control of air pollution. The award of this
assistance is intended not only to aid in the continued development
of these programs but is also structured to support those agencies
demonstrating their ability to maintain a comprehensive control scheme
for reduction and abatement of air pollution. The Federal Register,
Title 40, Part 35, State and Local Program Assistance Grants, provides
authority, criteria, and eligibility requirements for awarding of
these grants. Three types of grants will be awarded in support of an
air pollution control program—pre-maintenance, maintenance, and
interstate planning. Grants may be made in amounts up to two-thirds
of the cost for pre-maintenance programs, three-fifths of the costs
supporting a maintenance program, and for amounts up to 75 percent
of the estimated air quality planning program costs. The agencies
qualifying for support under the program can be municipal, regional,
state, or interstate. However, limitations are placed on the number
of years that an agency can receive pre-maintenance support based on
the type of agency and number of budget periods !(three or more or less
than 3) the agency received support for between'the years July 1, 1968
and July 1, 1972.
-------
A survey of State and local air pollution control agencies
conducted in the spring of 1971 indicates approximately 264 agencies
are operating programs that could receive some type of grant support.
\
\
The number of agencies currently receiving grant Support range from
16 in Region X to 45 in Region V.
In determining the desirability of continuation of support, a
program evaluation must be made of these agencies' program objectives
and program performance. These program evaluations are to be scheduled
at least annually by the Environmental Protection Agency Regional
Offices and are to be conducted no later than 120 days before the
beginning of a new grant period. The effectiveness of these evaluations
*
and ultimately their impact on the agency and the meeting of the Clean
Air Act objectives depends critically on the purpose for which they
are conducted as well as how they are conducted.
The level of effort at which the Federal government,in cooperation
i
with State and local control agencies, conduct these evaluations could
consist of either (1) a checklist review of the application with
i
control agency personnel to be sure all items mentioned as criteria
in the regulations are addressed or a combination of limited review
of the submitted application; (2) an agency financial audit; and/or
(3) a complete study of agency operations with recommendations for
•improvement.
APPLICABILITY OF AGENCY EVALUATION .
The application of the evaluation program'could thus take many
varied forms, entailmany different procedures for conduct of the
evaluations, and could generally result in a pon-uniform application
-------
of the grant's criteria and subsequent grant funding between the EPA
regional offices. It is important, therefore, that established procedures
and available criteria be provided so that consistent evaluation techniques
are applied to ARC agencies. The primary responsibility for conducting
i
these agency evaluations is with the regional offices. The time involved,
personnel required, writing of reports with recommendations and follow-up
of these recommendations will probably require expertise and personnel
training requirements beyond that presently available in the regional
offices for making such evaluations. The purpose of this document then
is to provide some guidance and definition to the regions concerning the
type of evaluations they should conduct and the procedures and criteria
to be utilized in any evaluation.-
COMPONENTS OF EVALUATION PROCEDURE
The regional office evaluation procedure should result in the
resolution of conflicting interests between the pollution control agencies
within a state; an allocation of available funds and other resources that
will achieve the most effective contol of enforcement operation; establish-
ment of a priority system for review and analysis of agency programs; and
of course cooperation between the Federal, State, and local governments
in conducting specific programs that will result in eventual overall
program improvement and achievement of national goals.
The overall mission of the grant process is to provide resources to
State and local agencies to assist them in solving their air pollution
I
problems. The specific evaluations and analyses made of an air
i
pollution agency must be directed toward improving the use of those
-------
resources to provide efficient operations that achieve control of air
pollution sources within the control program's jurisdiction.
/
In establishing an evaluation procedure to effectively and
efficiently serve this purpose, the Regional Office should consider-
the following components as a part of its evaluation process:
1. Development of a screening mechanism to select agencies
for review.
2. The use of adequate guidelines (criteria) for evaluation of
agencies that:
a. Reduces the time involved in the review process
b. Assures an adequate estimate of the level of the
effectiveness of the agency.
3. The setting up of more adequate instruments for the collection
of data pertaining to the evaluation process.
4. The assessment of factors involved in evaluation of an agency
that affects national standards, implementation plan accomplish-
ment, budgetary considerations, and best means of assuring
i
i
control.
5. The extension of the evaluation procedure beyond appraisal
of effectiveness by assuring, identification of agency needs
I
and formulation of programs for assistance to the agency.
i
Agency evaluations should not be exclusively the outgrowth of the
grants program, but should extend to the ARC agency an opportunity and
recommendations on which to base program development, planning objectives,
-------
resource needs, and criteria for reporting and measuring effectiveness.
A specific objective can be related to the different levels of analysis
to be performed for an agency.
The regional office should select a review technique, based on
i
criteria, that has as its aims:
1. The continual development of a review technique suitable for
selecting agencies for intensive evaluation based on criteria
that provides for follow-up to the agencies.
2: Includes the optimum means of evaluation, including subjective,
judgmental, or statistical methods.
3. Documents the problems involved in the evaluation process as
well as documenting the ARC agencies' shortcomings.
In general, the types of evaluation and analysis assistance that
will be made available to the State and local air pollution control
agencies will fall into the following categories:
1. Grant evaluation - The evaluation made in conjunction with the
agency's grant application to review the agency's goals and
program. This type of assistance should be geared to providing
the agencies assistance in defining goals, outlining project
milestones and accomplishments, and coordinating Federal program
requirements with the agency so that the;grant application can
provide an accurate estimate of each agency's needs and goals.
These evaluations are performed for each agency on a yearly
>
basis.
-------
2. Agency Performance Review - This evaluation is a complex
analysis and detailed review of the functions and activities
of an air pollution control agency. It is intended to serve
the agency by providing recommendations for improvement of
operations; assist the agency in qualifying for maintenance
support; and when necessary, provide new direction, organization,
or program planning concepts on which to base a long-term
viable air pollution control program. This type of evaluation
should be continuous, and if possible-, performed yearly to
j
supplement the grant evaluation review.
Section 2. EVALUATION AND ANALYSIS GUIDELINES
*
The purpose of this document is to present'the.general guide of
factors that should be considered and the types of studies of an
agency that should be undertaken to assure that proper evaluation and
assistance is being provided to improve the agency's operation.
The guides present an indication of minimum investigative elements
connected with the review of agency program progress. It should be
recognized that each type of evaluation can be associated with the
agency's acceptability for grant award and its ability to carry out
enforcement provisions as outlined in the Clean Air Act.
-------
Federally funded programs will be held more accountable for
demonstrating the effectiveness and efficiency of their operations. The
program evaluation then, either through mechanisms of review of the grant
i
application or actual observation of program operations, should provide
information to assist in making informed judgments concerning their
termination, continuation, modification or the refocusing of an agency's
programs. The evaluation itself should provide the agency with a review
report;on which it can take corrective action concerning the strengths >
and weaknesses in its plans and activities.
Consideration of which evaluation to use at which time in an agency's
stage of development will depend upon your ability (regional office
personnel) to recognize such need and the particular agency's ability
to adopt and adjust to any requirements or recommendations that might
result' from such an evaluation. These guides will cover the following
areas of concern: (1) frequency of review criteria, (2) basic agency
evaluation criteria, (3) specific program criteria, and (4) use of report.
FREQUENCY OF REVIEW CRITERIA
Reviews of control agencies should be made to improve program operation
and efficiency, to determine level of grant support, and to measure the
i
overall capability of the agency to adequately implement a program of
enforcement to achieve national ambient air standards. The reports can
then be broken into classifications, based on these reviews, such as
A' '• *<:'v.<&)
progranj improvement (grant evaluations), and agency program operation
(performance).- Under the classification of reviews for "program
-------
improvement could be placed planning assistance, pre-maintenance
reviews, and agency analysis reports. Agency program operations
(performance) would be (1) review of reports submitted by the agency
to fulfill implementation requirements, and (2) program analysis to
determine capability of the agency to adequately carry out and enforce
applicable implementation plans, new source performance standards, or
hazardous air pollutant emission standards. The type of report can
thus be classified as follows:
Report Program improvement Program operations
(grant evaluation)
Planning X
Pre-maintenance X
Maintenance X
Analysis X X
The frequency at which the various reports should be conducted
generally depends on the stage of development of the agency programs.
However, Federal Register requirements for grant support, and imple-
mentation plan timetables provide some guidelines on which to base the
frequency of reports for grant evaluation and program performance.
Evaluation, including application assistance, should be a continuing
program of the regions and should provide assistance to the agency in
determining the requirements of the Federal government. In addition,
factors of circumstance, opportunity, and individual agency requests
must be considered. The criteria for frequency of agency review for
each type of report can then be summarized as in Table II.
10
-------
COOPERATIVE RESPONSIBILITY FOR PROGRAM REVIEWS AND ANALYSES
The responsibility for awarding program grants lies with each of
the ten EPA regional offices. The primary responsibility for carrying
out enforcement tasks also rests with the regional offices. The assistance
to the regional offices from cooperating groups should be directed to
providing a basis for regional office decision-making in these areas.
Thus, the reasons for cooperative efforts with a regional office
centralized group in formalizing evaluation reports are to (1) provide
uniform guidelines and procedures for each regional office, (2) consolidate
expertise in a small centralized group that has program, administrative,
and technical skills for conducting such evaluations, and (3) provide
an independent viewpoint to agency operations within a region, that may
either reinforce or alter the recommendations, considerations, or views
of that agency as held by EPA enforcement and technical regional office
personnel.
A centralized regional office group cannot be expected to have the
personnel, expertise, and resources necessary to perform all the
evaluations and types of evaluations necessary for the effective program
development of State and local agencies. To provide some guide for the
workload that can be expected and the criteria that must be considered
the following table is presented to represent the cooperative relationship
that each regional office should consider as useful in making a review
of a control agency program.
11
-------
f ••' Cooperative responsibility
I \ Report writing
I Request Visit Interviews ! recommendations
Grant !
evaluation E.G. E.G. E.G. :> E.G. & T.P.
Program! 1
operation E.G. & E.G. & '
evaluation E.G. T.P. T.P. E.G. & T.P.
E.G. -jEvaluation Group „ '
T.P. -'Technical personnel
The participation level in the evaluations between the evaluation
group and other technical personnel depends on the lines of communication
established in the regional office and the support provided the
r
evaluation group by the other divisions. The level of participation
guidelines are the following:
i
1. Evaluation group coordinates and requests all evaluations.
Evaluation group prepares preliminary information concerning
problems and needs for specific type of evaluations at selected
agencies.
2.1 Evaluation group has responsibility for reporting on all
pre-maintenance evaluations. Any assistance provided by technical
personnel would be directed at a specific program area such as
i
air monitoring and data handling, identified during review of grant
applications or visits to the agency.
3.j The evaluation office should probably request outside review
assistance for all agencies requesting maintenance support to
reinforce or to provide program support regarding conditions for
12
-------
rejection or acceptance of maintenance grant award. Criteria
; !
i • ' i
for maintenance grant awards are set forth in regulations, so the
I '
evaluation office could theoretically provide this evaluation of
t
ari agency. However, judgment and interpretation of specific
' • 1
program operations may require assistance from individuals skilled
r
I
in enforcement, engineering, or technical services activities.
4i Agency operation reviews should be considered as a cooperative
effort with maximum participation of regional office personnel,
i1
i
both qualified technical personnel, and evaluation personnel.
These evaluations are intended to develop statewide programs
that will result in effective enforcement of State implementation
plans or be a part of the procedure for involving EPA state
strategy for enforcement of implementation plans.
5l Assistance on specific program areas identified during any
evaluation can be obtained also from headquarters components. As
expertise and documentation in areas of enforcement, permit
systems, laboratory operations, and other agency functions are
developed, they are generally made available in form of procedure
documents and/or guidelines.
i
An'agency evaluation essentially requires the scheduling of
activities as shown on Table I. If the evaluation involves only a
i
review of the grant application then such items as field visits and
i
field work may be eliminated for that activity. Although it is strongly
!
recommended that conferences be held with the State and local agency
13
-------
during the grant evaluation review to define goals, discuss agency
problems, and coordinate Federal program requirements. In some regions,
it may be important to establish a priority system which assists in
determining the specific evaluation activity that will be conducted and
i !
provides for the explicit type of data that will be collected for any
evaluation review [operations (performance), or program improvement
(grant1 application review)]. In establishing this priority system,
i
factors that also pertain to overall criteria associated with program
objectives should be considered in addition to the guidelines summarized
in Table II pertaining to purpose, frequency of review and basis for review.
BASIC [PROGRAM CRITERIA
In the evaluation of any air pollution control agency, consideration
must be given by the evaluator to the following basic influences that
affect control program operations:
i
1. The requirements of the approved implementation plan which
must be satisfactorily implemented to support national ambient
air quality standards.
2I The structure of State and local activities designed to achieve
the objectives in the State plan and to meet the Federal
i reporting requirements for enforcement of the plan.
When making visits to local agencies, for the purpose of evaluation,
it is important that representatives of the State agency participate to
assure'the above influences are satisfactorily addressed and any
problems that exist are fully discussed. Likewise, if possible, it
14
-------
would be'profitable to have local agency representatives present during
: . v/ . 'I
the evaluation visits to the State agencies, as the exchange of infor-
mation often resolves conflicting areas of program responsibility.
, '•• \
In general, when reviewing the agency, the following criteria
i '
objectij/es should be considered as being important to satisfactorily
i
implement the State's plan:
l.j Appropriate role of State agency in areas of direct involvement
* in enforcement activities.
i
21 Eliminate inefficient duplication of equipment, personnel,
\ and activities by seeing that agreements of understanding
!•
I exist between State and local agencies.
!
S. Determine that objectives move toward establishing minimum
'. monitoring network for State and that special monitoring
! requirements above the minimum network are adequately justified.
4. Develop reporting data techniques on a statewide basis that
integrate into Federal reporting requirements.
5. Adequately define role of State and local agencies in regard
to items such as enforecement authorities, permit systems, etc.
Criteria are provided in Table I LI and IV to assist in classification
of regional air pollution control programs and in defining the roles of
i
State and local agencies pertaining to specific agency operations.
Complicating the overall evaluation of local air pollution control
i
programs is the reorganization of State programs usually a consolidation
i
of environmental programs. Program consolidations often result in a
redirection; of priorities and sometimes change or mesh the familiar
15
-------
activities of the air program. Generally then, the level of effort in
I !
terms of, resources, objectives, and program activity are difficult to
; i
obtain in the traditional program concepts by the evaluator. However,
\
applying the above criteria objectives enables the evaluator to at best
i . ;
inform the agency of what is expected in carrying put activities to
achieve the implementation plan. In addition, when evaluating either
5
the grant application or performance of an agency the evaluator must
encourage and look for objectives that may accomplish long-range goals
i
and fos-ter the continued development and enforcement of the SIP. These
include programs associated with:
l.i Land use and transportation controls.
2. Review and development of impact statements.
3.1 Operation of permit systems.
4.J Planning and growth considerations affecting air pollution
control.
5. Data systems that will assist in efficient agency operations
and provide for adequate reporting of data on statewide basis
i
i compatible with Federal requirements.
Agency effectiveness is an essential consideration in the evaluation
of any air pollution control agency. Once a decision has been made that
an agency has developed adequate goals and objectives (perhaps the first
step of 'an evaluation), then key indicators of success (or failure) are
the measurable results which occur through actions taken to achieve
i
such goals and objectives. Therefore, measurements of agency effectiveness
i
should precede a comprehensive agency evaluation.
16
-------
The ultimate achievement of air quality goals is theoretically
i " ' 1
schedul-ed for!975, or 1977, or sooner. Obviously, attempts to measure
results prior to then must focus on intermediate objectives, or mile-
stones, that must be met. Progress reports, now required on a periodic
basis from each State, provide an excellent means of 'documenting program
achievements which may subsequently be measured.
Ijf an agency's objectives are closely examined, three classes of
i
measurable results can generally be formulated—air quality improvement,
emissions reduction, and source compliance. Realistically, each class
has some cause and effect relationship to .the remaining two, but from
r-
an evaluation standpoint there is measurable data which can be readily
obtained and which can be attributed to specific agency functions and
actions.
For example, improvements in air quality will be documented by
i
data collected through air monitoring activities. While air quality
improvements may be indirectly inferred from actions resulting in
emission reductions and source compliance, air monitoring data provides
a direct measure of improvement. Similarly, it may be reasoned that a
widespread improvement in air quality should infer that emissions are
being successfully reduced; however, the direct measure of reduced
i
emissions is provided through source testing activities and the develop-
ment of an emission inventory.
The type of program evaluation, and the degree of effort which
shouldjbe devoted, can be determined by analyzing each agency's progress
(measurable1;agency output), revealing difficulties or insufficiencies,
17
-------
and determining the relative urgencies of all agencies within a given
state, region, etc. Furthermore, it is possible to direct the evaluation
effort towards specific agency operations on the basis of any given area
of ineffectiveness where agency output does not demonstrate positive
progress or favorable results.
Program Elements for which State is generally responsible:
1. Basic statewide plan development and evaluation.
2. Basic operating procedures (forms, reporting formats, etc.).
3. Training (smoke schools, skills upgrading, etc.).
4. Public Information program. (Partly.)
5. Special engineering skills. (Some localities require specialized
inspection and engineering skills.)
6. Laboratory support (in particular special hardware and analytical
facilities). Local activities may be required for collection
of samples, and maintenance of sampling equipment, etc.
7. Major data handling facilities. Local data collection,
analysis, and reporting also require facilities and personnel.
8. Meteorological support.
9. Progress evaluation.
18
-------
Table I. AGENCY EVALUATION ACTIVITY SCHEDULE
PRELIMINARY
PREPARATION
SETTING
EVALUATION
PRIORITIES
SCHEDULE
FIELD VISIT
OFFICE
PREPARATION
-Federal Register Requirements (Grants)
-Implementation Plan deficiencies
(Program development)
I •
-Coordination with State and local
agencies for visit
-Preparation of. agendas, request for
preliminary information regarding
agency activities, schedule for
interviews
•Selection of team members
-Study of agency information—review
and summarize--SIP, Quarterly &
semi-annual report, compliance
reports, and grant.
FIELD WORK
4. j SITE VISITi
ANALYSIS AND
REPORT
DEVELOPMENT
5.
6.
IMPLEMENTATION
7.
8.
•EVALUATION
CRITIQUE
REPORT PREPARATION
A. SORT AND CATALOGUE
INFORMATION
B. ANALYSIS AND
DIAGNOSIS
C. PRELIMINARY DRAFT
PROGRAM
ASSISTANCE
-Thorough study of agency operations,
interviews with agency support
personnel
-Obtain appropriate information and
material
-Meet on final day of visit with "key"
agency personnel to summarize
preliminary finding and conclusion
-Additional information is obtained
and existing information verified
wherever questionable
-Review recommendations in regional
office
-Copies of draft report are forwarded
to agencies involved
-Consultation and discussion with
agency. Joint review of report
increases agency receptivity,
adds clarification, and improves
report effectiveness
•Follow-up on final report recommendatior
to provide additional assistance
and guidance in implementing
program recommendations
19
-------
fable II. GUIDELINES FOR EVALUATION OF AGENCIES
Grant
purpose
.1
Pre-Maintenance
i
i.
ii
t
j
Maintenance
(i
Efficiency
review j
i
j
Purpose
Program Improvement
Grant Evaluation
Program Improvement
Grant Evaluation
Program Operation
Frequency of
Review
Once per year
Once per agency
and updating of
baseline review
on a yearly
basis
Once per year
with follow-up
of recommenda-
tions and SUD-
sequent reviews
as necessary
and requested
to improve pro-
gram operations
,
Basis for Review
i
1) Approval of grant
2) Consistency with
implementation plan
1) Perform baseline
review of agency
operations 18 months
prior to grant sub-
mittal
2) Review for grant
approval 120 days
prior to grant
period
1) Program development
of local and State
program on Statewide
basis
2) Review of Statewide
program to measure
enforcement effec-
tiveness
3) Reorganization on a
Statewide basis
4) State programs
requesting maintenance
or extension of pre-
maintenance grant
time
20
-------
Table III. CRITERIA FOR PROGRAM OPERATION BASED
ON POPULATION AND MANPOWER STAFFING REQUIREMENTS
Population Range
< 1 million
i
!
1 million to 3 million
3 million to 5 million
5 million to 6 million
6 million +
Number of
States
15
. . * .
13
11
3
8
State Staffing
Requirements
. Man-Year Range .
16-75
76 - 150
151 - 250
251 - 500
501 - 1000
Maximum Potential of Local
• —Agencies- In Man-Years ,_,
0-15
16 - 25
26 - 140 -
33 - 178
153 - 500
Type of
.,._ , Program^ Operation
State Operated
(Local Minimal)
Stnte Comprehensive
(Local Minimal "to
moderate)
State Comprehensive
(Local Moderate to
comprehensive)
State Comprehensiv"
(Local Moderate to
comprehensive)
State Comprehensive
Local Comprehensiv<
Moderate
Minimal
-------
Table IV. CRITERIA FOR PROGRAM ORGANIZATION
AND ASSIGNMENT OF RESPONSIBILITIES
Program
level
Comprehensive
State and/or local
Moderate: local
Minimal : local
taffinga
- \level
(Staffing level > 20)
(Staffing level=6 to 20)
(Staffing level < 6
Program
element
Management
•Administration
; (supervisory and
fiscal matters)
•Policy and planning
•Program Evaluation
•Public and inter-
governmental
relations
•Long-range studies
(land-use and
transporcation)
•Evaluate environmental
impact studies
•Training
'Promulgate regulations
•Administer hearing
and variances
•Grant applications
•Lecml support
•Episode and emergency
operations
•Clerical
•Minimal staff time
allocated to management
•Activity limited c
direct staff
category. Mostly involves supervisors and
direct staff supervision
on program matters
•Some local and State
intergovernmental
liaison
•Grant applications
•Clerical
clerical support
•Liaison with State-
agency
•Grant applications
22
-------
Table IV. (con't.)
Comprehensive D
State and/or local
Program
activity
Moderate: local
Minimal c: local
Enforce-
ment
•Area surveillance
Source registration
Source inspections
and reports
Source data gathering
•Case documentation
•Legal actions
•Permit support
activities
•Complaint investigation
•Episode and emergency
activities
•Clerical support
•Area survoillance
Source registration .
Source inspection
and reports
Souce data gathering
•Case documentation
•Legal actions
•Limited permit support
activities
•Complaint investigation
•Episode and emergency
activities
•Reporting as required by
State
•Clerical support
•Area surveillance
Source registration
Specified source
inspections and
reports
Data gathering
•Case documentation
•Legal actions
•Initial complaint
investigation
i
•Emergency and episode
activities
•Reporting as required
by State
•Clerical support
Engi-
neering
•Permit systems
construction permits
operation permits
•Plan review
•Emission inventory
'Source testing
•Site inspections . and
consul taLion
•Regulation development
•Limited permit activity
for specified
pollutant source
categories
•Emission inventory
•Site inspections and
consultation for
specified source
categories
•Report.ing as required
by State
•No activities or
responsibilities
c 5 . i r.'.
•Special technical
studies
•Cleric.-l support
23
-------
Table IV. (con1t.)
'rogram
activity
Comprehensive
State and/or local
Moderate: local
Minimal c: local
Technical
support
•Laboratory operations
( «Air monitoring
' network
•Instrument maintenance
and calibration
•Source testing support
•Data handling
f
•Meteorology
•Special studies
•Legal documentation
•Clerical support
•Laboratory operations
(very limited)
•Air monitoring network
Primarily servicing and
sample collection
Instrument maintenance
Limited calibration
•Limited data reporting
as required by State
•Assistance to State in
special studies
'Sample collection
limited instrument
maintenance
Staffing level is the total equivalent man-years of effort in each
designated agency.
The degree of activitiy in local comprehensive programs for certain
functions would be specified (for example, all laboratory operations
would probably not be undertaken by a local comprehensive agency and
certain large source enforcement activities may be preempted by the
State agency).
The allocation of responsibilities would be specifically detailed by
the State agency. In general program activities would b« limited 1.0
enforcement activities associated witn area sources and the less
complex industrial sources.
-------
FUNCTIONAL ^.ALLOCATION OF MANPOWER RESOURCES
ON A STATEWIDE BASIS
' . . • Percent, of Total
Functional Area Staff
Management ; 20 to 25
Enforcement • 25 to 35
Engineering 20 to 30
Technical Operations 15 to 20
10 to 20 percent of each area is
for clerical support
25
-------
In either a review-of an agency's application for grant funds or an ,
'in depth" evaluation, the evaluator should seek to answer some basic
questions that pertain to operations. These questions can be structured
)y functional activity as given by the example protocol document Appendix 1
(n evaluating an agency, answers to the following overall questions should-
>e sought:
1. Statistics - Does the agency have the programs that define the
extent of the problems and program operation? This question requires
an agency to have programs such as emission inventory and air
monitoring networks. These data can be provided by the agency or
performed in conjunction with other agencies, but the system should
include proper reporting of the data.
2. Planning - Does the agency have the capability to adequately
analyze the data and develop a program with definitive program goals
and objectives?
3. Available Resources - Does the agency have the skilled manpower,
budget, facilities and/or a plan to obtain these resources to carry
out its role and responsibility under the SIP?
4. Necessary Action - Does the agency have the authority (rules
and regulations), legal assistance, and procedures required to take
enforcement action? Does the agency operate the preventive and
enforcement programs that are required to achieve ambient air
standards? These activities include planning programs associated
with growth characteristics of an area, land use and transportation
activities, impact statements, preparation and review, inspection
programs, and permit system operation.
26
-------
5. Assumption of National Programs - Has the agency asked for
and received the delegation to carry out national programs
, such as the enforcement of national new source performance
standards, and hazardous pollution standards.7 What Federal
action has been necessary by the Federal Government to enforce
the ambient air standards, promulgated regulations, or specific
implementation plan strategy (transportation controls) in the
State or within the local area? The local and State control
.. agencies should be encouraged to avoid Federal enforcement
except in partnership considerations beneficial to both
governmental levels.
Rating the above areas undoubtedly are judgmental on the part of
the evaluator unless he carefully uses the definitive criteria to measure
the agency against. An experienced evaluator skilled and knowledgeable
in a control agency's operation can develop important recommendations by
asking a few critical questions. However, uniformity and perhaps eventually
measures of cost effectiveness can be obtained from agency evaluations
by structuring specific questions and formulating data collection systems.
!
REPORTS
As the regional offices develop their evaluation, analysis and
review techniques, the State and local agencies should be encouraged to
i
document their records, procedures, and guidelines. In any on-site visit
to an agency the evaluator should ask to see copies of records, agency
activities, and of statistical concrete results. Agency documentation
should include such things as:
27
-------
-Description of functions and activities carried on by the agency.
-State and local agreements as to respective functions and
responsibilities.
-The agency's goals and objectives with dates for achievement.
-A compilation of agency policy statements. j
-A compilation of agency procedures used in program operation.
The agency should be requested to keep these records, procedures
in a standard operating manual. This manual should be compiled and
updated in a manner that keeps it available for review. Records of
this type serve as an evaluation tool for external evaluation and it
also aids the agency in conducting a self-evaluation. Records of this
type, if presented properly, would give an adequate picture of the
agency's general plan, delineate responsibilities, and set schedules
and assignments. It would be helpful if the regional EPA offices suggest
to the agencies a general format for the manual which could be set up
along the following lines:
1. Definition of the scope and objectives of the effort.
2. Definition of roles and responsibilities under the SIP.
3. Categorical program description that defines measurements made
and data output specifications. Include:
a. Air quality measurements
b. Meteorology measurements
c. Calibration procedure
d. Emission inventory
e. Permit system
f. .Management data.
28
-------
4. Description of personnel, equipment, and resources committed to
effort and effort contributed by other organizational components.
In case of a State agency this may be district offices or agreements
with local agencies.
/
5. Schedule of operation pertaining to the activity including:
i
a. "In-house" and field operations (inspections)
b. Number and frequency of samples, analyze
c. Data reduction and summary
d. Frequency and dates for report.
6. Procedures that describe roles,.responsibilities;:and
relationship for the.activity including:
a. Administrative support-obligation of the respective agencies
in terms of personnel, budget.
b. Respective duties assigned to each agency in regard to
maintenance and operation of equipment, inspection priority
(class or type of industry or equipment) and enforcement .a'
authority)'.
*•*
c. Liaison performed between respective agencies and overall
responsibility for.project supervision. .
f
7. Outline of procedures and responsibilities of data collection
and interpretation including: !
a. Forms • • ,:
b. Tabulation j
c. Distribution procedures and schedule of distribution
d. Major uses of data I
/
e. Brief description of data output and;if data system involves
participation of other agencies, a description of what those
data are.
29
-------
When the agencies have this type of information available, the
agency will find that they operate more efficiently. Also, the job of
the evaluator becomes much easier. In order to give emphasis to this
type of reporting system by the agencies, the regional offices could
format their findings and recommendations in a similar manner.
Thus, the suggested format for evaluation reports would follow
along the same lines as the operational manual and consist of the
following components:
1. Comparison of objectives against SIP and Federal program
requirements and regional or national operating norms.
Recommendations for new objectives and timetables are
•
necessary.
2. Recommendations regarding cooperative operation of activities,
new and more efficient organization of activities, joint
protocol documents, allocation of resources and elimination
of unproductive duplicative efforts.
3. Analysis of resources committed to effort.
4. Recommendations regarding data collection and handling systems.
5. Recommendations regarding scheduling of operations.
6. Analysis of district and local operations, if they reflect
on agency. j
7. Recommendations concerning data reporting and collection system.
8. If appropriate recommendations associated with long-range
i
planning for meeting anticipated Federal requirements regarding
stationary and mobile pollution sources.
30
-------
Section II,
• .AIR POLLUTION CONTROL AGENCY EVALUATION
.' 1
i . '
,' ;
Specific Criteria \
i
* - i
As the State and local control agencies move closer to the deadlines
imposed for meeting ambient air standards it will become increasingly
important; to have data which measures an agency's effectiveness in
utilizing; their program operations and resources. It appears that the
funding for such agency operations from the Federal viewpoint will not
be sufficient to support all activities that State and local programs
have need for or would like to carry on. As pointed out previously in
these guidelines, the Federal guidance and support to State and local
agencies ;should then be directed to strengthening those skills and
capabilities that will achieve national standards, and enforce regulations
and strategies designed to carry out implementation plans.
The specific evaluation procedures should be designed to identify
agency operations and make recommendations for improvement. Eventually,
it should be possible to allocate the available resources, State and local,
on the basis of priority and need. Therefore, the specific criteria
given for each area of an air pollution control agency's operation,
administration, engineering, enforcement, and technical services are
intended to provide information on a comparable manner from agency to
agency. Eventual refinement of the system, if used either on a nationwide
or regional basis, should provide a data base for comparison of agencies
against a regional or national norm, provide a measure of efficiency,
and as historical, air monitoring, emission data, and growth data trends
are established give some picture of the cost effectiveness of the dollar
expenditure by the control program.
In Section I, basic program criteria were provided to give the
evaluator'an understanding of the influence national priorities should
play in the evaluation of State and local programs. These objectives
31
-------
must be kept in mind and are indicated as priority objectives on the
report form for each program area. The evaluation data collection
procedure obtains data in several ways:
1. Quantitatively - for measurement against operating norms
of the programs within a region or on a nationwide base.
2. Qualitatively - subjective judgments made by the evaluator, and
3. Use of rating number or priority for operation and comparison
to its importance for achieving implementation plans, or
other Federal priorities.
32
-------
Schedule A
AIR POLLUTION CONTROL AGENCY
EVALUATION DATA REPORT
General Information
Agency Name
Fiscal Year
AQCR Name
(if not state agency)
to
EPA'.Region
Evaluation Date:
Review
Visit
AGENCY DESCRIPTORS:
I. Jurisdiction
A. Population served _
i
B. [Growth rate %
H. Population
I (annual)
i2. Sources (annual)
Industrial
II.
Operating Resources
A. Total Staff
1. Positions
2. Man-year estimate
B. Total Budget
Mobile
C. ;Area served (mi
D. ^Value mfg. $ x
1. Personnel
2. Operating exp.
3. Equip. & contractual
services
Implementation Plan Resource
1975 1977
1. Staff
2. Budget
Criteria Measurement
Operating Cost/Man-year
PROBLEM DESCRIPTORS:
I. Air Quality
I
Annual Highest
average value
Particulate
Sulfur dioxide
Nitrogen dioxide
Carbon monoxide
i
Hydrocarbons
Oxidantsj
Other-specify
ugm/m
ugm/m
i
ugm/nf
i
ugm/nT
ugm/mw
f
ugm/nf
ugm/m"
II. . Emission Data
(Annual tons/yr.)
Particulate
Sulfur dioxide
Nitrogen dioxide
Carbon monoxide
Hydrocarbons
Other-specify
33
-------
Schedule A, pg. 2
./ Schedule A
General Information
Suggested Sources of Data
AGENCY DESCRIPTORS:
I. Jurisdiction - Department of Commerce, Bureau of; Census
,:; Population Reports - 1970 and Interim Year Reports.
I Department of Commerce, Bureau of Census
I Manufacturing'Census - 1967 Data - Update with
| data from Agency Emission Inventory Permit System
' and other local sources.
II. ^Operating Resources - Agency grant application - update
, ;• through contacts with agency.
PROBLEM; DESCRIPTORS:
i
I. Air Quality - Semi-annual and annual reports - quarterly reports
are principal source of air quality data.
EPA data reporting system SAROAD, .CDHS, etc.
* Agency reporting records.
II. 'Emission Data - EPA data reporting system - NEDS (State)
Agency Emission Inventory.
34
-------
Schedule B AIR POLLUTION CONTROL AGENCY
/ EVALUATION DATA REPORT
! . Management Function
Definition of Responsibilities: •
r. Program responsibilities defined for agency activities' in: (encircle one)
Implementation plan and interagency agreements - 5
Implementation plan (approved) - 4 • . Grant application - 3
f
Interagency agreements - 2 Not defined or poorly defined - 0
I Criteria score
ji
II. Program Responsibility Summary Management:
Enterione: State Shared with Delegated Partial Other,
only 1 state 2 by state 3 delegation 4 specify 5
Planning Services Evaluation
SIP review, revision, etc. Training by Impact reviews
Land use controls Special Action Plan
initiation ___^
Transportation controls Outside (emergency, SCS)
agency
Complex Source Siting Program Review
Informational
\ - (library, public, and
inf. -retrieval and
dissemination)
i Region's Estimate of Responsibility Priority I, II, III,
' IV, V
35
-------
Schedule B, pg. 2
Program Objectives: (Select one)
i
>
I. Objectives are consistent with following: ,
Achieving and maintaining Federal or other applicable air quality
standards as defined in the approved implementation plan, and
maintenance support as defined in the Federal Register 5
! '
(
Achieving only the Federal or applicable air quality standards 4
r
i
i
Achieving only maintenance support as defined in the
Federal Register 3
i-
Achieving none of the above 0
I Criteria score
II. Review and Evaluation of Objectives: (subjective) Select one.
Clearly defined 5
Poorly defined (indicate areas) 3
Incomplete or unrealistic 1
Not defined 0 Criteria score
III. Program Plans for Achieving Goals indicate following:
Specific target dates and resource requirements 3
Resource requirements only 1
Target'dates only 1
None of above 0
Criteria score
. 36
-------
Schedule B, pg. 3
Resource Criteria: Enter estimate of resources needed by agency as given
in SIP or by other estimating procedure. Circle approximate % that
current resources represent. 1
I. Needs (Total agency)
% Estimate of Achievement of Need
SIP
1975
1977
Other (manpower model, etc.)
1975 1977
% Budget
Budgfet
Man-years
% man-years
50, 75, 90
50, 75, 90
50, 75, 90
50, 75, 90
50, 75, 90
50, 75, 90
50, 75, 90
50, 75, 90
If 100% indicate here
Also( describe needs outside implementation plan
i
II. Personnel
Criteria score
>75% - 4
<75% - 0
Administrative Services Staffing Pattern (man-years)
Total
General
Training
Legal _
Information_
Financial
Policy
Planning & eval.
Other
III. Vacancy rate - (Total agency)
Number
vacant
% of total positions
vacant
Budgeted positions
1 Man-years
[Turnover rate - circle one
i 5% 10%.
Criteria score: 5 4
15%
3
15%
0
37
-------
Schedule B, pg. 4
IV. Organization
Agency has:
1. Organization chart
2. Functional statements
3. Agency procedures that clearly
define responsibilities
Yes
1
1
No
0
0
Criteria score
V. Facilities - circle applicable criteria score for each agency operation
Sq. footage per person
150
Administration (per person)
Engineering
Enforcement
Technical Services lab
Data handling
Enter total ft2 available
5
. 4
4
3
5
100
4
3
3
2
4
75
0
2
2
0
0
Administrative Services
I. Training - Current Fiscal Year
A. Total number man-weeks
New employees
Existing employees
B. Program
New staff
Criteria score
Man-weeks avg.
Existing staff
(1-2)
(3-5)
5
Written, formal training program
Orientation for new employees
Outside training program
Identifies new training needs & skills
Yes
•1
1
1
1
1-3)
3
No
0
0
0
0
1
1
3
0
0
0
38
-------
Schedule B, pg. 5
II. Public Information and Education(circle answer)
A. Program Criteria
Yes No
Budgeted, with functional statement 1
Full-time or part-time employee assigned 1
B. Program Services
0
1
Criteria score
Reports
Annual
Quarterly
Monthly
Yes No
0
0
0
News Media
Yes No Number
News release 1 0
Radio & TV release 1 0
Radio & TV
appearance 1 0
Distribution List
Speakers
Civic & school
Citizen groups
Industry
Yes No Number
0
0
0
Publications
Number distributed
Number of kinds
Special Status Reports Yes j
Compliance 1 0
Air monitoring 1 0
Special problems
(new regulations) 1 0
Criteria measurement
Number/100,000 population
39
-------
Schedule C .'
! AIR POLLUTION CONTROL AGENCY
EVALUATION DATA REPORT
Engineering Activities
Program Responsibility \
!
t
I. Program Responsibility Summary Engineering
|
Enter one: State Shared with Delegated Partial Other,
only 1 state 2 by state 3 delegation 4 specify 5
i1
i
Information gathering Enforcement Support Planning
Emission inventory Source testing Regulations
Permit system Source inspections Special Studies
Other, specify Permit Land use
;, Transp.
Compliance sched. Complex source
review & preparation Other
II. Engineering Services Staffing Pattern (man-years)
Total Emission Inventory Permit System
Sourcei testing Source Construction Planning
Other (Insp. & compliance)
Program Operations
i
I. Emission Inventory Sources inventoried F.Y. Total
% Sources not inventoried 0-10% 10-50% 50-100%
'criteria score 5 3 0
40
-------
Schedule C, pig.2
II. Permit System Agency responsibility review I approve
Total permits issued FY .Total sources
% Sources operating without permit 0-10% 10-50%'; 50-100%
j criteria score 5 3 ( 0
!
Review:' Enter total number Const. Operation Limitations*
I '. Size Class
Type: 'Incinerator
?Fuel burning
,tInd. process
I Other
|Criteria score 5 ' 5
* Limitation on agency by agreement, reg., responsibility, etc. for review
or approval of permit.
Review Criteria:
% pt. sources reviewed 100% 75% 50% <25%
(calculations, consultations,
emissions, collection, system,
control system)
Criteria score 543 1
New construction caught by checkoff system Yes No
, Criteria score 1 0
Where: Ex. Bldg. Dept., Zoning agency, other
III. Source Tests Number source tests
Conducted by this agency for agency by
State Contractor Other
I Criteria measure
Enter % sources tested in compliance
41
-------
Schedule C, pg. 3
IV. Compliance Activities
Number of sources on compliance schedules
% of sources required to reduce
Emissions (Implementation Plan) on schedules ' 100% 75
-------
Schedule C, pg. 4
Program Information Criteria
I. Emission Inventory: Check if applicable. Recommendations should be made
By evaluator concerning these areas, if not satisfactory.
Data source: permit agency survey other (specify)
National Emission Data System (NEDS)
Emission factors used:
EPA AP-42 factors Other (specify)
Area source emission estimated by:
Rapid Survey Techniques (AP-29) APTD 1135 Other
Formats
Appendices D, E, F, G of Federal Register (11/25/71)
(*) National Emission Data Systems (NEDS)
Other (specify
»
Point sources included over: 100 tons/yr 50 25 other
Inventory is updated by means of:
Permit system application
Surveillance and investigation activities
Other (specify)
Data is computerized:
By agency
By state I
Other (specify) _
E.I. is used for:
Control strategy
Abatement action
Inspection priorities
Other (specify)
Recommend, if not used by agency.
43
-------
Schedule C, pg. 5
II. Permit System
Reviews and calculations are made to determine volume and
composition of emissions '
The capacity of the air pollution collection system
Effectiveness of control equipment
Other (specify)
Source tests are prerequisite for new or modified sources
Inspections are prerequisite for renewals
Fees are collected '
(Fee schedule (specify)) '
44
-------
C. • EVALUATION OF ENGINEERING FUNCTION
Introduction
An evaluation of engineering should determine!performance, in
terms of activities carried out and results achieved—including air
quality levels and emission rates. It should also determine capability,
in terms of staff experience and education, understanding of Federal,
state and local requirements and objectives, and knowledge and skill
in applying techniques for achieving these objectives. Five basic
activities should be evaluated: (1) emission inventory, (2) permit
system, (3) source testing, (4) regulations development, and (5) special
studies and reports.
Since a limited amount of time will be availab.le for on-site
interviews, much of the information needed for the evaluation should be
obtained in advance. This will enable the evaluator to make the best
use of his time at the agency and concentrate on areas which need
clarification. It also gives the agency an opportunity to review their
activities, organize their information and respond effectively to the
interview. j
The evaluation process involves three steps: (1) information
gathering, (2) site interviews, and (3) appraisal' and report. .
Step 1: Information Gathering
A summary of emission inventory and source compliance data can be
t
tabulated on a form similar to that shown in Figure 1. This provides a
•t
summary of emissions by source categories as well as a summary of
45
-------
work to be done. Compliance, data should be relative to requirements
of the SIP. /
Somej of this information can be obtained from the; agency's semi-
annual Reports and grant applications. Some may navel to be obtained
by means,of a questionnaire directed to the agency and completed prior
to the site visit. |
Step 2.i The Site Visit and Interviews
' I '.
Interviews should be limited to investigating specific problems
confronting the agency and determining the dimensions of and reasons
!
for these problems. The evaluator must adapt his questions accordingly.
For example, if information received in Step 1 above shows that there
I .
are a large number of sources requiring emissions reductions, but a
'»r*f •
small number of these are on compliance schedules, the reasons for the
discrepancy must be found and corrective action taken.
AS another example, if the emission inventory is not yet complete,
it should be determined specifically what plans there are for completing
it and how much manpower will be assigned.
Not all kinds of information can be reduced adequately to a data
i
form, such as: How are job priorities .established, work assignments
made, and how are engineering records filed and retrieved. It is
important to ask these types of questions during the interview, especially
when trying to discover the reasons for poor performance in specific areas.
The evaluator should be thoroughly familiar with the agency's
' i'
responsibilities under the implementation plan. He should determine,
'" !
in the course of the interview, how well the agency understands their
role and what specific plans they have for meeting these responsibilities.
• ''
(i.e., How they are going to complete the/emission inventoryj bring all
sources into'compliance, etc.)
46
-------
Step 3. Appraisal and Report
=
i
Basically the objective in evaluating the agency should be to
/ ' ;
examine areas in which the agency is having or will have problems in
meeting the requirements of the state implementation plan, and then
recommend the steps necessary to meet those requirements.
L
The,iresults of the evaluation will'depend partly on judgments
formed during the interviews as well as appraisal of the data submitted
prior to that time.
It
numerical information.
be of help.to the evaluator to use some system of rating
47
-------
EMISSION INV
IGURE I
RY AND COMPLIANCE DATA
Date
E E
0)
to >
Q) E
O •«-
S-
3 -l->
O O)
00 >,
19—
19—
o 3 re
£>
o
E~
O)
0 >
4J E
•r-
4J O>
re 4->
•O O)
^_
+-> Q.
to E
re o
—J 0
.
IO
E
O
OVi-
E -M
•*— O
i. 3
•r- -a
3 0)
a- s-
Q}
E
to o
a> •!-
u to
i- to
3 •!-
0 E
oo a>
CT>
E
+j ^J
fO »r—
i. E
a; s-
Q. (U
o a.
10 -l->
OJ 3
u o
S- _E
3 4->
O -r-
oo S
-
0>
E
4J
re E
O. (13
O •!—
IO (O
0) >
u
i. ^r
™"3 1 _?
O -r-
oo s
to
a*
3
•o
fi*
ti^
O
to
OJ
(J 4->
E O
re -r-
Q. to 3
Ez fO
O V*
-E O
S- O CJ>
O •!- -(-> ai a.
to re E
re Q co
• j
5
.<->...
Tons Pollutants
-••-••——.. _ oer vear
~---r - t.J'.V-.
O)
re
3
o
-u
S-
re
Q.
19—
,
19-
CM
O
00
19-
19—
-
X
o
19—
19—
~~~~—- - _^_
o
<_)
19--
-"."""
19-V
CJ
3:
19—
19-
-------
Schedule D
AIR-POLLUTION CONTROL AGENCY
EVALUATION DATA REPORT
Enforcement Activities
Program Responsibility:
I. Program responsibility summary - Enforcement
Enter one: State Shared with Delegated Partial Other,
only 1 state 2 by state 4 delegation 4 specify 5
Area Source Surveillance Complaint handling
Transportation control Episode program
Source inspection SCS implementation
Incineration Input for preparation
processes of legal case
fuel burning Prosecution of violators
II. Enforcement Services Staffing Pattern (man-years)
Total staff Field patrol . Complaint handling
Legal proceedings Source inspection
Data & report preparation (consultation) Other
Program Operations ;
I. Source identification
Number of sources under surveillance (total)
open burning
fuel burning
incinerators
process (total)
process particulate
gaseous
other sources (specify)
49
-------
Schedule D, pg. 2
Criteria:
Source listing (circle only one)
Includes all sources subject to regulations and SIP 5
Includes sources over 25 tons/yr. emissions 3
No source listing 0
Source listing update (circle only one)
Annually 3
Every 2 years 2
Every 5 years 1
Never 0
Source listing usage (circle both if applicable)
Develop and/or update emission inventory 3
Start and continue permit system 3
II. Inspections
(Enter number of sources)
Subject to emission regs. j
In violation of emission
regulations (total)
On compliance schedules
Legal action has been
started on
In compliance
In violation with no
compliance schedule and
no legal action started
Enter no. FY corrections by:
Staff
Board (if applicable)
Court
Fuel .Inciner- Industrial^ Open Other,
burning ation burning specify
Enter particulate sources $Q (a) and gaseous sources ir> (b)
-------
Schedule D, pg. 3
Criteria:.
Compliance Schedule Activity (circle all applicable)
Agency enters into legally enforceable compliance,
schedule agreements (milestone dates) ',. 5
Conducts on-site inspections.for progress
.verification '3
Scheduled inspections (circle all applicable)
Conducts periodic scheduled inspections 5
Inspection frequency based on industry type
] and/or emission potential 3
Comprehensiveness of inspection (circle all applicable)
Flow diagrams are made, reviewed or changed 1
Production figures obtained 1
Equipment list compiled, changed, and updated 1
Operating parameters noted 1
Maintenance program reviewed 1
Do not conduct comprehensive on-site inspections 0
III. Field Patrol and Complaint Handling
Number of complaints received per year
*Number of complaints acknowledged .
Number of violations observed
*Number of violations corrected'_
*Criteria for scoring
, Acknowledged % corrections
i >75% - 5 >90% - 5
50% - 74% - 4 75% - 90% - 3
! ! ,£50% - 0 Z.75% - 0
51
-------
Schedule .D, pg. 4
Criteria;
Observed violations are written up and documentedT 5
Area divided into districts for uniform coverage 4
24 hour response capability j 3
A communication system between headquarter and
field personnel 2
/
IV. Emergency Episode Program (Criteria)
A(communication system that can locate a responsible
i'agency official is in effect. _____ 4
An episode manual outlining responsibility of
;agency personnel has been prepared. _____ 3
The episode program has been coordinated with law
enforcement and civil defense agencies. ' ' 2
V. Administrative Procedures (Criteria)
Wrfitten standard operating procedures for
field enforcement personnel ____4
Standardized field forms in use ____3
Granting of a variance requires a compliance
schedule 2
l
Enforcement Records (check all applicable)
Master file on all sources 5
Fifes in alphabetical order 4
Files are cross-referenced 2
No^master file on all sources 0
52
-------
Schedule D, pg. 5
Prosecution \
-• • ' , .''"''."" »
Fijeld enforcement personnel provide input for i
-preparation of legal cases __L__5
Agency has a staff attorney 4
Agency has access to a pool attorney 3
Agency personnel and legal counsel have a close
^working relationship ______ 2
f
!j
Percentage of prosecutions in the agency's favor
80 - 100% 5
50 - 79% 3
25 - 49% ; 1
Less than 25% 0
53
-------
D. . EVALUATION OF ENFORCEMENT ACTIVITIES
• •> ' •
Enforcement is the "activities conducted by an air pollution control
agency to secure certain and continuing control over the sources of air
pollution." The mission of an enforcement program then must be to implement
the plans that have been adopted- to achieve acceptable levels of air quality.
Enforcement therefore overlaps many agency program activities such as
the permit system, source inventory, source testing, and air monitoring if
property line concentrations are a part of the regulations.
Since successful enforcement plays such a key role in improving air
quality, it is important that a careful evaluation be made of this activity.
An agency may have the best air monitoring network in the country or
»
the best laboratory, or the best source inventory, but if it is not bringing
sources into compliance with the emission regulations, then the agency
cannot justify the use of Federal grant money on its program.
An agency's program grant application must reflect SIP goals with
milestone dates, otherwise the evaluator has little to measure the
agency's performance against.
i
- A field operations (enforcement) program consists of key activities
for which objectives should be specified in the grant application. These
program activities and their significance are discussed below.
i
Source Identification - The agency must know how many sources it has and
where they are located before effective planning can take place. Source
identification is even a prerequisite to a comprehensive emission
t
inventory. Source identification must be considered the highest priority
54
-------
/ •'
activity of a new or young agency because it is the basic starting point
or common denominator from which all other agency activities should be
built upon.
Established agencies without a comprehensive source listing could
i
experience difficulty in their emission inventory, permit system operations,
and inspection programs.
Compliance Schedule Inspection - An on-site inspection of sources
to verify milestone dates in the legally enforceable compliance schedule.
Through a permit system or emission inventory the agency should have a
close estimate of sources not meeting the emission regulations. The agency
should then have on file compliance schedules for these sources. The
compliance schedules should include milestone dates to reflect periodic
progress toward completion of the schedule. The field enforcement officers
should verify this by on-site visits to the source shortly after each
milestone date has passed.
Scheduled Inspections - Inspections conducted by a certain date on
an annual or other frequency basis. Inspection scheduling may be
bas'ed on source registration, emission inventory, permit system, complaints,
or other information systems. Ideally, scheduled inspection frequencies
should be proportional to the emission potentials of the source. The
i !A '
scheduling process should not be completely automated'./ It must rely
i . ' •
heavily on the recommendations of the field enforcement officer.
Field Patrol - Sometimes referred :to as surveillance activities. Vehicle
patrol is the principal surveillance method. 'The enforcement officer
should patrol in "a" manner that will bring the greatest area of his district
55.
-------
under view while taking the shortest route. When a questionable visible
emission or violation is observed, the enforcement officer takes a reading
and/or investigates. All observed violations are written up according to
prescribed agency procedure. A communication system will allow the agency
to respond quickly to complaints and emergencies. If necessary, agencies
should have 24 hour a day field patrol capability.
Complaint Handling - An agency should respond quickly to complaints received.
The number of complaints received is influenced by many factors, such as
number of sources, agency publicity, air quality levels, public awareness
of agency, magnitude of emissions, meteorological conditions, etc. Many
of these factors cannot be related to the agency's effectiveness, therefore,
evaluation of complaint handling cannot be related to number of complaints
received. The capability to respond, response time, percent of complaints
responded to, and use of the complaint information, are the important
considerations in evaluating the complaint handling activities of an agency.
Emergency Episode Program - An agency should have the ability to respond
to a meteorological episode or upset involving hazardous pollutants. This
program must be coordinated with law enforcement and civil defense agencies.
An emergency episode procedure manual should be available to all involved
personnel. '
jl
Administrative Procedures - This includes agency procedure manuals written
for field enforcement officers, field forms such as violation notices,
inspection reports, citations, etc,, and the entire spectrum of how the
agency conducts its enforcement program. An agency must have standard
operating procedures'for enforcement. Without.standard operating procedures
there cannot be consistent enforce;..-:,;: practices.
56
-------
Enforcement Records -.The agency should have documentation on all. violators.
Source records should be filed in such a way that access to the information
requires a minimum of effort. All inspection sheets should be on file
along with any documentation for legal action.
Legal Action - Successful prosecution of violators after all other remedies
for relief have failed. The state attorney general or local prosecuting
attorney will usually have responsibility for prosecution before an
administrative body or the courts. It is most important to the agency to
have attorneys familiar with environmental law assigned to air pollution
cases rather than have to start out with a new attorney on each case.
Agency personnel should be informed as. to what documentation is needed for
a legal brief and should work closely with the prosecutor on case
preparation.
Two basic questions that must be addressed in evaluating the legal
action aspects of an enforcement program are: (1) Does the agency follow
up on sources that are in violation of emission regulations?, and (2) Is
the agency able to successfully prosecute violators?
• . A numerical^ value can be assigned to sub-elements under each of the
above program functions. The total points given an agency can then be used
to rank the enforcement activities of all agencies under consideration.
The importance of the total enforcement points scored by an agency '
would have to be considered in light of the enforcement role assigned that
agency by the state implementation plan. .
57
-------
Schedule E
AIR POLLUTION CONTROL AGENCY
EVALUATION DATA REPORT
Technical Services Activities
Program Responsibility
I. Program responsibility summary - Technical Services
Enterj one: State
only 1
Shared with
state 2
Delegated Partial Other,
by state 3 delegation 4 specify 5
Laboratory Services
Pollutants Analyzed
Griterial
I
Hazardous
Trace elements _
^aterial, fuel
qua!., etc. _
Special studies
Equipment
maintenance
Instrument
calibtation
Enforcement Support
Source testing
Special Analytical
studies
Source ident.
Part, ident.
Odors, etc.
Air Monitoring
Minimum F.R.
network
Special, complex
source,
transportation
control
Data Handling
Collection _
Storage and
analysis _
Report prep.
Meteorology
Data acquisition
Forecasts
SCS (modeling)
Eval. of source
Eval. area source
Land Use planning
II. Summary of Staff - Technical Services (man-years)
Total Air monitoring network Laboratory operations
Data processing
Special study
Other
58
-------
Schedule E,: pg. 2
Program Operations
I. Air Monitoring
Type: -(check one)
State
AQCR
Portion of AQCR
Not under SIP
Other monitoring
Enter*number in network: (FY'_
Tape samplers S09 cont.
Hi-vols
Bubbler & sequential
Ozone cont.
HC cont.
CO cont.
Other (specify)
Enter, number of days equipment out of operation due to failure:
Hi-vols Bubbler & sequential Tape samplers
S09 cont. Ozone cont.'_ HC cont.
c.
CO ;cont. Other (specify) '
II. Laboratory Operations
Number of samples analyzed: Total
Hazardous materials Fuel '_
Other
_ Criteria pollutants
Special (FL, Pb, etc.) _
Criteria:
Amount of data validated and reported
90% of possible
90% to 70%
70% or less
Don't know
i
_Good 5
rair 3
Unacceptable 0
Unacceptable 0
If unacceptable is marked the evaluator should go on to the following:
59
-------
Schedule E, pg. 3
Instrument maintenance and calibration is performed:
1. On a predetermined schedule, based on past experience,
to minimize the data from being out of control good 5
2. Every 3 to 6 months routinely fair 3
3. Once per year, or whenever equipment malfunctions poor 2
'4. Never unacceptable 0
Quality Control
Agency
Maintains Quality Control by (check all applicable)
replicate samples blanks_ spiking_
Constructs control chars on critical parameters. Yes No_
Criteria:
The Quality Control Program is:
1. Routinely carried out and adequately documented 5
2. Haphazardly carried out and/or poorly documented 3
3. Not done at all 0
III. Data Processing
Data is reported in format acceptable to EPA for quarterly and
semi-annual reports. Yes 5 No 0
Format used - SAROAD State accepted Other
Data report identifies: (check applicable descriptors)
Site Number of samples Sampling interval maximum
Averages Deviations
Data report summaries identify: (check applicable descriptors)
Relationship of values to standards Trends
Agency provides daily pollution report to public. Yes No_
Criteria Score:
Turnaround time on data reports:
<1 month -5"" 1 to 3 months - 3 >3 months - 0
60
-------
Schedule E, pg. 4
IV. Special Studies and Reports
A.
B.
C.
D.
E.
Mobile source emissions
Transportation controls
Land use alternatives
(complex sources)
Other (specify) SCS, and
large point source (S02)
Technical guides written
(List)
Number of
studies ;
being made'
Network
No. Comp.
Sites Pollutant date
Emergency Episodes
Number of sites designated Frequency of operation
Are meteorological forecasts being made? Yes No
Are supplementary control systems planned?
Can sampling be affected for sources? '
61
------- |