EPA-350-R-98-002
          \   Office of Inspector General
          /   Audit Report
            The Effectiveness and Efficiency
                 of EPA's Air Program
              Report No. E1KAE4-05-0246-8100057
                     February 27, 1998

-------
Inspector General Division
 Conducting the Audit:
Inspector General Divisions
 Contributing to the Issue Area:
Northern Audit Division
  Chicago, IL
Headquarters Audit Division
  Washington, DC

Eastern Audit  Division
  Boston, MA
  New York, NY

Mid-Atlantic Audit Division
  Philadelphia, PA

Southern Audit Division
  Research Triangle Park, NC

Central Audit  Division
  Dallas, TX

Western Audit Division
  San Francisco, CA

Office of Internal Audits
  Washington, DC

Engineering & Science Staff
  Washington, DC
 Program Offices Involved:
 Office of Air and Radiation

 Office of Research and
 Development

 Office of Communications,
 Education, and Public Affairs

-------
              UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
                              WASHINGTON, D.C. 20460
                                   February 27,1998
                                                                          OFFICE OF
                                                                     THE INSPECTOR GENERAL
MEMORANDUM
SUBJECT:
FROM:
TO:
             The Effectiveness and Efficiency of EPA's Air Program,
             Report No. E1KAE4-05-0246-8 100057
             Michael Simmons
             Deputy Assistant Inspector General
                for Internal Audits

             Richard D. Wilson
             Acting Assistant Administrator
                for Air and Radiation

             Henry L. Longest II
             Acting Assistant Administrator
                for Research and Development
             Loretta M. Ucelli
             Associate Administrator
                for Communications, Education, and Public Affairs

       Attached is a copy of our report entitled "The Effectiveness and Efficiency of EPA's Air
Program," Report No. El KAE4-05 -0246-8 100057. This report provides important findings and
recommendations that should significantly improve the efficiency of the Air Program and its
relations with other offices.  The final report incorporates the comments each of your offices
provided on position papers we previously issued.

       We appreciate your staffs' efforts in working with us to develop this report. We
particularly want to thank the Office of Air and Radiation management and staff for their
cooperation and the high level of attention they gave to our work in the Air Program over the past
several years. Without that cooperation and attention,  we could not have produced this report.

-------
Action Required

       In accordance with EPA Order 2750, we have designated the Acting Assistant
Administrator for Air and Radiation as the action official for this report.  As the action official,
he is required to provide us with a written response within 90 days of the date of this report.  The
response should incorporate actions from the Offices of Research and Development, and
Communications, Education, and Public Affairs. The response should also address all
recommendations.  For corrective actions planned but not completed by the response date, please
describe the actions that  are ongoing and provide a timetable for completion. This information
will assist us in deciding whether to close this report.

       We have no objections to the release of this report to the public. Should your staff have
any questions, please contact Kimberly O'Lone, Audit Manager, at 312-886-3186 or Janice
Miller, Team Leader, at 312-886-3084.

Attachment

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                          EXECUTIVE SUMMARY
INTRODUCTION
Since 1993, the Office of Inspector General (OIG) has been
conducting a coordinated body of audit work in the U.S.
Environmental Protection Agency's (EPA) Air Program, that led to
this overall report. We began this effort with a general survey in
1993 and issued a strategic plan in May 1994. We have issued
reports on air grants, toxics, enforcement, state implementation
plans, emission factors, and voluntary programs.  This report
culminates our efforts, pulling together overall issues from OIG
and U.S. General Accounting Office work.

The Office of Air and Radiation (OAR) is responsible for carrying
out EPA's Air Program. Its mission is to protect and enhance the
quality of the nation's air resources and protect human health and
the environment from airborne pollutants and radiation. OAR
carries out this mission by implementing the Clean Air Act, as
amended in 1990 (Act), and the 1993 Climate Change Action Plan.
OAR also develops programs to reduce risk from indoor air
pollution and radiation. This report primarily focuses on OAR's
activities under the Act.
OBJECTIVE
                          The objective of this audit was to determine whether the Air
                          Program was working effectively and efficiently to make the
                          nation's air cleaner.

                          This report is separated into three parts based on the above
                          objective.  Part 1 is the introduction, providing background
                          information on the Air Program and our audit work. Part 2
                          discusses whether the Air Program has been effective. Part 3
                          summarizes our conclusions on whether the Air Program has been
                          efficient.
                                                                      Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
RESULTS IN BRIEF
The Air Program Has
Been Effective in
Reducing Emissions
The Air Program Could
Be More Efficient
OAR data show that the Air Program has been effective in cleaning
the air and reducing the potential for depleting the ozone layer.1
We also concluded that, while the Air Program has generally
operated efficiently, it could also be more efficient.

Between 1987 and 1996, U.S. emissions of all criteria pollutants
declined.  Much of this reduction was due to emission controls
placed on motor vehicles and utilities.  OAR estimates that,
without the emission reductions, there would have been more
health problems, such as heart disease and respiratory illnesses.
Also, because of OAR's Acid Rain Program, the damage to lakes
and forests has been reduced.

Emissions of ozone depleting chemicals have decreased, through
the phase out of the production of certain chemicals, such as
chlorofluorocarbons (CFCs). Without the phase out of CFCs,
OAR predicts that the depletion of the stratospheric ozone layer
would increase, accompanied by an increase in skin cancer.

The Air Program has generally operated efficiently. Several prior
audit reports found that OAR had good management practices and
good internal controls over operations reviewed.  OAR officials,
however, could increase their program efficiency in several ways.
For instance, Air Program officials could improve their relations
with other parties that OAR needs to carry out its mission. These
parties include the Office of Research and Development (ORD),
the Office of Communications, Education, and Public Affairs
(OCEPA), and outside stakeholders, such as states. OAR officials
could also direct more attention to several ongoing activities to
improve their efficiency.  These issues are summarized in the
following paragraphs.
        1 We did not independently verify information relating to emissions or the health effects of emissions.
 Instead, we relied on EPA or other published sources for information in this part of our report.
                                            11
                                                                         Report No. 8100057

-------
                                                       The Effectiveness and Efficiency
                                                                of EPA's Air Program
OAR and ORD Needed    One way the Air Program could be more efficient is if its research
to Ensure That Air       and monitoring needs were adequately met. To do this, officials
Research and Monitoring  from OAR and ORD agreed they needed to work together.
Needs Were Met         Although the officials generally worked well together and agreed
                        on most of their activities, some areas of disagreement existed.
                        The disagreements mainly focused on how ORD used its budget
                        for air activities. ORD has not established a consistent process for
                        making Agency-wide research planning and budgeting decisions.
                        Both offices agreed, however, that the fiscal year 1999 process was
                        an improvement over prior years.  The two offices also did not
                        have  a method for resolving disagreements or sharing decisions
                        with all managers and staff in both offices. Disagreements
                        between the two offices have resulted in impaired working
                        relationships. If the offices do not work well together to resolve
                        disagreements, they may not be making decisions to use their joint
                        resources in the most efficient ways.
OAR and OCEPA
Needed to Work
Together to Raise
Public Awareness
OAR Needed to Give
Stakeholders
Sufficient Feedback
The Air Program could also be more efficient if it worked with
OCEPA to raise public awareness about air pollution.  The public
is important to the success of OAR programs. Officials from the
two offices agreed they have not communicated or coordinated
sufficiently with each other in the past. As a result, OAR did not
always use its resources efficiently.  Recently, officials in both
offices have taken steps to improve their working relationship.
They must continue working together to resolve past differences.

A third way the Air Program could be more efficient is if OAR
was more responsive to stakeholders, such as EPA regional offices,
states, and industry and environmental groups, in implementing the
Act. OAR frequently consulted stakeholders for ideas; however,
some stakeholder relationships could have benefitted from  more
feedback from OAR.  Stakeholders may have perceived a lack of
feedback because (1) they may not have had a clear understanding
of how the stakeholder process worked and entered the partnership
with high expectations, (2) there was not always a formal
mechanism to provide feedback, and (3) the process for working
with stakeholders was lengthy. As a result, some stakeholders
were reluctant to work with OAR again. OAR cannot efficiently
                                       111
                                                                    Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
   OAR Needed to
   Give Attention to
   Several Activities
carry out the Act without the cooperation and help of its
stakeholders.

Finally, the Air Program needs to devote attention to several
ongoing activities to improve efficiency. These activities have
been discussed in prior audit reports and include: leading the state
implementation plan process, developing and improving emission
factors, and issuing air toxic standards.  OAR has not considered
these activities high priorities, compared with other program areas.
This resulted in delays and limited funding.  Consequently, these
activities have operated inefficiently and may not be achieving
their desired results. For example, state plans to achieve emission
reductions could be delayed. The plans may also be incorrect if the
state's estimates of emissions from major sources are based on
inaccurate emission factors. Also, industry deadlines for installing
controls over emissions of air toxics will be delayed if OAR is late
in issuing standards.
RECOMMENDATIONS
                           We recommend that the Acting Assistant Administrator for Air and
                           Radiation:

                           1.     Along with the Acting Assistant Administrator for
                                  Research and Development continue the improvements
                                  made in the fiscal year 1999 research planning process.

                           2.     Work with the Associate Administrator for
                                  Communications, Education, and Public Affairs to establish
                                  procedures to ensure that the offices work up-front with
                                  each other when developing projects to raise public
                                  awareness of air pollution.

                           3.     Establish a process to ensure that feedback is provided to
                                  all stakeholders when OAR has not addressed their
                                  concerns, or has not used the input.

                           Additional and expanded recommendations are included at the end
                           of each chapter, beginning in Part 3.
                                            IV
                                                                         Report No. 8100057

-------
                                                      The Effectiveness and Efficiency
                                                              of EPA's Air Program
AGENCY COMMENTS
AND OIG EVALUATION  The Acting Assistant Administrators for OAR and ORD and the
                         Associate Administrator for OCEPA concurred with our
                         recommendations. They need to provide specific corrective
                         actions and milestone dates for implementing the
                         recommendations. See appendices 1,2, and 3 for the OAR, ORD,
                         and OCEPA responses.
                                                                 Report No. 8100057

-------
                                               The Effectiveness and Efficiency
                                                      of EPA's Air Program
                          Table of Contents

EXECUTIVE SUMMARY	i

ABBREVIATIONS 	«

PART 1: INTRODUCTION	1

INTRODUCTION	2
      Purpose	2
      Background 	2
      Scope and Methodology 	5

PART 2: WAS THE AIR PROGRAM WORKING EFFECTIVELY TO
MAKE THE NATION'S AIR CLEANER?                               7

THE AIR PROGRAM HAS BEEN EFFECTIVE IN REDUCING EMISSIONS  	8
      Criteria Pollutants 	8
      Stratospheric Ozone Depletion  	10
      Conclusion	11

PART 3: WAS THE AIR PROGRAM WORKING EFFICIENTLY TO
MAKE THE NATION'S AIR CLEANER?                              12

THE AIR PROGRAM COULD BE MORE EFFICIENT	13

 CHAPTERS

1     OAR AND ORD NEEDED TO ENSURE THAT AIR RESEARCH AND MONITORING
      NEEDS WERE MET 	I4
           Background 	14
           OAR and ORD Generally Worked Well Together 	15
           Some Disagreements Existed	-	16
           Causes of Disagreements	17
           Disagreements Impaired Relations and Efficiency 	17
           Conclusion	1°
           Recommendations	18
           Agency Comments and OIG Evaluation	19

                                   vi
                                                          Report No. 8100057

-------
                                                   The Effectiveness and Efficiency
                                                           of EPA's Air Program
2     OAR AND OCEPA NEEDED TO WORK TOGETHER TO
      RAISE PUBLIC AWARENESS	20
            Background 	20
            Changing Working Relationship for OAR and OCEPA	20
            OAR and OCEPA Did Not Always Work Well Together	21
            Relationship Impacted Efficiency  	21
            Conclusion	23
            Recommendations 	23
            Agency Comments and OIG Evaluation	23

3     OAR NEEDED TO GIVE STAKEHOLDERS SUFFICIENT FEEDBACK 	24
            Background 	24
            OAR Asked for Input From Stakeholders	26
            OAR Did Not Always Give Feedback	26
            Stakeholders Perceived Lack of Feedback	27
            Stakeholders Hesitant to Work With OAR 	28
            Conclusion	29
            Recommendations	29
            Agency Comments and OIG Evaluation	30

4     OAR NEEDED TO GIVE ATTENTION TO SEVERAL ACTIVITIES	31
            Background 	31
            SIP Processing Needed to be Improved	33
            Emission Factors Needed to be  Developed and Improved 	34
            Timely Air Toxic Standards Needed	35
            Conclusion	37
            Recommendations from Prior Audit Reports  	38

EXHIBITS

1     EPA OIG AND GAO REPORTS 	40

2     SCOPE, METHODOLOGY, AND PRIOR AUDIT COVERAGE 	42

3     ISSUES NEEDING FURTHER STUDY  	45

APPENDICES

1     OFFICE OF AIR AND RADIATION RESPONSE  	47
                                     vu
                                                              Report No. 8100057

-------
                                              The Effectiveness and Efficiency
                                                      of EPA's Air Program
2    OFFICE OF RESEARCH AND DEVELOPMENT RESPONSE 	49

3    OFFICE OF COMMUNICATIONS, EDUCATION, AND
     PUBLIC AFFAIRS RESPONSE	51

4    DISTRIBUTION	53
                                  Vlll
                                                         Report No. 8100057

-------
                                                      The Effectiveness and Efficiency
                                                              of EPA's Air Program
Act

CCAP

CFC

EPA

GAO

MACT

NOAA

OAR

OCEPA

OIG

OMB

QMS

ORD

SIP

Subcommittee


UV
             Abbreviations

Clean Air Act, as amended in 1990

Climate Change Action Plan

Chlorofluorocarbon

Environmental Protection Agency

U.S. General Accounting Office

Maximum Achievable Control Technology

National Oceanic and Atmospheric Administration

Office of Air and Radiation

Office of Communications, Education, and Public Affairs

Office of Inspector General

Office of Management and Budget

Office of Mobile Sources

Office of Research and Development

State Implementation Plan

Clean Air Act Advisory Committee subcommittee for Ozone, Paniculate
Matter, and Regional Haze Implementation Programs

Ultraviolet
                                        IX
                                                                  Report No. 8100057

-------
                    The Effectiveness and Efficiency
                           of EPA's Air Program
   PARTI

Introduction
                               Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                                     PARTI
                                   Introduction
PURPOSE
BACKGROUND
Since 1993, the Office of Inspector General (OIG) has been
conducting a coordinated body of audit work in the U.S.
Environmental Protection Agency's (EPA) Air Program. We
began this effort with a general survey in 1993 and issued a
strategic plan in May 1994. We have issued reports on air grants,
toxics, enforcement, state implementation plans, emission factors,
and voluntary programs. This report culminates our efforts,
pulling together overall issues from OIG and U.S. General
Accounting Office (GAO) work. See exhibit 1.

The objective of this audit was to determine whether the Air
Program was working effectively and efficiently to make the
nation's air cleaner.
                          The Office of Air and Radiation (OAR) is responsible for carrying
                          out EPA's Air Program. OAR's mission is to protect and enhance
                          the quality of the nation's air resources and protect human health
                          and the environment from airborne pollutants and radiation. OAR
                          carries out this mission by implementing the Clean Air Act, as
                          amended in 1990 (Act), and the 1993 Climate Change Action Plan.
                          OAR also develops and implements programs to reduce risk from
                          indoor air pollution and radiation.

                          OAR is headquartered in Washington, D.C., and is comprised of
                          four main program offices, as shown in figure 1.
                                                                     Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
Figure 1: Office of Air and Radiation
Organization Chart


Air Quality
Planning and Standards
(Research Triangle
Park, NC)

Atmospheric Programs
(Washington, DC)







Mobile Sources
(Washington, DC
and Ann Arbor, MI)

Radiation and
Indoor Air
(Washington, DC, with
labs in Las Vegas, NV
and Montgomery, AL)

OAR Responsibilities
OAR has defined four areas of fundamental responsibility:
implementing the Act; carrying out the Climate Change Action
Plan (CCAP); reducing risks from radon and other indoor air
pollutants; and protecting health and the environment from
radiation. Figure 2 shows OAR's fiscal 1996 resources (excluding
grants), as allocated among these responsibilities.
                                    Figure 2:  OAR FY 1996 Resources
                                              (in Millions)
                                                                 Other
                                                                 $8.4
                                                                  Radiation
                                                                  $13.9
                                                                  Indoor Air
                                                                  $18.1
                            OAR spent the bulk of its resources on responsibilities associated
                            with implementing the Act.  For instance, in fiscal 1996, OAR
                            spent about $175 million on its own programs to:

                             •     set health-based national ambient air quality standards for
                                  specific pollutants, and help state, local, and tribal
                                  governments develop and implement programs to prevent
                                  and control those pollutants;
                                                                         Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
                            •     develop standards to control the release of air toxics, which
                                  are pollutants that are known or suspected to cause cancer
                                  or other serious health effects;

                            •     set standards to reduce emissions from motor vehicles
                                  through tailpipe standards, evaporative emissions controls,
                                  on-board vapor recovery systems, and cleaner fuels;

                            •     operate the Acid Rain program, a market-based program to
                                  reduce sulfur dioxide and nitrogen oxide emissions from
                                  utilities; and

                            •     oversee U.S. efforts to stop stratospheric ozone depletion
                                  by regulating the production, use, and disposal of ozone-
                                  depleting substances, and implementing U.S.
                                  responsibilities under the revised Montreal Protocol.2

                           Another OAR fundamental responsibility is to implement
                           provisions of the CCAP. OAR's role is to develop and implement
                           programs to stimulate  and transform the markets for technologies
                           that reduce emissions for carbon dioxide, methane, and other
                           compounds that contribute to global warming. OAR allocated over
                           $69 million from its fiscal 1996 budget to implement the climate
                           change programs.

                           A third area of fundamental responsibility is aimed at reducing the
                           public health risks from radon and other indoor pollutants.  OAR
                           spent almost $18 million on its indoor environments programs.
                           These activities are authorized by the Indoor Radon Abatement Act
                           and the Superfund Amendments and Reauthorization Act.

                           OAR's remaining fundamental responsibility is to protect the
                           public health and the environment from radiation exposure, upon
                           which it spent about $14 million. Because our work focused on the
                           Air Program, we did not evaluate any of OAR's radiation
                           programs.
       2The Montreal Protocol is an international agreement on ozone-depleting substances. The agreement was
originally signed in 1987, and was substantially amended in 1990 and 1992.

                                           4
                                                                        Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
State Responsibilities
SCOPE AND
METHODOLOGY
State and local agencies are responsible for carrying out many of
the Act's provisions.  The Act had states take the lead in carrying
out its provisions, because pollution control problems often require
special understanding of local industries, geography, housing
patterns, and other factors. While the law requires OAR to set
limits on how much of a pollutant can be in the air anywhere in the
U.S., it also allows individual states to establish stronger pollution
controls. States are not, however, allowed to have weaker
pollution controls than those set for the whole country.

OAR provides funding to state and local agencies to assist in their „
programs. In Section 105 of the Act, OAR is authorized to provide
grants to state and local agencies.  These grants are designed to
help the agencies establish and operate air pollution control
programs. In fiscal 1996, OAR allocated just over $163 million to
these grants. OAR also provided $6.6 million to state and local
agencies for indoor environments programs.
We performed our audit in accordance with the U.S. GAO's
Government Auditing Standards, as issued by the Comptroller
General of the United States (1994 Revision).

As part of this audit, we relied on reports OIG and GAO auditors
prepared. A complete listing of audit reports is included in exhibit
1. The reports we used for this audit were also performed in
accordance with the Government Auditing Standards with the
following exceptions:

       OIG Special Reviews3

       EPA's Air State Implementation Plan Program
         Consolidated Report
       EPA's Development of its Proposed Open Market
         Trading Rule
       3These special reviews were more limited in scope than an audit and were conducted in accordance with
 OIG Manual Chapter 150.
                                                                         Report No. 8100057

-------
                                                            The Effectiveness and Efficiency
                                                                     of EPA's Air Program
                                  PIG Surveys4

                                  Region 3 Title V State Operating Permit Program
                                  Acid Rain Allowance Trading Program

                           The standards the OIG and GAO followed gave us reasonable
                           assurance of the quality and accuracy of the information. We
                           relied on the information in the reports to reach conclusions on
                           how the Air Program was managed and cited examples from the
                           prior reports in this report.

                           For our audit objectives, we performed a limited assessment of
                           internal controls, which included reviewing OAR's Assurance
                           Letters for fiscal years 1994,1995, and 1996.  The 1994 report
                           identified inaccurate data in the state implementation plan tracking
                           system as an Agency level weakness, but this weakness was shown
                           as corrected in 1995. We did not detect any material internal
                           control weaknesses during this audit. However, we continue to
                           believe that the emission factor program is a material weakness, as
                           identified in our September 30,1996 report, Emission Factor
                           Development, EPA OIG Report No. 6100318. We discuss this
                           issue further in chapter 4.

                           See exhibit 2 for scope and methodology details, as well as prior
                           audit coverage. See exhibit 3 for issues needing further study.
       4These surveys did not identify potential issues warranting additional audit work. As such, reports were
not issued to the Air Program. They were performed in accordance with Government Auditing Standards, with the
exception of the reporting requirement.
                                                                         Report No. 8100057

-------
                           The Effectiveness and Efficiency
                                  of EPA's Air Program
            PART 2

Was the Air Program Working
      Effectively to Make
   the Nation's Air Cleaner?
                                     Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                   of EPA's Air Program
                                      PART 2
                    The Air Program Has Been Effective
                             In Reducing Emissions
CRITERIA
POLLUTANTS

Background
                           Office of Air and Radiation (OAR) data show that the Air Program
                           has been effective in cleaning the air and reducing the potential for
                           depleting the ozone layer.5 Between 1987 and 1996, U.S.
                           emissions of all criteria pollutants declined. Much of this
                           reduction was due to emission controls placed on motor vehicles
                           and utilities.  OAR estimates that, without the emission reductions,
                           there would have been more health problems, such as heart disease
                           and respiratory illnesses.  Also, because of OAR's Acid Rain
                           Program, the damage to lakes and forests has been reduced.

                           Emissions of ozone depleting chemicals have decreased, through
                           the phase out of the production of certain chemicals, such as
                           chlorofluorocarbons (CFCs). Without the phase out of CFCs,
                           OAR predicts that the depletion of the stratospheric ozone layer
                           would increase, accompanied by an increase in skin cancer.
The Clean Air Act of 1970 required EPA to identify air pollutants
that it anticipated endangered public health. EPA identified six
"criteria" pollutants, for which it has promulgated the National
Ambient Air Quality Standards.  These pollutants are:  (1) lead, (2)
carbon monoxide, (3) sulfur dioxide, (4) particulate matter, (5)
ground-level ozone, and (6) nitrogen dioxide. The standards
specify acceptable air pollution concentrations for a geographic
area. The Clean Air Act, as amended in 1990 (Act), classified as
non-attainment areas those places that exceeded the standards and
established deadlines for states to achieve them.
       We did not independently verify information relating to emissions or the health effects of emissions.
Instead, we relied on EPA or other published sources for information in this part of our report.
                                           8
                                                                       Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
Emissions Have
Gone Down
According to the most current OAR data, the concentrations of all
six criteria pollutants have gone down, as shown in table 1.6
                           Table 1:
                      Criteria Pollutants
Pollutant
Lead
Carbon Monoxide
Sulfur Dioxide
Particulate Matter
(1988 -1996 data)
Ozone/Volatile
Organic Compounds
Nitrogen Dioxide
Air quality
concentrations
% decrease
1987 - 1996
75%
37%
37%
25%
15%
10%
                           These reductions have occurred in a time of growth. Since the Act
                           was passed in 1970, the U.S. economy, population, and on-road
                           miles driven have all increased. These factors are all traditional
                           causes of air quality problems.

                           OAR also reported that some emissions have been reduced ahead
                           of schedule.  Sulfur dioxide emissions, which form one of the
                           components of acid rain, were reduced from the 1980 level of
                           almost 11 million tons to just over 5 million tons in 1995.  This
                           was 39% below the 1995 allowable emissions level of 8.7 million
                           tons the Act required.
        National Air Quality and Emissions Trends Report, 1996, EPA-454/R-97-013, January 1998.

                                            9
                                                                        Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
The Act Has
Led to the
Reductions
Reductions Help
People and the
Environment
In the National Air Quality and Emissions Trends Reports from
1995 and 1996, OAR attributed the reductions of criteria pollutants
to steps taken under the Act. OAR concluded that changes in
gasoline (oxygenated fuels and unleaded and reformulated
gasoline) and other motor vehicle controls resulted in decreases in
carbon monoxide, lead, and ground-level ozone.  Utilities using
new control technologies, such as scrubbers, and switching to low
sulfur fuels, affected sulfur dioxide and nitrogen dioxide
emissions.  Finally, a decrease in residential wood burning resulted
in reduced particulate matter emissions.

According to OAR, the emission reductions achieved may help
improve public health and the environment. Exposure to
carbon monoxide, particulate matter, lead, and sulfur dioxide can
result in serious health threats to people with heart disease.
Nitrogen dioxide, ground-level ozone, particulate matter, and
sulfur dioxide can also cause or aggravate respiratory illnesses.
Exposure to lead may also result in neurological impairments, such
as mental retardation and behavioral disorders.
                           A study prepared for the U.S. Geological Survey found that rainfall
                           in the Eastern half of the country was 10 to 25% less acidic
                           because of OAR's Acid Rain Program.  As a result, lakes and
                           streams affected by acid rain will be able to recover, restoring fish
                           and other life. Forests will also be restored and visibility will
                           improve.
STRATOSPHERIC
OZONE DEPLETION

Background
Under the Act and the 1987 Montreal Protocol, OAR was required
to write regulations to phase-out the production and sale of CFCs
and several other chemicals.7 These chemicals may be causing the
destruction of the stratospheric ozone layer.
       7The Montreal Protocol is an international agreement on ozone-depleting substances. The agreement was
originally signed in 1987, and was substantially amended in 1990 and 1992.
                                           10
                                                                        Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                 ofEPA'sAir Program
Emissions Have
Gone Down
Production Bans
Have Led to Reductions
Reductions Will
Help People and
the Environment
CONCLUSION
Emissions of ozone depleting chemicals have gone down. For
example, scientists from the Department of Commerce's National
Oceanic and Atmospheric Administration (NOAA) reported that,
in 1995, the total amount of ozone-destroying chemicals in the
lower atmosphere declined for the first time.

Production bans under the Act led to decreases in ozone depleting
chemicals. NOAA scientists concluded that the decline in ozone
destroying chemicals in the atmosphere occurred because many
nations have limited the production of those chemicals. In July
1992, OAR issued its final rule that limited the production and
consumption of these chemicals. OAR issued allowances or
permits for producing and importing the chemicals. Most major
ozone depleting substances were phased out by December 1995. A
remaining chemical, methyl bromide, may be produced and
consumed through the year 2000, when it will also be phased out.

Halting the depletion of the stratospheric ozone layer will reduce
harmful effects of exposure to the sun, such as skin cancer.  Ozone
depletion allows more ultraviolet (UV)-b radiation to reach the
Earth's surface. According to OAR, increased UV-b radiation
causes human skin cancers, cataracts, and impairs human immune
systems. Increased UV-b radiation also reduces crop yields and
threatens plant and animal life.
                          OAR data show that the Air Program has been effective in cleaning
                          the air and reducing the potential for depleting the ozone layer.
                                          11
                                                                      Report No. 8100057

-------
                           The Effectiveness and Efficiency
                                  of EPA's Air Program
            PART 3

Was the Air Program Working
      Efficiently to Make
   the Nation's Air Cleaner?
                12
                                     Report No. 8100057

-------
                                         The Effectiveness and Efficiency
                                                  of EPA's Air Program
                     PART 3
The Air Program Could Be More Efficient
          The Air Program has generally operated efficiently. Several prior
          audit reports found that the Ah" Program had good management
          practices and good internal controls over operations reviewed. For
          instance:

           •     The voluntary programs for Radon and ENERGY STAR®
                 used good management practices and developed ways to
                 estimate their environmental results.

           •     The Green Lights program had adequate program
                 operations and internal controls, and it had direct and
                 specific plans for addressing program barriers.

           •     EPA regions (1) had adequate procedures in place for
                 obtaining, reviewing, and approving state implementation
                 plans (SIPs), (2) established a good working relationship
                 with Headquarters and states, and (3) worked up-front with
                 states to get SIPs submitted on time.

           •     The Office of Air and Radiation's (OAR) Acid Rain
                 Division developed procedures and effectively
                 implemented the Acid Rain Permits Program, Allowance
                 Tracking System, and Continuous Emissions Monitoring
                 System.

          While the Air Program has been efficient in many ways, it can still
          improve its efficiency.  For instance, Air Program officials could
          improve their relations with other parties that OAR needs to carry
          out its mission. These parties include the Office of Research and
          Development, the Office of Communications, Education, and
          Public Affairs, and outside stakeholders, such as states.  OAR
          could also be  more efficient if it directed more attention to several
          ongoing activities.  The following chapters discuss in detail how
          the Air Program could work  more efficiently to make the nation's
          air cleaner.


                          13
                                                      Report No. 8100057

-------
                                                        The Effectiveness and Efficiency
                                                                 of EPA's Air Program
                                     PART 3
                                  CHAPTER 1
                  OAR and ORD Needed to Ensure That
             Air Research and Monitoring Needs Were Met


                          One way the Air Program could be more efficient is if its research
                          and monitoring needs were adequately met. To do this, officials
                          from the Offices of Air and Radiation (OAR) and Research and
                          Development (ORD) agreed they needed to work together.
                          Although the officials generally worked well together and agreed
                          on most of their activities, some areas of disagreement existed.
                          The disagreements mainly focused on how ORD used its budget
                          for air activities. ORD had not established a consistent process for
                          making Agency-wide research planning and budgeting decisions.
                          Both offices agreed, however, that the fiscal year 1999 process was
                          an improvement over prior years. The two offices also did not
                          have a method for resolving disagreements or sharing decisions
                          with all managers and staff in both offices. Disagreements
                          between the two offices have resulted in impaired working
                          relationships. If the offices do not work well together to resolve
                          disagreements, they may not be making decisions to use their joint
                          resources in the most efficient ways.
BACKGROUND
                          OAR has many research and monitoring needs to support its air
                          programs. Air Program officials perform some of this research and
                          monitoring, while outside organizations, such as ORD, other
                          government agencies, or contractors, also perform some. ORD's
                          activities include a range of functions: technical support, such as
                          monitoring to detect air pollution trends; long-term research to find
                          new ways to prevent and control ah* pollution; risk assessments for
                          new standards and other pollutants; developing measurement and
                          reference methods; research on the health and ecological effects of
                          air pollutants; and working on Air Program reports, such as the
                          Mercury Report to Congress.
                                         14
                                                                    Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                          In fiscal year 1997, ORD allocated just over 20% of its Enacted
                          Operating Plan budget to Air Program activities.  ORD used $91.4
                          million of its $439.4 million budget for air-related research and
                          technical support activities. As shown in figure 3, of the $91.4
                          million, about three quarters was for research activities and one
                          quarter was for technical support.
                                   Figure 3: ORD's Air-Related Resources
                                                 FY 1997
                                      Research
                                                             Technical Support
                          The past several years have been a period of change for ORD and
                          the Agency's research, development, and technical support
                          planning process.  In July 1994, an Agency-wide Steering
                          Committee issued a report to the Administrator, Research,
                          Development, and Technical Services at EPA: A New Beginning.
                          To implement the report's recommendations, in 1996, ORD issued
                          a Strategic Plan after obtaining input from both within and outside
                          the Agency.  The plan identified priorities, as well as areas for
                          disinvestment. ORD also established a new planning process
                          based on priorities identified by the program and regional offices
                          and risk-based decision-making.

OAR AND ORD
GENERALLY WORKED OAR and ORD generally worked well together and agreed on most
WELL TOGETHER      of their activities, such as then" coordination on the newly revised
                          ambient air standards for ozone and particulate matter.  While the
                          two offices have an overall positive working relationship, officials
                          from both offices agreed that their working relationships could

                                          15
                                                                       Report No. 8100057

-------
                                                             The Effectiveness and Efficiency
                                                                      of EPA's Air Program
                            improve if they better resolved disagreements. They need to build
                            upon this relationship to address the issues cited in this chapter.
SOME
DISAGREEMENTS
EXISTED
Some areas of disagreement existed between the two offices. The
disagreements mainly focused on (1) how ORD used what it
described as its limited resources, and (2) which of the two offices
would perform needed monitoring activities.  For instance, ORD
officials decided that, with declining resources, their office could
not continue performing both research activities and all monitoring
activities. Therefore, consistent with its 1996/1997 Strategic Plan,
ORD chose to disinvest in "routine" monitoring activities, while
maintaining the more technical or research-oriented monitoring
networks.8  Air Program officials disagreed with ORD's decision
to phase out some air monitoring activities.

The two offices also disagreed about which office should perform
the monitoring activities. ORD officials believed that, as the
monitoring activities evolved from being research in nature to
more routine, the Air Program should take over the activities. This
transition would allow ORD's limited resources to be devoted to
research. However, OAR expected ORD to continue its traditional
role in providing monitoring support and believed much of the
monitoring in question was not "routine."  Air Program officials
stated that they did not have enough time to adequately reallocate
their resources to handle the monitoring when ORD  presented
them with a disinvestment plan. OAR also believed that, if
monitoring functions were to be transferred, they should be
accompanied by the associated resources.
        One example of a distinction between routine and research-oriented monitoring involved the Agency's
ultraviolet-b radiation monitoring network. Although ORD had considered transferring responsibility for the
network to the Air Program, ORD instead decided to maintain the network, because calibration and operation of the
current state-of-the-art monitors were very technical and, as such, did not constitute routine monitoring.
                                            16
                                                                          Report No. 8100057

-------
                                                        The Effectiveness and Efficiency
                                                                 of EPA's Air Program
CAUSES OF
DISAGREEMENTS       The disagreements between the two offices occurred for several
                          reasons. For instance:

                           •     OAR was concerned that ORD had not established a
                                 consistent process for making planning and budgeting
                                 decisions that provided clear opportunities for input by
                                 stakeholder offices, such as OAR.  Although an Air
                                 Program official said the just-completed planning process
                                 for fiscal year 1999 was better than it had been in prior
                                 years, he did not think it was well-defined. Other Air
                                 Program managers also were not sure how, or even if, they
                                 had input to ORD's decisions.

                           •     ORD expressed concern that OAR had not always set
                                 priorities for its research and monitoring needs, or indicated
                                 what activities ORD could eliminate if sufficient funding
                                 was not available to meet all of the needs. According to
                                 ORD, program offices need to convey and prioritize their
                                 research, development, and technical support needs, so
                                 ORD can put the needs in context with those of other
                                 offices.

                           •     Neither office had a mechanism in place for resolving
                                 disagreements over the decisions they made. They also did
                                 not have a way of sharing decisions affecting air research
                                 and monitoring between the two offices and among the
                                 managers and staff within each office.

                           •     The two offices had not agreed on the definition of routine
                                 monitoring and which office should perform such
                                 monitoring.
DISAGREEMENTS
IMPAIRED RELATIONS Disagreements between the two offices impaired working
AND EFFICIENCY       relationships and efficiency.  Air Program managers were very
                          frustrated with ORD, because they were not always aware of, or
                                         17
                                                                     Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                   of EPA's Air Program
                           did not understand, ORD's decisions on air research and
                           monitoring activities.  ORD officials, in turn, were frustrated with
                           Air Program officials, because they requested many research and
                           monitoring projects, without establishing priorities, especially
                           across the different OAR offices.

                           If the two offices do not work well together, they may not be
                           making decisions to use their joint resources in the most efficient
                           ways.  For example, because OAR did not clearly identify its top
                           priorities, ORD officials could not always make informed
                           decisions about which Air Program research and monitoring
                           activities were the most important.  If ORD chose not to perform
                           some activities OAR considered critical, then Air Program officials
                           would need to find other resources to fund those activities. If the
                           two offices worked together to establish their joint priorities, then
                           they could conduct the most important activities, while less
                           important activities could wait until resources were available.
CONCLUSION
RECOMMENDATIONS
                           If OAR and ORD worked together to resolve their disagreements
                           over how air research and monitoring resources should be spent,
                           and what activities each office should do, they could improve the
                           efficiency of their joint efforts.
                           We recommend that the Acting Assistant Administrator for Air and
                           Radiation and the Acting Assistant Administrator for Research and
                           Development work together to:

                           1 -1.   Continue the improvements made in the fiscal year 1999
                                 research planning process including making sure that in
                                 future processes:

                                 a.     ORD provides a consistent process for making
                                        planning and budgeting decisions, which identifies
                                        opportunities for input from the program offices and
                                        regions at key decision points.
                                          18
                                                                       Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                                 b.     OAR identifies up-front its cross-office priorities,
                                       and both offices agree on the lower priority
                                       activities that will be eliminated if resources are
                                       limited.

                                 c.     each office informs its managers and staff at each
                                       critical stage of the planning and budgeting process.

                                 d.     both offices commit to resolve differences on short-
                                       term or time-critical issues expeditiously, seeking
                                       higher resolution of disagreements only as a last
                                       resort and doing so together.

                           1 -2.   Use the Agency's Environmental Monitoring Management
                                 Council as a forum for addressing multi-office technical
                                 support issues, such as defining routine monitoring and
                                 which offices should perform it.

AGENC^OMMENl?"
AND OIG EVALUATION  Following our briefing on the results of our fieldwork, OAR and
                           ORD officials concurred that they needed to work together to
                           resolve their disagreements.  In a conference call on September 3,
                           1997, officials from the two  offices proposed several actions they
                           could take to resolve these concerns. These actions are included in
                           the recommendations.

                           Both Acting Assistant Administrators for OAR and ORD
                           concurred with our findings  and recommendations. They need to
                           provide specific corrective actions and milestone dates for
                           implementing the recommendations.
                                          19
                                                                      Report No. 8100057

-------
                                                       The Effectiveness and Efficiency
                                                                of EPA's Air Program
                                    PART 3
                                 CHAPTER 2
                        OAR and OCEPA Needed to
                Work Together to Raise Public Awareness
BACKGROUND
CHANGING WORKING
RELATIONSHIP FOR
OAR AND OCEPA
                         A second way the Air Program could be more efficient is to work
                         with the Office of Communications, Education, and Public Affairs
                         (OCEPA) to raise public awareness about air pollution.  The public
                         is important to the success of Office of Air and Radiation (OAR)
                         programs. Officials from the two offices agreed they have not
                         communicated or coordinated sufficiently with each other in the
                         past. As a result, OAR did not always use its resources  efficiently.
                         Recently, officials in both offices have taken steps to improve their
                         working relationship. They must continue working together to
                         resolve past differences.
                         EPA's program offices individually develop education and
                         marketing materials, but are to coordinate them through OCEPA.
                         For example, OCEPA reviews, approves, and distributes education
                         and marketing materials that the Air Program develops.

                         Education and awareness are pivotal for OAR's voluntary
                         programs, such as those to reduce radon exposure, greenhouse
                         gases, and energy consumption. OAR's Office of Mobile Sources
                         sees public education as a key to convincing people to reduce
                         future automobile emissions and accept other alternatives to
                         driving. The Organization for Economic Cooperation and
                         Development agrees, and recommended in 1996 that the U.S.
                         strengthen education to  increase public understanding about the
                         effects of air pollution caused by vehicle use.
Recently, OAR and OCEPA officials have agreed they needed to
improve their working relationship and have begun taking steps to
do so. OCEPA has had a change in management, and plans to
reorganize soon.  Anticipation of the reorganization and other
                                        20
                                                                   Report No. 8100057

-------
                                                       The Effectiveness and Efficiency
                                                                of EPA's Air Program
OARANDOCEPA
DID NOT ALWAYS
WORK WELL
TOGETHER
RELATIONSHIP
IMPACTED
EFFICIENCY
                         changes have helped OCEPA officials streamline their efforts,
                         allowing them to concentrate on areas where they will have the
                         most impact. OAR officials reported that OCEPA staff members
                         are now more responsive and helpful. Officials from both offices
                         also meet together frequently.  Communication is more open and
                         constructive, fostering a cooperative relationship, instead of an
                         adversarial one.
Conflicts between OAR and OCEPA caused tension and resulted
in staff members not working well together. The two offices had
not always coordinated on projects to raise public awareness of air
pollution. For instance, OCEPA officials said that OAR should
involve OCEPA early in educational and outreach projects.
Because Air Program officials did not always do this, OCEPA did
not review the products until they were completed.  OCEPA then
questioned the effectiveness or cost of some products. OAR
officials thought OCEPA needlessly delayed the projects with
extensive questions and did not always add value to the projects.
The lack of communication and coordination between OAR and
OCEPA resulted in OAR not always using its resources in the most
efficient way.  Staff in both offices were frustrated and had
difficulty working together.  Officials felt they were working
against each other, not together to accomplish OAR's mission.
Among other instances, tension was created when:

 •     OCEPA was not involved with OAR's development of a
       refrigerator-style magnet for the ENERGY STAR program.
       (See figure 4 for a representation of the magnet.) OAR
       created the magnet for contractors to give to the public,
       providing information about types of equipment that have
       met the ENERGY STAR energy efficiency criteria. OCEPA
       officials were disturbed because (1) they thought OAR
       should have consulted them about information to include
       on the magnet, and (2) the magnet was printed without
       OCEPA's approval.  An OCEPA official stated that the
                                         21
                                                                    Report No. 8100057

-------
                              The Effectiveness and Efficiency
                                       of EPA's Air Program
      magnet was difficult to read and did not contain useful
      information, such as a phone number. OAR officials stated
      that it was not clear that OCEPA was supposed to review
      magnets before they were released.

           Figure 4: OAR Energy Star Magnet
                 f «*S4=18£B HEA.TPUMPS* "FURNACES
 •     The two offices could not arrange to work together on a
      fast-track project with the American Lung Association.
      The American Lung Association and OAR planned to enter
      a cooperative agreement to develop a public service
      announcement about ground-level ozone. To complete the
      announcement in time for ozone season, the project needed
      to proceed quickly. OCEPA officials, however, could not
      accommodate OAR's request for a fast-track review.
      OCEPA's workload would not allow it to look at the
      announcement until after the ozone season, in September.
      OAR officials were frustrated that OCEPA officials could
      not make time to review the project, even though it was
      important to them.  OAR eventually did not finalize the
      planned cooperative agreement with the American Lung
      Association because the project could not be completed
      timely.

By not working together, the two programs were not operating as
efficiently as possible.
               22
                                            Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
CONCLUSION
RECOMMENDATIONS
                          Recognizing that they needed to improve their communication and
                          coordination, OAR and OCEPA officials have taken steps to begin
                          improving their working relationship. Working together, OCEPA
                          and OAR can inform the public about air pollution causes, effects,
                          and remedies, in common sense ways.
                          We recommend that the Acting Assistant Administrator for Air and
                          Radiation work with the Associate Administrator for
                          Communications, Education, and Public Affairs to:

                          2-1.   Ensure that product review requirements are followed.

                          2-2.   Establish procedures to ensure that the offices work up-
                                 front with each other when developing projects to raise
                                 public awareness of air pollution.

                          We also recommend that the Associate Administrator for
                          Communications, Education, and Public Affairs:

                          2-3.   Establish concise product review procedures that serve as a
                                 clear and simple guide to the product review process.
AGENCY COMMENTS
AND OIG EVALUATION Following our briefing on the results of our fieldwork, OAR and
                          OCEPA officials agreed they needed to improve their coordination
                          and communication. In August 1997, officials from the two
                          offices committed to work together.  Several of their proposed
                          actions are included in the recommendations.

                          Both the Acting Assistant Administrator for OAR and the
                          Associate Administrator for OCEPA concurred with our
                          recommendations. OCEPA provided corrective actions, but needs
                          to provide a milestone date for recommendation 2-2. OAR needs
                          to provide specific corrective actions and milestone dates.
                                         23
                                                                      Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                                     PART 3
                                  CHAPTER 3
                             OAR Needed to Give
                      Stakeholders Sufficient Feedback
                          A third way the Air Program could be more efficient is if the
                          Office of Air and Radiation (OAR) was more responsive to
                          stakeholders, such as EPA regional offices, states, industry, and
                          environmental groups, in implementing the Clean Air Act, as
                          amended in 1990 (Act).  OAR frequently consulted stakeholders
                          for ideas; however,  some stakeholder relationships could have
                          benefitted from more feedback from OAR.  Stakeholders may have
                          perceived a lack of feedback because (1) they may not have had a
                          clear understanding of how the stakeholder process worked and
                          entered the partnership with high expectations, (2) there was not
                          always a formal mechanism to  provide feedback, and (3) the
                          process for working with stakeholders was lengthy. As a result,
                          some stakeholders were reluctant to work with OAR again. OAR
                          cannot efficiently carry out the Act without the cooperation and
                          help of its stakeholders.
BACKGROUND
                          OAR officials recognize that working with stakeholders is critical
                          to the success of their programs.  According to OAR, "by working
                          together as stakeholders with common goals, federal, state and
                          local agencies can help each other in attaining and preserving
                          clean air in the United States."9

                          OAR does not have the mandate or the resources to carry out the
                          Act itself. Instead, the Act encouraged OAR to consult
                          stakeholders and made state and local entities, with EPA oversight,
                          largely responsible for implementation. With that responsibility,
                          Federal funds have also gone to state and local entities. For
                          example, as shown in figure 5, of OAR's fiscal year 1996
       9OAR Stakeholders Page, September 1997.

                                          24
                                                                      Report No. 8100057

-------
                                                      The Effectiveness and Efficiency
                                                               of EPA's Air Program
                     resources, 37% went to state and local entities as grants to establish
                     and operate air pollution control programs.10
                                  Figure 5: OAR Air Grants to State
                                          and Local Entities
                            500
                            400
                           e
                           .2300
                            100
                                                OAR Resources

                                                Air Grants
                                  *«'•'
                                            $411.6
                                                      $448.2
                                                                $441.1
                                   1993
                                             1994
                                                      1995
                                                                1996
                     State and local entities also contribute their own resources toward
                     cleaning the air.  In September 1995, the State and Territorial Air
                     Pollution Program Administrators/Association of Local Air
                     Pollution Control Officials reported that Section 105 grant funding
                     accounted for between 4 and 65% of overall air program funding
                     for selected state and local entities.
                              Table 2: OAR Air Grants As a Percentage
                                   of Overall Air Program Funding
Number Sampled
14 States
1 1 Locals
High
65%
32%
Low
8%
4%
Average
29%
19%
                     OAR and its stakeholders share a common goal, working to make
                     the nation's air cleaner. Together, OAR and its stakeholders can
                     achieve more than if they worked independently.
10These grants are referred to as Section 105 grants, based on the Act section in which they are authorized.

                                     25
                                                                   Report No. 8100057

-------
                                                        The Effectiveness and Efficiency
                                                                 of EPA's Air Program
OAR ASKED FOR
INPUT FROM
STAKEHOLDERS
OAR often turned to stakeholders for input. According to OAR
officials, one of their most important stakeholders is the Clean Air
Act Advisory Committee, which they were required to establish
under the Act. The committee provides advice to OAR on policy
and technical issues associated with implementation of the Act.
Committee members include a variety of people representing:
states, local entities, tribes, public interest groups, environmental
groups, industry and trade groups, consultants, academia, and
Federal Government agencies.

OAR also involved stakeholders in a wide variety of projects. For
example, the Office of Mobile Sources (QMS) worked with
automobile manufacturers on proposed regulations and to develop
new technology. OMS had also established partnerships with other
governmental agencies, such as the Department of Energy and the
Department of Transportation.  OAR worked with 37 states
through the Ozone Transport and Assessment Group to develop
solutions for ozone transport.  OAR also solicited input from the
regions on planning and priority setting.
OAR DID NOT ALWAYS
GIVE FEEDBACK
 Some stakeholders, however, did not believe they received
 sufficient feedback from OAR. Several stakeholders told us that
 OAR frequently asked them to provide input, but did not tell them
 of the results. For example, members of the Clean Air Act
 Advisory Committee subcommittee for Ozone, Paniculate Matter,
 and Regional Haze Implementation Programs (subcommittee) said
 EPA did not keep them informed of progress on the
 implementation plan for the new ozone and particulate matter
 standards. OAR received subcommittee recommendations on how
 to implement the new standards and later, without further
 consultation or input from the subcommittee, released a broad
 outline describing how it planned to implement the new standards.
 The subcommittee members were initially disappointed with
 OAR's outline, and that they were "never briefed on the strategy"
                                          26
                                                                     Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                   of EPA's Air Program
STAKEHOLDERS
PERCEIVED LACK
OF FEEDBACK
Stakeholders may have perceived a lack of feedback from OAR
because (1) they may not have had a clear understanding of how
the stakeholder process worked and entered the partnership with
high expectations, (2) there was not always a formal mechanism to
provide feedback, and (3) the process for working with
stakeholders was lengthy.

First, OAR and its stakeholders may have entered the stakeholder
relationship with different expectations, when stakeholders did not
clearly understand the process. A U.S. General Accounting Office
report found that stakeholders often worked with EPA expecting to
reach a consensus.11  However, from OAR officials' perspective,
when they ask for input from stakeholders, especially on a
proposed rule, consensus could not always be achieved.  In one
example, the State of Washington and OAR worked together on
the Maximum Achievable Control Technology (MACT) standard
for aluminum mills.  A state official said that she learned OAR's
expectation of the state role in the MACT development was
different from the state's initial expectation.  In this case, state
officials were disappointed that OAR did not use all of their input
when developing the final standard. While OAR officials
considered the state comments, they could not reflect all of the
comments in the final rule, because they were not technically
feasible. Because of their varied expectations, state staff were
frustrated.

Second, stakeholders provide input to OAR and some may expect
to be kept informed of the status. When a stakeholder provides
formal comments to EPA on proposed rules, a mechanism is in
place to address the comments, by either including them in the
final rule or at least responding to the comments, if changes are not
made to the rule. However, there is no such formal mechanism to
       1 ^Environmental Protection: Challenges Facing EPA's Efforts to Reinvent Environmental Regulation,
GAO/RCED-97-155, July 2, 1997.
                                          27
                                                                       Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                 of EPA's Air Program
                          provide feedback to stakeholders at other stages of the rulemaking
                          process.

                          Finally, participating with OAR as a stakeholder is a very lengthy
                          process. If stakeholders want to ensure their comments are
                          included in the final rule, they must submit formal comments
                          throughout the rulemaking process. This process can take up to
                          two years. OAR often asks stakeholders to provide input when
                          developing a proposed rule. For example, OAR consults the Clean
                          Air Act Advisory Committee or the appropriate subcommittee to
                          get stakeholder input prior to developing a proposal. OAR then
                          writes the proposed rule and submits it to the Office of
                          Management and Budget (OMB).  OMB can take up to 90 days to
                          review a proposed rule, and sometimes does not take any action
                          during the first 60 days. At this time, OMB usually works only
                          with OAR on the proposed rule. This results in a long waiting
                          period where stakeholders are left out and do not know what is
                          going on with the rule.
STAKEHOLDERS
HESITANT TO WORK
WITH OAR
OAR cannot efficiently implement the Act without the
cooperation and help of its stakeholders. Stakeholder expectations
are often higher than what OAR can or intends to meet.  These
high expectations can create misunderstandings and hurt OAR's
stakeholder relationships. Relationships that lack sufficient
feedback can leave stakeholders feeling misguided and hesitant to
work with OAR again. For example:

       Clean Air Act Advisory Committee subcommittee
       members were upset at not being briefed on OAR's final
       implementation plan for the new ozone and particulate
       matter standards.  Several subcommittee members initially
       thought "EPA must either redefine the group's role in the
       process or disband it entirely."  The members were left
       wondering why EPA officials asked for input if they did
       not intend to use it. The members also wanted EPA to
       refine the group's mission, to better direct resources to areas
       where EPA would be receptive to suggestions.
                                          28
                                                                      Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
                            •     A state air director said he saw his staffs work on the
                                  MACT standard for aluminum mills as being of limited
                                  value. His staff is currently working on other MACT
                                  standards with OAR, but with a lower amount of effort than
                                  was spent on the aluminum mill MACT. He also has a
                                  lower expectation of results and of what the state's input
                                  can accomplish.

                           Poor stakeholder relationships result in OAR potentially alienating
                           a vital resource. If stakeholders enter a partnership with high
                           expectations, they will likely be disappointed when those
                           expectations are not met.  Stakeholders that do not feel their input
                           was addressed in a final regulation are more likely to challenge the
                           regulation through lawsuits.
CONCLUSION
RECOMMENDATIONS
                           OAR officials need to see working with stakeholders as a process
                           that involves much more than just accepting input.  OAR and the
                           stakeholders need to have a clear understanding of their
                           relationship up-front. OAR also needs to provide stakeholders
                           with sufficient feedback, especially when unable to  incorporate the
                           stakeholders' suggestions. Informal feedback is crucial when there
                           is no formal mechanism available.  Assuring solid relationships
                           with stakeholders is vital to efficiently achieving the common goal
                           of cleaning the nation's  air.
                           We recommend that the Acting Assistant Administrator for Air and
                           Radiation:

                           3-1.   Develop a process to work with stakeholders up-front to
                                  define expectations of the results or outcome of the
                                  stakeholder relationship.

                           3-2.   Require OAR officials to explain to stakeholders the
                                  process for proposing a regulation, including:  OMB
                                  involvement, that comments at some stages will not
                                  formally be addressed, and that it can be a lengthy process.
                                           29
                                                                        Report No. 8100057

-------
                                                       The Effectiveness and Efficiency
                                                                of EPA's Air Program
                         3-3.   Establish a process to ensure that feedback is provided to
                                all stakeholders when OAR has not addressed their
                                concerns, or has not used the input.
AGENCY COMMENTS
AND OIG EVALUATION The Acting Assistant Administrator for OAR concurred with our
                         findings and recommendations. He needs to provide specific
                         corrective actions and milestone dates for implementing the
                         recommendations.
                                         30
                                                                    Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                          	ofEPA'sAir Program
                                       PART 3
                                   CHAPTER 4
                        OAR Needed to Give Attention
                               to Several Activities
                           The fourth way in which the Air Program could be more efficient
                           is to devote attention to several ongoing activities. These activities
                           have been discussed in prior audit reports and include:  leading the
                           state implementation plan (SIP) process, developing and improving
                           emission factors, and issuing air toxic standards. The Office of Air
                           and Radiation (OAR) has not considered these activities high
                           priorities, compared with other program areas. This resulted in
                           delays and limited funding.  Consequently, these activities have
                           operated inefficiently and may not be achieving their desired
                           results.  For example, state plans to achieve emission reductions
                           could be delayed. The plans may also be incorrect if the state's
                           estimates of emissions from major sources  are based on inaccurate
                           emission factors. Also, industry deadlines  for installing controls
                           over emissions of air toxics will be delayed if OAR is late in
                           issuing standards.
BACKGROUND
                           SIPs, emission factors, and air toxic standards are vital to
                           achieving some of OAR's fundamental responsibilities, as they
                           have defined them. For example, to meet its responsibility of
                           helping state, local, and tribal governments develop and implement
                           programs to achieve the air quality standards, OAR must assure
                           that approved SIPs are in place and emission factors are reliable.
                           To meet its responsibility to develop standards to control the
                           release of air toxics, OAR must assure that it has a current,
                           practical plan to develop the standards.

SIPs                       SIPs are the major mechanisms used to attain air quality standards.
                           A SIP is a plan made up of the regulations a state will use to clean
                           up polluted air. The Clean Air Act, as amended in 1990, (Act)
                           established deadlines for states to submit the plans to EPA for
                           review and action. States develop a separate plan for each standard

                                           31
                                                                        Report No. 8100057

-------
Emission Factors
Air Toxics
                                                           The Effectiveness and Efficiency
                                                                     of EPA's Air Program
                           or air pollutant of concern, such as: ozone, carbon monoxide, and
                           particulate matter.
Emission factors are sometimes used to estimate emissions from
major sources of air pollution.  They are primarily used when more
reliable emissions data, such as monitoring data from continuous
emission monitors or stack tests, are not available for a source.
EPA and states use these emission estimates in many parts of their
air pollution control programs, including: (1) preparation of
emission inventories used in air quality models to develop and
evaluate pollution control strategies, (2) permitting and fee
programs, and (3) emission trading programs.

Air toxic standards, called Maximum Achievable Control
Technology standards, establish pollution controls for all sources
that emit significant amounts of toxic substances into the air.
Potential sources of air toxics include printers, dry cleaners,  and oil
and natural gas producers. The standards require all major sources
to install control equipment or change manufacturing processes to
reduce toxic emissions to levels at least as stringent as those
already achieved by the best-performing facilities in a source
category.12 The Act established deadlines for OAR to issue air
toxic standards, requiring OAR to regulate certain percentages of
source categories by November of 1994,1997, and  2000, as shown
in table 3.
                      Table 3: Air Toxic
                        Standards Due
                     (Shown Cumulatively)
Year Due
1994
1997
2000
Percentage of
Standards Due
25%
50%
100%
        I2A source category is a group of facilities that generally employ similar manufacturing processes or
 produce similar products.
                                            32
                                                                         Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                         	ofEPA'sAir Program
SIP PROCESSING
NEEDED TO BE
IMPROVED
While regional offices had several good management practices
in the SIP program, OAR could improve parts of the program on a
national basis. For instance, OAR and states had not always met
Act deadlines for submitting and processing SIPs. Late OAR
guidance was the greatest single cause for states submitting their
SIPs late.13 State legislatures often have to pass SIPs into law.
Because this can be a lengthy process, some states need a long lead
tune to develop and put their SIPS in place.

Another major cause of late SIPs was OAR delays in bringing
policy issues to  closure. Both our review and an OAR SIP
Improvement workgroup found that some issues were left
unresolved for a long time. For example, OAR needed to resolve
national issues before regions could process some SIPs. In several
instances, Region 2 officials delayed processing state SIPs because
they identified a national issue that needed to be resolved. They
had to wait for a decision from an OAR workgroup before
processing the SIPs.  The OAR SIP Improvement workgroup cited
low priority with management as a root cause of this delay.

While states have already submitted many of the original SIPs
required under the Act, they will be required to submit new SIPs to
meet the ambient air quality standards for ozone and particulate
matter that were revised in 1997. As states begin to design
programs to achieve these new standards, they will again need
OAR's guidance and prompt policy decisions to develop their SIPs
efficiently. The longer it takes to get SIPs processed and in place,
the longer it will take to achieve cleaner air.
       13EPA's Air State Implementation Plan Program Consolidated Report, EPA DIG Report No. 6400100,
September 30, 1996.
                                          33
                                                                      Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                   of EPA's Air Program
EMISSION FACTORS
NEEDED TO BE
DEVELOPED
AND IMPROVED
Emission factor development had not met users' demands for new
and revised emission factors.14 Emission factors were unavailable
for many sources of air pollution, and those that were available
were often unreliable.  Moreover, OAR had not met the Act's
requirements to develop emission factors for all sources of ozone
precursors. State and industry officials had to use the unreliable
factors to estimate emissions. In cases where no factors were
available, users were forced to rely on "best engineering
judgement," or an educated guess.

OAR cited resource limitations as causing problems with emission
factors. Developing or revising emission factors can be expensive
and can take several years to complete. For example, one revision
cost OAR about $1 million in contract funds and took over five
years to complete.  An OAR official said that recent budget cuts
significantly hindered OAR's efforts to revise and develop high
quality emission factors. Resources for emission factors decreased
from $5.8 million in contract dollars  and 17.6 full time equivalents
(FTEs) in fiscal year 1992 to $1.4 million and 8.8 FTEs in fiscal
year 1996. This is shown in figure 6.
                                       Figure 6: Emission Factor Funding
                                            FY 1992 through FY 1996
                                               FTEs
                                               Contract Dollars (in Millions)
                                        1992
                                               1993
                                                      1994
                                                             1995
                                                                     1996
        14Emission Factor Development, EPA OIG Report No. 6100318, September 30, 1996.

                                           34
                                                                        Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                   of EPA's Air Program
                           Without reliable emission factors, OAR and the states who use
                           them cannot be sure that their: (1) air pollution control strategies
                           target the right industries or products, (2) permitting programs
                           include all required sources and establish proper emission limits,
                           and (3) emission trading programs are effective in reducing air
                           pollution.  If these programs are not effective, the nation's air
                           quality could be adversely affected and people could be subjected
                           to the health hazards associated with excessive exposure to air
                           pollutants.
TIMELY AIR TOXIC
STANDARDS NEEDED
OAR has been behind schedule for promulgating air toxic
standards. As shown in figure 7, OAR missed its statutory
deadlines for most of the 21 air toxic standards due in 1992 and
1994. OAR also missed more than 85% of its 26 statutory
deadlines for 1997.

                  Figure 7: Air Toxic
                  Standard Deadlines
                                     30

                                     25

                                     20

                                     15

                                     10

                                     5
                                                  E3  Standards Due
                                                     Deadlines Missed
                                        26
                            19
                                       V////////////A
                                 13
                                              23
                                            1992
                                                        1994
                                                                      1997
                           Although the Agency was behind schedule for promulgating air
                           toxic standards, in 1995, OAR officials actively sought and put to
                           use many initiatives to speed and improve the process. An Office
                           of Inspector General (OIG) review of various OAR initiatives
                           revealed that many had already proven effective at speeding up the
                                           35
                                                                        Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                   of EPA's Air Program
                          process and others had merit.15 However, to meet the deadlines set
                          by Congress for future standards (58 are due in November 2000),
                          OAR must continue and even accelerate the issuance of
                          regulations.

                          U.S. General Accounting Office (GAO) reports have identified
                          limited funding as a reason the Air Toxics Program has had limited
                          success.  In 1991, OAR officials stated that they planned to have
                          larger budgets for the program in future years. However, those
                          plans never materialized. For instance, GAO reported that OAR's
                          1994 budget was reduced by $32 million.  The Air Toxic Standards
                          Program faced a significant reduction to its funding request, when
                          it was reduced by 51 percent. See figure 8.
                                          Figure 8: Air Toxic Funding
                                                   FY 1994
15
n
n
= 10
i
5
n
$14.5






<
\ \ \
' <. ••

•• \ x




<;
—

',
s^-


51%
? $7.1


, • ', 4-
" ^**"*'


                                           Requested
                                                               Fnnded
                           If OAR does not issue the remaining air toxic standards timely, it
                           will (1) allow industry additional time to operate without using the
                           best methods available to reduce the air toxics it emits and (2) face
                           challenges to the standards.

                           Issuing standards late allows industry more time before it has to
                           change its methods to reduce emissions. Industry does not have to
                           change its processes until an air toxic standard becomes effective.
                           If OAR does not issue an air toxic standard within 18 months of its
       ^Development of Maximum Achievable Control Technology Standards, EPA OIG Report No. 6100140,
March 19, 1996.
                                           36
                                                                        Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
                           statutory due date, states are then required to set emission limits for
                           sources through the state operating permit. State officials are
                           supposed to determine, on a case-by-case basis, what limit they
                           think would apply, had a standard been timely issued. If OAR
                           later issues a different or more stringent air toxic standard, the new
                           standard  does not apply to the permitted source until the next
                           permit renewal. The Act allows up to eight years for this to occur.

                           Because  of problems with the air toxic standard program, OAR
                           may be vulnerable to lawsuits.  For instance, environmental
                           organizations may sue OAR to force it to issue standards that are
                           already late. These groups and the regulated industry may also  sue
                           OAR, because budget problems may impact the quality of the
                           newer standards.  For example, in a December 1994 memorandum,
                           an OAR official stated that the fiscal year 1995 budget level would
                           have a significant impact on the Office's ability to meet its
                           requirements.  He said this was especially true for developing and
                           issuing the 1997 and 2000 standards.16 In an attempt to meet as
                           many deadlines as possible, OAR reduced the amount of data
                           collected and analyzed in developing the standards. However,
                           OAR officials expressed concern that the quality of the newer air
                           toxic standards may suffer.
CONCLUSION
                           Because OAR had not considered its ongoing activities of:

                            •     leading the SIP process,

                            •     developing and improving emission factors, and

                            •     issuing air toxic standards

                           as high priorities, the activities had not operated as efficiently as
                           possible, experiencing delays and limited funding. If these
                           activities continue to be low priorities, it is likely that the problems
       ^Information on EPA's Air and Radiation Program's Budget, 1990-95, GAO/RCED-96-201R, July 1,
1996.

                                           37
                                                                        Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                   of EPA's Air Program
                           identified in this chapter will also continue. SIPS will still be
                           delayed and will be based on unreliable emission factors. Industry
                           deadlines for installing air toxic controls will also be delayed. As a
                           result, OAR and state air programs may not be as efficient or
                           effective as they could be, resulting in continued air pollution.
RECOMMENDATIONS
FROM PRIOR AUDIT     Prior audit reports included recommendations to address the
REPORTS                 concerns discussed in this chapter. For instance, in our 1996 report
                           on SIPs, we recommended that the OAR Assistant Administrator:

                           (1)    place priority on implementing the SIP workgroup's
                                 recommendations for developing guidance and bringing
                                 policy issues to closure;

                           (2)    place priority on promptly resolving national SIP issues,
                                 including those that impact guidance documents; and

                           (3)    establish realistic deadlines for future SIPs that allow EPA
                                 time to develop useful guidance.

                           In her April 1997 response to that report, the Assistant
                           Administrator agreed with the recommendations and provided
                           steps for implementing them. She also stated that OAR had
                           emphasized up-front planning for the new ozone and particulate
                           matter SIPs to ensure that the SIP revisions were submitted in a
                           timely manner. She identified several actions that she believed
                           would substantially ease the SIP development effort and result in
                           SIPs being submitted more timely and being fully approvable.
                           These actions, when fully implemented, should address our
                           concerns with late guidance and unresolved national issues. As a
                           result, we are not making additional recommendations in this area.

                           In the prior OIG audit of emission factors, we recommended that
                           OAR declare the program as a material weakness, until adequate
                           resources were obtained to assure satisfactory progress.  OAR
                           submitted its emission factor strategy to the EPA Senior
                           Leadership Council and the OIG  on November 25,1997. The
                           strategy provided some actions for correcting the program
                                          38
                                                                       Report No. 8100057

-------
                                The Effectiveness and Efficiency
                                         of EPA's Air Program
weaknesses. For instance, beginning in fiscal year 1998, OAR will
focus its efforts on upgrading and developing factors for air toxics
and the revised particulate matter standard, as well as those for
non-road mobile sources. We accepted that these actions, when
fully implemented, will address our concerns with emission factor
development.  However, we still believe that the program should
be declared a material weakness, until OAR implements the
corrective actions. We do not have any additional
recommendations in this area.

GAO recommended, in 1991, that OAR revise its air toxics
strategy to include all actions, activities, and tasks mandated or
reasonably believed to be necessary to carry  out the objectives of
the Act. As of October 1997, this had not been completed. We
believe OAR should still update the air toxics strategy for issuing
the remaining  air toxic standards.
                 39
                                             Report No. 8100057

-------
                                                  The Effectiveness and Efficiency
                                                         of EPA's Air Program
                                                                   Exhibit 1
                                                                 Page 1 of 2
                     EPA OIG and GAO Reports
           EPA OIG Reports Included in the Scope of this Report
Report Name,
Issuing Office
Survey of EPA Green Lights Program,
Headquarters Audit Division
Regional Management of CAA Section 105 Air Grant Program,
Southern Audit Division
Acid Rain Allowance Trading Program,
Eastern Audit Division
Region 3 Title V State Operating Permit Program,
Mid-Atlantic Audit Division
Development of Maximum Achievable Control Technology
Standards,
Headquarters Audit Division
EPA's Development of its Proposed Open Market Trading Rule,
Eastern Audit Division
Emission Factor Development,
Southern Audit Division
EPA's Air State Implementation Plan Program Consolidated
Report,
Northern Audit Division
Risk Reduction Through Voluntary Programs,
Northern Audit Division
Report
Date
01/17/95
09/29/95
12/12/95
02/06/96
03/19/96
03/28/96
09/30/96
09/30/96
03/19/97
Report
Number
5700002
5100510
(1)
(1)
6100140
6400046
6100318
6400100
7100130
(1) No report number was assigned, because it was an internal survey report.
                                    40
                                                             Report No. 8100057

-------
                                       The Effectiveness and Efficiency
                                      	   of EPA's Air Program
                                                         Exhibit 1
                                                        Page 2 of2
GAO Reports Included in the Scope of this Report
Report Name
Air Pollution: EPA's Strategy and Resources May Be
Inadequate to Control Air Toxics
Indoor Air Pollution: Federal Efforts Are Not Effectively
Addressing a Growing Problem
Asbestos Removal and Disposal: EPA Needs to Improve
Compliance with its Regulations
Air Pollution: Unresolved Issues May Hamper Success of
EPA's Proposed Emissions Program
Air Pollution: Actions to Promote Radon Testing
Air Pollution: Difficulties in Implementing a National Air
Permit Program
Air Pollution: EPA's Progress in Determining the Costs and
Benefits of Clean Air Legislation
Air Pollution: Reductions in EPA's 1994 Air Quality
Program's Budget
Air Pollution: Allowance Trading Offers an Opportunity to
Reduce Emissions at Less Cost
Air Pollution: EPA Data Gathering Efforts Would Have
Imposed a Burden on States
Information on EPA's Air and Radiation Program's Budget,
1990-95
Global Warming: Difficulties Assessing Countries' Progress
Stabilizing Emissions of Greenhouse Gases
Peer Review: EPA's Implementation Remains Uneven
Environmental Protection: Challenges Facing EPA's Efforts
to Reinvent Environmental Regulation
Date
06/26/91
10/15/91
02/25/92
09/25/92
12/24/92
02/23/93
02/1 1/94
1 1/29/94
12/16/94
08/07/95
07/01/96
09/04/96
09/24/96
07/02/97
Number
GAO/RCED-91-143
GAO/RCED-92-8
GAO/RCED-92-83
GAO/RCED-92-288
GAO/RCED-93-20
GAO/RCED-93-59
GAO/RCED-94-20
GAO/RCED-95-31BR
GAO/RCED-95-30
GAO/AIMD-95-160
GAO/RCED-96-201R
GAO/RCED-96-188
GAO/RCED-96-236
GAO/RCED-97-155
                        41
                                                   Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                                    of EPA's Air Program
                                                                               Exhibit 2
                                                                              Page 1 of3
              Scope, Methodology, and Prior Audit Coverage
SCOPE
We began this work with a general survey of the Air Program in
November 1993.  In May 1994, we developed a strategic plan for
the air area. The plan identified ten reviews needed for a
comprehensive program evaluation.  The suggested reviews and
the work completed are shown in table 4.
                                 Table 4: Audits Included in the Air Strategic Plan
Planned Audits
Title V Permits
Green Programs
Section 105 Grants
State Implementation Plans
Enforcement17
Acid Rain Emissions Trading
Non-Acid Rain Emissions
Trading
Air Toxics
Indoor Air18
Emission Inventories/Factors
Work Completed
Survey without report
Survey with report
Audit report
Special review report
Audit reports
Survey without report
Special review report
Audit report
Audit report
Audit report
       17We originally planned to include air enforcement in this report. Because OAR is no longer responsible
for enforcement, it was not included in this report. Instead, the OIG issued a report (7100306) on ah- enforcement to
the Office of Enforcement and Compliance Assurance on 09/30/97. The OIG also issued reports on validation of air
enforcement data in the states of Pennsylvania, Arkansas, Maryland, and Massachusetts.

       18We refocused this audit from indoor air to an audit of Air Voluntary Programs.
                                           42
                                                                         Report No. 8100057

-------
                               The Effectiveness and Efficiency
                                        of EPA's Air Program
                                                   Exhibit 2
                                                 Page 2 of 3

After completing the individual assignments, in October 1996, we
began fieldwork to prepare this report on the Air Program's
effectiveness and efficiency. We issued fully-developed position
papers to OAR, ORD, and OCEPA on November 4, 1997. We
held meetings with each office in December to obtain comments
on the position papers. Because we had agreement on the
recommendations included in the position papers, we did not issue
a draft report, and instead obtained formal written responses on the
position papers. Those responses are included as appendices 1 , 2,
and 3.

Following is the methodology we used to answer our objective for
this report.
The first part of our objective was to determine if the Air
Program was operating effectively to make the nation's air cleaner.
To accomplish this objective, we obtained and reviewed EPA,
international, and other government reports. We did not perform
direct tests of the data contained in these reports, because it was
not practical to do so. As a result, we cannot be certain of the
data's validity and reliability.

The second part of our objective was to determine if the Air
Program was operating efficiently to make the nation's air cleaner.
To accomplish this objective, we:

 •     reviewed prior OIG and GAO audit reports, issued since
       November 1990, to identify common concerns.

 •     discussed these concerns with Air Program management
       and reviewed relevant documents they provided to
       determine why the similar concerns existed and whether
       they were indicators of program- wide problems.
                43
                                            Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                                                                            Exhibit 2
                                                                           Page 3 of3

                           •    interviewed officials from regional and state air programs,
                                and environmental groups, to obtain their opinions on the
                                efficiency of the national Air Program.

                           •    interviewed officials from other EPA program offices that
                                OAR works with, including the Offices of Research and
                                Development; Communications, Education, and Public
                                Affairs; and Enforcement and Compliance Assurance. We
                                also reviewed documents they provided, to evaluate how
                                well the offices worked together to achieve the Air
                                Program's goals.

PRIOR AUDIT
COVERAGE              The OIG and GAO have performed many audits on various
                          activities of the Air Program, as shown in exhibit 1. We relied on
                          these audit reports for general information in preparing this report.
                          We also highlighted four of these audits as containing issues
                          needing attention, as discussed in chapter 4.  Those reports are:

                          1.    EPA's Air State Implementation Plan Program
                                Consolidated Report, 09/30/96, Report No. 6400100.

                          2.    Emission Factor Development, 09/30/96, Report No.
                                6100318.

                          3.    Development of Maximum Achievable Control Technology
                                Standards, 03/19/96, Report No. 6100140.

                          4.    EPA's Strategy and Resources May Be Inadequate to
                                Control Air Toxics, 06/26/91, Report No. GAO/RCED-91-
                                 143.

                          For specific details on the findings and recommendations in these
                          reports,  see chapter 4.
                                          44
                                                                      Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                                                                             Exhibit 3
                                                                           Page 1 of2
                        Issues Needing Further Study
Acid Rain
Emissions Trading
Non-Acid Rain
Emissions Trading
Monitoring Data
Achieving Attainment
                          During our work in the Air Program, we identified several areas
                          that may need future study. A brief summary of those areas
                          follows.
When we performed a survey of this program in 1995, the program
was not yet fully operational. We may want to perform additional
work on the results of the "True-Opt" reconciliation process, where
actual utility emissions are reconciled to the allowances held. This
reconciliation began in January 1996. We may also want to review
the status of states receiving approval of their acid rain permit
programs, which were due in July 1996.
We performed our work on this program in 1996, but EPA had not
established its final open market trading rule, so no trades had been
completed. Since the time of our review, EPA decided not to issue
a rule, but instead issued guidance. We may want to perform
additional work once some trades have been completed under this
guidance to evaluate the internal controls over the process.

This is an area we may want to consider looking at, because it is
critical to how EPA evaluates the results of its air pollution
programs.

We may want to determine whether non-attainment areas are
achieving attainment on schedule. For instance, moderate carbon
monoxide and ozone non-attainment areas were supposed to meet
the standards by December 1995 and November 1996,
respectively.  If these areas have not met the standards, we should
evaluate whether the states have fully implemented their state
implementation plans and whether the implementation has resulted
in anticipated reductions in pollution. Furthermore, we should
                                          45
                                                                      Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                                  of EPA's Air Program
                                                                             Exhibit 3
                                                                           Page 2 of2

                          look at what actions EPA took when areas did not meet the
                          standards timely.

Title V Permits            After states have had time to implement the Title V state operating
                          permit program, we should consider reviewing it.  We performed a
                          survey in 1996, but the program was not fully operational at that
                          time. Potential audit issues include (1) EPA and state review of
                          companies that changed from major sources to synthetic minor
                          sources, thus avoiding the need for a Title V permit, and (2) how
                          EPA and states have addressed companies who admitted they had
                          been out of compliance on their permit applications.
                                          46
                                                                       Report No. 8100057

-------
                                                           The Effectiveness and Efficiency
                                                           	ofEPA's Air Program
                                                                               Appendix 1
                                                                                Page 1 of2
                Office of Air and Radiation Response
                   UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
                                 WASHINGTON, D.C. 20460

                                    FEB 12 1998
  -•i PROI«-&                                                          OFFICE OF
                                                              AIR AND RADIATION
 MEMORANDUM

 SUBJECT:      OAR's Comments on the Air Theme Report - "The Effectiveness and Efficiency
                ofEPA's Air Program" OIG Report ElKAE4-05-0246-xxxxxx
 FROM:         Richard D. Wilson
                Acting Assistant Administrator
                   for Air and Radiation  (6101)
 TO:            Michael D. Simmons
                Deputy Assistant Inspector General
                   for Internal Audits    (2421)

        Thank you for the opportunity to comment on the Office of the Inspector General's Air
 Theme Report - "The Effectiveness and Efficiency ofEPA's Air Program" (OIG Report
 E1KAE4-05-0246-XXXXXX). We have reviewed the OIG's position papers and concur with the
 findings and recommendations.

        I am pleased that OAR and ORD have made great strides in identifying research priorities
 with then- respective programs in the current budget cycle. We will continue to work with ORD
 to strengthen OAR's presence and results with the research community. Similarly it is
 encouraging that OAR and OCEPA's roles have been clarified and we are proceeding with a
 better working relationship between our offices.
        As you can  see, even after half a decade with this Air Theme Audit, there remains much
 work to be done. So I ask not only for the OIG's continued cooperation but also for OCEPA and
 ORD to work with OAR to develop plans for implementing the report's recommendations under
 the subsequent ninety day review. Please have your offices continue to coordinate any responses
 through Pete Cosier (260-7755) of my staff. Thank you. I believe with everyone's cooperation
 we will soon be able to complete this theme audit effort.

 cc:     Henry L. Longest II (8101)
        Loretta M. Ucelli (1701)
        Kevin Teichman (8104R)
        Arnold Bloom (8102R)
        Ernest Ragland (2421)
        Anthony Carrollo, Director OIG, Northern Audit Division
	   Recycled/Recyclable • Printed with Vegetable Oil Based Inks on 100% Recycled Paper (40% Postconsumer)
                Note: The original response was signed by Richard D. Wilson.
                                         47
                                                                         Report No. 8100057

-------
                                                 The Effectiveness and Efficiency
                                                           of EPA's Air Program
                                                                    Appendix 1
                                                                     Page 2 of2
Kimberly O'Lone, OIG, Northern Audit Division
Janice Miller, OIG, Northern Audit Division
Rob Brenner (6103)
Margo Oge (6401)
JohnSeitz(MD-lO)
Paul Stolpman, (6201J)
Larry Weinstock, (661J)
Jerry Kurtzweg (6102)
Steve Page (6101)
Carl Mazza (6101)
Kevin Teichman (8104R)
Omayra Salgado (6102)
Pete Cosier (6102)
Arnold Bloom (8102R)
                                 48
                                                                Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                    of EPA's Air Program
                                                                              Appendix 2
                                                                              Page 1 of2
        Office of Research and Development Response
ssy
                 UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
       S                        WASHINGTON, D.C. 20460
                                   FEE-4 1998
                                                                OFFICE OF
                                                        RESEARCH AND DEVELOPMENT
MEMORANDUM
SUBJECT:      Response to DIG Position Papers - "The Effectiveness and Efficiency of
               EPA's Air Program"

FROM:         Henry L. Longest II
               Acting Assistant Administrator
                 for Research and Development (8101)

TO:            Michael Simmons
               Deputy Assistant Inspector General
                 for Internal Audit (2421)

Purpose

       The purpose of this memorandum is to convey ORD's response to the Office of
Inspector General's Position Papers - "The Effectiveness and Efficiency of EPA's Air
Program," issued to us in final form on February 3, 1998.

Discussion

       We have reviewed the OIG position papers and concur with the findings and
recommendations pertaining to ORD. We are pleased to note that ORD has already
taken certain steps, as part of its FY 2000 planning process, to implement the OIG's
recommendations. Specifically, last month, ORD met with OAR to discuss OAR's
strategic priorities for ORD research and technical support. We will continue to
work with OAR to forge our corrective action plan to the OIG's report
recommendations. We understand that this corrective action plan must be
submitted to the OIG within 90 days from the issuance of the final audit report.

       We appreciate the opportunity to work with the OIG and OAR representatives
in developing and reviewing various iterations of these position papers.  We believe
this process was insightful and important.  We particularly appreciate the extensive

           Recycled/Recyclable • Printed with Vegetable Oil Based Inks on 100% Recycled Paper (40% Postconsumer)
    Note: The original response was signed by Joseph K. Alexander for Henry L. Longest II.
                                        49
                                                                        Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                                    of EPA's Air Program
                                                                              Appendix 2
                                                                               Page 2 of2
and outstanding efforts of the OIG Northern Audit Division Team Leader in working
with us on this important assignment.

        Once again, thank you for your responsiveness to our suggestions on this
effort.  Should your staff have any questions or require further information, they
may contact Arnold Bloom at (202) 564-6687.

cc:      Richard Wilson (6101)
        CarlMazza(6101)
        Pete Cosier (6102)
        Joseph Alexander (8101R)
        Thomas Clark (8101R)
        Kevin Teichman (8104R)
        PeterDurant(8102R)
        LekKadeli(8102R)
        Colleen Lentini (8102R)
        Arnold Bloom (8102R)
        Ernest Ragland (2421)
        Janice Miller (OIG/OA/NAD)
                                         50
                                                                         Report No. 8100057

-------
                                                          The Effectiveness and Efficiency
                                                         	     of EPA's Air Program
                                                                             Appendix 3
                                                                              Pagel of 2
              Office of Communications, Education,
             	and Public Affairs Response
                  UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
                                WASHINGTON, D.C. 20460

                                                               OFFICE OF
                                                        COMMUNICATIONS, EDUCATION AND
                              DEC 16 1997                    PUBLIC AFFAIRS

MEMORANDUM

SUBJECT:     OCEPA Comments for the Air Theme Report

FROM:        Loretta M. Ucelli
               Associate Administrator

TO:           Michael Simmons
               Deputy Assistant Inspector General
                For Internal Audits (2421)

        We have reviewed and discussed your position papers from the Air Theme audit.
OCEPA is in agreement with the recommendations that effect our office ( Part 2, Chapter 2).

        We met with your staff and suggested a few minor revisions, which Janice Miller
assures me are being made to the position papers.

        Recommendations and proposed actions:

        2-1. Carry out and work within the planned OCEPA reorganization to ensure that
            product review requirements are followed and discourage OAR officials from
            bypassing the product review requirements.

OCEPA's proposed reorganization plan is being reviewed hi the Office of the Administrator.
We anticipate sign-off before the first of the year and clearance by the Agency by the end of
January. An integral part of the reorganization is the placement of the product review system
within the new Office of Communication. We believe that the program offices will benefit from
a new approach to planning and approving the Agency's information products.  OCEPA also
intends to conduct a two day work shop, in mid-to-late February, with representatives from the
program offices and outside stakeholders to assist the Agency in defining and producing the
types of information products that will be useful to various audiences and  the general public.

        2-2. Establish procedures to ensure that the offices work up-front with each other
            when developing projects to raise public awareness of air pollution.

OCEPA intends to update all communications planning procedures to ensure that initial contact
       Note: The original response was signed by Diane H. Esanu for Loretta M. Ucelli.


                                        51
                                                                        Report No. 8100057

-------
                                                            The Effectiveness and Efficiency
                                                                      of EPA's Air Program
                                                                                Appendix 3
                                                                                 Page 2 of2
between ourselves and any program office begins early and is a collaborative effort that will lead
to increased public awareness.

        2-3. Establish concise product review procedures that serve as a clear and simple
            guide to the product review process for program offices.

An additional benefit of the February product review workshop will be discussions that will lead
to new guidelines (procedures) that program offices will reference when planning to develop
information products.  We anticipate that written guidelines, after program input and review,
should be finalized by May, 1998.

        If you or your staff have other questions or need additional information, please contact
Diane Esanu, Acting Deputy Associate Administrator at (202) 260-2190.
                                           52
                                                                            Report No. 8100057

-------
                                                         The Effectiveness and Efficiency
                                                         	     of EPA's Air Program
                                                                           Appendix 4
                                                                           Page 1 of 1
                                   Distribution
Headquarters

Assistant Administrator for Air and Radiation (6101)
Assistant Administrator for Research and Development (8101R)
Associate Administrator for Communications, Education, and Public Affairs (1701)
Agency Followup Official (2710)
 Arm: Chief Financial Officer, OCFO
Agency Followup Coordinator (2724)
Audit Followup Coordinator, Office of Air and Radiation (6102)
Audit Followup Coordinator, Office of Research and Development (8102R)
Audit Followup Coordinator, Office of Communications, Education, and Public Affairs (1104)
Associate Administrator for Congressional and Legislative Affairs (1301)
Headquarters Library (3401)

Office of Inspector General

Inspector General (2410)

Regional Offices

Regional Administrators
Regional Air Directors
Regional Audit Followup Coordinators
Regional Directors of Public Affairs
Regional Libraries

All State Directors
                                         53
                                                                      Report No. 8100057

-------
U.S. Environments! P/oteuio,} ,VvnC
K** f'Utow (PL-12J)
£32 {

-------