V
    00
                                  U,S EPA Headquarters Library
                                      Mail code 3404T
                                  1200 Pennsylvania Avenue NW
                                    Washington, DC 20460
                                      202-566-0556
                  OFFICE OF INSPECTOR GENERAL
         Evaluation Report
»* •
    \
JS.
      OOO
EPA's Method for Calculating
Air Toxics Emissions for Reporting
Results Needs Improvement
                  Report No. 2004-P-00012
                  March 31, 2004

-------
Report Contributors:
                        Patrick Milligan
                        Frank Martinsky
                        Lorraine Fleury
                        Michelle Brown
Abbreviations

EIIP

EPA

GPRA

MACT

NTI

OAR

OIG

OMB

ORD

OTAQ
Emission Inventory Improvement Program

Environmental Protection Agency

Government Performance and Results Act

Maximum Achievable Control Technology

National Toxics Inventory

Office of Air and Radiation

Office of Inspector General

Office of Management and Budget

Office of Research and Development

Office of Transportation and Air Quality
Photo (clockwise, from top left): (1) Cars and trucks emitting exhaust fumes
(weathersmith.com); (2) forest fire burning uncontrolled (US Forest Service web site);
(3) agricultural equipment emissions (weathersmith.com); and (4) pollution from an industrial
smokestack (New York State Department of Environmental Conservation).

-------
                            UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
                                         WASHINGTON. D.C. 20460
                                                                                OFFICE OF
                                                                             INSPECTOR GENERAL
                                          March 31,2004

       MEMORANDUM

       SUBJECT:        EPA's Method for Calculating Air Toxics Emissions for
                         Reporting Results Needs Improvement
                         Report No. 2004-P-00012
i
ij       FROM:           J. RickBeusse  /s/
I                         Director for Program Evaluation, Air Quality Issues

       TO:               Jeffrey R. Holmstead
                         Assistant Administrator for Air and Radiation (6101 A)

       Attached is our final report regarding the Environmental Protection Agency (EPA) efforts to
       develop air toxics emissions data for use as a Government Performance and Results Act
       performance measure. This report contains findings regarding EPA's methods for calculating air
       toxics emissions.  Also, the report contains corrective actions the Office of Inspector General
       (OIG) recommends. This report represents the opinion of the OIG and the findings contained in
       this report do not necessarily represent the final EPA position. Final determination on matters in
       this report will be made by EPA managers in accordance with established procedures.

       EPA's Office of Air and Radiation (OAR) provided us with a draft response on March 24,2004,
       that included comments from the OAR's Office of Air Quality and Planning Standards and
       OAR's Office of Transportation and Air Quality. Also, officials from the Office of Research and
       Development (ORD) provided draft comments.  Because these responses are draft and may not
       represent EPA's final position on the issues and recommendations in this report, the draft
       responses are not included as appendices in this report.

       Action Required

       In accordance with EPA Manual 2750, as the action official, you are required to provide this
       office with a written response within 90 days of the final report date. Since this report deals
       primarily with OAR's Air Toxics Program, the Assistant Administrator for Air and Radiation
       was designated the primary action official.  As such, he should take the lead in coordinating the

-------
Agency's response.  The response should address all recommendations. For the corrective
actions planned but not completed by the response date, please describe the actions that are
ongoing and provide a timetable for completion. If you do not concur with a recommendation,
please provide alternative actions addressing the findings reported. We appreciate the efforts of
EPA officials and staff, as well as external stakeholders, in working with us to develop this
report. For your convenience, this report will be available at http: //www. epa. gov/oig.
If you or your staff have any questions regarding this report, please contact me at (919) 541-5747
or Patrick Milligan, Assignment Manager, at (215) 814-2326.

-------
                             Executive Summary
      Purpose

                   Toxic air pollutants are harmful substances that are known or suspected to cause
j|                  cancer and other serious health problems, and can also have adverse environmental
I                  effects. The Clean Air Act identifies 188 toxic air pollutants and directs EPA to
i|                  regulate the sources emitting these pollutants.

li                •  EPA has been tasked with reducing toxic air pollutants and the resulting health
ij                  effects. The Government Performance and Results Act (GPRA) of 1993 required
1                  EPA, like other Federal agencies, to measure progress in achieving results.  EPA
jj                  measures progress of its Air Toxics Program by calculating how many tons of
|!                  emissions are reduced annually. A vital component for assessing progress and
J                  directing future efforts of the program is the National Toxics Inventory (NTI), an
i'                  estimate made once every 3 years of the total amount of air toxics emitted annually
1                  nationwide.  Accordingly,  our objectives were to answer me following:

                    •  What are the key  assumptions and limitations underlying EPA's NTI and how
                       do these impact its use as a GPRA performance measure?

                    •  What actions could EPA take to improve its current  air toxics GPRA
                       performance measure for assessing progress toward  achieving the air toxics
                       objective?

      Results in Brief

                    Although the methods by which air toxics emissions are estimated have improved
                    substantially in the past 6 years, invalidated assumptions and other limitations
                    underlying the NTI continue to impact its use as a GPRA performance measure.  As
                    a result, EPA is not certain how much progress stationary, area, and mobile sources
                    have actually made in reducing air toxics emissions from the EPA-established 1993
                    baseline. For example, EPA recently finalized its third NTI, which also resulted in
                    revisions to the first two inventories:

                    •  For the 1993 baseline NTI, EPA increased the estimated emissions inventory
                       from 6.1 million tons to 7.2 million tons (an 18-percent increase).
                    •  For the 1996 NTI, EPA increased the estimated emissions inventory from
                       4.6 million tons to 5.0 million tons (a 9-percent increase).
                   '•  For the 1999 NTI, EPA estimated 5.1 million tons, an increase of 100,000 tons
                       when compared to the updated 1996 inventory.

                    EPA is not certain whether reductions or increases may  have resulted due, at least in
                    part, to the Agency's change in the way it estimated the inventory, rather than real
                    reductions or increases in emissions.  While emissions estimating techniques have

-------
              improved, broad assumptions about the behavior of sources and serious data
              limitations still exist. Furthermore, due to the changes mat resulted in
              improvements in later inventories, the meaningfulness of comparing new
              inventories against the 1993 baseline is questionable. For example, although State-
              validated emissions data is EPA's preferred calculation methodology:

              •   For the 1993 baseline NTI, only three States provided data to EPA.
              •   For the 1996 NTI, 36 States provided data to EPA.
              •   For the 1999 NTI, 39 States provided data to EPA.

              Additionally, States were not required to verify their emissions data for any of the
              above inventories, due largely to EPA's concerns about adding more reporting
              requirements on States and the lack of a clear statutory mandate. Agency officials
              said that the Clean Air Act only required limited scope studies on a select number of
              toxic air pollutants found primarily in urban areas.  As such, only 33  of the 188 air
              toxics have received more in-depth study. Also, despite some improvements in
              emissions estimating techniques for certain sources of air toxics, available usage
              estimates and emission factors often are not reliable.  Further improvements are
              needed in the methods for calculating air toxics emissions if EPA is to accurately
              gauge the extent to which its programs have actually reduced emissions. Otherwise,
              EPA cannot be assured that the full reductions actually occurred or were, at least in
              part, the result of changes in estimation methods.

              EPA could also improve its current air toxics GPRA performance measure by
              including toxicity information with tonnage measures. Currently, EPA aggregates
              the tonnage reductions of emissions for all 188 toxic air pollutants but does not
              account for the varying levels of toxicity among these pollutants.  While useful, this
              aggregate measure does not provide EPA, Congress, or the public with specific  ,
              information on those pollutants that are the most dangerous to human health.
              During me course of our work, EPA officials told us that they see merit in adding a
              risk component to their GPRA reporting, and they recently began exploring
              possibilities for doing so.  This supplemental  measure would better enable EPA to
              track its progress in reducing those air toxics  that are most harmful to human health
              and, in so doing, more closely meet the intent of GPRA.
Recommendations
              We recommend that the Assistant Administrator for Air and Radiation improve the
              accuracy, reliability, and usefulness of the Agency's current emissions-based air
              toxics GPRA performance measure by providing increased funding and support for
              development of the NTI, and supplement the existing measure of comparing air
              toxics emissions reductions to the 1993 baseline inventory by also comparing
              emission totals between all other completed inventories. We also recommend that
              the Assistant Administrator supplement the existing tonnage measure for air toxics
              with a measure of progress toward reduction in human exposure and health risk.

-------
Agency Comments and OIG Evaluation

             Officials from the Office of Air and Radiation (OAR) and the Office of Research and
             Development (ORD) provided detailed draft comments on the report.  Because these
             responses are draft and may not represent the Agency's final position on the issues
             and recommendations in this report, the draft responses are not included  as
             appendices to this report. Nonetheless, we made changes to the draft report based on
             both sets of draft comments, as well as our exit conference discussions with Agency
             officials.

             OAR's detailed draft comments agreed with the recommendations in this report, with
             one exception. OAR intends to continue using the 1993 NTI as the baseline, instead
             of implementing our recommendation to use the more reliable 1996 NTI as the
             baseline for measuring progress under GPRA. Agency officials explained that if
             EPA were to use the 1996 NTI as the baseline, it would not enable them  to capture
             the emissions reductions achieved by stationary and mobile source standards
             implemented between 1993 and 1996. While we recognize this point, we continue to
             believe that, due to the unreliability of the 1993  NTI, EPA should use the 1996 NTI
             as the baseline. Additional discussion of this point is provided at the end of
             Chapter 2.

             With regard to ORD, officials of this office concurred with our draft report
             recommendations, stated that the draft report was reasonable, and provided several
             detailed comments intended to sharpen the quality and accuracy of the report.  These
             changes were made.

-------
iv

-------
                    Table of Contents
Executive Summary	   i
 Chapters
     1    Introduction [[[  1

     2    Improvements Needed to Air Toxics inventory
          for Use in GPRA Reporting .........................................  7

     3    GPRA Measure Needs to Address Risk of Air Toxics Exposure .......... 23
 Appendix
     A    Details on Scope and Methodology ................................. 27

     B    EPA's Recent Update to the 1 993 and 1996 Inventories ................. 29


-------

-------
                                 Chapter 1
                                 introduction
Purpose
              Toxic air pollutants are harmful substances that are known or suspected to cause
              cancer and other serious health problems, and can also have adverse environmental
              effects. The Clean Air Act identifies 188 toxic air pollutants and directs EPA to
              regulate the sources emitting these pollutants.

              EPA has been tasked with reducing toxic air pollutants and the resulting health
              effects caused by exposure to these pollutants. The 1993 Government Performance
              and Results Act (GPRA) required EPA, like other Federal agencies, to measure
              progress in achieving results. EPA measures progress of its Air Toxics Program by
              calculating how many tons of air toxics emissions are reduced annually. A vital
              component for assessing progress and directing future efforts of the program is the
              National Toxics Inventory (NTI), an estimate made once every 3 years of the total
              amount of air toxics emitted annually nationwide. Accordingly, our objectives were
              to answer the following:

              •  What are the key assumptions and limitations underlying EPA's NTI and how
                 do these impact its use as a GPRA performance measure?

              •  What actions could EPA take to improve its current air toxics GPRA
                 performance measure for assessing progress toward achieving the air toxics
                 objective?
Background
              What Are Adverse Health Effects of Air Toxics?

              Toxic air pollutants are known or suspected to cause various serious health
              problems. The adverse health effects include: increased risk of cancer; effects on
              the neurological, cardiovascular, respiratory, and immune systems; liver and kidney
              damage; and effects on fetal and child development.

              Immediate effects of toxics exposure may be minor, such as watery eyes; greater
              health problems, such as cancer, may not appear until years after exposure. In
              addition to exposure through breathing air toxics, some toxic air pollutants, such as
              mercury, can deposit onto soils or surface waters, where they are ingested by plants
              and animals and eventually transmitted through the food chain. Exposure to some
              air toxics before birth or during childhood may interfere with normal development,
              while other air toxics may affect the ability to conceive or give birth to a healthy
              child. Studies have found that exposure to benzene from less than 5 years to more

-------
              than 30 years has resulted in individuals developing and ultimately dying from
              leukemia. Studies of smelter workers identified an association between
              occupational arsenic exposure and lung cancer mortality.

              Where Do Air Toxics Come From?

              Air toxics come from a complex variety of sources, which poses a challenge to EPA
              when it compiles  the air toxics emissions inventory. For example, toxic air
              pollutants include benzene, which is found in gasoline; perchlorethlyene, which is
              emitted from some dry cleaning facilities; and methylene  chloride, which is used as
              a solvent and paint stripper by a number of industries. Toxics coming from other
              sources include asbestos; dioxin; toluene; and metals such as cadmium, mercury,
              chromium, and lead compounds.  Table 1.1 describes some of the types of sources
              emitting air toxics and the amount of emissions estimated to be generated by each
              category of sources, according to  EPA's 19931 baseline inventory.

                               Table 1.1: Categories of Sources of Air Toxics

;.(,.., ,
••-:•.'•-: - -:• "
fliH;
•.-S.:-S :.?•
;i«£
::'-o."'o:J
1
:' -:S " -:•"'-
!i
&.&•?'&
.••.. •.- ...
y.-,:: _x :
..StwnaK
Categories
Major
Area*
Mobile
A. On-road
B. Non-road
Subtotal
BefiniBan
Emissions of 10 tons per year or more
of any one hazardous air pollutant, or
25 tons per year or more of any
combination of hazardous pollutants
Emissions of less than 10 tons per year
of any one hazardous air pollutant, or
less than 25 tons per year of any
combination of hazardous pollutants
Emissions from motorized vehicles
normally operated on public roadways
Emissions from a diverse collection of
engines, equipment, vehicles, and
vessels
Sxamptesoi types j
, OfSWifce*
Utilities, Refineries,
Car Manufacturers,
Chemical
Manufacturers
Dry Cleaners,
Gas Stations,
Wood Burning Stoves
Cars, Buses, Sport
Utility Vehicles, Light-
& Heavy-Duty Trucks
Construction and
Agricultural
Equipment,
Personal Watercraft
Total Emissions
19$38as«fM« ,:
flnmiflionterw)
2.7
1.1
1.7
.6
2.3
6.1**
              * Area sources include a sub-category entitled "Other" for those sources that are not otherwise regulated, such as wildfires
              and open burning.
              ** This total represents the 2002 version of the 1993 baseline NTI.
              The 1993 toxics emissions inventory is the baseline measure EPA selected to gauge
              progress for the Air Toxics Program.
       JA large part of the 1993 baseline inventory is based on data obtained from 1990 and 1993. For simplicity,
and because EPA has traditionally referred to it as such, we refer to this data as the 1993 baseline inventory.

-------
              What Are the Air Toxics GPRA Goals and Measures?

              GPRA, passed in 1993, holds Federal agencies accountable for achieving results by
              requiring them to measure program outcomes, including for EPA's Air Toxics
              Program. By 1997, GPRA required agencies to establish outcome goals to measure
              their progress and report on their accomplishments.  The three main components of
              GPRA require EPA to: (1) develop a 5-year strategic plan defining its long-term
              goals and how the Agency intends to achieve them; (2) prepare annual performance
              plans with goals and measures that relate to either outputs or outcomes that
              quantitatively measure the Agency's results and demonstrate their linkage to
              longer-term goals; and (3) prepare annual performance reports on accomplishments.
              As shown in Table 1.2, EPA's Strategic Plan objective for the Air Toxics Program
              (updated every 3 years) relies on emission reductions.

                             Table 1.2: Strategic Plan Objectives for Air Toxics
₯«*f
1997
2000
2003
; Objective
By 2010, reduce air toxics emissions by 75 percent from 1993 levels to
significantly reduce the risk to Americans for cancer and other serious
adverse health effects caused by airborne toxics.
By 2020, EPA will eliminate unacceptable risks of cancer and other
significant health problems from air toxics emissions for at least
95 percent of the population, with particular attention to children and
other sensitive subpopulations, and substantially reduce or eliminate
adverse effects on our natural environment.
By 201 0, working with partners, reduce air toxics emissions and
implement area-specific approaches to reduce the risk to public health
and the environment from air toxic pollutants.
swdget ;
(a)
$95 million
(17% of Air
Budget)
$11 8 million
(19% of Air
Budget)
              (a): EPA's air toxics budget request for 1997 was not comparable to 2000 and 2003; the total air program budget request
                 for 1997 was $304 million.

              What Is the Air Toxics Emissions Inventory?

              EPA compiles an NTT every 3 years that consists of an estimate of the total
              emissions of the 188 air toxics identified in the Clean Air Act. Table 1.3 depicts the
              years for which four inventories were or are in Ihe process  of being developed and
              when each was completed.
                          Table 1.3: Air Toxics Emissions Inventories Undertaken
iWS^^^WY*w^lt?is|l|?|l;
1993
1996
1999
2002
SihillaiYeir Inventory Completed^ |
1998
2001
2003
In progress
       2EPA referred to the 1993 and 1996 inventories as the NTL For the 1999 inventory, EPA combined the air
toxics inventory with the criteria pollutant emissions inventory and changed the name to the National Emissions
Inventory, or NEI. For purposes of this report, we will refer to all 3 inventories (1993,1996, and 1999) as the NTL

-------
              Chart 1.1 provides a scheduled timeline of key steps in developing the 1999 air
              toxics inventory, which took place after 1999 ended.

                      Chart 1.1: Scheduled Timeline for Developing 1999 toxics Inventory
              I
                        States send Surveys to Point Sources
                             Federal Agencies develop activity data - VMT
                                     States quality control and record data
                                             EPA receives first reported data
                                                   EPA prepares draft Inventory
                              Reviewer's comments
                   EPA receives new data, along with comments
                              EPAprepares second draft Inventory
2nd Round of comments and Inventory revisions
             (a) Development of the NTI begins after the designated year has ended.
              In addition to the activities listed in Chart 1.1, State and other Federal agencies must
              obtain data from other sources (such as the Federal Highway Administration, which
              provides EPA with information on national vehicle miles traveled for the inventory
              year). These data are not available until a year or so after the inventory year has
              ended. After the new inventory is developed, EPA compares it to the 1993 baseline
              inventory to determine the amount of emissions reductions achieved. Although the
              1993 and 1996 inventories took over 4 years to complete, EPA hopes to reduce this
              time in the future.

              What Are The Uses of the Inventory?

              Although EPA is in the early stages of developing an ambient (outdoor) air toxics
              monitoring network3 capable of measuring air toxics in ambient air on a national
              basis, the Agency does not yet have a formal network and therefore relies primarily
              on emissions data for measuring progress. The NTI is the primary source of
              national air toxics emissions data used by major stakeholders - such as scientists,
              researchers, lawmakers, and the public - to gauge progress. Uses of the NTI
              include rulemaking efforts, modeling, and risk assessments.  More details on the
              NTI are in Chapter 2.
        EPA's National Air Toxics Trends Sites (NATTS) network consists of 22 sites. Agency officials said that
they plan to implement the network in fiscal year 2004.

-------
Scope and Methodology
              To assess the adequacy of EPA's GPRA measure for air toxics, we reviewed
              documentation related to: GPRA objectives and measures; the development of the
              1993,1996, and 1999 inventories; Quality Assurance/Quality Control efforts for the
              development of the three inventories; and regulations, guidance, policy, and
              assessments applicable to the development of the inventories. We also conducted
              numerous interviews with officials from: the Office of Air Quality Planning and
              Standards, the Office of Transportation and Air Quality, EPA Region 3, and the
              Office of Research and Development (ORD). Budget information was obtained
              from the Office of the Chief Financial Officer.  We also reviewed Section 112 of the
              Clean Air Act to identify the requirements of EPA to identify, control, and reduce
              air toxics emissions.

              Our field work was conducted from February 2003 to November 2003. We
              conducted this evaluation in accordance with Government Auditing Standards,
              issued by the Comptroller General of the United States.  Additional details on our
              scope and methodology are in Appendix A.  On March 30,2004, we met with
              Office of Air and Radiation (OAR) officials to discuss their draft comments and we
              made additional modifications to the report based on our discussions.

-------

-------
                                Chapter 2

        Improvements Needed to Air Toxics  Inventory

	      for Use in GPRA Reporting	

              Although the methods used to estimate air toxics emissions have improved
              substantially in the past 6 years, EPA is not certain of how much progress
              stationary, area, and mobile sources have actually made in reducing those
              emissions. Emissions estimating techniques have undergone continuous change and
              improvement since the 1993 baseline was developed, and because subsequent
              inventories were amassed quite differently, comparison to the 1993 baseline has
              become less meaningful over time.  For example, only 3 States provided data for the
              1993 baseline, while 36 and 39 States provided data for the 1996 and 1999
              inventories, respectively. Of 62 authorized local programs, only one reported air
              toxics emissions data for 1993; three for 1996; and nine for the 1999 inventory.
              Even with increased State and local agency participation,  these agencies have not
              been required to report air toxics emissions data, nor have they been required to
              verify it. Further, despite improvements in estimation techniques, usage estimates
              and emission factors often are not reliable. Due to the limitations noted, EPA
              cannot be assured that the full amount of reductions reported actually occurred or
              whether they were, at least in part, the result of improved  estimation methods.
              Improvements in the methods for calculating air toxics emissions are needed if the
              Agency is to accurately gauge the extent to which its programs have reduced the
              public's health risk to air toxics as called for under GPRA.

Difficulties in Comparing EPA's Air Toxics Emissions Inventories

              To develop major and area source emission estimates, EPA primarily uses four
              types of data (listed in order of preference based on data reliability):

              •   State, Local, and Tribal-Submitted Data: State, local, and tribal agencies, on
                 a voluntary basis, submit air toxics emissions data.

              •   Maximum Achievable Control Technology (MACT)/ResiduaI Risk: Each
                 time EPA develops a control for a source category (a source of air toxics), it
                 gathers emissions data.

              •   Toxics Release Inventory: This is an EPA-managed  database used to inform
                 the public and government officials about the total release and transfer of toxic
                 chemicals, and is based on information reported by a select group of industries.

              •   National Estimates: When State information is not available, EPA estimates
                 emissions on a national level and allocates down to a  local (county) level.

-------
              Although States have not been required to validate or even report air toxics
              emissions data, EPA considers State-submitted data to be the most reliable.  In its
              absence, the Agency uses the less reliable MACT, Toxics Release Inventory, and
              national estimates. EPA also uses data gathered from mobile source rulemaking
              efforts, industry surveys, and other Federal agency information, such as vehicle
              mileage data from the Federal Highway Administration, each with its own set of
              limitations.  Table 2.1 provides a brief qualitative assessment of the sources of
              information used for each triennial inventory, highlighting the significant changes
              made to more recent inventories.

                             Table 2.1: Key Differences in the Three Inventories
Sources of
Pollution
Major
Area
Mobile
(On-Road)
Mobile
(Non-Road)
', .1603
8*4«8n«
3 States and 1 local agency
provided data; few quality
checks performed. EPA
used less reliable data such
as MACT, Toxics Release
Inventory, and national
estimates. (Mainly non-
facility specific data
aggregated up to the county
level.)
Emission estimates largely
based on national activity
data and national emission
factors. Emissions allocated
down to the county level.
Five toxics estimated using
early on-road model, while
other toxics calculated using
less sophisticated methods.
Fuel types not accounted for.
Emissions estimated using
non-road study of emissions.
This study was first attempt
at identifying non-road
equipment types. Much of
the activity data and
emission factors unverified.
Fuel types partially
accounted for.
1W$ ' '-:' -
36 States and 3 local
agencies reported data -
46 States reviewed the data.
Quality check efforts made to
eliminate duplicate or ,
missing facilities. MACT and
Toxics Release Inventory
used less. Estimates based
on point emissions.
Began to use more State and
MACT data for non-point
sources. Emission estimates
still largely based on national
activity data and national
emission factors.
On-road model (MOBILE 5b)
used for all toxics estimates.
Fuel types accounted for.
New model {NONROAD)
developed to more
accurately depict types of
equipment. Much of the •
activity data and emission
factors still unverified. Fuel
types partially accounted for.
193$ '/ - \
39 States and 9 local
agencies reported data -
46 States reviewed the
data. For the first time,
3 tribal agencies reported
data. Some additional
quality checks performed
on data. MACT and
Toxics Release Inventory
used even less.
Some area source
locations identified and
tracked. Improved use of
non-point source data.
New on-road model
(MOBILE 6) accounts for
many variables such as
vehicle type, vehicle
speed, road type, and fuel
types.
Equipment-type
information again
improved. Model
(updated NONROAD) also
attempted to account for
changes in engine power
levels related to pollution.
Much of the activity data
and emission factors still
unverified. Fuel types
partially accounted for.
Limitations of the Air Toxics Inventories

               For the 1993 baseline inventory, EPA received very little State participation.  Over
               the past 10 years, States have progressively submitted more toxics emissions data,
               of increasingly improved quality. EPA officials said that, due primarily to States'
               increased awareness and need for toxics emissions data for purposes other than the
               inventory, the States wanted to rely more on their own estimates than the estimates

-------
generated by EPA. Some of the assumptions and limitations associated with the
1993,1996, and 1999 inventories are discussed below.

1993 Baseline Inventory

EPA developed the 1993 baseline (the 1993 NTI) as a repository of information
collected by EPA to meet the requirements of Clean Air Act Sections 112(c)(6) and
112(k), which required limited scope studies on a select number of toxic air
pollutants found in urban areas. In total, EPA studied 33 of the 188 air toxics as a
starting point for developing the NTI, but recognized that there were information
gaps.

Because EPA's authority to require States to report air toxics emissions was limited,
only three States and one local agency provided useable input to the 1993 emission
estimates. For the remaining 47 States and 61 local agencies that did not submit air
toxics emissions information, EPA's only viable option was to use national data and
apportion it down to the county level (the "top-down" approach).  Sources of this
national data included the EPA-developed national estimates and information
gathered from the development of the MACT standards. Because studies have
found that air toxics often are a localized problem, the top-down approach is
considered one of the least desirable methods because it does not accurately depict
local variations.

Subsequent to developing the 1993 baseline, EPA sometimes revised the baseline
when it received more  accurate 1993 information. However, Agency officials stated
that due to limited resources and because the baseline inventory is only an estimate,
their efforts are typically geared toward obtaining better data for current and future
air toxics inventories rather than updating the 1993 baseline.

1996 Inventory

For the 1996 inventory, some of gaps in the 1993 baseline inventory were filled to
provide a more complete picture of air toxics emissions. As compared to the 1993
baseline inventory, EPA used fewer top-down approaches because 36 States and
3 local agencies submitted air toxics emissions estimates, and MACT and Toxics
Release Inventory data was used for the remaining 14 States and 59 local agencies
that did not submit air toxics emissions information. However, there was still not
full State and local participation for reporting, and the State and local figures
provided were not verified by EPA. Further, the 1996 emission estimates for the
Area and Mobile Non-Road sources of pollution were still largely based on national
activity data and emission factors.

-------
              1999 Inventory

              Fertile 1999 inventory, 39 States, 9 local agencies, and 3 tribal agencies submitted
              emissions estimates, and there were improved Quality Assurance/Quality Control
              efforts by EPA and the States, who performed the following four activities:

              •  States used a Quality Control formatting tool for file format and data field
                 verification prior to sending the estimates to EPA.
              •  EPA used a software tool to identify duplicate facilities.
              •  EPA sent States draft summaries of emission data for States to review for
                 accuracy and completeness.
              •  EPA performed Quality Assurance/Quality Control checks to identify gross
                 errors with the emission estimates.

              However, there are still limitations with the 1999 NTI.  EPA once again had to use
              MACT, Toxics Release Inventory, or apportion emissions in a top down manner for
              the 11 States and 53 local agencies that did not submit air toxics emissions
              information. Also, Agency officials said that the Quality Assurance/Quality Control
              check was designed to identify emission estimates substantially outside the
              historical range of reported emissions.  These checks are not used to identify
              inconsistencies reported from independent sources of data or against national
              estimates. Additionally, these checks do not include sampling State data to  verify
              the adequacy of emission estimation methodologies. These additional checks would
              substantiate State-reported data.

Progress In Reducing Emissions Uncertain

              As shown in Chart 2.1, as of September 2003, the 1993 baseline inventory indicated
              that a total of 6.1 million tons of toxics emissions was released in 1993, while the
              1996 inventory indicated that a total of 4.6 million tons of toxics emissions was
              released in  1996, or total reductions of 1.5 million tons.  Recently EPA finalized the
              1999 inventory, which showed 5.1 million tons of emissions, or an increase of
              500,000 tons when compared to the 1996 inventory.  However, this reduction (1993
              to 1996) and this increase (1996 to 1999) may have resulted due, at least in part, to
              EPA's change in the way it estimated the inventory, rather than a real reduction in
              emissions.  EPA was unable to provide us with reliable data that would accurately
              show how much air toxics emissions were actually reduced from 1993 to 1996, and
              how much of the reported change may  have been due to a change in methodology.
                                           10

-------
                           Chart 2.1:
       Emissions for the 1993,1996 and 1999 NTIs
                       (in millions of tons)
 Non-road
On-road
           1993
         Total: 6.1
                     On-road
  1996
Total: 4.6
                                         On-road
         1999
        Tola!: 5.1
                  Major
 Area
Mobile
  EPA credited the 1.5-million-ton reduction to its programs designed to control air
  toxics emissions. Almost all of these reductions occurred-in the major source
  category, where emissions decreased from 2.7 million tons in 1993 to 1.2 million
  tons in 1996. During this 3-year period, EPA issued 11 MACT standards
  representing 664,000 tons of emission reductions.  However, there still remains
  836,000 tons of emissions reductions achieved by controlling sources not subject to
  MACT standards. During this same time period, EPA received improved data and
  substantially changed its method for calculating emission estimates.  Specifically,
  the 1996 NTI had much more State-submitted data on major source emissions, and
  the accuracy of the mobile emission estimates improved.

  Regarding the recently completed 1999 NTI, again there is a significant change in
  the calculation methods used, which may impact the amount of emissions reported.
  For example, for the 1999 NTI, improved mobile models were used for both the
  on-road and non-road estimates, there was slight increase in State and local agency
  participation, and there were improved Quality Assurance/Quality Control efforts by
  EPA and the States.  These changes in calculation methods may have caused, at
  least in part, the 500,000 ton increase in emissions from 1996 to 1999. Recent
  revisions to the 1993 and 1996 NTIs demonstrate a significant improvement in
  characterizing EPA's progress in reducing air toxics emissions. However, this
  increase again demonstrates that EPA is not certain whether changes in the
  inventory may be due, at least in part, to the Agency's change in the way it
  estimated the inventories.
                              11

-------
              The above figures for the 1993 and 1996 inventories were used recently in key EPA
              reports to the Congress and the public on the Agency's progress in reducing air
              toxics emissions, including:

              •   EPA's November 2003 "FY 2003 Annual Report" to the Congress and the
                  President, which serves as the Agency's program performance report as required
                  by GPRA;

              •   EPA's August 2003 "Latest Findings on National Air Quality: 2002 Status and
                  Trends Report," which provides the public with the latest information on air
                  quality nationwide; and

              •   EPA's June 2003 "Draft Report on the Environment 2003," which provides the
                  nation with EPA's first ever national picture of the environment and represents
                  EPA's first step at developing a set of comprehensive environmental indicators.

              However, in February 2004, after the conclusion of our field work, EPA provided us
              with revised totals for the  1993 and 1996 inventories (prior to this time, no figures
              had been provided for the  1999 inventory). For example, the Agency's revised
              2004 estimate of the 1993  baseline is now 7.2 million tons.  Appendix B provides
              further details on EPA's recent update of the 1993 and 1996 inventories. In our
              opinion, these revised totals further substantiate our position that EPA is uncertain
              of its progress in reducing emissions and needs to improve its measures of air toxics
              progress.

Status of EPA Efforts to Improve Recent Inventories

              While improvements to estimation methods were made to the more current
              inventories, EPA acknowledges that there are still deficiencies in the estimation
              methods. These deficiencies vary from source to source and affect the reliability
              and accuracy .of the data. Because development and maintenance of air toxic
              emission inventories is not explicitly required by the Clean Air Act, most of the data
              used for the emissions inventories were obtained and developed for other reasons
              and then secondarily used  to compile the NTI. Below is a description of how the
              inventory is amassed and examples of the assumptions and limitations associated
              with the data for the primary source categories of pollution (major, area, and
              mobile).

              Major and Area Sources

              Because major and area source categories include stationary sources and often use
              the same data sources, they are presented together. Major sources comprised
              26 percent of the 1996 inventory, and area sources comprised 24 percent.  Some
              area sources, such as wood burning stoves, cannot be cost-effectively inventoried as
              individual sources because they are comprised of a large number of sources spread
              across the country, so emissions are calculated at a national level and then  allocated

                                           12

-------
down to the county level. Also, some source categories have both major and area
emitters. For example, a small neighborhood dry cleaner may not emit enough toxic
emissions annually to reach the threshold of a major source, but a large regional dry
cleaner may reach this threshold and thus be classified a major source.

Following is a description of the four primary sources of information used to
estimate both major and area source emissions, from most reliable to least reliable.

   State-Reported  Emissions: Because there are no reporting requirements for
   States, the data submitted by States are inconsistent and States are not required
   to verify the accuracy of the data. To improve State data, EPA in 2000 included
   in a proposed regulation specific reporting requirements that would standardize
   the  type of toxics data gathered by States, as well as the methods used to
   calculate estimates.  However, the proposed air toxics reporting requirements
   were deleted from the final regulation.  This was due largely to EPA's concerns
   about adding more reporting requirements on States, the Agency's belief that the
   Clean Air Act is  not clear on air toxics reporting, and its belief that there would
   be lawsuits if specific reporting requirements were imposed. Seventeen of the
   22 State and local agencies that commented on the proposed regulation said that
   States should be  required to report air toxics emissions data. However, some
   said they would not support increased reporting requirements without increased
   funding. In 2003, EPA obligated approximately $181 million in Clean Air Act
   Section 105 grant funds to States for carrying out a variety of Clean Air Act
   goals, but the Agency did not use these funds to negotiate commitments with
   States for consistent, validated air          ,
   toxics emissions data                        TRie emissions tn«enteiy

   EPA has also attempted to improve
   air toxics emissions reporting by
   working with the States through the
   Emission Inventory Improvement
   Program (EIIP),  which was intended
   to promote the development and use
   of standard procedures for collecting
   and reporting emissions data.
   Initially, this program worked to
   develop standardized methods for
   collecting data for both criteria4  and
   toxic pollutants.  However, EIIP
   workgroup officials decided that
   developing better air toxics emissions
   estimation methodologies was beyond
   the  intended scope of the workgroup
                                                          sM Oflfee
                                                       Tip goal ft achieved fay developing « $&t
                                                       arprefarted arid vX&tr&frta method*" *?r
                                                       all invent9ry«s*l»«3teq| t*sk$. This
                                                       Bferp&tii&ttiBn jrfipWSwe* #» q&«t$te*jiqy
                                                       Of caHUsctetf tfirffcdraf result* in ftjareasBd
       4Six common air pollutants found nationwide that harm human health and the environment, called criteria
pollutants because EPA sets standards for these pollutants by first developing health-based criteria.
                                           13

-------
effort, and focused only on criteria pollutants.  According to EPA officials,
beginning in 2004, the EIIP was no longer active because of limited funding for
the program. However, EPA officials told us that there is an ongoing
reorganization in the Office of Air Quality Planning and Standards that will
likely include a group dedicated to improving emission factors and emission
estimation methods for air toxics.

MACT:  EPA initially gathered data to learn as much as possible about the
industry before a MACT emission limitation was established. In doing so, EPA
relied on many different sources of data, including: national activity data applied
to emission factors, facility level emissions reported to EPA, and facility level
emissions obtained from the Toxics Release Inventory database. Of the
70 MACT area source categories that reported emissions data:

•  19 used facility-level emissions reported to EPA.
•  36 used national activity estimates.
•  15 relied on Toxics Release Inventory data.

Only 35 of the 70 MACT area source categories calculated emission estimates
for 1996. For example, of the  19 facility level estimates reported to EPA, only
9 estimated 1996 emissions, while the remaining 10 estimated emissions for 1 or
more calendar years ranging from 1991 to 1997. For 1 of the 36 sources using
national activity data - halogenated solvent cleaners - EPA developed an
estimate of the total mass for various toxic air pollutants, but did not know how
much of each air toxic comprised the total. As a result, EPA divided the mass
equally among a list of applicable air toxics.

Toxics Release Inventory: The Toxics Release Inventory database is mandated
by the Emergency Planning and Community Response Act, which requires a
select group of industries meeting specific requirements to report toxics release
information to EPA, including releases to air, land, and water. In those instances
where EPA used the database to develop MACT standards, the Agency used this
same data for the toxics inventory. Because it was the only available
information, EPA used the database despite the significant limitations associated
with the data. Not all facilities that emit air toxics meet the requirements to
report Toxics Release Inventory emissions.  As a result, some sources of air
toxics would not be identified and the emissions would not be recorded or
reported to EPA. The data is also compromised because facilities are allowed to
report reasonable but unverified estimates of toxic amounts instead of monitored
releases.  Finally, calculation methods can vary annually and, as a result, some
facilities  may report emission reductions not because of actual reductions but
because a different calculation method was used, which may result in EPA
overstating or understating the actual amount of reductions.

-------
                                                     y, Hair cafe prOductst;:
                                                     H, *m) rei ear* ftpguiife
                                      &* ihdtidOrf In this Area SOUttH eatery, £PA
                                      •Ori ppwtefci uspSB: and population tfatp ferfte
                                      ytef 1$a& T* patoufate «w tafei em&sians
                                                             IUJS. population
                                      10 ^et^rraineaip total arrissiOnsfrOrmpesrsOnat!  .
                                      ttes county tevet feasati on U.S, CeniijH 8ur?a«
National Estimates: When EPA
could not obtain emissions data
from the above three activities, the
Agency reverted to estimating the
emissions from these source
categories on a national scale and
then allocating them down to a
local level.  When national-level
emissions are used, it does not
account for significant local
variations, which can have a large
effect on the health of the local
population exposed to those toxic
emissions.  For example, EPA's
estimates assume each person in
the United States, regardless of
age, sex, and other differences,
uses the.same amount of personal
care products (see box).
 When there is no data available for the year targeted based on the above four
 primary sources of information, the Agency as a last resort will occasionally use
 data derived from either a prior or subsequent year's inventory. This affects the
 inventory's ability to accurately depict emissions reduction trends because, at
 times, the same data are being used for multiple years.

Mobile Sources

While major stationary sources of emissions include many types of pollution-
creating equipment and processes, the number of mobile source emissions points far
exceed stationary sources in that they are produced by millions of engines of varying
types and sizes that move around the country.  Due to the quantity, diversity, and
mobility of these on-road and non-road sources, sophisticated methods using
complex calculations that include a host of assumptions were developed to estimate
emissions.

Since the inception of the EPA mobile source program, EPA has historically focused
its efforts on understanding and controlling on-road emissions because they
represented the majority of the pollution from mobile sources. These efforts have
led to increased knowledge of on-road emission factors and better activity data
EPA has made substantial progress in reducing on-road emissions through the
production of "cleaner" vehicles and the implementation of on-road vehicle control
programs. This, combined with the ever-increasing popularity of non-road sources,
has resulted in EPA becoming more concerned about non-road emissions.
Furthermore, non-road sources was the only category to show an increase in
emissions from 1993 to 1996.  However, the nature of non-road sources presents
                             15

-------
EPA with considerable challenges, due to the large number of equipment types and
manufacturers.

   On-Road Sources: On-road mobile emissions comprised 28 percent of
   emissions for the 1993 baseline inventory. The predominant source of emissions
   in this category was gasoline-fueled automobiles. There are two general
   approaches to calculating on-road emission estimates for the 21 toxic air
   pollutants emitted by motor vehicles:

   •  For 15 of the on-road air toxics emissions, estimates are derived from either
      particulate matter or volatile organic compound emissions data For example,
      the Agency takes the particulate matter data for on-road emissions and then
      estimates how much air toxics are in those emissions. In  doing so, EPA
      assumes that air toxics emissions from vehicles are directly proportional to
      vehicle particulate matter emissions.

   •  For six of the on-road air toxics emissions, total vehicle miles traveled
      (obtained from the Federal Highway Administration's database) for each class
     •of vehicle are multiplied by an emission factor to develop the air toxics
      estimate.

   EPA currently develops emission estimates that account for specific vehicle
   activities, such as vehicle type and speed, and emission-related factors, such as
   ambient temperature. For example, while the 1996 mobile estimate included .
   7 vehicle types and 1 vehicle speed, the 1999 estimate included 28 vehicle types
   and 12 vehicle speeds. However, with these more complex calculations come
   increased assumptions and limitations. In 2000, the National Research Council
   performed a review of EPA's on-road model, entitled Modeling Mobile Source
   Emissions, and identified the following assumptions and limitations:

   •  EPA's selection of vehicles for testing does not sufficiently represent high-
      emitting vehicles. Emissions from high-emitting vehicles are a relatively
      small fraction of the on-road fleet, but contribute a large fraction of total
      vehicle emissions, and thus require a large sampling fraction. If high-emitters
      are not properly characterized, emission factors can be seriously biased.

   •  EPA's calculation method does not consider road-grade effects on emissions.
      Road-grade influences vehicle emissions in that higher emissions occur when
      vehicles negotiate steep road grades.

   •  Light-duty truck emissions are sometimes estimated from passenger
      automobile test data, even though some trucks are regularly used as working
      vehicles, which results in higher emissions due to the increased load.

   •  EPA lacks sufficient data to estimate effects of high emitters and sometimes
      fills in missing data by  assuming high emitters behave the same as normal
      emitters.
                             16

-------
The National Research Council review stated that EPA needs to better
understand and quantity the uncertainty related to sources of data used in the
model, such as the use of small and nonrepresentative emissions data  The
Council believes that the quantification of these uncertainties is critical for
understanding the weaknesses of the model and identifying the most critical
needs for improving emissions test data.  In response to the Council's findings,
EPA has initiated a $4 million test program to collect data on high-emitting
vehicles, and the Agency is developing a new model to better quantify the
uncertainty in on-road emissions estimates. In our view, these are steps in the
right direction. Nonetheless, at the present time, the Agency has limited
assurance that data reported in its GPRA performance measure is accurately
assessing progress.

Non-Road Sources: Non-road mobile emissions comprised about 10 percent of
emissions for the 1993 baseline inventory and increased to more than 19 percent,
or nearly one-fifth of the total, for the 1996 inventory.  Until the mid-1990s,
emissions from these engines were largely uncontrolled. This category covers a
diverse collection of engines for equipment, vehicles, and vessels, ranging from
leaf blowers to earth-moving equipment. As compared to on-road methods,
emissions estimation methods for non-road sources are much less developed
because they lack adequate basic data. In developing non-road estimates, EPA
typically develops a usage estimate such as the number of hours a piece of
equipment operates during the year.  EPA then multiplies that number by an
emission factor, which essentially calculates how much pollution is emitted for
each hour the equipment is operating. Both the non-road usage estimates and
emission factors have significant limitations.

Usage Estimates - Currently, EPA cannot verify the accuracy of the data. Usage
estimates come from a marketing research firm tfiat gathers usage information
from equipment manufacturers through telephone surveys.  The marketing
research firm takes this data and uses a proprietary process to project non-road
population totals. Because EPA cannot assess the accuracy of the process, EPA
officials expressed interest in developing their own strategy for collecting usage
data.  For example, a more valid estimate would be generated if EPA could place
a monitor on equipment to measure the hours of operation to better verity the
accuracy of their data. Despite the limitation described above, EPA uses the
survey data because it is the only available source of non-road equipment usage
information.

Emission Factors - EPA officials have stated that there are few studies that have
developed toxics emission factors for non-road equipment. An emission factor
is developed by monitoring emissions from several test sites and computing an
average emission rate. However, for many of the existing studies, engine
emissions have not been tested while equipment is operated under field
conditions (see example in following box).  In lieu of developing its own
emission factors, EPA has largely relied on certified emissions information
                          17

-------
                                                                             Factors
  provided by engine manufacturers.  Also, the non-road calculation does not
.  account for engine deterioration, which results in higher emissions in later years.
  Although EPA calculates deterioration for on-road vehicles by using State
  vehicle registration information, similar information does not exist for most non-
  road equipment. However, EPA has        ,    /    tr         *  ',    's
  efforts underway to develop
  improved air toxics estimates for
  some categories of non-road
  equipment. For example, EPA just
  completed a test program that
  characterized air toxics emissions for
  14 non-road diesel engines under a
  variety of operating conditions with
  different fuels, and is also testing
                                                                               •, rwn*
                                                                           So For--    - i
                                                      exampte^, wNn » tewfetote soopps a feuetet
                                                      of soil, rag»'twjS9poww*an aswef fc
                  2- and 4-stroke gasoline lawn and
                  garden equipment at its mobile
                  source laboratory in ORD's
                  National Exposure Research
                  Laboratory. According to Agency          '"      "     "  *  5          -
                  officials, the efforts should result in substantial improvement in estimating air
                  toxics emissions for some categories of non-road equipment.

Additional Uses of the Air Toxics Emissions Inventory Data

               In addition to needing quality information for reporting results in GPRA, there are
               several other important needs for a valid, reliable, and accurate inventory of air
               toxics emissions, such as:

               Rulemaking Efforts - The 1993 baseline and 1996 inventory were relied upon
               extensively in EPA's development of the 2001 Final Mobile Rule.5 In developing
               this rule, EPA used data from these inventories to determine the controls that would
               be needed to reduce mobile  emissions to the desired level.

               National Air Toxics Assessment - In carrying out its National Air Toxics
               Assessment, EPA used the 1996 NTI to develop a subset of 33 air toxics posing the
               greatest risk to urban areas to better understand their priority as related to risk, as
               well as their effects on the nation's population. The emissions data were input into
               models that projected ambient air concentrations and ultimately estimated the level
               of human exposure to the 33 toxics. EPA then followed risk characterization
               guidelines to estimate both the cancer and non-cancer effects. The Science Advisory
               Board identified weaknesses in the methods used to estimate emissions (1996 NTI)
               and determined that improved accuracy and reliability of the NTI was vital to the
               National-Scale Air Toxics Assessment effort.
        540 CFR Parts 80 and 86, Control of Emissions of Hazardous Air Pollutants From Mobile Sources; Final
  Rule, March 29,2001
                                            18

-------
               Residual Risk Assessments - After industry has installed required MACT controls
               to reduce air toxics emissions, the Clean Air Act requires EPA to assess the
               remaining risks to public health. The NTI is critical to measuring the effectiveness
               of these controls.  If, after 8 years from MACT promulgation (or 9 years after
               promulgation of the 2-year MACT source categories), the MACT controls have not
               reduced emissions to an acceptable risk level to protect public health with "an ample
               margin of safety," EPA must promulgate health-based standards for that source
               category to address this risk.

               Modeling - In addition to air toxics being modeled for the National-Scale Air
               Toxics Assessment effort, EPA also uses the NTI as the basis for numerous
               modeling efforts at the local and regional levels. For example, EPA will model a
               series of toxics for a community or smaller geographical area EPA, State, and local
               agencies use such model-derived data to estimate exposures and focus limited
               resources.  ORD officials also cited an additional need for air toxics data in
               attempting to understand how pollutants behave once released into the environment.
Conclusions
              EPA is likely to rely on emissions data for gauging its progress in reducing health
              risks from airborne toxics for many years to come. However, the Agency faces
              considerable challenges in improving this measure.  The most reliable approach
              would be to require State and local agencies to report validated emissions once
              every 3 years. EPA proposed this approach in 2000 but did not follow through on
              this plan due to concerns about adding more reporting requirements on States, as
              well as the belief that the 1990 Clean Air Act is not clear on air toxics reporting.
              However, in our opinion, GPRA is clear in its mandate that Federal agencies
              demonstrate they are achieving results, and GPRA also provides them with the
              authority to establish appropriate measures for gauging their progress. Nonetheless,
              if EPA does not wish to include State reporting requirements in a rulemaking, the
              Agency can achieve the same results by judicious use of its Section 105 grants to
              State and local agencies.

              Regarding rne other limitations discussed in this chapter, EPA currently does not
              have a systematic approach for improving air toxics emissions estimation techniques
              similar to the approach that the National Research Council recommends EPA use to
              improve its mobile model.  Such an approach would allow EPA to identify the
              various uses of the inventory, and the level of accuracy needed for those uses. This
              will enable the Agency to gauge the effectiveness of the existing inventory. Further,
              EPA will be able to use this assessment to make future improvements to the
              inventory by identifying and improving the emissions estimates that have the largest
              impact on the accuracy of the inventory. We believe the Emissions Inventory
              Improvement Program has the organization, expertise, and funding to guide and
              oversee such improvements.
                                            19

-------
              While improvements to the inventory are important to strengthening EPA's air
              toxics GPRA performance measure, these improvements will also benefit EPA and
              the States in developing and implementing effective toxics emissions control
              strategies to protect the public from the dangers of air toxics. In addition to
              comparing reductions to the baseline inventory, the Agency should also compare
              emission totals between all other completed inventories, which would better enable
              EPA to verify the actual emissions reductions achieved. This revised approach
              would become more meaningful as future inventories are developed.
Recommendations
              In order to obtain reliable air toxics emissions data once every 3 years, we
              recommend that the Assistant Administrator for Air and Radiation:

              2-1.    Improve the accuracy and reliability of the air toxics GPRA performance
                     measure by providing increased funding and support for development of the
                     NTI by:

                     (a)  requiring all authorized State and local agencies to report annually air
                         toxics emissions data for stationary, area, and mobile sources;

                     (b)  developing air toxics emissions reporting policies, procedures, and
                         guidance for facilitating standardized State and local agency reporting of
                         air toxics emissions data.  After the reporting requirements have been
                         imposed, use Clean Air Act Section 105 grant funds to ensure that State
                         and local agencies comply with the requirements; and

                     (c)  developing more accurate emissions factors and activity data for
                         stationary, area, and mobile sources by either reviving the EIIP program
                         and utilizing the organization and expertise of this program, or by
                         establishing a similar program in-house.

              2-2.    Because of the difficulties encountered and resources involved in periodically
                     attempting to ascertain what the actual air toxics emissions totals were for the
                     1993 NTI, EPA should use the 1996 NTI as the baseline year for measuring
                     progress.

              2-3.    Supplement the existing measure of comparing air toxics emissions
                     reductions to the 1993 baseline inventory by also comparing emission totals
                     between all other completed inventories.

Agency Comments and OIG  Evaluation

              Office of Air and Radiation (OAR) and Office of Research and Development (ORD)
              officials provided detailed draft comments to our draft report and, where
              appropriate, we made revisions. Because these responses are draft and may not
                                           20

-------
represent the Agency's final position on the issues and recommendations in this
report, the draft responses are not included as appendices to this report.
Nonetheless, we made changes to the draft report based on both sets of draft
comments, as well as our exit conference discussions with Agency officials.

Many of OAR's comments involved changing the 1993 and 1996 NTIs to reflect
EPA's methods and assumptions used in developing the recently issued 1999 NTI.
As shown in the report, we believe it is important to discuss the 1999 NTI and its
impact on the prior two inventories'  (1993 baseline and 1996 NTI) emission totals.
However, due to EPA's delay in issuing the 1999 NTI, we were only able to
evaluate  the factors influencing the reliability of the 1993 and 1996 inventories and,
without an in-depth examination of the factors impacting the reliability of the 1999
NTI similar to the efforts we undertook with regard to the 1993 and 1996 NTIs, we
do not believe it would be appropriate to reflect EPA's most recent assumptions en
masse. Instead, we have presented both sets of figures as appropriate in the report.

OAR's detailed draft comments agreed with the recommendations in this report,
with one exception.  OAR intends to continue using the 1993 NTI as the baseline,
instead of implementing our recommendation to use the more reliable 1996 NTI as
the baseline for measuring progress under GPRA.  Agency officials explained that if
EPA were to use the 1996 NTI as the baseline, it would not enable them to capture
the emissions reductions achieved by stationary and mobile source standards
implemented between 1993 and 1996. While we recognize this point, we continue to
believe-that, due to the unreliability of the 1993 NTI, EPA should use the 1996 NTI
as the baseline.

As noted in our report, emissions estimating techniques have undergone continuous
change and improvement since the 1993 baseline was developed, and subsequent
inventories have been amassed quite differently, making comparison to the 1993
NTI much less reliable as a measure of GPRA progress as compared to the  1996
NTI. For example, only 3 States provided data for the 1993 baseline, while 36
States provided data for the 1996 inventory. Also, as discussed in our report,
significant uncertainties continue to hamper use of the 1993 NTI as a baseline. For
example, these uncertainties caused EPA recently to revise the 1993 baseline from
6.1 million tons to 7.2 million tons, more than an 18 percent increase. These revised
totals further substantiate our position that EPA is uncertain of its progress in
reducing emissions and needs to improve its measures of air toxics progress. We
agree that subsequent inventories have improved; however, EPA's limited resources
may be better used in improving the 1996 NTI as a baseline in lieu of the 1993 NTI.
We continue to believe that,  until ambient air toxics monitoring data is available to
replace emissions data, the best available baseline for measuring Agency progress
for GPRA purposes is the 1996 NTI.

With regard to ORD, officials of this office concurred with our draft report
recommendations, stated that ihe draft report was reasonable, and provided several
                             21

-------
detailed comments intended to sharpen the quality and accuracy of the report. These
changes were made.
                             22

-------
                                Chapter 3
             GPRA Measure Needs to Address Risk

	    of Air Toxics  Exposure	,

             EPA currently measures progress in its Air Toxics Program by reporting on tonnage
             reductions of emissions for all 188 toxic air pollutants collectively but does not
             account for the varying levels of toxicity.  This aggregate GPRA measure, although
             useful, does not provide EPA, Congress, and the public with information on the
             specific subset of air toxics that are the most dangerous to human health.  Agency
             officials said that they see merit in adding a risk component to their GPRA
             reporting, and they recently began exploring possibilities of doing so. This
             supplemental risk measure would better enable EPA to track its progress in reducing
             those air toxics that are most harmful to human health and, in so doing, more closely
             meet the intent of GPRA.

EPA Needs to Supplement Emission Reduction Measure

             The emissions reduction measure is an important component of EPA's Air Toxics
             Program. In addition to improving data, as discussed in Chapter 2, the measure
             needs to be supplemented with information related to toxicity or risk. The emission
             reduction measure alone cannot ensure risk reduction occurs, because it does not
             specify which emissions are being reduced or if those reduced are in fact the most
             harmful to human health.  If the Agency continues to only measure total emission
             reductions without adding a risk component to its GPRA measure, there is limited
             assurance that EPA is maximizing its efforts to reduce risk because air toxic health
             risk reductions are not proportionate to the tonnage of emission reductions. For
             example, toluene and xylene emissions represent about 30 percent (2 million tons)
             of the total toxics emissions (6.1 million tons) in the 1993 baseline. However, if
             EPA were to eliminate toluene and xylene entirely, it would not result in as much of
             a reduction in risk to human health as would a similar tonnage reduction in benzene
             and formaldehyde, which are more harmful air toxics.

Risk-Based Information Already Exists

             Although much more needs to be learned about the health effects of various air
             toxics, we believe EPA has sufficient risk information to begin adding a risk-based
             component to its air toxics emissions reduction measure. To date, EPA has not
             included a risk element to its measure because Agency officials believe adequate
             risk data are not available. However, EPA has completed two efforts that should
             enable the Agency to stratify the 188 air toxics into several categories of risk and
             allow it to report on emission reductions for these specified subsets of air toxics.
             EPA's Integrated Urban Strategy and a study on mobile air toxics emissions
                                         23

-------
illustrate how EPA has already stratified some of the more prevalent and hazardous
air toxics.

Integrated Urban Strategy

The Clean Air Act directed EPA to identify at least 30 air toxics from emissions of
area sources that represented the greatest threat to public health, along with actions
the Agency is taking to reduce exposure.  Further, the Act directs a reduction in
public health risks of not less than 75 percent in the incidence of cancer attributable
to area sources. In 1999, EPA identified 33 urban air toxics from major, area, and
mobile sources. Some of EPA's emission estimation techniques included:
evaluating the health effects of the 188 air toxics; assessing air quality monitoring
data; reviewing existing studies; and producing a list of pollutants based on the
relative hazards they pose in urban areas while considering toxicity, emissions, and
related characteristics. EPA used three ranking methods to produce a final list of
priority urban air toxics:

*   Risk Assessment/Hazard Ranking Studies - Fourteen urban studies were
    reviewed, and results were sorted by pollutant and ranked numerically.
•   Cumulative Exposure Project - Modeled ambient concentrations of air toxics
    were compared to health benchmarks to estimate the level of toxic risk.
•   Exposure/Toxicity Indicators Ranking - Seven indices were prepared for the
    188 pollutants; each pollutant was numerically  ranked within each index.

EPA is beginning to make efforts to prioritize the list of 188 air toxics. Using the
1999 NTI, for each air toxic, EPA has estimated the tons of emissions reduced and
then weighed each air toxic by health criteria. EPA officials stated that they plan to
conduct this analysis every 3 years to coincide with new emissions inventories.
Additionally, EPA plans to expand the National Air Toxics Assessment to include
each of the 188 air toxics for which the Agency has established health criteria.  EPA
officials said utilizing these efforts, along with data collected from the air toxics
monitoring network, will enable the Agency to develop a relative risk ranking of the
most harmful air toxics. We believe that EPA could use this risk information to
stratify the 188 air toxics into several categories of risk and report under GPRA on
emission reductions  for these specified subsets of air toxics, in addition to reporting
aggregate tonnage reductions for all toxic air pollutants collectively.

Mobile Air Toxics Effort

EPA developed a list of air toxics emitted from mobile sources by searching through
available databases and studies that speciated the emissions from motor vehicles and
their fuels.  By comparing the list of air toxics to the toxicity information available
in EPA's Integrated  Risk Information System, EPA was able to identify 21 air toxics
that are possible, probable, or known human carcinogens. Thirteen of these 21 air
toxics were also identified as part of the 33 air toxics studied in the Integrated Urban
Strategy. Moreover, EPA routinely  gathers more information on five mobile air
                             24

-------
              toxics they consider the most dangerous: benzene; 1,3 butadiene; methyl tertiary
              butyl ether (MTBE); formaldehyde; and acetaldehyde.
OMB Review of EPA's Air Toxics Program Cited Similar Problems

              As part of its efforts to review agencies' implementation of GPRA, the Office of
              Management and Budget (OMB) developed a rating tool6 to identify common
              performance challenges found in Federal agency programs. Using this newly
              created tool, in 2003 OMB evaluated EPA's Air Toxics Program. OMB found that
              the program's purpose was clear and the management of the program good, but the
              program had not been able to demonstrate that it was maximizing the program's net
              benefits. Furthermore, linkages were insufficient between annual performance goals
              and the long-term performance goal of protecting 95 percent of the United States
              population from unacceptable risks of cancer and other significant health problems
              from air toxic emissions.
              OMB's evaluation rated the areas of program results and accountability as
              "Results Not Demonstrated," and cited the following concerns with EPA's GPRA
              goals and measures:

                 ....Although the long-term cancer reduction goal is clearly outcome-related,
                  "unacceptable risks" are not defined, the relationship between emissions
                 changes and actual health outcomes is not known because there are no
                 efficiency measures	there are inadequate linkages between annual
                 performance and long-term goals that prevent it [EPA] from demonstrating its
                 impact on human health	

              OMB's observations regarding EPA's results measures are consistent with our
              observations, particularly as they relate to measuring reductions in air toxics risk to
              human health. In response to these findings, the Administration took measures to
              implement OMB's recommendations by requesting $7 million in increased funding
              for the Air Toxics Program.  The funding was to be for State grants involving
              ambient air toxics monitoring that would help fill these data gaps.  In addition, EPA
              indicated it would focus on maximizing the program's net benefits by minimizing
              the cost of each harmful health effect avoided, and establishing better performance
              measures.  OMB plans to review EPA's Air Toxics Program every year until
              adequate results are demonstrated. According to Agency officials, it will be some
              years before EPA has a national ambient air toxics monitoring network capable of
              measuring air toxics in ambient air on a widespread basis. Until that time, the
              Agency could improve its focus on health risk by stratifying the 188 air toxics into
              several categories of risk for GPRA reporting.
        6Known as the Program Assessment Rating Tool, this Tool was created to make ratings more consistent,
  objective, and credible.
                                           25

-------
Conclusion
              EPA's mission is to protect public health, not just to reduce emissions. As the EPA
              OIG and OMB have identified, there presently is insufficient linkage between health
              risk and tons of emissions reduced. As such, we believe EPA needs to add a risk
              component to its air toxics measure that will enable the Agency to better
              demonstrate that it is managing for results, as called for under GPRA.
Recommendation

              3-1.   We recommend that the Assistant Administrator for Air and Radiation
                    supplement the existing tonnage reduction measure of progress for the EPA
                    air toxics program with measures that address progress toward reductions in
                    human exposure and health risk.  .   -

Agency Comments and OIG Evaluation

              OAR officials commented that this chapter did not fully characterize the extent of
              the Agency's efforts to collect data for prioritizing hazardous air pollutants as it
              relates to risk. We incorporated OAR officials' comments as appropriate. However,
              because up to this time the Agency has not used this data to measure progress under
              GPRA by reporting on the varying levels of toxicity, we believe our
              recommendation is appropriate. ORD recommended changing the phrase "health
              risk component" to "reduction in human exposure and health risk" in
              Recommendation 3-1. We agreed, and have  revised this recommendation
              accordingly.
                                         26

-------
                                                                        Appendix A
                Details on Scope and Methodology
To assess the appropriateness of EPA's current air toxics GPRA performance measure for assessing
progress towards achieving the air toxics objective, we reviewed documentation that pertained to
GPRA measures and objectives, including:

        •   The Government Performance and Results Act (GPRA) of 1993
        •   EPA's Strategic Plans for 1997,2000, and 2003
        •   EPA's Fiscal 2003 Annual Plan
        •   EPA's Fiscal 2002 Annual Report
        •   OMB's Program Assessment Rating Tool Evaluation of EPA's Air Toxics Program

To assess the accuracy and reliability of data supporting EPA's air toxics GPRA performance
measure, we conducted interviews with EPA Regional, OAR, and ORD officials; researched
documents; and attended training in support of these efforts. Budget information was obtained
through the Office of the Chief Financial Officer and OAR's Office of Transportation and Air
Quality.

We reviewed Section 112 of the Clean Air Act to understand EPA's obligations to control and
reduce air toxics emissions.

To gain a better understanding of the history and development of the inventories, we reviewed EPA's
documentation, which included:

           Development of the 1993 Baseline National Toxics Inventory
           Development of the 1996 National Toxics Inventory
           Development of the 1999 National Toxics Inventory
           Toxics Release Inventory requirements for reporting emissions
           Emissions efforts developed under MACT programs and rulemakings
           Both the models for on-road (MOBILE 5b and MOBILE 6) and the model for off-road
           mobile emissions (NONROAD).

To determine the possibility of requiring State and local agencies to report emissions, we researched
the Consolidated Emissions Reporting Rule.

To understand EPA's guidance for implementing Quality Assurance/Quality Control procedures for
data and EPA's effort to implement this guidance, we reviewed:

           EPA Order 5360.1
           EPA's 1999 National Emission Inventory Preparation Plan
           EPA's Emissions Inventory Improvement Program
           EPA's Information Quality Guidelines
           The Challenge of Meeting New EPA Data Standards and Information Quality
           Guidelines in the Development of the 2002 NEI Point Source Data for HAPs

                                         27

-------
We researched documentation regarding the Integrated Risk Information System to determine how
this database is used in conjunction with the risk assessment phase of MACT standards. We also
reviewed the National-Scale Air Toxics Assessment for 1996.

Because the amount of emissions for the source category Other Mobile was minimal (17,000 tons),
we did not include that category in the scope of our evaluation. Types of sources in this category
include locomotive, aircraft, and commercial marine vessels.

Prior Coverage

        General Accounting Office

        •   Major Management Challenges and Program Risks - EPA (GAO-03-112), January 2003
        •   Few Agencies Reporting on the Completeness and Reliability of Performance Data
           (GAO-02-372), April 2002
        •   EPA Should Improve Oversight of Emissions Reporting by Large Facilities
           (GAO-01-46), April 2001
        •   EPA Faces Challenges in Developing Results-Oriented Performance Goals and
           Measures (GAO/RCED-00-77), April 2000
        •   Air Pollution - Limitations of EPA's Motor Vehicle Emissions Model and Plans to
           Address Them (GAO/RCED-97-210), September 1997

        EPA OIG

        •   EPA's Progress in Using the Government Performance and Results Act to Manage for
           Results (2001-B-000001), June 2001
        •   Emission Factor Development (6100306), September 1996
                                         28
                                                                                               i

-------
                                                                         Appendix B

  EPA's Recent Update to the 1993 and 1996 Inventories

The three charts below show EPA's estimates of air toxics emissions by major sector for the 1993,
1996, and 1999 NTI as of February 2004.
         Updated Emissions for the 1993,1996 and 1999 NTIs
                               (in mil lions of tons)
   Non-road
                              Non-road
                                                       Non-road
  On-i
According to Agency officials, the use of updated models to estimate mobile source emissions
resulted in a significant change in the mobile emissions total. EPA decided that both the 1993
baseline and the 1996 NTI needed to be revised to reflect these updated models, and as such, the
inventory totals changed. For example,

•  the 1993 baseline inventory increased from 6.1 million to 7.2 million tons.
•  the 1996 inventory increased from 4.6 million to 5.0 million tons.

EPA had not previously reported the air toxics emissions inventory totals for the 1999 inventory,
which is now reported to be 5.1 million tons, or an increase of 500,000 tons above the previously
reported 4.6 million tons for the 1996 inventory.  The Agency's newly revised air toxics emissions
inventory totals indicate a decrease in overall emissions, from 7.2 million tons in 1993 to 5.0 million
tons in 1996.  However, analysis of the new totals from the revised 1996 inventory (5.0 million) to
the new 1999 inventory (5.1 million) still shows an increase of 100,000 tons of air toxics emissions
from 1996 to 1999. These revised totals were provided in February 2004 after our field work was
completed. We did not verify the revised figures or independently determine the reasons for the
                                         29

-------
changes.  These revised numbers were not reflected in recent Agency reports to the Congress and the
public on EPA's progress in reducing air toxics emissions, including:

•  EPA's FY 2003 Annual Report to the Congress and the President, which serves as the Agency's
   program performance report as required by GPRA, which reported a 1993 baseline of 6 million
   tons in November 2003;

•  EPA's Latest Findings on National Air Quality:  2002 Status and Trends Report, which reported
   a 1993 baseline of 6.1 million tons and a 1996 inventory of 4.7 million tons in August 2003; and

•  EPA's Draft Report on the Environment 2003: Technical Document, which reported a 1993
   baseline of 6.11 million tons and a 1996 inventory of 4.67 million tons in June 2003.

EPA officials told us that they plan to include the revised totals in next year's reports.
                                            30

-------
                                                                           Appendix C
                                  Distribution
EPA Headquarters

Assistant Administrator for Air and Radiation (6101 A)
Assistant Administrator for Research and Development (8101R)
Comptroller (2731 A)
Agency Follow-up Official (2710A)
Agency Follow-up Coordinator (2724A)
Audit Follow-up Coordinator, Office of Air and Radiation (6102A)
Audit Follow-up Coordinator, Office of Research and Development (8102A)
Associate Administrator for Congressional and Intergovernmental Relations (1301A)
Director, Office of Regional Operations (1108 A)
Associate Administrator for Public Affairs (1101 A)
Director, Office of Air Quality Planning and Standards (C404-04)
Deputy Director, Office of Air Quality Planning and Standards (C404-04)
Director, Emissions Standards Division (C504-03)
Acting Director, Emissions, Monitoring and Analysis Division (C304-02)
Director, Office of Transportation and Air Quality (6401 A)
Deputy Director, Office of Transportation and Air Quality (6401 A)
Director, National Exposure Research Laboratory (MD-75)
Leader, Emission Inventory & Factors Group (C304-02)
National Exposure Research Laboratory Audit Liaison (MD-343-01)
Audit Liaison, Office of Air Quality Planning and  Standards (C404-2)
Audit Liaison, Office of Research and Development (8102R)
Inspector General (2410)
EPA Regions

Regional Air Program Directors
                                           31

-------

19nnD
120PA Pennsylvania Avenue NW
   Washington, DC 20460
      202-56&-0556
                                                               f
                                                                  ^
                                                               •«
                                                                4

-------