V
00
U,S EPA Headquarters Library
Mail code 3404T
1200 Pennsylvania Avenue NW
Washington, DC 20460
202-566-0556
OFFICE OF INSPECTOR GENERAL
Evaluation Report
»*
\
JS.
OOO
EPA's Method for Calculating
Air Toxics Emissions for Reporting
Results Needs Improvement
Report No. 2004-P-00012
March 31, 2004
-------
Report Contributors:
Patrick Milligan
Frank Martinsky
Lorraine Fleury
Michelle Brown
Abbreviations
EIIP
EPA
GPRA
MACT
NTI
OAR
OIG
OMB
ORD
OTAQ
Emission Inventory Improvement Program
Environmental Protection Agency
Government Performance and Results Act
Maximum Achievable Control Technology
National Toxics Inventory
Office of Air and Radiation
Office of Inspector General
Office of Management and Budget
Office of Research and Development
Office of Transportation and Air Quality
Photo (clockwise, from top left): (1) Cars and trucks emitting exhaust fumes
(weathersmith.com); (2) forest fire burning uncontrolled (US Forest Service web site);
(3) agricultural equipment emissions (weathersmith.com); and (4) pollution from an industrial
smokestack (New York State Department of Environmental Conservation).
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON. D.C. 20460
OFFICE OF
INSPECTOR GENERAL
March 31,2004
MEMORANDUM
SUBJECT: EPA's Method for Calculating Air Toxics Emissions for
Reporting Results Needs Improvement
Report No. 2004-P-00012
i
ij FROM: J. RickBeusse /s/
I Director for Program Evaluation, Air Quality Issues
TO: Jeffrey R. Holmstead
Assistant Administrator for Air and Radiation (6101 A)
Attached is our final report regarding the Environmental Protection Agency (EPA) efforts to
develop air toxics emissions data for use as a Government Performance and Results Act
performance measure. This report contains findings regarding EPA's methods for calculating air
toxics emissions. Also, the report contains corrective actions the Office of Inspector General
(OIG) recommends. This report represents the opinion of the OIG and the findings contained in
this report do not necessarily represent the final EPA position. Final determination on matters in
this report will be made by EPA managers in accordance with established procedures.
EPA's Office of Air and Radiation (OAR) provided us with a draft response on March 24,2004,
that included comments from the OAR's Office of Air Quality and Planning Standards and
OAR's Office of Transportation and Air Quality. Also, officials from the Office of Research and
Development (ORD) provided draft comments. Because these responses are draft and may not
represent EPA's final position on the issues and recommendations in this report, the draft
responses are not included as appendices in this report.
Action Required
In accordance with EPA Manual 2750, as the action official, you are required to provide this
office with a written response within 90 days of the final report date. Since this report deals
primarily with OAR's Air Toxics Program, the Assistant Administrator for Air and Radiation
was designated the primary action official. As such, he should take the lead in coordinating the
-------
Agency's response. The response should address all recommendations. For the corrective
actions planned but not completed by the response date, please describe the actions that are
ongoing and provide a timetable for completion. If you do not concur with a recommendation,
please provide alternative actions addressing the findings reported. We appreciate the efforts of
EPA officials and staff, as well as external stakeholders, in working with us to develop this
report. For your convenience, this report will be available at http: //www. epa. gov/oig.
If you or your staff have any questions regarding this report, please contact me at (919) 541-5747
or Patrick Milligan, Assignment Manager, at (215) 814-2326.
-------
Executive Summary
Purpose
Toxic air pollutants are harmful substances that are known or suspected to cause
j| cancer and other serious health problems, and can also have adverse environmental
I effects. The Clean Air Act identifies 188 toxic air pollutants and directs EPA to
i| regulate the sources emitting these pollutants.
li EPA has been tasked with reducing toxic air pollutants and the resulting health
ij effects. The Government Performance and Results Act (GPRA) of 1993 required
1 EPA, like other Federal agencies, to measure progress in achieving results. EPA
jj measures progress of its Air Toxics Program by calculating how many tons of
|! emissions are reduced annually. A vital component for assessing progress and
J directing future efforts of the program is the National Toxics Inventory (NTI), an
i' estimate made once every 3 years of the total amount of air toxics emitted annually
1 nationwide. Accordingly, our objectives were to answer me following:
What are the key assumptions and limitations underlying EPA's NTI and how
do these impact its use as a GPRA performance measure?
What actions could EPA take to improve its current air toxics GPRA
performance measure for assessing progress toward achieving the air toxics
objective?
Results in Brief
Although the methods by which air toxics emissions are estimated have improved
substantially in the past 6 years, invalidated assumptions and other limitations
underlying the NTI continue to impact its use as a GPRA performance measure. As
a result, EPA is not certain how much progress stationary, area, and mobile sources
have actually made in reducing air toxics emissions from the EPA-established 1993
baseline. For example, EPA recently finalized its third NTI, which also resulted in
revisions to the first two inventories:
For the 1993 baseline NTI, EPA increased the estimated emissions inventory
from 6.1 million tons to 7.2 million tons (an 18-percent increase).
For the 1996 NTI, EPA increased the estimated emissions inventory from
4.6 million tons to 5.0 million tons (a 9-percent increase).
' For the 1999 NTI, EPA estimated 5.1 million tons, an increase of 100,000 tons
when compared to the updated 1996 inventory.
EPA is not certain whether reductions or increases may have resulted due, at least in
part, to the Agency's change in the way it estimated the inventory, rather than real
reductions or increases in emissions. While emissions estimating techniques have
-------
improved, broad assumptions about the behavior of sources and serious data
limitations still exist. Furthermore, due to the changes mat resulted in
improvements in later inventories, the meaningfulness of comparing new
inventories against the 1993 baseline is questionable. For example, although State-
validated emissions data is EPA's preferred calculation methodology:
For the 1993 baseline NTI, only three States provided data to EPA.
For the 1996 NTI, 36 States provided data to EPA.
For the 1999 NTI, 39 States provided data to EPA.
Additionally, States were not required to verify their emissions data for any of the
above inventories, due largely to EPA's concerns about adding more reporting
requirements on States and the lack of a clear statutory mandate. Agency officials
said that the Clean Air Act only required limited scope studies on a select number of
toxic air pollutants found primarily in urban areas. As such, only 33 of the 188 air
toxics have received more in-depth study. Also, despite some improvements in
emissions estimating techniques for certain sources of air toxics, available usage
estimates and emission factors often are not reliable. Further improvements are
needed in the methods for calculating air toxics emissions if EPA is to accurately
gauge the extent to which its programs have actually reduced emissions. Otherwise,
EPA cannot be assured that the full reductions actually occurred or were, at least in
part, the result of changes in estimation methods.
EPA could also improve its current air toxics GPRA performance measure by
including toxicity information with tonnage measures. Currently, EPA aggregates
the tonnage reductions of emissions for all 188 toxic air pollutants but does not
account for the varying levels of toxicity among these pollutants. While useful, this
aggregate measure does not provide EPA, Congress, or the public with specific ,
information on those pollutants that are the most dangerous to human health.
During me course of our work, EPA officials told us that they see merit in adding a
risk component to their GPRA reporting, and they recently began exploring
possibilities for doing so. This supplemental measure would better enable EPA to
track its progress in reducing those air toxics that are most harmful to human health
and, in so doing, more closely meet the intent of GPRA.
Recommendations
We recommend that the Assistant Administrator for Air and Radiation improve the
accuracy, reliability, and usefulness of the Agency's current emissions-based air
toxics GPRA performance measure by providing increased funding and support for
development of the NTI, and supplement the existing measure of comparing air
toxics emissions reductions to the 1993 baseline inventory by also comparing
emission totals between all other completed inventories. We also recommend that
the Assistant Administrator supplement the existing tonnage measure for air toxics
with a measure of progress toward reduction in human exposure and health risk.
-------
Agency Comments and OIG Evaluation
Officials from the Office of Air and Radiation (OAR) and the Office of Research and
Development (ORD) provided detailed draft comments on the report. Because these
responses are draft and may not represent the Agency's final position on the issues
and recommendations in this report, the draft responses are not included as
appendices to this report. Nonetheless, we made changes to the draft report based on
both sets of draft comments, as well as our exit conference discussions with Agency
officials.
OAR's detailed draft comments agreed with the recommendations in this report, with
one exception. OAR intends to continue using the 1993 NTI as the baseline, instead
of implementing our recommendation to use the more reliable 1996 NTI as the
baseline for measuring progress under GPRA. Agency officials explained that if
EPA were to use the 1996 NTI as the baseline, it would not enable them to capture
the emissions reductions achieved by stationary and mobile source standards
implemented between 1993 and 1996. While we recognize this point, we continue to
believe that, due to the unreliability of the 1993 NTI, EPA should use the 1996 NTI
as the baseline. Additional discussion of this point is provided at the end of
Chapter 2.
With regard to ORD, officials of this office concurred with our draft report
recommendations, stated that the draft report was reasonable, and provided several
detailed comments intended to sharpen the quality and accuracy of the report. These
changes were made.
-------
iv
-------
Table of Contents
Executive Summary i
Chapters
1 Introduction [[[ 1
2 Improvements Needed to Air Toxics inventory
for Use in GPRA Reporting ......................................... 7
3 GPRA Measure Needs to Address Risk of Air Toxics Exposure .......... 23
Appendix
A Details on Scope and Methodology ................................. 27
B EPA's Recent Update to the 1 993 and 1996 Inventories ................. 29
-------
-------
Chapter 1
introduction
Purpose
Toxic air pollutants are harmful substances that are known or suspected to cause
cancer and other serious health problems, and can also have adverse environmental
effects. The Clean Air Act identifies 188 toxic air pollutants and directs EPA to
regulate the sources emitting these pollutants.
EPA has been tasked with reducing toxic air pollutants and the resulting health
effects caused by exposure to these pollutants. The 1993 Government Performance
and Results Act (GPRA) required EPA, like other Federal agencies, to measure
progress in achieving results. EPA measures progress of its Air Toxics Program by
calculating how many tons of air toxics emissions are reduced annually. A vital
component for assessing progress and directing future efforts of the program is the
National Toxics Inventory (NTI), an estimate made once every 3 years of the total
amount of air toxics emitted annually nationwide. Accordingly, our objectives were
to answer the following:
What are the key assumptions and limitations underlying EPA's NTI and how
do these impact its use as a GPRA performance measure?
What actions could EPA take to improve its current air toxics GPRA
performance measure for assessing progress toward achieving the air toxics
objective?
Background
What Are Adverse Health Effects of Air Toxics?
Toxic air pollutants are known or suspected to cause various serious health
problems. The adverse health effects include: increased risk of cancer; effects on
the neurological, cardiovascular, respiratory, and immune systems; liver and kidney
damage; and effects on fetal and child development.
Immediate effects of toxics exposure may be minor, such as watery eyes; greater
health problems, such as cancer, may not appear until years after exposure. In
addition to exposure through breathing air toxics, some toxic air pollutants, such as
mercury, can deposit onto soils or surface waters, where they are ingested by plants
and animals and eventually transmitted through the food chain. Exposure to some
air toxics before birth or during childhood may interfere with normal development,
while other air toxics may affect the ability to conceive or give birth to a healthy
child. Studies have found that exposure to benzene from less than 5 years to more
-------
than 30 years has resulted in individuals developing and ultimately dying from
leukemia. Studies of smelter workers identified an association between
occupational arsenic exposure and lung cancer mortality.
Where Do Air Toxics Come From?
Air toxics come from a complex variety of sources, which poses a challenge to EPA
when it compiles the air toxics emissions inventory. For example, toxic air
pollutants include benzene, which is found in gasoline; perchlorethlyene, which is
emitted from some dry cleaning facilities; and methylene chloride, which is used as
a solvent and paint stripper by a number of industries. Toxics coming from other
sources include asbestos; dioxin; toluene; and metals such as cadmium, mercury,
chromium, and lead compounds. Table 1.1 describes some of the types of sources
emitting air toxics and the amount of emissions estimated to be generated by each
category of sources, according to EPA's 19931 baseline inventory.
Table 1.1: Categories of Sources of Air Toxics
;.(,.., ,
-:.'-: - -: "
fliH;
.-S.:-S :.?
;i«£
::'-o."'o:J
1
:' -:S " -:"'-
!i
&.&?'&
... .- ...
y.-,:: _x :
..StwnaK
Categories
Major
Area*
Mobile
A. On-road
B. Non-road
Subtotal
BefiniBan
Emissions of 10 tons per year or more
of any one hazardous air pollutant, or
25 tons per year or more of any
combination of hazardous pollutants
Emissions of less than 10 tons per year
of any one hazardous air pollutant, or
less than 25 tons per year of any
combination of hazardous pollutants
Emissions from motorized vehicles
normally operated on public roadways
Emissions from a diverse collection of
engines, equipment, vehicles, and
vessels
Sxamptesoi types j
, OfSWifce*
Utilities, Refineries,
Car Manufacturers,
Chemical
Manufacturers
Dry Cleaners,
Gas Stations,
Wood Burning Stoves
Cars, Buses, Sport
Utility Vehicles, Light-
& Heavy-Duty Trucks
Construction and
Agricultural
Equipment,
Personal Watercraft
Total Emissions
19$38as«fM« ,:
flnmiflionterw)
2.7
1.1
1.7
.6
2.3
6.1**
* Area sources include a sub-category entitled "Other" for those sources that are not otherwise regulated, such as wildfires
and open burning.
** This total represents the 2002 version of the 1993 baseline NTI.
The 1993 toxics emissions inventory is the baseline measure EPA selected to gauge
progress for the Air Toxics Program.
JA large part of the 1993 baseline inventory is based on data obtained from 1990 and 1993. For simplicity,
and because EPA has traditionally referred to it as such, we refer to this data as the 1993 baseline inventory.
-------
What Are the Air Toxics GPRA Goals and Measures?
GPRA, passed in 1993, holds Federal agencies accountable for achieving results by
requiring them to measure program outcomes, including for EPA's Air Toxics
Program. By 1997, GPRA required agencies to establish outcome goals to measure
their progress and report on their accomplishments. The three main components of
GPRA require EPA to: (1) develop a 5-year strategic plan defining its long-term
goals and how the Agency intends to achieve them; (2) prepare annual performance
plans with goals and measures that relate to either outputs or outcomes that
quantitatively measure the Agency's results and demonstrate their linkage to
longer-term goals; and (3) prepare annual performance reports on accomplishments.
As shown in Table 1.2, EPA's Strategic Plan objective for the Air Toxics Program
(updated every 3 years) relies on emission reductions.
Table 1.2: Strategic Plan Objectives for Air Toxics
₯«*f
1997
2000
2003
; Objective
By 2010, reduce air toxics emissions by 75 percent from 1993 levels to
significantly reduce the risk to Americans for cancer and other serious
adverse health effects caused by airborne toxics.
By 2020, EPA will eliminate unacceptable risks of cancer and other
significant health problems from air toxics emissions for at least
95 percent of the population, with particular attention to children and
other sensitive subpopulations, and substantially reduce or eliminate
adverse effects on our natural environment.
By 201 0, working with partners, reduce air toxics emissions and
implement area-specific approaches to reduce the risk to public health
and the environment from air toxic pollutants.
swdget ;
(a)
$95 million
(17% of Air
Budget)
$11 8 million
(19% of Air
Budget)
(a): EPA's air toxics budget request for 1997 was not comparable to 2000 and 2003; the total air program budget request
for 1997 was $304 million.
What Is the Air Toxics Emissions Inventory?
EPA compiles an NTT every 3 years that consists of an estimate of the total
emissions of the 188 air toxics identified in the Clean Air Act. Table 1.3 depicts the
years for which four inventories were or are in Ihe process of being developed and
when each was completed.
Table 1.3: Air Toxics Emissions Inventories Undertaken
iWS^^^WY*w^lt?is|l|?|l;
1993
1996
1999
2002
SihillaiYeir Inventory Completed^ |
1998
2001
2003
In progress
2EPA referred to the 1993 and 1996 inventories as the NTL For the 1999 inventory, EPA combined the air
toxics inventory with the criteria pollutant emissions inventory and changed the name to the National Emissions
Inventory, or NEI. For purposes of this report, we will refer to all 3 inventories (1993,1996, and 1999) as the NTL
-------
Chart 1.1 provides a scheduled timeline of key steps in developing the 1999 air
toxics inventory, which took place after 1999 ended.
Chart 1.1: Scheduled Timeline for Developing 1999 toxics Inventory
I
States send Surveys to Point Sources
Federal Agencies develop activity data - VMT
States quality control and record data
EPA receives first reported data
EPA prepares draft Inventory
Reviewer's comments
EPA receives new data, along with comments
EPAprepares second draft Inventory
2nd Round of comments and Inventory revisions
(a) Development of the NTI begins after the designated year has ended.
In addition to the activities listed in Chart 1.1, State and other Federal agencies must
obtain data from other sources (such as the Federal Highway Administration, which
provides EPA with information on national vehicle miles traveled for the inventory
year). These data are not available until a year or so after the inventory year has
ended. After the new inventory is developed, EPA compares it to the 1993 baseline
inventory to determine the amount of emissions reductions achieved. Although the
1993 and 1996 inventories took over 4 years to complete, EPA hopes to reduce this
time in the future.
What Are The Uses of the Inventory?
Although EPA is in the early stages of developing an ambient (outdoor) air toxics
monitoring network3 capable of measuring air toxics in ambient air on a national
basis, the Agency does not yet have a formal network and therefore relies primarily
on emissions data for measuring progress. The NTI is the primary source of
national air toxics emissions data used by major stakeholders - such as scientists,
researchers, lawmakers, and the public - to gauge progress. Uses of the NTI
include rulemaking efforts, modeling, and risk assessments. More details on the
NTI are in Chapter 2.
EPA's National Air Toxics Trends Sites (NATTS) network consists of 22 sites. Agency officials said that
they plan to implement the network in fiscal year 2004.
-------
Scope and Methodology
To assess the adequacy of EPA's GPRA measure for air toxics, we reviewed
documentation related to: GPRA objectives and measures; the development of the
1993,1996, and 1999 inventories; Quality Assurance/Quality Control efforts for the
development of the three inventories; and regulations, guidance, policy, and
assessments applicable to the development of the inventories. We also conducted
numerous interviews with officials from: the Office of Air Quality Planning and
Standards, the Office of Transportation and Air Quality, EPA Region 3, and the
Office of Research and Development (ORD). Budget information was obtained
from the Office of the Chief Financial Officer. We also reviewed Section 112 of the
Clean Air Act to identify the requirements of EPA to identify, control, and reduce
air toxics emissions.
Our field work was conducted from February 2003 to November 2003. We
conducted this evaluation in accordance with Government Auditing Standards,
issued by the Comptroller General of the United States. Additional details on our
scope and methodology are in Appendix A. On March 30,2004, we met with
Office of Air and Radiation (OAR) officials to discuss their draft comments and we
made additional modifications to the report based on our discussions.
-------
-------
Chapter 2
Improvements Needed to Air Toxics Inventory
for Use in GPRA Reporting
Although the methods used to estimate air toxics emissions have improved
substantially in the past 6 years, EPA is not certain of how much progress
stationary, area, and mobile sources have actually made in reducing those
emissions. Emissions estimating techniques have undergone continuous change and
improvement since the 1993 baseline was developed, and because subsequent
inventories were amassed quite differently, comparison to the 1993 baseline has
become less meaningful over time. For example, only 3 States provided data for the
1993 baseline, while 36 and 39 States provided data for the 1996 and 1999
inventories, respectively. Of 62 authorized local programs, only one reported air
toxics emissions data for 1993; three for 1996; and nine for the 1999 inventory.
Even with increased State and local agency participation, these agencies have not
been required to report air toxics emissions data, nor have they been required to
verify it. Further, despite improvements in estimation techniques, usage estimates
and emission factors often are not reliable. Due to the limitations noted, EPA
cannot be assured that the full amount of reductions reported actually occurred or
whether they were, at least in part, the result of improved estimation methods.
Improvements in the methods for calculating air toxics emissions are needed if the
Agency is to accurately gauge the extent to which its programs have reduced the
public's health risk to air toxics as called for under GPRA.
Difficulties in Comparing EPA's Air Toxics Emissions Inventories
To develop major and area source emission estimates, EPA primarily uses four
types of data (listed in order of preference based on data reliability):
State, Local, and Tribal-Submitted Data: State, local, and tribal agencies, on
a voluntary basis, submit air toxics emissions data.
Maximum Achievable Control Technology (MACT)/ResiduaI Risk: Each
time EPA develops a control for a source category (a source of air toxics), it
gathers emissions data.
Toxics Release Inventory: This is an EPA-managed database used to inform
the public and government officials about the total release and transfer of toxic
chemicals, and is based on information reported by a select group of industries.
National Estimates: When State information is not available, EPA estimates
emissions on a national level and allocates down to a local (county) level.
-------
Although States have not been required to validate or even report air toxics
emissions data, EPA considers State-submitted data to be the most reliable. In its
absence, the Agency uses the less reliable MACT, Toxics Release Inventory, and
national estimates. EPA also uses data gathered from mobile source rulemaking
efforts, industry surveys, and other Federal agency information, such as vehicle
mileage data from the Federal Highway Administration, each with its own set of
limitations. Table 2.1 provides a brief qualitative assessment of the sources of
information used for each triennial inventory, highlighting the significant changes
made to more recent inventories.
Table 2.1: Key Differences in the Three Inventories
Sources of
Pollution
Major
Area
Mobile
(On-Road)
Mobile
(Non-Road)
', .1603
8*4«8n«
3 States and 1 local agency
provided data; few quality
checks performed. EPA
used less reliable data such
as MACT, Toxics Release
Inventory, and national
estimates. (Mainly non-
facility specific data
aggregated up to the county
level.)
Emission estimates largely
based on national activity
data and national emission
factors. Emissions allocated
down to the county level.
Five toxics estimated using
early on-road model, while
other toxics calculated using
less sophisticated methods.
Fuel types not accounted for.
Emissions estimated using
non-road study of emissions.
This study was first attempt
at identifying non-road
equipment types. Much of
the activity data and
emission factors unverified.
Fuel types partially
accounted for.
1W$ ' '-:' -
36 States and 3 local
agencies reported data -
46 States reviewed the data.
Quality check efforts made to
eliminate duplicate or ,
missing facilities. MACT and
Toxics Release Inventory
used less. Estimates based
on point emissions.
Began to use more State and
MACT data for non-point
sources. Emission estimates
still largely based on national
activity data and national
emission factors.
On-road model (MOBILE 5b)
used for all toxics estimates.
Fuel types accounted for.
New model {NONROAD)
developed to more
accurately depict types of
equipment. Much of the
activity data and emission
factors still unverified. Fuel
types partially accounted for.
193$ '/ - \
39 States and 9 local
agencies reported data -
46 States reviewed the
data. For the first time,
3 tribal agencies reported
data. Some additional
quality checks performed
on data. MACT and
Toxics Release Inventory
used even less.
Some area source
locations identified and
tracked. Improved use of
non-point source data.
New on-road model
(MOBILE 6) accounts for
many variables such as
vehicle type, vehicle
speed, road type, and fuel
types.
Equipment-type
information again
improved. Model
(updated NONROAD) also
attempted to account for
changes in engine power
levels related to pollution.
Much of the activity data
and emission factors still
unverified. Fuel types
partially accounted for.
Limitations of the Air Toxics Inventories
For the 1993 baseline inventory, EPA received very little State participation. Over
the past 10 years, States have progressively submitted more toxics emissions data,
of increasingly improved quality. EPA officials said that, due primarily to States'
increased awareness and need for toxics emissions data for purposes other than the
inventory, the States wanted to rely more on their own estimates than the estimates
-------
generated by EPA. Some of the assumptions and limitations associated with the
1993,1996, and 1999 inventories are discussed below.
1993 Baseline Inventory
EPA developed the 1993 baseline (the 1993 NTI) as a repository of information
collected by EPA to meet the requirements of Clean Air Act Sections 112(c)(6) and
112(k), which required limited scope studies on a select number of toxic air
pollutants found in urban areas. In total, EPA studied 33 of the 188 air toxics as a
starting point for developing the NTI, but recognized that there were information
gaps.
Because EPA's authority to require States to report air toxics emissions was limited,
only three States and one local agency provided useable input to the 1993 emission
estimates. For the remaining 47 States and 61 local agencies that did not submit air
toxics emissions information, EPA's only viable option was to use national data and
apportion it down to the county level (the "top-down" approach). Sources of this
national data included the EPA-developed national estimates and information
gathered from the development of the MACT standards. Because studies have
found that air toxics often are a localized problem, the top-down approach is
considered one of the least desirable methods because it does not accurately depict
local variations.
Subsequent to developing the 1993 baseline, EPA sometimes revised the baseline
when it received more accurate 1993 information. However, Agency officials stated
that due to limited resources and because the baseline inventory is only an estimate,
their efforts are typically geared toward obtaining better data for current and future
air toxics inventories rather than updating the 1993 baseline.
1996 Inventory
For the 1996 inventory, some of gaps in the 1993 baseline inventory were filled to
provide a more complete picture of air toxics emissions. As compared to the 1993
baseline inventory, EPA used fewer top-down approaches because 36 States and
3 local agencies submitted air toxics emissions estimates, and MACT and Toxics
Release Inventory data was used for the remaining 14 States and 59 local agencies
that did not submit air toxics emissions information. However, there was still not
full State and local participation for reporting, and the State and local figures
provided were not verified by EPA. Further, the 1996 emission estimates for the
Area and Mobile Non-Road sources of pollution were still largely based on national
activity data and emission factors.
-------
1999 Inventory
Fertile 1999 inventory, 39 States, 9 local agencies, and 3 tribal agencies submitted
emissions estimates, and there were improved Quality Assurance/Quality Control
efforts by EPA and the States, who performed the following four activities:
States used a Quality Control formatting tool for file format and data field
verification prior to sending the estimates to EPA.
EPA used a software tool to identify duplicate facilities.
EPA sent States draft summaries of emission data for States to review for
accuracy and completeness.
EPA performed Quality Assurance/Quality Control checks to identify gross
errors with the emission estimates.
However, there are still limitations with the 1999 NTI. EPA once again had to use
MACT, Toxics Release Inventory, or apportion emissions in a top down manner for
the 11 States and 53 local agencies that did not submit air toxics emissions
information. Also, Agency officials said that the Quality Assurance/Quality Control
check was designed to identify emission estimates substantially outside the
historical range of reported emissions. These checks are not used to identify
inconsistencies reported from independent sources of data or against national
estimates. Additionally, these checks do not include sampling State data to verify
the adequacy of emission estimation methodologies. These additional checks would
substantiate State-reported data.
Progress In Reducing Emissions Uncertain
As shown in Chart 2.1, as of September 2003, the 1993 baseline inventory indicated
that a total of 6.1 million tons of toxics emissions was released in 1993, while the
1996 inventory indicated that a total of 4.6 million tons of toxics emissions was
released in 1996, or total reductions of 1.5 million tons. Recently EPA finalized the
1999 inventory, which showed 5.1 million tons of emissions, or an increase of
500,000 tons when compared to the 1996 inventory. However, this reduction (1993
to 1996) and this increase (1996 to 1999) may have resulted due, at least in part, to
EPA's change in the way it estimated the inventory, rather than a real reduction in
emissions. EPA was unable to provide us with reliable data that would accurately
show how much air toxics emissions were actually reduced from 1993 to 1996, and
how much of the reported change may have been due to a change in methodology.
10
-------
Chart 2.1:
Emissions for the 1993,1996 and 1999 NTIs
(in millions of tons)
Non-road
On-road
1993
Total: 6.1
On-road
1996
Total: 4.6
On-road
1999
Tola!: 5.1
Major
Area
Mobile
EPA credited the 1.5-million-ton reduction to its programs designed to control air
toxics emissions. Almost all of these reductions occurred-in the major source
category, where emissions decreased from 2.7 million tons in 1993 to 1.2 million
tons in 1996. During this 3-year period, EPA issued 11 MACT standards
representing 664,000 tons of emission reductions. However, there still remains
836,000 tons of emissions reductions achieved by controlling sources not subject to
MACT standards. During this same time period, EPA received improved data and
substantially changed its method for calculating emission estimates. Specifically,
the 1996 NTI had much more State-submitted data on major source emissions, and
the accuracy of the mobile emission estimates improved.
Regarding the recently completed 1999 NTI, again there is a significant change in
the calculation methods used, which may impact the amount of emissions reported.
For example, for the 1999 NTI, improved mobile models were used for both the
on-road and non-road estimates, there was slight increase in State and local agency
participation, and there were improved Quality Assurance/Quality Control efforts by
EPA and the States. These changes in calculation methods may have caused, at
least in part, the 500,000 ton increase in emissions from 1996 to 1999. Recent
revisions to the 1993 and 1996 NTIs demonstrate a significant improvement in
characterizing EPA's progress in reducing air toxics emissions. However, this
increase again demonstrates that EPA is not certain whether changes in the
inventory may be due, at least in part, to the Agency's change in the way it
estimated the inventories.
11
-------
The above figures for the 1993 and 1996 inventories were used recently in key EPA
reports to the Congress and the public on the Agency's progress in reducing air
toxics emissions, including:
EPA's November 2003 "FY 2003 Annual Report" to the Congress and the
President, which serves as the Agency's program performance report as required
by GPRA;
EPA's August 2003 "Latest Findings on National Air Quality: 2002 Status and
Trends Report," which provides the public with the latest information on air
quality nationwide; and
EPA's June 2003 "Draft Report on the Environment 2003," which provides the
nation with EPA's first ever national picture of the environment and represents
EPA's first step at developing a set of comprehensive environmental indicators.
However, in February 2004, after the conclusion of our field work, EPA provided us
with revised totals for the 1993 and 1996 inventories (prior to this time, no figures
had been provided for the 1999 inventory). For example, the Agency's revised
2004 estimate of the 1993 baseline is now 7.2 million tons. Appendix B provides
further details on EPA's recent update of the 1993 and 1996 inventories. In our
opinion, these revised totals further substantiate our position that EPA is uncertain
of its progress in reducing emissions and needs to improve its measures of air toxics
progress.
Status of EPA Efforts to Improve Recent Inventories
While improvements to estimation methods were made to the more current
inventories, EPA acknowledges that there are still deficiencies in the estimation
methods. These deficiencies vary from source to source and affect the reliability
and accuracy .of the data. Because development and maintenance of air toxic
emission inventories is not explicitly required by the Clean Air Act, most of the data
used for the emissions inventories were obtained and developed for other reasons
and then secondarily used to compile the NTI. Below is a description of how the
inventory is amassed and examples of the assumptions and limitations associated
with the data for the primary source categories of pollution (major, area, and
mobile).
Major and Area Sources
Because major and area source categories include stationary sources and often use
the same data sources, they are presented together. Major sources comprised
26 percent of the 1996 inventory, and area sources comprised 24 percent. Some
area sources, such as wood burning stoves, cannot be cost-effectively inventoried as
individual sources because they are comprised of a large number of sources spread
across the country, so emissions are calculated at a national level and then allocated
12
-------
down to the county level. Also, some source categories have both major and area
emitters. For example, a small neighborhood dry cleaner may not emit enough toxic
emissions annually to reach the threshold of a major source, but a large regional dry
cleaner may reach this threshold and thus be classified a major source.
Following is a description of the four primary sources of information used to
estimate both major and area source emissions, from most reliable to least reliable.
State-Reported Emissions: Because there are no reporting requirements for
States, the data submitted by States are inconsistent and States are not required
to verify the accuracy of the data. To improve State data, EPA in 2000 included
in a proposed regulation specific reporting requirements that would standardize
the type of toxics data gathered by States, as well as the methods used to
calculate estimates. However, the proposed air toxics reporting requirements
were deleted from the final regulation. This was due largely to EPA's concerns
about adding more reporting requirements on States, the Agency's belief that the
Clean Air Act is not clear on air toxics reporting, and its belief that there would
be lawsuits if specific reporting requirements were imposed. Seventeen of the
22 State and local agencies that commented on the proposed regulation said that
States should be required to report air toxics emissions data. However, some
said they would not support increased reporting requirements without increased
funding. In 2003, EPA obligated approximately $181 million in Clean Air Act
Section 105 grant funds to States for carrying out a variety of Clean Air Act
goals, but the Agency did not use these funds to negotiate commitments with
States for consistent, validated air ,
toxics emissions data TRie emissions tn«enteiy
EPA has also attempted to improve
air toxics emissions reporting by
working with the States through the
Emission Inventory Improvement
Program (EIIP), which was intended
to promote the development and use
of standard procedures for collecting
and reporting emissions data.
Initially, this program worked to
develop standardized methods for
collecting data for both criteria4 and
toxic pollutants. However, EIIP
workgroup officials decided that
developing better air toxics emissions
estimation methodologies was beyond
the intended scope of the workgroup
sM Oflfee
Tip goal ft achieved fay developing « $&t
arprefarted arid vX&tr&frta method*" *?r
all invent9ry«s*l»«3teq| t*sk$. This
Bferp&tii&ttiBn jrfipWSwe* #» q&«t$te*jiqy
Of caHUsctetf tfirffcdraf result* in ftjareasBd
4Six common air pollutants found nationwide that harm human health and the environment, called criteria
pollutants because EPA sets standards for these pollutants by first developing health-based criteria.
13
-------
effort, and focused only on criteria pollutants. According to EPA officials,
beginning in 2004, the EIIP was no longer active because of limited funding for
the program. However, EPA officials told us that there is an ongoing
reorganization in the Office of Air Quality Planning and Standards that will
likely include a group dedicated to improving emission factors and emission
estimation methods for air toxics.
MACT: EPA initially gathered data to learn as much as possible about the
industry before a MACT emission limitation was established. In doing so, EPA
relied on many different sources of data, including: national activity data applied
to emission factors, facility level emissions reported to EPA, and facility level
emissions obtained from the Toxics Release Inventory database. Of the
70 MACT area source categories that reported emissions data:
19 used facility-level emissions reported to EPA.
36 used national activity estimates.
15 relied on Toxics Release Inventory data.
Only 35 of the 70 MACT area source categories calculated emission estimates
for 1996. For example, of the 19 facility level estimates reported to EPA, only
9 estimated 1996 emissions, while the remaining 10 estimated emissions for 1 or
more calendar years ranging from 1991 to 1997. For 1 of the 36 sources using
national activity data - halogenated solvent cleaners - EPA developed an
estimate of the total mass for various toxic air pollutants, but did not know how
much of each air toxic comprised the total. As a result, EPA divided the mass
equally among a list of applicable air toxics.
Toxics Release Inventory: The Toxics Release Inventory database is mandated
by the Emergency Planning and Community Response Act, which requires a
select group of industries meeting specific requirements to report toxics release
information to EPA, including releases to air, land, and water. In those instances
where EPA used the database to develop MACT standards, the Agency used this
same data for the toxics inventory. Because it was the only available
information, EPA used the database despite the significant limitations associated
with the data. Not all facilities that emit air toxics meet the requirements to
report Toxics Release Inventory emissions. As a result, some sources of air
toxics would not be identified and the emissions would not be recorded or
reported to EPA. The data is also compromised because facilities are allowed to
report reasonable but unverified estimates of toxic amounts instead of monitored
releases. Finally, calculation methods can vary annually and, as a result, some
facilities may report emission reductions not because of actual reductions but
because a different calculation method was used, which may result in EPA
overstating or understating the actual amount of reductions.
-------
y, Hair cafe prOductst;:
H, *m) rei ear* ftpguiife
&* ihdtidOrf In this Area SOUttH eatery, £PA
Ori ppwtefci uspSB: and population tfatp ferfte
ytef 1$a& T* patoufate «w tafei em&sians
IUJS. population
10 ^et^rraineaip total arrissiOnsfrOrmpesrsOnat! .
ttes county tevet feasati on U.S, CeniijH 8ur?a«
National Estimates: When EPA
could not obtain emissions data
from the above three activities, the
Agency reverted to estimating the
emissions from these source
categories on a national scale and
then allocating them down to a
local level. When national-level
emissions are used, it does not
account for significant local
variations, which can have a large
effect on the health of the local
population exposed to those toxic
emissions. For example, EPA's
estimates assume each person in
the United States, regardless of
age, sex, and other differences,
uses the.same amount of personal
care products (see box).
When there is no data available for the year targeted based on the above four
primary sources of information, the Agency as a last resort will occasionally use
data derived from either a prior or subsequent year's inventory. This affects the
inventory's ability to accurately depict emissions reduction trends because, at
times, the same data are being used for multiple years.
Mobile Sources
While major stationary sources of emissions include many types of pollution-
creating equipment and processes, the number of mobile source emissions points far
exceed stationary sources in that they are produced by millions of engines of varying
types and sizes that move around the country. Due to the quantity, diversity, and
mobility of these on-road and non-road sources, sophisticated methods using
complex calculations that include a host of assumptions were developed to estimate
emissions.
Since the inception of the EPA mobile source program, EPA has historically focused
its efforts on understanding and controlling on-road emissions because they
represented the majority of the pollution from mobile sources. These efforts have
led to increased knowledge of on-road emission factors and better activity data
EPA has made substantial progress in reducing on-road emissions through the
production of "cleaner" vehicles and the implementation of on-road vehicle control
programs. This, combined with the ever-increasing popularity of non-road sources,
has resulted in EPA becoming more concerned about non-road emissions.
Furthermore, non-road sources was the only category to show an increase in
emissions from 1993 to 1996. However, the nature of non-road sources presents
15
-------
EPA with considerable challenges, due to the large number of equipment types and
manufacturers.
On-Road Sources: On-road mobile emissions comprised 28 percent of
emissions for the 1993 baseline inventory. The predominant source of emissions
in this category was gasoline-fueled automobiles. There are two general
approaches to calculating on-road emission estimates for the 21 toxic air
pollutants emitted by motor vehicles:
For 15 of the on-road air toxics emissions, estimates are derived from either
particulate matter or volatile organic compound emissions data For example,
the Agency takes the particulate matter data for on-road emissions and then
estimates how much air toxics are in those emissions. In doing so, EPA
assumes that air toxics emissions from vehicles are directly proportional to
vehicle particulate matter emissions.
For six of the on-road air toxics emissions, total vehicle miles traveled
(obtained from the Federal Highway Administration's database) for each class
of vehicle are multiplied by an emission factor to develop the air toxics
estimate.
EPA currently develops emission estimates that account for specific vehicle
activities, such as vehicle type and speed, and emission-related factors, such as
ambient temperature. For example, while the 1996 mobile estimate included .
7 vehicle types and 1 vehicle speed, the 1999 estimate included 28 vehicle types
and 12 vehicle speeds. However, with these more complex calculations come
increased assumptions and limitations. In 2000, the National Research Council
performed a review of EPA's on-road model, entitled Modeling Mobile Source
Emissions, and identified the following assumptions and limitations:
EPA's selection of vehicles for testing does not sufficiently represent high-
emitting vehicles. Emissions from high-emitting vehicles are a relatively
small fraction of the on-road fleet, but contribute a large fraction of total
vehicle emissions, and thus require a large sampling fraction. If high-emitters
are not properly characterized, emission factors can be seriously biased.
EPA's calculation method does not consider road-grade effects on emissions.
Road-grade influences vehicle emissions in that higher emissions occur when
vehicles negotiate steep road grades.
Light-duty truck emissions are sometimes estimated from passenger
automobile test data, even though some trucks are regularly used as working
vehicles, which results in higher emissions due to the increased load.
EPA lacks sufficient data to estimate effects of high emitters and sometimes
fills in missing data by assuming high emitters behave the same as normal
emitters.
16
-------
The National Research Council review stated that EPA needs to better
understand and quantity the uncertainty related to sources of data used in the
model, such as the use of small and nonrepresentative emissions data The
Council believes that the quantification of these uncertainties is critical for
understanding the weaknesses of the model and identifying the most critical
needs for improving emissions test data. In response to the Council's findings,
EPA has initiated a $4 million test program to collect data on high-emitting
vehicles, and the Agency is developing a new model to better quantify the
uncertainty in on-road emissions estimates. In our view, these are steps in the
right direction. Nonetheless, at the present time, the Agency has limited
assurance that data reported in its GPRA performance measure is accurately
assessing progress.
Non-Road Sources: Non-road mobile emissions comprised about 10 percent of
emissions for the 1993 baseline inventory and increased to more than 19 percent,
or nearly one-fifth of the total, for the 1996 inventory. Until the mid-1990s,
emissions from these engines were largely uncontrolled. This category covers a
diverse collection of engines for equipment, vehicles, and vessels, ranging from
leaf blowers to earth-moving equipment. As compared to on-road methods,
emissions estimation methods for non-road sources are much less developed
because they lack adequate basic data. In developing non-road estimates, EPA
typically develops a usage estimate such as the number of hours a piece of
equipment operates during the year. EPA then multiplies that number by an
emission factor, which essentially calculates how much pollution is emitted for
each hour the equipment is operating. Both the non-road usage estimates and
emission factors have significant limitations.
Usage Estimates - Currently, EPA cannot verify the accuracy of the data. Usage
estimates come from a marketing research firm tfiat gathers usage information
from equipment manufacturers through telephone surveys. The marketing
research firm takes this data and uses a proprietary process to project non-road
population totals. Because EPA cannot assess the accuracy of the process, EPA
officials expressed interest in developing their own strategy for collecting usage
data. For example, a more valid estimate would be generated if EPA could place
a monitor on equipment to measure the hours of operation to better verity the
accuracy of their data. Despite the limitation described above, EPA uses the
survey data because it is the only available source of non-road equipment usage
information.
Emission Factors - EPA officials have stated that there are few studies that have
developed toxics emission factors for non-road equipment. An emission factor
is developed by monitoring emissions from several test sites and computing an
average emission rate. However, for many of the existing studies, engine
emissions have not been tested while equipment is operated under field
conditions (see example in following box). In lieu of developing its own
emission factors, EPA has largely relied on certified emissions information
17
-------
Factors
provided by engine manufacturers. Also, the non-road calculation does not
. account for engine deterioration, which results in higher emissions in later years.
Although EPA calculates deterioration for on-road vehicles by using State
vehicle registration information, similar information does not exist for most non-
road equipment. However, EPA has , / tr * ', 's
efforts underway to develop
improved air toxics estimates for
some categories of non-road
equipment. For example, EPA just
completed a test program that
characterized air toxics emissions for
14 non-road diesel engines under a
variety of operating conditions with
different fuels, and is also testing
, rwn*
So For-- - i
exampte^, wNn » tewfetote soopps a feuetet
of soil, rag»'twjS9poww*an aswef fc
2- and 4-stroke gasoline lawn and
garden equipment at its mobile
source laboratory in ORD's
National Exposure Research
Laboratory. According to Agency '" " " * 5 -
officials, the efforts should result in substantial improvement in estimating air
toxics emissions for some categories of non-road equipment.
Additional Uses of the Air Toxics Emissions Inventory Data
In addition to needing quality information for reporting results in GPRA, there are
several other important needs for a valid, reliable, and accurate inventory of air
toxics emissions, such as:
Rulemaking Efforts - The 1993 baseline and 1996 inventory were relied upon
extensively in EPA's development of the 2001 Final Mobile Rule.5 In developing
this rule, EPA used data from these inventories to determine the controls that would
be needed to reduce mobile emissions to the desired level.
National Air Toxics Assessment - In carrying out its National Air Toxics
Assessment, EPA used the 1996 NTI to develop a subset of 33 air toxics posing the
greatest risk to urban areas to better understand their priority as related to risk, as
well as their effects on the nation's population. The emissions data were input into
models that projected ambient air concentrations and ultimately estimated the level
of human exposure to the 33 toxics. EPA then followed risk characterization
guidelines to estimate both the cancer and non-cancer effects. The Science Advisory
Board identified weaknesses in the methods used to estimate emissions (1996 NTI)
and determined that improved accuracy and reliability of the NTI was vital to the
National-Scale Air Toxics Assessment effort.
540 CFR Parts 80 and 86, Control of Emissions of Hazardous Air Pollutants From Mobile Sources; Final
Rule, March 29,2001
18
-------
Residual Risk Assessments - After industry has installed required MACT controls
to reduce air toxics emissions, the Clean Air Act requires EPA to assess the
remaining risks to public health. The NTI is critical to measuring the effectiveness
of these controls. If, after 8 years from MACT promulgation (or 9 years after
promulgation of the 2-year MACT source categories), the MACT controls have not
reduced emissions to an acceptable risk level to protect public health with "an ample
margin of safety," EPA must promulgate health-based standards for that source
category to address this risk.
Modeling - In addition to air toxics being modeled for the National-Scale Air
Toxics Assessment effort, EPA also uses the NTI as the basis for numerous
modeling efforts at the local and regional levels. For example, EPA will model a
series of toxics for a community or smaller geographical area EPA, State, and local
agencies use such model-derived data to estimate exposures and focus limited
resources. ORD officials also cited an additional need for air toxics data in
attempting to understand how pollutants behave once released into the environment.
Conclusions
EPA is likely to rely on emissions data for gauging its progress in reducing health
risks from airborne toxics for many years to come. However, the Agency faces
considerable challenges in improving this measure. The most reliable approach
would be to require State and local agencies to report validated emissions once
every 3 years. EPA proposed this approach in 2000 but did not follow through on
this plan due to concerns about adding more reporting requirements on States, as
well as the belief that the 1990 Clean Air Act is not clear on air toxics reporting.
However, in our opinion, GPRA is clear in its mandate that Federal agencies
demonstrate they are achieving results, and GPRA also provides them with the
authority to establish appropriate measures for gauging their progress. Nonetheless,
if EPA does not wish to include State reporting requirements in a rulemaking, the
Agency can achieve the same results by judicious use of its Section 105 grants to
State and local agencies.
Regarding rne other limitations discussed in this chapter, EPA currently does not
have a systematic approach for improving air toxics emissions estimation techniques
similar to the approach that the National Research Council recommends EPA use to
improve its mobile model. Such an approach would allow EPA to identify the
various uses of the inventory, and the level of accuracy needed for those uses. This
will enable the Agency to gauge the effectiveness of the existing inventory. Further,
EPA will be able to use this assessment to make future improvements to the
inventory by identifying and improving the emissions estimates that have the largest
impact on the accuracy of the inventory. We believe the Emissions Inventory
Improvement Program has the organization, expertise, and funding to guide and
oversee such improvements.
19
-------
While improvements to the inventory are important to strengthening EPA's air
toxics GPRA performance measure, these improvements will also benefit EPA and
the States in developing and implementing effective toxics emissions control
strategies to protect the public from the dangers of air toxics. In addition to
comparing reductions to the baseline inventory, the Agency should also compare
emission totals between all other completed inventories, which would better enable
EPA to verify the actual emissions reductions achieved. This revised approach
would become more meaningful as future inventories are developed.
Recommendations
In order to obtain reliable air toxics emissions data once every 3 years, we
recommend that the Assistant Administrator for Air and Radiation:
2-1. Improve the accuracy and reliability of the air toxics GPRA performance
measure by providing increased funding and support for development of the
NTI by:
(a) requiring all authorized State and local agencies to report annually air
toxics emissions data for stationary, area, and mobile sources;
(b) developing air toxics emissions reporting policies, procedures, and
guidance for facilitating standardized State and local agency reporting of
air toxics emissions data. After the reporting requirements have been
imposed, use Clean Air Act Section 105 grant funds to ensure that State
and local agencies comply with the requirements; and
(c) developing more accurate emissions factors and activity data for
stationary, area, and mobile sources by either reviving the EIIP program
and utilizing the organization and expertise of this program, or by
establishing a similar program in-house.
2-2. Because of the difficulties encountered and resources involved in periodically
attempting to ascertain what the actual air toxics emissions totals were for the
1993 NTI, EPA should use the 1996 NTI as the baseline year for measuring
progress.
2-3. Supplement the existing measure of comparing air toxics emissions
reductions to the 1993 baseline inventory by also comparing emission totals
between all other completed inventories.
Agency Comments and OIG Evaluation
Office of Air and Radiation (OAR) and Office of Research and Development (ORD)
officials provided detailed draft comments to our draft report and, where
appropriate, we made revisions. Because these responses are draft and may not
20
-------
represent the Agency's final position on the issues and recommendations in this
report, the draft responses are not included as appendices to this report.
Nonetheless, we made changes to the draft report based on both sets of draft
comments, as well as our exit conference discussions with Agency officials.
Many of OAR's comments involved changing the 1993 and 1996 NTIs to reflect
EPA's methods and assumptions used in developing the recently issued 1999 NTI.
As shown in the report, we believe it is important to discuss the 1999 NTI and its
impact on the prior two inventories' (1993 baseline and 1996 NTI) emission totals.
However, due to EPA's delay in issuing the 1999 NTI, we were only able to
evaluate the factors influencing the reliability of the 1993 and 1996 inventories and,
without an in-depth examination of the factors impacting the reliability of the 1999
NTI similar to the efforts we undertook with regard to the 1993 and 1996 NTIs, we
do not believe it would be appropriate to reflect EPA's most recent assumptions en
masse. Instead, we have presented both sets of figures as appropriate in the report.
OAR's detailed draft comments agreed with the recommendations in this report,
with one exception. OAR intends to continue using the 1993 NTI as the baseline,
instead of implementing our recommendation to use the more reliable 1996 NTI as
the baseline for measuring progress under GPRA. Agency officials explained that if
EPA were to use the 1996 NTI as the baseline, it would not enable them to capture
the emissions reductions achieved by stationary and mobile source standards
implemented between 1993 and 1996. While we recognize this point, we continue to
believe-that, due to the unreliability of the 1993 NTI, EPA should use the 1996 NTI
as the baseline.
As noted in our report, emissions estimating techniques have undergone continuous
change and improvement since the 1993 baseline was developed, and subsequent
inventories have been amassed quite differently, making comparison to the 1993
NTI much less reliable as a measure of GPRA progress as compared to the 1996
NTI. For example, only 3 States provided data for the 1993 baseline, while 36
States provided data for the 1996 inventory. Also, as discussed in our report,
significant uncertainties continue to hamper use of the 1993 NTI as a baseline. For
example, these uncertainties caused EPA recently to revise the 1993 baseline from
6.1 million tons to 7.2 million tons, more than an 18 percent increase. These revised
totals further substantiate our position that EPA is uncertain of its progress in
reducing emissions and needs to improve its measures of air toxics progress. We
agree that subsequent inventories have improved; however, EPA's limited resources
may be better used in improving the 1996 NTI as a baseline in lieu of the 1993 NTI.
We continue to believe that, until ambient air toxics monitoring data is available to
replace emissions data, the best available baseline for measuring Agency progress
for GPRA purposes is the 1996 NTI.
With regard to ORD, officials of this office concurred with our draft report
recommendations, stated that ihe draft report was reasonable, and provided several
21
-------
detailed comments intended to sharpen the quality and accuracy of the report. These
changes were made.
22
-------
Chapter 3
GPRA Measure Needs to Address Risk
of Air Toxics Exposure ,
EPA currently measures progress in its Air Toxics Program by reporting on tonnage
reductions of emissions for all 188 toxic air pollutants collectively but does not
account for the varying levels of toxicity. This aggregate GPRA measure, although
useful, does not provide EPA, Congress, and the public with information on the
specific subset of air toxics that are the most dangerous to human health. Agency
officials said that they see merit in adding a risk component to their GPRA
reporting, and they recently began exploring possibilities of doing so. This
supplemental risk measure would better enable EPA to track its progress in reducing
those air toxics that are most harmful to human health and, in so doing, more closely
meet the intent of GPRA.
EPA Needs to Supplement Emission Reduction Measure
The emissions reduction measure is an important component of EPA's Air Toxics
Program. In addition to improving data, as discussed in Chapter 2, the measure
needs to be supplemented with information related to toxicity or risk. The emission
reduction measure alone cannot ensure risk reduction occurs, because it does not
specify which emissions are being reduced or if those reduced are in fact the most
harmful to human health. If the Agency continues to only measure total emission
reductions without adding a risk component to its GPRA measure, there is limited
assurance that EPA is maximizing its efforts to reduce risk because air toxic health
risk reductions are not proportionate to the tonnage of emission reductions. For
example, toluene and xylene emissions represent about 30 percent (2 million tons)
of the total toxics emissions (6.1 million tons) in the 1993 baseline. However, if
EPA were to eliminate toluene and xylene entirely, it would not result in as much of
a reduction in risk to human health as would a similar tonnage reduction in benzene
and formaldehyde, which are more harmful air toxics.
Risk-Based Information Already Exists
Although much more needs to be learned about the health effects of various air
toxics, we believe EPA has sufficient risk information to begin adding a risk-based
component to its air toxics emissions reduction measure. To date, EPA has not
included a risk element to its measure because Agency officials believe adequate
risk data are not available. However, EPA has completed two efforts that should
enable the Agency to stratify the 188 air toxics into several categories of risk and
allow it to report on emission reductions for these specified subsets of air toxics.
EPA's Integrated Urban Strategy and a study on mobile air toxics emissions
23
-------
illustrate how EPA has already stratified some of the more prevalent and hazardous
air toxics.
Integrated Urban Strategy
The Clean Air Act directed EPA to identify at least 30 air toxics from emissions of
area sources that represented the greatest threat to public health, along with actions
the Agency is taking to reduce exposure. Further, the Act directs a reduction in
public health risks of not less than 75 percent in the incidence of cancer attributable
to area sources. In 1999, EPA identified 33 urban air toxics from major, area, and
mobile sources. Some of EPA's emission estimation techniques included:
evaluating the health effects of the 188 air toxics; assessing air quality monitoring
data; reviewing existing studies; and producing a list of pollutants based on the
relative hazards they pose in urban areas while considering toxicity, emissions, and
related characteristics. EPA used three ranking methods to produce a final list of
priority urban air toxics:
* Risk Assessment/Hazard Ranking Studies - Fourteen urban studies were
reviewed, and results were sorted by pollutant and ranked numerically.
Cumulative Exposure Project - Modeled ambient concentrations of air toxics
were compared to health benchmarks to estimate the level of toxic risk.
Exposure/Toxicity Indicators Ranking - Seven indices were prepared for the
188 pollutants; each pollutant was numerically ranked within each index.
EPA is beginning to make efforts to prioritize the list of 188 air toxics. Using the
1999 NTI, for each air toxic, EPA has estimated the tons of emissions reduced and
then weighed each air toxic by health criteria. EPA officials stated that they plan to
conduct this analysis every 3 years to coincide with new emissions inventories.
Additionally, EPA plans to expand the National Air Toxics Assessment to include
each of the 188 air toxics for which the Agency has established health criteria. EPA
officials said utilizing these efforts, along with data collected from the air toxics
monitoring network, will enable the Agency to develop a relative risk ranking of the
most harmful air toxics. We believe that EPA could use this risk information to
stratify the 188 air toxics into several categories of risk and report under GPRA on
emission reductions for these specified subsets of air toxics, in addition to reporting
aggregate tonnage reductions for all toxic air pollutants collectively.
Mobile Air Toxics Effort
EPA developed a list of air toxics emitted from mobile sources by searching through
available databases and studies that speciated the emissions from motor vehicles and
their fuels. By comparing the list of air toxics to the toxicity information available
in EPA's Integrated Risk Information System, EPA was able to identify 21 air toxics
that are possible, probable, or known human carcinogens. Thirteen of these 21 air
toxics were also identified as part of the 33 air toxics studied in the Integrated Urban
Strategy. Moreover, EPA routinely gathers more information on five mobile air
24
-------
toxics they consider the most dangerous: benzene; 1,3 butadiene; methyl tertiary
butyl ether (MTBE); formaldehyde; and acetaldehyde.
OMB Review of EPA's Air Toxics Program Cited Similar Problems
As part of its efforts to review agencies' implementation of GPRA, the Office of
Management and Budget (OMB) developed a rating tool6 to identify common
performance challenges found in Federal agency programs. Using this newly
created tool, in 2003 OMB evaluated EPA's Air Toxics Program. OMB found that
the program's purpose was clear and the management of the program good, but the
program had not been able to demonstrate that it was maximizing the program's net
benefits. Furthermore, linkages were insufficient between annual performance goals
and the long-term performance goal of protecting 95 percent of the United States
population from unacceptable risks of cancer and other significant health problems
from air toxic emissions.
OMB's evaluation rated the areas of program results and accountability as
"Results Not Demonstrated," and cited the following concerns with EPA's GPRA
goals and measures:
....Although the long-term cancer reduction goal is clearly outcome-related,
"unacceptable risks" are not defined, the relationship between emissions
changes and actual health outcomes is not known because there are no
efficiency measures there are inadequate linkages between annual
performance and long-term goals that prevent it [EPA] from demonstrating its
impact on human health
OMB's observations regarding EPA's results measures are consistent with our
observations, particularly as they relate to measuring reductions in air toxics risk to
human health. In response to these findings, the Administration took measures to
implement OMB's recommendations by requesting $7 million in increased funding
for the Air Toxics Program. The funding was to be for State grants involving
ambient air toxics monitoring that would help fill these data gaps. In addition, EPA
indicated it would focus on maximizing the program's net benefits by minimizing
the cost of each harmful health effect avoided, and establishing better performance
measures. OMB plans to review EPA's Air Toxics Program every year until
adequate results are demonstrated. According to Agency officials, it will be some
years before EPA has a national ambient air toxics monitoring network capable of
measuring air toxics in ambient air on a widespread basis. Until that time, the
Agency could improve its focus on health risk by stratifying the 188 air toxics into
several categories of risk for GPRA reporting.
6Known as the Program Assessment Rating Tool, this Tool was created to make ratings more consistent,
objective, and credible.
25
-------
Conclusion
EPA's mission is to protect public health, not just to reduce emissions. As the EPA
OIG and OMB have identified, there presently is insufficient linkage between health
risk and tons of emissions reduced. As such, we believe EPA needs to add a risk
component to its air toxics measure that will enable the Agency to better
demonstrate that it is managing for results, as called for under GPRA.
Recommendation
3-1. We recommend that the Assistant Administrator for Air and Radiation
supplement the existing tonnage reduction measure of progress for the EPA
air toxics program with measures that address progress toward reductions in
human exposure and health risk. . -
Agency Comments and OIG Evaluation
OAR officials commented that this chapter did not fully characterize the extent of
the Agency's efforts to collect data for prioritizing hazardous air pollutants as it
relates to risk. We incorporated OAR officials' comments as appropriate. However,
because up to this time the Agency has not used this data to measure progress under
GPRA by reporting on the varying levels of toxicity, we believe our
recommendation is appropriate. ORD recommended changing the phrase "health
risk component" to "reduction in human exposure and health risk" in
Recommendation 3-1. We agreed, and have revised this recommendation
accordingly.
26
-------
Appendix A
Details on Scope and Methodology
To assess the appropriateness of EPA's current air toxics GPRA performance measure for assessing
progress towards achieving the air toxics objective, we reviewed documentation that pertained to
GPRA measures and objectives, including:
The Government Performance and Results Act (GPRA) of 1993
EPA's Strategic Plans for 1997,2000, and 2003
EPA's Fiscal 2003 Annual Plan
EPA's Fiscal 2002 Annual Report
OMB's Program Assessment Rating Tool Evaluation of EPA's Air Toxics Program
To assess the accuracy and reliability of data supporting EPA's air toxics GPRA performance
measure, we conducted interviews with EPA Regional, OAR, and ORD officials; researched
documents; and attended training in support of these efforts. Budget information was obtained
through the Office of the Chief Financial Officer and OAR's Office of Transportation and Air
Quality.
We reviewed Section 112 of the Clean Air Act to understand EPA's obligations to control and
reduce air toxics emissions.
To gain a better understanding of the history and development of the inventories, we reviewed EPA's
documentation, which included:
Development of the 1993 Baseline National Toxics Inventory
Development of the 1996 National Toxics Inventory
Development of the 1999 National Toxics Inventory
Toxics Release Inventory requirements for reporting emissions
Emissions efforts developed under MACT programs and rulemakings
Both the models for on-road (MOBILE 5b and MOBILE 6) and the model for off-road
mobile emissions (NONROAD).
To determine the possibility of requiring State and local agencies to report emissions, we researched
the Consolidated Emissions Reporting Rule.
To understand EPA's guidance for implementing Quality Assurance/Quality Control procedures for
data and EPA's effort to implement this guidance, we reviewed:
EPA Order 5360.1
EPA's 1999 National Emission Inventory Preparation Plan
EPA's Emissions Inventory Improvement Program
EPA's Information Quality Guidelines
The Challenge of Meeting New EPA Data Standards and Information Quality
Guidelines in the Development of the 2002 NEI Point Source Data for HAPs
27
-------
We researched documentation regarding the Integrated Risk Information System to determine how
this database is used in conjunction with the risk assessment phase of MACT standards. We also
reviewed the National-Scale Air Toxics Assessment for 1996.
Because the amount of emissions for the source category Other Mobile was minimal (17,000 tons),
we did not include that category in the scope of our evaluation. Types of sources in this category
include locomotive, aircraft, and commercial marine vessels.
Prior Coverage
General Accounting Office
Major Management Challenges and Program Risks - EPA (GAO-03-112), January 2003
Few Agencies Reporting on the Completeness and Reliability of Performance Data
(GAO-02-372), April 2002
EPA Should Improve Oversight of Emissions Reporting by Large Facilities
(GAO-01-46), April 2001
EPA Faces Challenges in Developing Results-Oriented Performance Goals and
Measures (GAO/RCED-00-77), April 2000
Air Pollution - Limitations of EPA's Motor Vehicle Emissions Model and Plans to
Address Them (GAO/RCED-97-210), September 1997
EPA OIG
EPA's Progress in Using the Government Performance and Results Act to Manage for
Results (2001-B-000001), June 2001
Emission Factor Development (6100306), September 1996
28
i
-------
Appendix B
EPA's Recent Update to the 1993 and 1996 Inventories
The three charts below show EPA's estimates of air toxics emissions by major sector for the 1993,
1996, and 1999 NTI as of February 2004.
Updated Emissions for the 1993,1996 and 1999 NTIs
(in mil lions of tons)
Non-road
Non-road
Non-road
On-i
According to Agency officials, the use of updated models to estimate mobile source emissions
resulted in a significant change in the mobile emissions total. EPA decided that both the 1993
baseline and the 1996 NTI needed to be revised to reflect these updated models, and as such, the
inventory totals changed. For example,
the 1993 baseline inventory increased from 6.1 million to 7.2 million tons.
the 1996 inventory increased from 4.6 million to 5.0 million tons.
EPA had not previously reported the air toxics emissions inventory totals for the 1999 inventory,
which is now reported to be 5.1 million tons, or an increase of 500,000 tons above the previously
reported 4.6 million tons for the 1996 inventory. The Agency's newly revised air toxics emissions
inventory totals indicate a decrease in overall emissions, from 7.2 million tons in 1993 to 5.0 million
tons in 1996. However, analysis of the new totals from the revised 1996 inventory (5.0 million) to
the new 1999 inventory (5.1 million) still shows an increase of 100,000 tons of air toxics emissions
from 1996 to 1999. These revised totals were provided in February 2004 after our field work was
completed. We did not verify the revised figures or independently determine the reasons for the
29
-------
changes. These revised numbers were not reflected in recent Agency reports to the Congress and the
public on EPA's progress in reducing air toxics emissions, including:
EPA's FY 2003 Annual Report to the Congress and the President, which serves as the Agency's
program performance report as required by GPRA, which reported a 1993 baseline of 6 million
tons in November 2003;
EPA's Latest Findings on National Air Quality: 2002 Status and Trends Report, which reported
a 1993 baseline of 6.1 million tons and a 1996 inventory of 4.7 million tons in August 2003; and
EPA's Draft Report on the Environment 2003: Technical Document, which reported a 1993
baseline of 6.11 million tons and a 1996 inventory of 4.67 million tons in June 2003.
EPA officials told us that they plan to include the revised totals in next year's reports.
30
-------
Appendix C
Distribution
EPA Headquarters
Assistant Administrator for Air and Radiation (6101 A)
Assistant Administrator for Research and Development (8101R)
Comptroller (2731 A)
Agency Follow-up Official (2710A)
Agency Follow-up Coordinator (2724A)
Audit Follow-up Coordinator, Office of Air and Radiation (6102A)
Audit Follow-up Coordinator, Office of Research and Development (8102A)
Associate Administrator for Congressional and Intergovernmental Relations (1301A)
Director, Office of Regional Operations (1108 A)
Associate Administrator for Public Affairs (1101 A)
Director, Office of Air Quality Planning and Standards (C404-04)
Deputy Director, Office of Air Quality Planning and Standards (C404-04)
Director, Emissions Standards Division (C504-03)
Acting Director, Emissions, Monitoring and Analysis Division (C304-02)
Director, Office of Transportation and Air Quality (6401 A)
Deputy Director, Office of Transportation and Air Quality (6401 A)
Director, National Exposure Research Laboratory (MD-75)
Leader, Emission Inventory & Factors Group (C304-02)
National Exposure Research Laboratory Audit Liaison (MD-343-01)
Audit Liaison, Office of Air Quality Planning and Standards (C404-2)
Audit Liaison, Office of Research and Development (8102R)
Inspector General (2410)
EPA Regions
Regional Air Program Directors
31
-------
19nnD
120PA Pennsylvania Avenue NW
Washington, DC 20460
202-56&-0556
f
^
«
4
------- |