V 00 U,S EPA Headquarters Library Mail code 3404T 1200 Pennsylvania Avenue NW Washington, DC 20460 202-566-0556 OFFICE OF INSPECTOR GENERAL Evaluation Report »* \ JS. OOO EPA's Method for Calculating Air Toxics Emissions for Reporting Results Needs Improvement Report No. 2004-P-00012 March 31, 2004 ------- Report Contributors: Patrick Milligan Frank Martinsky Lorraine Fleury Michelle Brown Abbreviations EIIP EPA GPRA MACT NTI OAR OIG OMB ORD OTAQ Emission Inventory Improvement Program Environmental Protection Agency Government Performance and Results Act Maximum Achievable Control Technology National Toxics Inventory Office of Air and Radiation Office of Inspector General Office of Management and Budget Office of Research and Development Office of Transportation and Air Quality Photo (clockwise, from top left): (1) Cars and trucks emitting exhaust fumes (weathersmith.com); (2) forest fire burning uncontrolled (US Forest Service web site); (3) agricultural equipment emissions (weathersmith.com); and (4) pollution from an industrial smokestack (New York State Department of Environmental Conservation). ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON. D.C. 20460 OFFICE OF INSPECTOR GENERAL March 31,2004 MEMORANDUM SUBJECT: EPA's Method for Calculating Air Toxics Emissions for Reporting Results Needs Improvement Report No. 2004-P-00012 i ij FROM: J. RickBeusse /s/ I Director for Program Evaluation, Air Quality Issues TO: Jeffrey R. Holmstead Assistant Administrator for Air and Radiation (6101 A) Attached is our final report regarding the Environmental Protection Agency (EPA) efforts to develop air toxics emissions data for use as a Government Performance and Results Act performance measure. This report contains findings regarding EPA's methods for calculating air toxics emissions. Also, the report contains corrective actions the Office of Inspector General (OIG) recommends. This report represents the opinion of the OIG and the findings contained in this report do not necessarily represent the final EPA position. Final determination on matters in this report will be made by EPA managers in accordance with established procedures. EPA's Office of Air and Radiation (OAR) provided us with a draft response on March 24,2004, that included comments from the OAR's Office of Air Quality and Planning Standards and OAR's Office of Transportation and Air Quality. Also, officials from the Office of Research and Development (ORD) provided draft comments. Because these responses are draft and may not represent EPA's final position on the issues and recommendations in this report, the draft responses are not included as appendices in this report. Action Required In accordance with EPA Manual 2750, as the action official, you are required to provide this office with a written response within 90 days of the final report date. Since this report deals primarily with OAR's Air Toxics Program, the Assistant Administrator for Air and Radiation was designated the primary action official. As such, he should take the lead in coordinating the ------- Agency's response. The response should address all recommendations. For the corrective actions planned but not completed by the response date, please describe the actions that are ongoing and provide a timetable for completion. If you do not concur with a recommendation, please provide alternative actions addressing the findings reported. We appreciate the efforts of EPA officials and staff, as well as external stakeholders, in working with us to develop this report. For your convenience, this report will be available at http: //www. epa. gov/oig. If you or your staff have any questions regarding this report, please contact me at (919) 541-5747 or Patrick Milligan, Assignment Manager, at (215) 814-2326. ------- Executive Summary Purpose Toxic air pollutants are harmful substances that are known or suspected to cause j| cancer and other serious health problems, and can also have adverse environmental I effects. The Clean Air Act identifies 188 toxic air pollutants and directs EPA to i| regulate the sources emitting these pollutants. li EPA has been tasked with reducing toxic air pollutants and the resulting health ij effects. The Government Performance and Results Act (GPRA) of 1993 required 1 EPA, like other Federal agencies, to measure progress in achieving results. EPA jj measures progress of its Air Toxics Program by calculating how many tons of |! emissions are reduced annually. A vital component for assessing progress and J directing future efforts of the program is the National Toxics Inventory (NTI), an i' estimate made once every 3 years of the total amount of air toxics emitted annually 1 nationwide. Accordingly, our objectives were to answer me following: What are the key assumptions and limitations underlying EPA's NTI and how do these impact its use as a GPRA performance measure? What actions could EPA take to improve its current air toxics GPRA performance measure for assessing progress toward achieving the air toxics objective? Results in Brief Although the methods by which air toxics emissions are estimated have improved substantially in the past 6 years, invalidated assumptions and other limitations underlying the NTI continue to impact its use as a GPRA performance measure. As a result, EPA is not certain how much progress stationary, area, and mobile sources have actually made in reducing air toxics emissions from the EPA-established 1993 baseline. For example, EPA recently finalized its third NTI, which also resulted in revisions to the first two inventories: For the 1993 baseline NTI, EPA increased the estimated emissions inventory from 6.1 million tons to 7.2 million tons (an 18-percent increase). For the 1996 NTI, EPA increased the estimated emissions inventory from 4.6 million tons to 5.0 million tons (a 9-percent increase). ' For the 1999 NTI, EPA estimated 5.1 million tons, an increase of 100,000 tons when compared to the updated 1996 inventory. EPA is not certain whether reductions or increases may have resulted due, at least in part, to the Agency's change in the way it estimated the inventory, rather than real reductions or increases in emissions. While emissions estimating techniques have ------- improved, broad assumptions about the behavior of sources and serious data limitations still exist. Furthermore, due to the changes mat resulted in improvements in later inventories, the meaningfulness of comparing new inventories against the 1993 baseline is questionable. For example, although State- validated emissions data is EPA's preferred calculation methodology: For the 1993 baseline NTI, only three States provided data to EPA. For the 1996 NTI, 36 States provided data to EPA. For the 1999 NTI, 39 States provided data to EPA. Additionally, States were not required to verify their emissions data for any of the above inventories, due largely to EPA's concerns about adding more reporting requirements on States and the lack of a clear statutory mandate. Agency officials said that the Clean Air Act only required limited scope studies on a select number of toxic air pollutants found primarily in urban areas. As such, only 33 of the 188 air toxics have received more in-depth study. Also, despite some improvements in emissions estimating techniques for certain sources of air toxics, available usage estimates and emission factors often are not reliable. Further improvements are needed in the methods for calculating air toxics emissions if EPA is to accurately gauge the extent to which its programs have actually reduced emissions. Otherwise, EPA cannot be assured that the full reductions actually occurred or were, at least in part, the result of changes in estimation methods. EPA could also improve its current air toxics GPRA performance measure by including toxicity information with tonnage measures. Currently, EPA aggregates the tonnage reductions of emissions for all 188 toxic air pollutants but does not account for the varying levels of toxicity among these pollutants. While useful, this aggregate measure does not provide EPA, Congress, or the public with specific , information on those pollutants that are the most dangerous to human health. During me course of our work, EPA officials told us that they see merit in adding a risk component to their GPRA reporting, and they recently began exploring possibilities for doing so. This supplemental measure would better enable EPA to track its progress in reducing those air toxics that are most harmful to human health and, in so doing, more closely meet the intent of GPRA. Recommendations We recommend that the Assistant Administrator for Air and Radiation improve the accuracy, reliability, and usefulness of the Agency's current emissions-based air toxics GPRA performance measure by providing increased funding and support for development of the NTI, and supplement the existing measure of comparing air toxics emissions reductions to the 1993 baseline inventory by also comparing emission totals between all other completed inventories. We also recommend that the Assistant Administrator supplement the existing tonnage measure for air toxics with a measure of progress toward reduction in human exposure and health risk. ------- Agency Comments and OIG Evaluation Officials from the Office of Air and Radiation (OAR) and the Office of Research and Development (ORD) provided detailed draft comments on the report. Because these responses are draft and may not represent the Agency's final position on the issues and recommendations in this report, the draft responses are not included as appendices to this report. Nonetheless, we made changes to the draft report based on both sets of draft comments, as well as our exit conference discussions with Agency officials. OAR's detailed draft comments agreed with the recommendations in this report, with one exception. OAR intends to continue using the 1993 NTI as the baseline, instead of implementing our recommendation to use the more reliable 1996 NTI as the baseline for measuring progress under GPRA. Agency officials explained that if EPA were to use the 1996 NTI as the baseline, it would not enable them to capture the emissions reductions achieved by stationary and mobile source standards implemented between 1993 and 1996. While we recognize this point, we continue to believe that, due to the unreliability of the 1993 NTI, EPA should use the 1996 NTI as the baseline. Additional discussion of this point is provided at the end of Chapter 2. With regard to ORD, officials of this office concurred with our draft report recommendations, stated that the draft report was reasonable, and provided several detailed comments intended to sharpen the quality and accuracy of the report. These changes were made. ------- iv ------- Table of Contents Executive Summary i Chapters 1 Introduction [[[ 1 2 Improvements Needed to Air Toxics inventory for Use in GPRA Reporting ......................................... 7 3 GPRA Measure Needs to Address Risk of Air Toxics Exposure .......... 23 Appendix A Details on Scope and Methodology ................................. 27 B EPA's Recent Update to the 1 993 and 1996 Inventories ................. 29 ------- ------- Chapter 1 introduction Purpose Toxic air pollutants are harmful substances that are known or suspected to cause cancer and other serious health problems, and can also have adverse environmental effects. The Clean Air Act identifies 188 toxic air pollutants and directs EPA to regulate the sources emitting these pollutants. EPA has been tasked with reducing toxic air pollutants and the resulting health effects caused by exposure to these pollutants. The 1993 Government Performance and Results Act (GPRA) required EPA, like other Federal agencies, to measure progress in achieving results. EPA measures progress of its Air Toxics Program by calculating how many tons of air toxics emissions are reduced annually. A vital component for assessing progress and directing future efforts of the program is the National Toxics Inventory (NTI), an estimate made once every 3 years of the total amount of air toxics emitted annually nationwide. Accordingly, our objectives were to answer the following: What are the key assumptions and limitations underlying EPA's NTI and how do these impact its use as a GPRA performance measure? What actions could EPA take to improve its current air toxics GPRA performance measure for assessing progress toward achieving the air toxics objective? Background What Are Adverse Health Effects of Air Toxics? Toxic air pollutants are known or suspected to cause various serious health problems. The adverse health effects include: increased risk of cancer; effects on the neurological, cardiovascular, respiratory, and immune systems; liver and kidney damage; and effects on fetal and child development. Immediate effects of toxics exposure may be minor, such as watery eyes; greater health problems, such as cancer, may not appear until years after exposure. In addition to exposure through breathing air toxics, some toxic air pollutants, such as mercury, can deposit onto soils or surface waters, where they are ingested by plants and animals and eventually transmitted through the food chain. Exposure to some air toxics before birth or during childhood may interfere with normal development, while other air toxics may affect the ability to conceive or give birth to a healthy child. Studies have found that exposure to benzene from less than 5 years to more ------- than 30 years has resulted in individuals developing and ultimately dying from leukemia. Studies of smelter workers identified an association between occupational arsenic exposure and lung cancer mortality. Where Do Air Toxics Come From? Air toxics come from a complex variety of sources, which poses a challenge to EPA when it compiles the air toxics emissions inventory. For example, toxic air pollutants include benzene, which is found in gasoline; perchlorethlyene, which is emitted from some dry cleaning facilities; and methylene chloride, which is used as a solvent and paint stripper by a number of industries. Toxics coming from other sources include asbestos; dioxin; toluene; and metals such as cadmium, mercury, chromium, and lead compounds. Table 1.1 describes some of the types of sources emitting air toxics and the amount of emissions estimated to be generated by each category of sources, according to EPA's 19931 baseline inventory. Table 1.1: Categories of Sources of Air Toxics ;.(,.., , -:.'-: - -: " fliH; .-S.:-S :.? ;i«£ ::'-o."'o:J 1 :' -:S " -:"'- !i &.&?'& ... .- ... y.-,:: _x : ..StwnaK Categories Major Area* Mobile A. On-road B. Non-road Subtotal BefiniBan Emissions of 10 tons per year or more of any one hazardous air pollutant, or 25 tons per year or more of any combination of hazardous pollutants Emissions of less than 10 tons per year of any one hazardous air pollutant, or less than 25 tons per year of any combination of hazardous pollutants Emissions from motorized vehicles normally operated on public roadways Emissions from a diverse collection of engines, equipment, vehicles, and vessels Sxamptesoi types j , OfSWifce* Utilities, Refineries, Car Manufacturers, Chemical Manufacturers Dry Cleaners, Gas Stations, Wood Burning Stoves Cars, Buses, Sport Utility Vehicles, Light- & Heavy-Duty Trucks Construction and Agricultural Equipment, Personal Watercraft Total Emissions 19$38as«fM« ,: flnmiflionterw) 2.7 1.1 1.7 .6 2.3 6.1** * Area sources include a sub-category entitled "Other" for those sources that are not otherwise regulated, such as wildfires and open burning. ** This total represents the 2002 version of the 1993 baseline NTI. The 1993 toxics emissions inventory is the baseline measure EPA selected to gauge progress for the Air Toxics Program. JA large part of the 1993 baseline inventory is based on data obtained from 1990 and 1993. For simplicity, and because EPA has traditionally referred to it as such, we refer to this data as the 1993 baseline inventory. ------- What Are the Air Toxics GPRA Goals and Measures? GPRA, passed in 1993, holds Federal agencies accountable for achieving results by requiring them to measure program outcomes, including for EPA's Air Toxics Program. By 1997, GPRA required agencies to establish outcome goals to measure their progress and report on their accomplishments. The three main components of GPRA require EPA to: (1) develop a 5-year strategic plan defining its long-term goals and how the Agency intends to achieve them; (2) prepare annual performance plans with goals and measures that relate to either outputs or outcomes that quantitatively measure the Agency's results and demonstrate their linkage to longer-term goals; and (3) prepare annual performance reports on accomplishments. As shown in Table 1.2, EPA's Strategic Plan objective for the Air Toxics Program (updated every 3 years) relies on emission reductions. Table 1.2: Strategic Plan Objectives for Air Toxics ₯«*f 1997 2000 2003 ; Objective By 2010, reduce air toxics emissions by 75 percent from 1993 levels to significantly reduce the risk to Americans for cancer and other serious adverse health effects caused by airborne toxics. By 2020, EPA will eliminate unacceptable risks of cancer and other significant health problems from air toxics emissions for at least 95 percent of the population, with particular attention to children and other sensitive subpopulations, and substantially reduce or eliminate adverse effects on our natural environment. By 201 0, working with partners, reduce air toxics emissions and implement area-specific approaches to reduce the risk to public health and the environment from air toxic pollutants. swdget ; (a) $95 million (17% of Air Budget) $11 8 million (19% of Air Budget) (a): EPA's air toxics budget request for 1997 was not comparable to 2000 and 2003; the total air program budget request for 1997 was $304 million. What Is the Air Toxics Emissions Inventory? EPA compiles an NTT every 3 years that consists of an estimate of the total emissions of the 188 air toxics identified in the Clean Air Act. Table 1.3 depicts the years for which four inventories were or are in Ihe process of being developed and when each was completed. Table 1.3: Air Toxics Emissions Inventories Undertaken iWS^^^WY*w^lt?is|l|?|l; 1993 1996 1999 2002 SihillaiYeir Inventory Completed^ | 1998 2001 2003 In progress 2EPA referred to the 1993 and 1996 inventories as the NTL For the 1999 inventory, EPA combined the air toxics inventory with the criteria pollutant emissions inventory and changed the name to the National Emissions Inventory, or NEI. For purposes of this report, we will refer to all 3 inventories (1993,1996, and 1999) as the NTL ------- Chart 1.1 provides a scheduled timeline of key steps in developing the 1999 air toxics inventory, which took place after 1999 ended. Chart 1.1: Scheduled Timeline for Developing 1999 toxics Inventory I States send Surveys to Point Sources Federal Agencies develop activity data - VMT States quality control and record data EPA receives first reported data EPA prepares draft Inventory Reviewer's comments EPA receives new data, along with comments EPAprepares second draft Inventory 2nd Round of comments and Inventory revisions (a) Development of the NTI begins after the designated year has ended. In addition to the activities listed in Chart 1.1, State and other Federal agencies must obtain data from other sources (such as the Federal Highway Administration, which provides EPA with information on national vehicle miles traveled for the inventory year). These data are not available until a year or so after the inventory year has ended. After the new inventory is developed, EPA compares it to the 1993 baseline inventory to determine the amount of emissions reductions achieved. Although the 1993 and 1996 inventories took over 4 years to complete, EPA hopes to reduce this time in the future. What Are The Uses of the Inventory? Although EPA is in the early stages of developing an ambient (outdoor) air toxics monitoring network3 capable of measuring air toxics in ambient air on a national basis, the Agency does not yet have a formal network and therefore relies primarily on emissions data for measuring progress. The NTI is the primary source of national air toxics emissions data used by major stakeholders - such as scientists, researchers, lawmakers, and the public - to gauge progress. Uses of the NTI include rulemaking efforts, modeling, and risk assessments. More details on the NTI are in Chapter 2. EPA's National Air Toxics Trends Sites (NATTS) network consists of 22 sites. Agency officials said that they plan to implement the network in fiscal year 2004. ------- Scope and Methodology To assess the adequacy of EPA's GPRA measure for air toxics, we reviewed documentation related to: GPRA objectives and measures; the development of the 1993,1996, and 1999 inventories; Quality Assurance/Quality Control efforts for the development of the three inventories; and regulations, guidance, policy, and assessments applicable to the development of the inventories. We also conducted numerous interviews with officials from: the Office of Air Quality Planning and Standards, the Office of Transportation and Air Quality, EPA Region 3, and the Office of Research and Development (ORD). Budget information was obtained from the Office of the Chief Financial Officer. We also reviewed Section 112 of the Clean Air Act to identify the requirements of EPA to identify, control, and reduce air toxics emissions. Our field work was conducted from February 2003 to November 2003. We conducted this evaluation in accordance with Government Auditing Standards, issued by the Comptroller General of the United States. Additional details on our scope and methodology are in Appendix A. On March 30,2004, we met with Office of Air and Radiation (OAR) officials to discuss their draft comments and we made additional modifications to the report based on our discussions. ------- ------- Chapter 2 Improvements Needed to Air Toxics Inventory for Use in GPRA Reporting Although the methods used to estimate air toxics emissions have improved substantially in the past 6 years, EPA is not certain of how much progress stationary, area, and mobile sources have actually made in reducing those emissions. Emissions estimating techniques have undergone continuous change and improvement since the 1993 baseline was developed, and because subsequent inventories were amassed quite differently, comparison to the 1993 baseline has become less meaningful over time. For example, only 3 States provided data for the 1993 baseline, while 36 and 39 States provided data for the 1996 and 1999 inventories, respectively. Of 62 authorized local programs, only one reported air toxics emissions data for 1993; three for 1996; and nine for the 1999 inventory. Even with increased State and local agency participation, these agencies have not been required to report air toxics emissions data, nor have they been required to verify it. Further, despite improvements in estimation techniques, usage estimates and emission factors often are not reliable. Due to the limitations noted, EPA cannot be assured that the full amount of reductions reported actually occurred or whether they were, at least in part, the result of improved estimation methods. Improvements in the methods for calculating air toxics emissions are needed if the Agency is to accurately gauge the extent to which its programs have reduced the public's health risk to air toxics as called for under GPRA. Difficulties in Comparing EPA's Air Toxics Emissions Inventories To develop major and area source emission estimates, EPA primarily uses four types of data (listed in order of preference based on data reliability): State, Local, and Tribal-Submitted Data: State, local, and tribal agencies, on a voluntary basis, submit air toxics emissions data. Maximum Achievable Control Technology (MACT)/ResiduaI Risk: Each time EPA develops a control for a source category (a source of air toxics), it gathers emissions data. Toxics Release Inventory: This is an EPA-managed database used to inform the public and government officials about the total release and transfer of toxic chemicals, and is based on information reported by a select group of industries. National Estimates: When State information is not available, EPA estimates emissions on a national level and allocates down to a local (county) level. ------- Although States have not been required to validate or even report air toxics emissions data, EPA considers State-submitted data to be the most reliable. In its absence, the Agency uses the less reliable MACT, Toxics Release Inventory, and national estimates. EPA also uses data gathered from mobile source rulemaking efforts, industry surveys, and other Federal agency information, such as vehicle mileage data from the Federal Highway Administration, each with its own set of limitations. Table 2.1 provides a brief qualitative assessment of the sources of information used for each triennial inventory, highlighting the significant changes made to more recent inventories. Table 2.1: Key Differences in the Three Inventories Sources of Pollution Major Area Mobile (On-Road) Mobile (Non-Road) ', .1603 8*4«8n« 3 States and 1 local agency provided data; few quality checks performed. EPA used less reliable data such as MACT, Toxics Release Inventory, and national estimates. (Mainly non- facility specific data aggregated up to the county level.) Emission estimates largely based on national activity data and national emission factors. Emissions allocated down to the county level. Five toxics estimated using early on-road model, while other toxics calculated using less sophisticated methods. Fuel types not accounted for. Emissions estimated using non-road study of emissions. This study was first attempt at identifying non-road equipment types. Much of the activity data and emission factors unverified. Fuel types partially accounted for. 1W$ ' '-:' - 36 States and 3 local agencies reported data - 46 States reviewed the data. Quality check efforts made to eliminate duplicate or , missing facilities. MACT and Toxics Release Inventory used less. Estimates based on point emissions. Began to use more State and MACT data for non-point sources. Emission estimates still largely based on national activity data and national emission factors. On-road model (MOBILE 5b) used for all toxics estimates. Fuel types accounted for. New model {NONROAD) developed to more accurately depict types of equipment. Much of the activity data and emission factors still unverified. Fuel types partially accounted for. 193$ '/ - \ 39 States and 9 local agencies reported data - 46 States reviewed the data. For the first time, 3 tribal agencies reported data. Some additional quality checks performed on data. MACT and Toxics Release Inventory used even less. Some area source locations identified and tracked. Improved use of non-point source data. New on-road model (MOBILE 6) accounts for many variables such as vehicle type, vehicle speed, road type, and fuel types. Equipment-type information again improved. Model (updated NONROAD) also attempted to account for changes in engine power levels related to pollution. Much of the activity data and emission factors still unverified. Fuel types partially accounted for. Limitations of the Air Toxics Inventories For the 1993 baseline inventory, EPA received very little State participation. Over the past 10 years, States have progressively submitted more toxics emissions data, of increasingly improved quality. EPA officials said that, due primarily to States' increased awareness and need for toxics emissions data for purposes other than the inventory, the States wanted to rely more on their own estimates than the estimates ------- generated by EPA. Some of the assumptions and limitations associated with the 1993,1996, and 1999 inventories are discussed below. 1993 Baseline Inventory EPA developed the 1993 baseline (the 1993 NTI) as a repository of information collected by EPA to meet the requirements of Clean Air Act Sections 112(c)(6) and 112(k), which required limited scope studies on a select number of toxic air pollutants found in urban areas. In total, EPA studied 33 of the 188 air toxics as a starting point for developing the NTI, but recognized that there were information gaps. Because EPA's authority to require States to report air toxics emissions was limited, only three States and one local agency provided useable input to the 1993 emission estimates. For the remaining 47 States and 61 local agencies that did not submit air toxics emissions information, EPA's only viable option was to use national data and apportion it down to the county level (the "top-down" approach). Sources of this national data included the EPA-developed national estimates and information gathered from the development of the MACT standards. Because studies have found that air toxics often are a localized problem, the top-down approach is considered one of the least desirable methods because it does not accurately depict local variations. Subsequent to developing the 1993 baseline, EPA sometimes revised the baseline when it received more accurate 1993 information. However, Agency officials stated that due to limited resources and because the baseline inventory is only an estimate, their efforts are typically geared toward obtaining better data for current and future air toxics inventories rather than updating the 1993 baseline. 1996 Inventory For the 1996 inventory, some of gaps in the 1993 baseline inventory were filled to provide a more complete picture of air toxics emissions. As compared to the 1993 baseline inventory, EPA used fewer top-down approaches because 36 States and 3 local agencies submitted air toxics emissions estimates, and MACT and Toxics Release Inventory data was used for the remaining 14 States and 59 local agencies that did not submit air toxics emissions information. However, there was still not full State and local participation for reporting, and the State and local figures provided were not verified by EPA. Further, the 1996 emission estimates for the Area and Mobile Non-Road sources of pollution were still largely based on national activity data and emission factors. ------- 1999 Inventory Fertile 1999 inventory, 39 States, 9 local agencies, and 3 tribal agencies submitted emissions estimates, and there were improved Quality Assurance/Quality Control efforts by EPA and the States, who performed the following four activities: States used a Quality Control formatting tool for file format and data field verification prior to sending the estimates to EPA. EPA used a software tool to identify duplicate facilities. EPA sent States draft summaries of emission data for States to review for accuracy and completeness. EPA performed Quality Assurance/Quality Control checks to identify gross errors with the emission estimates. However, there are still limitations with the 1999 NTI. EPA once again had to use MACT, Toxics Release Inventory, or apportion emissions in a top down manner for the 11 States and 53 local agencies that did not submit air toxics emissions information. Also, Agency officials said that the Quality Assurance/Quality Control check was designed to identify emission estimates substantially outside the historical range of reported emissions. These checks are not used to identify inconsistencies reported from independent sources of data or against national estimates. Additionally, these checks do not include sampling State data to verify the adequacy of emission estimation methodologies. These additional checks would substantiate State-reported data. Progress In Reducing Emissions Uncertain As shown in Chart 2.1, as of September 2003, the 1993 baseline inventory indicated that a total of 6.1 million tons of toxics emissions was released in 1993, while the 1996 inventory indicated that a total of 4.6 million tons of toxics emissions was released in 1996, or total reductions of 1.5 million tons. Recently EPA finalized the 1999 inventory, which showed 5.1 million tons of emissions, or an increase of 500,000 tons when compared to the 1996 inventory. However, this reduction (1993 to 1996) and this increase (1996 to 1999) may have resulted due, at least in part, to EPA's change in the way it estimated the inventory, rather than a real reduction in emissions. EPA was unable to provide us with reliable data that would accurately show how much air toxics emissions were actually reduced from 1993 to 1996, and how much of the reported change may have been due to a change in methodology. 10 ------- Chart 2.1: Emissions for the 1993,1996 and 1999 NTIs (in millions of tons) Non-road On-road 1993 Total: 6.1 On-road 1996 Total: 4.6 On-road 1999 Tola!: 5.1 Major Area Mobile EPA credited the 1.5-million-ton reduction to its programs designed to control air toxics emissions. Almost all of these reductions occurred-in the major source category, where emissions decreased from 2.7 million tons in 1993 to 1.2 million tons in 1996. During this 3-year period, EPA issued 11 MACT standards representing 664,000 tons of emission reductions. However, there still remains 836,000 tons of emissions reductions achieved by controlling sources not subject to MACT standards. During this same time period, EPA received improved data and substantially changed its method for calculating emission estimates. Specifically, the 1996 NTI had much more State-submitted data on major source emissions, and the accuracy of the mobile emission estimates improved. Regarding the recently completed 1999 NTI, again there is a significant change in the calculation methods used, which may impact the amount of emissions reported. For example, for the 1999 NTI, improved mobile models were used for both the on-road and non-road estimates, there was slight increase in State and local agency participation, and there were improved Quality Assurance/Quality Control efforts by EPA and the States. These changes in calculation methods may have caused, at least in part, the 500,000 ton increase in emissions from 1996 to 1999. Recent revisions to the 1993 and 1996 NTIs demonstrate a significant improvement in characterizing EPA's progress in reducing air toxics emissions. However, this increase again demonstrates that EPA is not certain whether changes in the inventory may be due, at least in part, to the Agency's change in the way it estimated the inventories. 11 ------- The above figures for the 1993 and 1996 inventories were used recently in key EPA reports to the Congress and the public on the Agency's progress in reducing air toxics emissions, including: EPA's November 2003 "FY 2003 Annual Report" to the Congress and the President, which serves as the Agency's program performance report as required by GPRA; EPA's August 2003 "Latest Findings on National Air Quality: 2002 Status and Trends Report," which provides the public with the latest information on air quality nationwide; and EPA's June 2003 "Draft Report on the Environment 2003," which provides the nation with EPA's first ever national picture of the environment and represents EPA's first step at developing a set of comprehensive environmental indicators. However, in February 2004, after the conclusion of our field work, EPA provided us with revised totals for the 1993 and 1996 inventories (prior to this time, no figures had been provided for the 1999 inventory). For example, the Agency's revised 2004 estimate of the 1993 baseline is now 7.2 million tons. Appendix B provides further details on EPA's recent update of the 1993 and 1996 inventories. In our opinion, these revised totals further substantiate our position that EPA is uncertain of its progress in reducing emissions and needs to improve its measures of air toxics progress. Status of EPA Efforts to Improve Recent Inventories While improvements to estimation methods were made to the more current inventories, EPA acknowledges that there are still deficiencies in the estimation methods. These deficiencies vary from source to source and affect the reliability and accuracy .of the data. Because development and maintenance of air toxic emission inventories is not explicitly required by the Clean Air Act, most of the data used for the emissions inventories were obtained and developed for other reasons and then secondarily used to compile the NTI. Below is a description of how the inventory is amassed and examples of the assumptions and limitations associated with the data for the primary source categories of pollution (major, area, and mobile). Major and Area Sources Because major and area source categories include stationary sources and often use the same data sources, they are presented together. Major sources comprised 26 percent of the 1996 inventory, and area sources comprised 24 percent. Some area sources, such as wood burning stoves, cannot be cost-effectively inventoried as individual sources because they are comprised of a large number of sources spread across the country, so emissions are calculated at a national level and then allocated 12 ------- down to the county level. Also, some source categories have both major and area emitters. For example, a small neighborhood dry cleaner may not emit enough toxic emissions annually to reach the threshold of a major source, but a large regional dry cleaner may reach this threshold and thus be classified a major source. Following is a description of the four primary sources of information used to estimate both major and area source emissions, from most reliable to least reliable. State-Reported Emissions: Because there are no reporting requirements for States, the data submitted by States are inconsistent and States are not required to verify the accuracy of the data. To improve State data, EPA in 2000 included in a proposed regulation specific reporting requirements that would standardize the type of toxics data gathered by States, as well as the methods used to calculate estimates. However, the proposed air toxics reporting requirements were deleted from the final regulation. This was due largely to EPA's concerns about adding more reporting requirements on States, the Agency's belief that the Clean Air Act is not clear on air toxics reporting, and its belief that there would be lawsuits if specific reporting requirements were imposed. Seventeen of the 22 State and local agencies that commented on the proposed regulation said that States should be required to report air toxics emissions data. However, some said they would not support increased reporting requirements without increased funding. In 2003, EPA obligated approximately $181 million in Clean Air Act Section 105 grant funds to States for carrying out a variety of Clean Air Act goals, but the Agency did not use these funds to negotiate commitments with States for consistent, validated air , toxics emissions data TRie emissions tn«enteiy EPA has also attempted to improve air toxics emissions reporting by working with the States through the Emission Inventory Improvement Program (EIIP), which was intended to promote the development and use of standard procedures for collecting and reporting emissions data. Initially, this program worked to develop standardized methods for collecting data for both criteria4 and toxic pollutants. However, EIIP workgroup officials decided that developing better air toxics emissions estimation methodologies was beyond the intended scope of the workgroup sM Oflfee Tip goal ft achieved fay developing « $&t arprefarted arid vX&tr&frta method*" *?r all invent9ry«s*l»«3teq| t*sk$. This Bferp&tii&ttiBn jrfipWSwe* #» q&«t$te*jiqy Of caHUsctetf tfirffcdraf result* in ftjareasBd 4Six common air pollutants found nationwide that harm human health and the environment, called criteria pollutants because EPA sets standards for these pollutants by first developing health-based criteria. 13 ------- effort, and focused only on criteria pollutants. According to EPA officials, beginning in 2004, the EIIP was no longer active because of limited funding for the program. However, EPA officials told us that there is an ongoing reorganization in the Office of Air Quality Planning and Standards that will likely include a group dedicated to improving emission factors and emission estimation methods for air toxics. MACT: EPA initially gathered data to learn as much as possible about the industry before a MACT emission limitation was established. In doing so, EPA relied on many different sources of data, including: national activity data applied to emission factors, facility level emissions reported to EPA, and facility level emissions obtained from the Toxics Release Inventory database. Of the 70 MACT area source categories that reported emissions data: 19 used facility-level emissions reported to EPA. 36 used national activity estimates. 15 relied on Toxics Release Inventory data. Only 35 of the 70 MACT area source categories calculated emission estimates for 1996. For example, of the 19 facility level estimates reported to EPA, only 9 estimated 1996 emissions, while the remaining 10 estimated emissions for 1 or more calendar years ranging from 1991 to 1997. For 1 of the 36 sources using national activity data - halogenated solvent cleaners - EPA developed an estimate of the total mass for various toxic air pollutants, but did not know how much of each air toxic comprised the total. As a result, EPA divided the mass equally among a list of applicable air toxics. Toxics Release Inventory: The Toxics Release Inventory database is mandated by the Emergency Planning and Community Response Act, which requires a select group of industries meeting specific requirements to report toxics release information to EPA, including releases to air, land, and water. In those instances where EPA used the database to develop MACT standards, the Agency used this same data for the toxics inventory. Because it was the only available information, EPA used the database despite the significant limitations associated with the data. Not all facilities that emit air toxics meet the requirements to report Toxics Release Inventory emissions. As a result, some sources of air toxics would not be identified and the emissions would not be recorded or reported to EPA. The data is also compromised because facilities are allowed to report reasonable but unverified estimates of toxic amounts instead of monitored releases. Finally, calculation methods can vary annually and, as a result, some facilities may report emission reductions not because of actual reductions but because a different calculation method was used, which may result in EPA overstating or understating the actual amount of reductions. ------- y, Hair cafe prOductst;: H, *m) rei ear* ftpguiife &* ihdtidOrf In this Area SOUttH eatery, £PA Ori ppwtefci uspSB: and population tfatp ferfte ytef 1$a& T* patoufate «w tafei em&sians IUJS. population 10 ^et^rraineaip total arrissiOnsfrOrmpesrsOnat! . ttes county tevet feasati on U.S, CeniijH 8ur?a« National Estimates: When EPA could not obtain emissions data from the above three activities, the Agency reverted to estimating the emissions from these source categories on a national scale and then allocating them down to a local level. When national-level emissions are used, it does not account for significant local variations, which can have a large effect on the health of the local population exposed to those toxic emissions. For example, EPA's estimates assume each person in the United States, regardless of age, sex, and other differences, uses the.same amount of personal care products (see box). When there is no data available for the year targeted based on the above four primary sources of information, the Agency as a last resort will occasionally use data derived from either a prior or subsequent year's inventory. This affects the inventory's ability to accurately depict emissions reduction trends because, at times, the same data are being used for multiple years. Mobile Sources While major stationary sources of emissions include many types of pollution- creating equipment and processes, the number of mobile source emissions points far exceed stationary sources in that they are produced by millions of engines of varying types and sizes that move around the country. Due to the quantity, diversity, and mobility of these on-road and non-road sources, sophisticated methods using complex calculations that include a host of assumptions were developed to estimate emissions. Since the inception of the EPA mobile source program, EPA has historically focused its efforts on understanding and controlling on-road emissions because they represented the majority of the pollution from mobile sources. These efforts have led to increased knowledge of on-road emission factors and better activity data EPA has made substantial progress in reducing on-road emissions through the production of "cleaner" vehicles and the implementation of on-road vehicle control programs. This, combined with the ever-increasing popularity of non-road sources, has resulted in EPA becoming more concerned about non-road emissions. Furthermore, non-road sources was the only category to show an increase in emissions from 1993 to 1996. However, the nature of non-road sources presents 15 ------- EPA with considerable challenges, due to the large number of equipment types and manufacturers. On-Road Sources: On-road mobile emissions comprised 28 percent of emissions for the 1993 baseline inventory. The predominant source of emissions in this category was gasoline-fueled automobiles. There are two general approaches to calculating on-road emission estimates for the 21 toxic air pollutants emitted by motor vehicles: For 15 of the on-road air toxics emissions, estimates are derived from either particulate matter or volatile organic compound emissions data For example, the Agency takes the particulate matter data for on-road emissions and then estimates how much air toxics are in those emissions. In doing so, EPA assumes that air toxics emissions from vehicles are directly proportional to vehicle particulate matter emissions. For six of the on-road air toxics emissions, total vehicle miles traveled (obtained from the Federal Highway Administration's database) for each class of vehicle are multiplied by an emission factor to develop the air toxics estimate. EPA currently develops emission estimates that account for specific vehicle activities, such as vehicle type and speed, and emission-related factors, such as ambient temperature. For example, while the 1996 mobile estimate included . 7 vehicle types and 1 vehicle speed, the 1999 estimate included 28 vehicle types and 12 vehicle speeds. However, with these more complex calculations come increased assumptions and limitations. In 2000, the National Research Council performed a review of EPA's on-road model, entitled Modeling Mobile Source Emissions, and identified the following assumptions and limitations: EPA's selection of vehicles for testing does not sufficiently represent high- emitting vehicles. Emissions from high-emitting vehicles are a relatively small fraction of the on-road fleet, but contribute a large fraction of total vehicle emissions, and thus require a large sampling fraction. If high-emitters are not properly characterized, emission factors can be seriously biased. EPA's calculation method does not consider road-grade effects on emissions. Road-grade influences vehicle emissions in that higher emissions occur when vehicles negotiate steep road grades. Light-duty truck emissions are sometimes estimated from passenger automobile test data, even though some trucks are regularly used as working vehicles, which results in higher emissions due to the increased load. EPA lacks sufficient data to estimate effects of high emitters and sometimes fills in missing data by assuming high emitters behave the same as normal emitters. 16 ------- The National Research Council review stated that EPA needs to better understand and quantity the uncertainty related to sources of data used in the model, such as the use of small and nonrepresentative emissions data The Council believes that the quantification of these uncertainties is critical for understanding the weaknesses of the model and identifying the most critical needs for improving emissions test data. In response to the Council's findings, EPA has initiated a $4 million test program to collect data on high-emitting vehicles, and the Agency is developing a new model to better quantify the uncertainty in on-road emissions estimates. In our view, these are steps in the right direction. Nonetheless, at the present time, the Agency has limited assurance that data reported in its GPRA performance measure is accurately assessing progress. Non-Road Sources: Non-road mobile emissions comprised about 10 percent of emissions for the 1993 baseline inventory and increased to more than 19 percent, or nearly one-fifth of the total, for the 1996 inventory. Until the mid-1990s, emissions from these engines were largely uncontrolled. This category covers a diverse collection of engines for equipment, vehicles, and vessels, ranging from leaf blowers to earth-moving equipment. As compared to on-road methods, emissions estimation methods for non-road sources are much less developed because they lack adequate basic data. In developing non-road estimates, EPA typically develops a usage estimate such as the number of hours a piece of equipment operates during the year. EPA then multiplies that number by an emission factor, which essentially calculates how much pollution is emitted for each hour the equipment is operating. Both the non-road usage estimates and emission factors have significant limitations. Usage Estimates - Currently, EPA cannot verify the accuracy of the data. Usage estimates come from a marketing research firm tfiat gathers usage information from equipment manufacturers through telephone surveys. The marketing research firm takes this data and uses a proprietary process to project non-road population totals. Because EPA cannot assess the accuracy of the process, EPA officials expressed interest in developing their own strategy for collecting usage data. For example, a more valid estimate would be generated if EPA could place a monitor on equipment to measure the hours of operation to better verity the accuracy of their data. Despite the limitation described above, EPA uses the survey data because it is the only available source of non-road equipment usage information. Emission Factors - EPA officials have stated that there are few studies that have developed toxics emission factors for non-road equipment. An emission factor is developed by monitoring emissions from several test sites and computing an average emission rate. However, for many of the existing studies, engine emissions have not been tested while equipment is operated under field conditions (see example in following box). In lieu of developing its own emission factors, EPA has largely relied on certified emissions information 17 ------- Factors provided by engine manufacturers. Also, the non-road calculation does not . account for engine deterioration, which results in higher emissions in later years. Although EPA calculates deterioration for on-road vehicles by using State vehicle registration information, similar information does not exist for most non- road equipment. However, EPA has , / tr * ', 's efforts underway to develop improved air toxics estimates for some categories of non-road equipment. For example, EPA just completed a test program that characterized air toxics emissions for 14 non-road diesel engines under a variety of operating conditions with different fuels, and is also testing , rwn* So For-- - i exampte^, wNn » tewfetote soopps a feuetet of soil, rag»'twjS9poww*an aswef fc 2- and 4-stroke gasoline lawn and garden equipment at its mobile source laboratory in ORD's National Exposure Research Laboratory. According to Agency '" " " * 5 - officials, the efforts should result in substantial improvement in estimating air toxics emissions for some categories of non-road equipment. Additional Uses of the Air Toxics Emissions Inventory Data In addition to needing quality information for reporting results in GPRA, there are several other important needs for a valid, reliable, and accurate inventory of air toxics emissions, such as: Rulemaking Efforts - The 1993 baseline and 1996 inventory were relied upon extensively in EPA's development of the 2001 Final Mobile Rule.5 In developing this rule, EPA used data from these inventories to determine the controls that would be needed to reduce mobile emissions to the desired level. National Air Toxics Assessment - In carrying out its National Air Toxics Assessment, EPA used the 1996 NTI to develop a subset of 33 air toxics posing the greatest risk to urban areas to better understand their priority as related to risk, as well as their effects on the nation's population. The emissions data were input into models that projected ambient air concentrations and ultimately estimated the level of human exposure to the 33 toxics. EPA then followed risk characterization guidelines to estimate both the cancer and non-cancer effects. The Science Advisory Board identified weaknesses in the methods used to estimate emissions (1996 NTI) and determined that improved accuracy and reliability of the NTI was vital to the National-Scale Air Toxics Assessment effort. 540 CFR Parts 80 and 86, Control of Emissions of Hazardous Air Pollutants From Mobile Sources; Final Rule, March 29,2001 18 ------- Residual Risk Assessments - After industry has installed required MACT controls to reduce air toxics emissions, the Clean Air Act requires EPA to assess the remaining risks to public health. The NTI is critical to measuring the effectiveness of these controls. If, after 8 years from MACT promulgation (or 9 years after promulgation of the 2-year MACT source categories), the MACT controls have not reduced emissions to an acceptable risk level to protect public health with "an ample margin of safety," EPA must promulgate health-based standards for that source category to address this risk. Modeling - In addition to air toxics being modeled for the National-Scale Air Toxics Assessment effort, EPA also uses the NTI as the basis for numerous modeling efforts at the local and regional levels. For example, EPA will model a series of toxics for a community or smaller geographical area EPA, State, and local agencies use such model-derived data to estimate exposures and focus limited resources. ORD officials also cited an additional need for air toxics data in attempting to understand how pollutants behave once released into the environment. Conclusions EPA is likely to rely on emissions data for gauging its progress in reducing health risks from airborne toxics for many years to come. However, the Agency faces considerable challenges in improving this measure. The most reliable approach would be to require State and local agencies to report validated emissions once every 3 years. EPA proposed this approach in 2000 but did not follow through on this plan due to concerns about adding more reporting requirements on States, as well as the belief that the 1990 Clean Air Act is not clear on air toxics reporting. However, in our opinion, GPRA is clear in its mandate that Federal agencies demonstrate they are achieving results, and GPRA also provides them with the authority to establish appropriate measures for gauging their progress. Nonetheless, if EPA does not wish to include State reporting requirements in a rulemaking, the Agency can achieve the same results by judicious use of its Section 105 grants to State and local agencies. Regarding rne other limitations discussed in this chapter, EPA currently does not have a systematic approach for improving air toxics emissions estimation techniques similar to the approach that the National Research Council recommends EPA use to improve its mobile model. Such an approach would allow EPA to identify the various uses of the inventory, and the level of accuracy needed for those uses. This will enable the Agency to gauge the effectiveness of the existing inventory. Further, EPA will be able to use this assessment to make future improvements to the inventory by identifying and improving the emissions estimates that have the largest impact on the accuracy of the inventory. We believe the Emissions Inventory Improvement Program has the organization, expertise, and funding to guide and oversee such improvements. 19 ------- While improvements to the inventory are important to strengthening EPA's air toxics GPRA performance measure, these improvements will also benefit EPA and the States in developing and implementing effective toxics emissions control strategies to protect the public from the dangers of air toxics. In addition to comparing reductions to the baseline inventory, the Agency should also compare emission totals between all other completed inventories, which would better enable EPA to verify the actual emissions reductions achieved. This revised approach would become more meaningful as future inventories are developed. Recommendations In order to obtain reliable air toxics emissions data once every 3 years, we recommend that the Assistant Administrator for Air and Radiation: 2-1. Improve the accuracy and reliability of the air toxics GPRA performance measure by providing increased funding and support for development of the NTI by: (a) requiring all authorized State and local agencies to report annually air toxics emissions data for stationary, area, and mobile sources; (b) developing air toxics emissions reporting policies, procedures, and guidance for facilitating standardized State and local agency reporting of air toxics emissions data. After the reporting requirements have been imposed, use Clean Air Act Section 105 grant funds to ensure that State and local agencies comply with the requirements; and (c) developing more accurate emissions factors and activity data for stationary, area, and mobile sources by either reviving the EIIP program and utilizing the organization and expertise of this program, or by establishing a similar program in-house. 2-2. Because of the difficulties encountered and resources involved in periodically attempting to ascertain what the actual air toxics emissions totals were for the 1993 NTI, EPA should use the 1996 NTI as the baseline year for measuring progress. 2-3. Supplement the existing measure of comparing air toxics emissions reductions to the 1993 baseline inventory by also comparing emission totals between all other completed inventories. Agency Comments and OIG Evaluation Office of Air and Radiation (OAR) and Office of Research and Development (ORD) officials provided detailed draft comments to our draft report and, where appropriate, we made revisions. Because these responses are draft and may not 20 ------- represent the Agency's final position on the issues and recommendations in this report, the draft responses are not included as appendices to this report. Nonetheless, we made changes to the draft report based on both sets of draft comments, as well as our exit conference discussions with Agency officials. Many of OAR's comments involved changing the 1993 and 1996 NTIs to reflect EPA's methods and assumptions used in developing the recently issued 1999 NTI. As shown in the report, we believe it is important to discuss the 1999 NTI and its impact on the prior two inventories' (1993 baseline and 1996 NTI) emission totals. However, due to EPA's delay in issuing the 1999 NTI, we were only able to evaluate the factors influencing the reliability of the 1993 and 1996 inventories and, without an in-depth examination of the factors impacting the reliability of the 1999 NTI similar to the efforts we undertook with regard to the 1993 and 1996 NTIs, we do not believe it would be appropriate to reflect EPA's most recent assumptions en masse. Instead, we have presented both sets of figures as appropriate in the report. OAR's detailed draft comments agreed with the recommendations in this report, with one exception. OAR intends to continue using the 1993 NTI as the baseline, instead of implementing our recommendation to use the more reliable 1996 NTI as the baseline for measuring progress under GPRA. Agency officials explained that if EPA were to use the 1996 NTI as the baseline, it would not enable them to capture the emissions reductions achieved by stationary and mobile source standards implemented between 1993 and 1996. While we recognize this point, we continue to believe-that, due to the unreliability of the 1993 NTI, EPA should use the 1996 NTI as the baseline. As noted in our report, emissions estimating techniques have undergone continuous change and improvement since the 1993 baseline was developed, and subsequent inventories have been amassed quite differently, making comparison to the 1993 NTI much less reliable as a measure of GPRA progress as compared to the 1996 NTI. For example, only 3 States provided data for the 1993 baseline, while 36 States provided data for the 1996 inventory. Also, as discussed in our report, significant uncertainties continue to hamper use of the 1993 NTI as a baseline. For example, these uncertainties caused EPA recently to revise the 1993 baseline from 6.1 million tons to 7.2 million tons, more than an 18 percent increase. These revised totals further substantiate our position that EPA is uncertain of its progress in reducing emissions and needs to improve its measures of air toxics progress. We agree that subsequent inventories have improved; however, EPA's limited resources may be better used in improving the 1996 NTI as a baseline in lieu of the 1993 NTI. We continue to believe that, until ambient air toxics monitoring data is available to replace emissions data, the best available baseline for measuring Agency progress for GPRA purposes is the 1996 NTI. With regard to ORD, officials of this office concurred with our draft report recommendations, stated that ihe draft report was reasonable, and provided several 21 ------- detailed comments intended to sharpen the quality and accuracy of the report. These changes were made. 22 ------- Chapter 3 GPRA Measure Needs to Address Risk of Air Toxics Exposure , EPA currently measures progress in its Air Toxics Program by reporting on tonnage reductions of emissions for all 188 toxic air pollutants collectively but does not account for the varying levels of toxicity. This aggregate GPRA measure, although useful, does not provide EPA, Congress, and the public with information on the specific subset of air toxics that are the most dangerous to human health. Agency officials said that they see merit in adding a risk component to their GPRA reporting, and they recently began exploring possibilities of doing so. This supplemental risk measure would better enable EPA to track its progress in reducing those air toxics that are most harmful to human health and, in so doing, more closely meet the intent of GPRA. EPA Needs to Supplement Emission Reduction Measure The emissions reduction measure is an important component of EPA's Air Toxics Program. In addition to improving data, as discussed in Chapter 2, the measure needs to be supplemented with information related to toxicity or risk. The emission reduction measure alone cannot ensure risk reduction occurs, because it does not specify which emissions are being reduced or if those reduced are in fact the most harmful to human health. If the Agency continues to only measure total emission reductions without adding a risk component to its GPRA measure, there is limited assurance that EPA is maximizing its efforts to reduce risk because air toxic health risk reductions are not proportionate to the tonnage of emission reductions. For example, toluene and xylene emissions represent about 30 percent (2 million tons) of the total toxics emissions (6.1 million tons) in the 1993 baseline. However, if EPA were to eliminate toluene and xylene entirely, it would not result in as much of a reduction in risk to human health as would a similar tonnage reduction in benzene and formaldehyde, which are more harmful air toxics. Risk-Based Information Already Exists Although much more needs to be learned about the health effects of various air toxics, we believe EPA has sufficient risk information to begin adding a risk-based component to its air toxics emissions reduction measure. To date, EPA has not included a risk element to its measure because Agency officials believe adequate risk data are not available. However, EPA has completed two efforts that should enable the Agency to stratify the 188 air toxics into several categories of risk and allow it to report on emission reductions for these specified subsets of air toxics. EPA's Integrated Urban Strategy and a study on mobile air toxics emissions 23 ------- illustrate how EPA has already stratified some of the more prevalent and hazardous air toxics. Integrated Urban Strategy The Clean Air Act directed EPA to identify at least 30 air toxics from emissions of area sources that represented the greatest threat to public health, along with actions the Agency is taking to reduce exposure. Further, the Act directs a reduction in public health risks of not less than 75 percent in the incidence of cancer attributable to area sources. In 1999, EPA identified 33 urban air toxics from major, area, and mobile sources. Some of EPA's emission estimation techniques included: evaluating the health effects of the 188 air toxics; assessing air quality monitoring data; reviewing existing studies; and producing a list of pollutants based on the relative hazards they pose in urban areas while considering toxicity, emissions, and related characteristics. EPA used three ranking methods to produce a final list of priority urban air toxics: * Risk Assessment/Hazard Ranking Studies - Fourteen urban studies were reviewed, and results were sorted by pollutant and ranked numerically. Cumulative Exposure Project - Modeled ambient concentrations of air toxics were compared to health benchmarks to estimate the level of toxic risk. Exposure/Toxicity Indicators Ranking - Seven indices were prepared for the 188 pollutants; each pollutant was numerically ranked within each index. EPA is beginning to make efforts to prioritize the list of 188 air toxics. Using the 1999 NTI, for each air toxic, EPA has estimated the tons of emissions reduced and then weighed each air toxic by health criteria. EPA officials stated that they plan to conduct this analysis every 3 years to coincide with new emissions inventories. Additionally, EPA plans to expand the National Air Toxics Assessment to include each of the 188 air toxics for which the Agency has established health criteria. EPA officials said utilizing these efforts, along with data collected from the air toxics monitoring network, will enable the Agency to develop a relative risk ranking of the most harmful air toxics. We believe that EPA could use this risk information to stratify the 188 air toxics into several categories of risk and report under GPRA on emission reductions for these specified subsets of air toxics, in addition to reporting aggregate tonnage reductions for all toxic air pollutants collectively. Mobile Air Toxics Effort EPA developed a list of air toxics emitted from mobile sources by searching through available databases and studies that speciated the emissions from motor vehicles and their fuels. By comparing the list of air toxics to the toxicity information available in EPA's Integrated Risk Information System, EPA was able to identify 21 air toxics that are possible, probable, or known human carcinogens. Thirteen of these 21 air toxics were also identified as part of the 33 air toxics studied in the Integrated Urban Strategy. Moreover, EPA routinely gathers more information on five mobile air 24 ------- toxics they consider the most dangerous: benzene; 1,3 butadiene; methyl tertiary butyl ether (MTBE); formaldehyde; and acetaldehyde. OMB Review of EPA's Air Toxics Program Cited Similar Problems As part of its efforts to review agencies' implementation of GPRA, the Office of Management and Budget (OMB) developed a rating tool6 to identify common performance challenges found in Federal agency programs. Using this newly created tool, in 2003 OMB evaluated EPA's Air Toxics Program. OMB found that the program's purpose was clear and the management of the program good, but the program had not been able to demonstrate that it was maximizing the program's net benefits. Furthermore, linkages were insufficient between annual performance goals and the long-term performance goal of protecting 95 percent of the United States population from unacceptable risks of cancer and other significant health problems from air toxic emissions. OMB's evaluation rated the areas of program results and accountability as "Results Not Demonstrated," and cited the following concerns with EPA's GPRA goals and measures: ....Although the long-term cancer reduction goal is clearly outcome-related, "unacceptable risks" are not defined, the relationship between emissions changes and actual health outcomes is not known because there are no efficiency measures there are inadequate linkages between annual performance and long-term goals that prevent it [EPA] from demonstrating its impact on human health OMB's observations regarding EPA's results measures are consistent with our observations, particularly as they relate to measuring reductions in air toxics risk to human health. In response to these findings, the Administration took measures to implement OMB's recommendations by requesting $7 million in increased funding for the Air Toxics Program. The funding was to be for State grants involving ambient air toxics monitoring that would help fill these data gaps. In addition, EPA indicated it would focus on maximizing the program's net benefits by minimizing the cost of each harmful health effect avoided, and establishing better performance measures. OMB plans to review EPA's Air Toxics Program every year until adequate results are demonstrated. According to Agency officials, it will be some years before EPA has a national ambient air toxics monitoring network capable of measuring air toxics in ambient air on a widespread basis. Until that time, the Agency could improve its focus on health risk by stratifying the 188 air toxics into several categories of risk for GPRA reporting. 6Known as the Program Assessment Rating Tool, this Tool was created to make ratings more consistent, objective, and credible. 25 ------- Conclusion EPA's mission is to protect public health, not just to reduce emissions. As the EPA OIG and OMB have identified, there presently is insufficient linkage between health risk and tons of emissions reduced. As such, we believe EPA needs to add a risk component to its air toxics measure that will enable the Agency to better demonstrate that it is managing for results, as called for under GPRA. Recommendation 3-1. We recommend that the Assistant Administrator for Air and Radiation supplement the existing tonnage reduction measure of progress for the EPA air toxics program with measures that address progress toward reductions in human exposure and health risk. . - Agency Comments and OIG Evaluation OAR officials commented that this chapter did not fully characterize the extent of the Agency's efforts to collect data for prioritizing hazardous air pollutants as it relates to risk. We incorporated OAR officials' comments as appropriate. However, because up to this time the Agency has not used this data to measure progress under GPRA by reporting on the varying levels of toxicity, we believe our recommendation is appropriate. ORD recommended changing the phrase "health risk component" to "reduction in human exposure and health risk" in Recommendation 3-1. We agreed, and have revised this recommendation accordingly. 26 ------- Appendix A Details on Scope and Methodology To assess the appropriateness of EPA's current air toxics GPRA performance measure for assessing progress towards achieving the air toxics objective, we reviewed documentation that pertained to GPRA measures and objectives, including: The Government Performance and Results Act (GPRA) of 1993 EPA's Strategic Plans for 1997,2000, and 2003 EPA's Fiscal 2003 Annual Plan EPA's Fiscal 2002 Annual Report OMB's Program Assessment Rating Tool Evaluation of EPA's Air Toxics Program To assess the accuracy and reliability of data supporting EPA's air toxics GPRA performance measure, we conducted interviews with EPA Regional, OAR, and ORD officials; researched documents; and attended training in support of these efforts. Budget information was obtained through the Office of the Chief Financial Officer and OAR's Office of Transportation and Air Quality. We reviewed Section 112 of the Clean Air Act to understand EPA's obligations to control and reduce air toxics emissions. To gain a better understanding of the history and development of the inventories, we reviewed EPA's documentation, which included: Development of the 1993 Baseline National Toxics Inventory Development of the 1996 National Toxics Inventory Development of the 1999 National Toxics Inventory Toxics Release Inventory requirements for reporting emissions Emissions efforts developed under MACT programs and rulemakings Both the models for on-road (MOBILE 5b and MOBILE 6) and the model for off-road mobile emissions (NONROAD). To determine the possibility of requiring State and local agencies to report emissions, we researched the Consolidated Emissions Reporting Rule. To understand EPA's guidance for implementing Quality Assurance/Quality Control procedures for data and EPA's effort to implement this guidance, we reviewed: EPA Order 5360.1 EPA's 1999 National Emission Inventory Preparation Plan EPA's Emissions Inventory Improvement Program EPA's Information Quality Guidelines The Challenge of Meeting New EPA Data Standards and Information Quality Guidelines in the Development of the 2002 NEI Point Source Data for HAPs 27 ------- We researched documentation regarding the Integrated Risk Information System to determine how this database is used in conjunction with the risk assessment phase of MACT standards. We also reviewed the National-Scale Air Toxics Assessment for 1996. Because the amount of emissions for the source category Other Mobile was minimal (17,000 tons), we did not include that category in the scope of our evaluation. Types of sources in this category include locomotive, aircraft, and commercial marine vessels. Prior Coverage General Accounting Office Major Management Challenges and Program Risks - EPA (GAO-03-112), January 2003 Few Agencies Reporting on the Completeness and Reliability of Performance Data (GAO-02-372), April 2002 EPA Should Improve Oversight of Emissions Reporting by Large Facilities (GAO-01-46), April 2001 EPA Faces Challenges in Developing Results-Oriented Performance Goals and Measures (GAO/RCED-00-77), April 2000 Air Pollution - Limitations of EPA's Motor Vehicle Emissions Model and Plans to Address Them (GAO/RCED-97-210), September 1997 EPA OIG EPA's Progress in Using the Government Performance and Results Act to Manage for Results (2001-B-000001), June 2001 Emission Factor Development (6100306), September 1996 28 i ------- Appendix B EPA's Recent Update to the 1993 and 1996 Inventories The three charts below show EPA's estimates of air toxics emissions by major sector for the 1993, 1996, and 1999 NTI as of February 2004. Updated Emissions for the 1993,1996 and 1999 NTIs (in mil lions of tons) Non-road Non-road Non-road On-i According to Agency officials, the use of updated models to estimate mobile source emissions resulted in a significant change in the mobile emissions total. EPA decided that both the 1993 baseline and the 1996 NTI needed to be revised to reflect these updated models, and as such, the inventory totals changed. For example, the 1993 baseline inventory increased from 6.1 million to 7.2 million tons. the 1996 inventory increased from 4.6 million to 5.0 million tons. EPA had not previously reported the air toxics emissions inventory totals for the 1999 inventory, which is now reported to be 5.1 million tons, or an increase of 500,000 tons above the previously reported 4.6 million tons for the 1996 inventory. The Agency's newly revised air toxics emissions inventory totals indicate a decrease in overall emissions, from 7.2 million tons in 1993 to 5.0 million tons in 1996. However, analysis of the new totals from the revised 1996 inventory (5.0 million) to the new 1999 inventory (5.1 million) still shows an increase of 100,000 tons of air toxics emissions from 1996 to 1999. These revised totals were provided in February 2004 after our field work was completed. We did not verify the revised figures or independently determine the reasons for the 29 ------- changes. These revised numbers were not reflected in recent Agency reports to the Congress and the public on EPA's progress in reducing air toxics emissions, including: EPA's FY 2003 Annual Report to the Congress and the President, which serves as the Agency's program performance report as required by GPRA, which reported a 1993 baseline of 6 million tons in November 2003; EPA's Latest Findings on National Air Quality: 2002 Status and Trends Report, which reported a 1993 baseline of 6.1 million tons and a 1996 inventory of 4.7 million tons in August 2003; and EPA's Draft Report on the Environment 2003: Technical Document, which reported a 1993 baseline of 6.11 million tons and a 1996 inventory of 4.67 million tons in June 2003. EPA officials told us that they plan to include the revised totals in next year's reports. 30 ------- Appendix C Distribution EPA Headquarters Assistant Administrator for Air and Radiation (6101 A) Assistant Administrator for Research and Development (8101R) Comptroller (2731 A) Agency Follow-up Official (2710A) Agency Follow-up Coordinator (2724A) Audit Follow-up Coordinator, Office of Air and Radiation (6102A) Audit Follow-up Coordinator, Office of Research and Development (8102A) Associate Administrator for Congressional and Intergovernmental Relations (1301A) Director, Office of Regional Operations (1108 A) Associate Administrator for Public Affairs (1101 A) Director, Office of Air Quality Planning and Standards (C404-04) Deputy Director, Office of Air Quality Planning and Standards (C404-04) Director, Emissions Standards Division (C504-03) Acting Director, Emissions, Monitoring and Analysis Division (C304-02) Director, Office of Transportation and Air Quality (6401 A) Deputy Director, Office of Transportation and Air Quality (6401 A) Director, National Exposure Research Laboratory (MD-75) Leader, Emission Inventory & Factors Group (C304-02) National Exposure Research Laboratory Audit Liaison (MD-343-01) Audit Liaison, Office of Air Quality Planning and Standards (C404-2) Audit Liaison, Office of Research and Development (8102R) Inspector General (2410) EPA Regions Regional Air Program Directors 31 ------- 19nnD 120PA Pennsylvania Avenue NW Washington, DC 20460 202-56&-0556 f ^ « 4 ------- |