OFFICE OF INSPECTOR GENERAL Evaluation Report EPA Can Improve Emissions Factors Development and Management Report No. 2006-P-00017 March 22, 2006 ------- Report Contributors: Patrick Milligan Frank Martinsky Kevin Good Bill Nelson Abbreviations CAFO Concentrated Animal Feeding Operation CO Carbon Monoxide EFPAG Emissions Factors and Policy Applications Group (within EPA) EPA Environmental Protection Agency FIRE Factor Information Retrieval FMFIA Federal Managers' Financial Integrity Act GAO Government Accountability Office kg kilograms MACT Maximum Achievable Control Technology NEI National Emissions Inventory PEI Probabilistic Emissions Inventory NOx Nitrogen Oxides OAQPS Office of Air Quality Planning and Standards OIG Office of Inspector General PMa.5 Fine Particulate Matter VOC Volatile Organic Compound ------- f'Stf'- U.S. Environmental Protection Agency Office of Inspector General At a Glance 2006-P-00017 March 22, 2006 C#fafyiirfttr fmptwhtg Why We Did This Review Wt: sought to determine whether the air emissions factors used by the Environmental Protection Agency (EPA) are of acceptable quality for making key environmental decisions, an<3 whether EPA's process foj developing, improving, and rating emissions factors is sufficient to meet users' nesds. Background Emissions factors are broad es.imates of the emissions generated from a source, such as a factory. Nationally, emissions factors are used for about 80 percent of emissions reporting. An emissions factor is a representative v* lue that attempts to relate the quantity of a pollutant released with an activity rate associated witfa the release. Emissions factors underlie marty environmental decisions. Recently, States and industry have been developing emissions factors attd submitting them to EPA. For further trtfiuinatior*, contact our Office of Congressional and Public liaison at (202) 566-2391. T > view the full report, click on the following link: www.epa.aov/oig/reports/20C6 /;0060322-2006-P-Q0017.pdf EPA Can Improve Emissions Factors Development and Management What We Found EPA has made progress in emissions factors development since our review of the program in 1996, but a large number of factors continue to be rated low. The number of EPA-rated factors increased by nearly 94 percent, from 8,838 in 1996 to 17,110 in 2004. However, the percentage of emissions factors rated below average or poor increased from 56 percent in 1996 to 62 percent in 2004. Emissions factors, intended for use in developing emissions inventories, have been inappropriately used for key environmental decisions beyond their intended use. For example, emissions factors have been used for non-inventory purposes, such as setting permit limits and reporting the level of air pollution control at specific facilities. For three industry sectors EPA examined, inappropriate use of emissions factors contributed to more than one million tons of pollutants not being controlled. Demand for emissions factors is increasing, and will continue for a broad array of environmental decisions, including measuring and reporting environmental progress. This pertains not only to existing factors but to those that still need to be developed, especially emissions factors for sources of fine particulate matter. If EPA can improve the quality of its factors, this should improve environmental decision-making for reducing air pollution. Improving the quality of emissions factors is an extremely challenging task that may take EPA years to address. The quality of many emissions factors remains low in part because EPA did not have a sufficient process for developing, improving, and rating emissions factors, nor did EPA have a comprehensive strategic plan. We found inconsistent emissions factors guidance, continuing reliance on a qualitative rating system when a quantitative range of uncertainty is needed, and insufficient program funding when needs are increasing. What We Recommend We are making a number of recommendations to EPA to, among other things, develop emissions factors guidance that addresses the development and appropriate use of emissions factors for non-inventory purposes; establish a rating system that provides the quantitative range of uncertainty for emissions factors for both inventory and non-inventory purposes; work with industry, State and local agencies, and others to leverage available resources for meeting increasing demands for new factors; and establish a workgroup to develop a comprehensive strategic plan for the Emissions Factors Program, and ensure that requested resources are used to achieve program goals. In response to the draft report, the Agency stated that our recommendations generally align with its current improvement efforts. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 OFFICE OF INSPECTOR GENERAL MEMORANDUM SUBJECT: FROM: TO: March 22, 2006 EPA Can Improve Emissions Factors Development and Management Report No. 2006-P-00017 J. Rick Beusse /s/ Director for Program Evaluation, Air Issues William L. Wehrum Acting Assistant Administrator for Air and Radiation This memorandum transmits the results of an Office of Inspector General (OIG) evaluation of emissions factors development and management. This report contains findings that describe how the U.S. Environmental Protection Agency (EPA) can improve emissions factors development and management, as well as corrective actions the OIG recommends. This report represents the opinion of the OIG and the findings contained in this report do not necessarily represent the final EPA position. Final determinations on matters in the report will be made by EPA managers in accordance with established procedures. Action Required In accordance with EPA Manual 2750, as the action official, you are required to provide a written response within 90 days of the final report date. The response should address all recommendations. For the corrective actions planned but not completed by the response date, please describe the actions that are ongoing and provide a timetable for completion. Where you disagree with a recommendation, please provide alternative actions for addressing the findings reported. We appreciate the efforts of EPA officials and staff in working with us to develop this report. If you or your staff have any questions regarding this report, please contact me at 919-541-5747 or Pat Milligan at 215-814-2326. ------- Table of Contents At a Glance Chapters Introduction Purpose Background Scope and Methodology. 1 1 6 Use of Unreliable Emissions Factors Adversely Impacts Key Environmental Decisions EPA Rated More Factors But Quality of Many Factors Remains Low.. Misuse of Emissions Factors Resulted in Significant Unidentified and Uncontrolled Emissions Increasing Demand for New Emissions Factors Illustrates Continued Importance of Emissions Factors Program Conclusion 3 EPA's Management of Emissions Factors Program Needs Improvement Conflicting Guidance Issued for Emissions Factors Rating System Does Not Define Appropriate Uses Inadequate Funding Provided and Used Recent EPA Efforts Made to Revamp Emissions Factors Program Comprehensive Plan Needed to Improve Data Collection and Set Priorities Conclusions Recommendations Agency Comments and OIG Evaluation 10 13 14 15 15 17 19 21 22 24 25 26 A Details on Scope and Methodology B Details on Concentrated Animal Feeding Operations Air Emissions. C Agency Response to Draft Report D OIG Evaluation of Agency Response E Distribution 28 31 32 35 37 ------- Chapter 1 Introduction Purpose Quantifying air emissions is a vital aspect of air pollution programs. Regulatory authorities and others use emissions values in: (1) developing emissions inventories, (2) identifying and evaluating control strategies, (3) determining applicability of permit and regulatory requirements, and (4) assessing risks. Emissions factors are broad estimates of the emissions generated from a source, such as a factory. These factors are the most commonly used estimate for establishing emission values, and are used nationally for about 80 percent of emissions reporting. The Environmental Protection Agency (EPA) has long recognized the importance of emissions factors, particularly for developing emissions inventories. Over the last 10 years, permitting authorities, source owners and operators, and a few Agency programs have begun using emissions factors for purposes other than generating a national emissions inventory. For example, emissions factors have been used to develop emissions control strategies, determine applicability of permitting and regulatory requirements, establish permit limits, ascertain the effects of sources, and develop emissions reduction strategies. Therefore, the impact of the quality of the factors is far greater than it would be if they were used only for the inventory. Because emissions factors underlie so many environmental decisions, the objectives of our evaluation were to determine whether: • Emissions factors are of an acceptable quality for use in key environmental decisions made by EPA and State and local agencies; and • The Agency's process for developing, improving, and rating emissions factors is sufficient to meet key users' needs. Background In the 1960's and early 1970's, emissions estimation methods came about due to the need to estimate air pollution emissions. EPA first developed emissions factors from source test data used to develop new emissions standards in the 1970's. At that time, factors were mostly used to develop emissions inventories. In the 1980's and 1990's, EPA expenditures on source testing declined, yet during the same time period the demand for emissions data expanded as emissions factors were increasingly used for ------- non-inventory decisions. As noted in our prior report, EPA's emissions factor program resources were surpassed by the need for more and improved emissions data.1 Recently, EPA reorganized the Agency's emissions factors program in an effort to address this challenge. What Are the Ways of Obtaining Air Emissions Information? Generally, air emissions information can be obtained through direct measures of emissions or by estimating emissions. Under ideal circumstances, all emissions data users would derive values from ongoing emissions testing, continuous emissions monitoring, or frequent calculation using well-accepted engineering principles. However, these methods are time and resource intensive. Continuous Emissions Monitoring and Source Testing represent ways to directly measure emissions, while the others involve estimating. The five basic means of obtaining emissions information are discussed below. • Continuous Emissions Monitoring: This involves continuously measuring pollutants emitted into the atmosphere from a single source, such as a smokestack, by placing a monitor at the source. This is one of the most reliable methods for measuring emissions, but has annual costs ranging from about $ 10,000 to $50,000 and is only required at the largest sources of air pollution. • Source Testing: Like Continuous Emissions Monitoring, source testing data are generated by placing a monitor at a source, but in this case only measures for a limited number of hours. The facility uses the monitoring data to calculate an annual emissions total. This is also generally more reliable than emissions factors but has annual costs of $20,000 for conducting a test every 5 years. While some facilities are required to periodically conduct source tests, many are not. • Material Balance: For some sources with Volatile Organic Compound (VOC) emissions, material balance (also known as mass balance) assumes that a percentage of the materials used in a process will evaporate to become air emissions. Therefore, the amount of emissions resulting from evaporation is based on the amount of evaporative material used. Recordkeeping to calculate material balance costs about $2,000 to $10,000 per year, per process. • Emissions Calculating Tools: These estimating methods represent a more advanced and complicated use of emissions 1 Emission Factor Development, EPA Office of Inspector General (OIG) Report No. 6100306, September 30, 1996. ------- factors and other data, and include models, databases, and other pollutant estimating software. • Emissions Factors: An emissions factor relates the emissions amount of a pollutant with an activity rate associated with its release. Emissions factors are essentially averages of available data from various sources, such as source tests. The cost for using an emissions factor depends on the time to locate the appropriate factor. The ability to rapidly locate emissions factors makes this one of the least expensive methods for estimating emissions. The general equation for emissions estimates is: Activity Rate x Emissions Factor = Emissions The following example illustrates one way that an emissions factor is used. Burning distillate oil in industrial boilers produces carbon monoxide (CO) and other emissions. The CO emissions factor for this process is 0.6 kilograms (kg) of CO emitted per 1,000 liters of oil burned. Assuming the boiler burns 90,000 liters of oil per day, the following calculation shows the estimated amount of CO emitted per day: 90,000 liters per day x 0.6 kg CO/1,000 liters = 54 kg ofCO/day In the absence of direct measures, emissions factors are frequently used as a quick, low cost way to estimate emissions. How Are Emissions Factors Developed? EPA's Emissions Factors and Policy Applications Group (EFPAG) oversees the Emissions Factors Program. EFPAG is part of the Office of Air Quality Planning and Standards (OAQPS) within EPA's Office of Air and Radiation. EPA issues the guidance for developing the factors, and initially was responsible for developing many of the factors. However, industry and States increasingly have been developing emissions factors and submitting them to EPA for inclusion in AP-42.2 EPA guidance states that the following five steps are to be carried out to develop emissions factors: 1. Data Collection - For a particular industry sector, EPA collects information related to facility process descriptions and source emissions test data, if known to exist and available. Sources of this information include existing data in EPA databases and 2 The Compilation of Air Pollutant Emission Factors, commonly referred to as the "AP-42 series" of emissions factors, is the primary guidance and source of rated emissions factors used by EPA. ------- 2. external requests to Federal, State, and local agencies and industry trade organizations. Data Evaluation - The data are evaluated to determine which data should be used to develop the emissions factor. The data selected are known as "test data" and are given test quality ratings. 3. Data Classification - The data are then grouped into clusters of similar processes, which will eventually be averaged into an emissions factor. 4. Data Calculation - EPA averages the groups and develops a numerical emissions factor. The factor is given an overall qualitative rating of "A" (excellent) through "E" (poor) based on the ratings assigned to the test data and production quality. 5. Published Emissions Factor - The group of processes represented by the emissions factor is assigned an existing or new source classification code, and the emissions factor is published in AP-42. How Are Emissions Factors Used? Emissions factors are used to develop the emissions data that are the cornerstone of a host of important environmental decisions made by EPA; State, local, and Tribal agencies; industries; environmental groups; and others. These decisions include facility permitting, development of control strategies, and compliance and enforcement decisions. According to a 2001 Government Accountability Office (GAO) report,3 EPA's data show that, nationally, emissions factors are used for about 80 percent of emissions determinations. Emissions factors data are also used to measure environmental progress and demonstrate program results under the Government Performance and Results Act of 1993.4 Table 1.1 provides some of the key uses of emissions factors. 3 April 200! GAO Report, EPA Should Improve Oversight of Emissions Reporting by Large Facilities (GAO-01-46) 4 The Government Performance and Results Act holds Federal agencies accountable for measuring program outcomes and reporting results annually to Congress and the public. ------- Table 1.1: Key Uses of Emissions Factors Emissions Factors Uses Determining Source Classification Establishing and Enforcing Permit Limits Permit Fees Issuing Maximum Achievable Control Technology Standards Plant-wide Applicability Limit and Emissions Trading Measuring Environmental Progress Description EPA classifies a facility to be a minor or major source by determining the facility's potential to emit; this impacts the level of air pollution control equipment needed. Based on emissions measures or estimates, EPA and States set emissions limits via operating permits. Using the emissions estimates, States calculate annual fees a facility must pay for the emissions released. EPA writes standards requiring major sources of hazardous air pollutants to install the best control technology available for an industry sector. Both of these regulatory approaches measure emissions from a more holistic viewpoint. For plant-wide applicability limits, some point sources that increase in emissions are offset by corresponding decreases from other point sources at the same facility. This same type of offsetting method is used when facilities trade emissions credits to other facilities emitting over the limit. The amount of emissions reduced remains a key measure of environmental progress. One of the most important databases for tracking the amount or emissions reduced is EPA's National Emissions Inventory (NEI). Emissions factors play a pivotal role in many decisions, especially in measuring environmental progress. For example, every 3 years, EPA prepares a national database of air emissions information - the NEI - based on emissions factors and other input from States, industry, and other stakeholders. This inventory depends heavily on emissions factors for stationary sources (such as factories) and mobile sources (such as trucks and automobiles). The NEI database contains information on sources that emit criteria air pollutants (six common air pollutants that harm human health and the environment for which specific standards are set), as well as hazardous air pollutants, also known as air toxics. Key uses of the inventory include: • Conducting air dispersion modeling and analysis; • Developing control strategies to reduce pollution levels; • Issuing air regulations; • Performing risk assessments; • Screening sources for compliance investigations; • Tracking short- and long-term trends in emissions; and • Measuring program results in EPA's annual performance plan. When an area of the country does not meet the National Ambient Air Quality Standards, EPA and the States gather information about the ------- emissions rates of various sources in that area (often derived from emissions factors) to target important sources for control. Through its annual Trends Report, EPA uses the emissions inventory data to gauge progress in meeting its goals of reducing air pollution. Emissions factors play a key role in assessing such annual progress. How Are Emissions Factors Maintained and Accessed? In the AP-42 series of emissions factors, EPA assigns a qualitative rating to the emissions factor based on the quantity and quality of the data used to develop the factor. The AP42 series is EPA's recommended source of air pollutant emissions factors for both criteria and hazardous air pollutant emissions, and contains over 17,000 rated emissions factors for more than 200 air pollution source categories. The Factor Information Retrieval (FIRE) Data System is a database containing both rated and unrated emissions factors. FIRE contains all AP-42-rated factors, as well as approximately 4,400 unrated emissions factors that EPA recommends for use. FIRE also contains a list of source classification codes, and information about industries' operating systems, processes, and chemicals emitted. FIRE incorporates new or revised emissions factors from AP-42. The Clearinghouse for Inventories and Emissions Factors is EPA's electronic repository of the most up-to-date information on inventories and emissions factors, including AP-42 and FIRE. The clearinghouse facilitates the exchange of emissions factors and information on emissions inventories between Federal, State, and local agencies; industry; private citizens; universities; contractors; and foreign governments. The clearinghouse also provides historical inventory information and emissions estimation guidance. Scope and Methodology To assess the adequacy of EPA's Emissions Factors Program, we reviewed documentation related to AP-42 emissions factors, including: the development and prioritization of emissions factors, the process used for rating emissions factors, key uses of emissions factors, and policies and procedures for implementing the Emissions Factors Program. We also conducted interviews with officials from EPA at both the Headquarters and regional levels; officials from several States and an air pollution association, and emissions factors experts. We conducted our field work from March 2005 to October 2005, in accordance with Government Auditing Standards, issued by the Comptroller General of the United States. ------- As part of our evaluation, we considered the results of a prior EPA O1G report, Emission Factor Development (Report No. 6100306), dated September 30,1996. Additional details on our scope and methodology are in Appendix A. ------- Use of Unreliable Emissions Factors Adversely Impacts Key Environmental Decisions EPA has made progress in emissions factors development since our 1996 review, primarily in the increased number of factors receiving ratings on quality. However, the large percentage of low quality factors has not diminished. EPA officials have identified the inappropriate use of emissions factors for key environmental decisions, such as permit limits and the level of air pollution control equipment installed at specific facilities, resulting in the release of significant amounts of unidentified and uncontrolled emissions. For example, according to EPA enforcement records, three industries - petroleum refineries, wood products, and ethanol production - operated with insufficient control equipment primarily because emissions limits were significantly underestimated due to the emissions factors used. EPA, through separate enforcement actions, required companies in these industries to install additional emissions controls, resulting in the combined reduction of over one million tons of pollutants. Absent EPA intervention through enforcement actions or stringent guidance, industries will have little incentive to ensure that the emissions factors used are of known and acceptable quality. This pertains not only to existing factors but to ones that still need to be developed, especially quality factors for sources of fine Particulate Matter. For example, factors will be used by EPA and States to develop fine Particulate Matter control strategies. As EPA continues to measure and report progress in reducing harmful air pollutants, it needs to improve the quality of emissions factors estimates. This will improve environmental decision-making related to air pollution and human health. EPA Rated More Factors But Quality of Many Factors Remains Low In our September 1996 OIG report, we noted that emissions factors were unavailable for many sources of air pollution and, when available, many were unreliable. This was attributed to significant funding cuts that materially affected EPA's ability to meet an increased demand for quality emissions factors. Table 2.1 shows that, as of March 1996, EPA rated 56 percent of the emissions factors as either below average or poor (24 percent plus 32 percent, respectively). As of September 2004, that number had increased to 62 percent (28 percent plus 34 percent, respectively). EPA based its ratings on test methods, quantity of data, and ------- whether the facilities tested represented a sufficient random sample of the industry. Table 2.1: Comparison of Emissions Factors Ratings (March 1996 and September 2004) Factor : Rating A B C D E Qualitative ; Description Excellent Above Average Average Below Average Poor Total As of March 1996 Number of Factors 1.270 1,190 1,513 2,077 2,788 8,838 Percent of Total 14% 13% 17% 24% 32% As of September 2004 Number of Factors 2,135 1,829 2,619 4,740 5,787 17,110 Percent of Total 12% 11% 15% 28% 34% Our 1996 OIG report emphasized the need for EPA to invest more resources in the emissions factors program and to develop alternative approaches to factor development by increasing industry and State involvement. The Agency did increase its emissions factors efforts, as evidenced by the increased number of factors it had rated since the 1996 OIG review. Since March 1996, EPA nearly doubled the number of rated emissions factors, from 8,838 to 17,110. As of September 2004, 4,409 emissions factors, or about 20 percent of the total of 21,519, were still listed as unrated. Overall, the number of rated emissions factors has increased, which reflects some improvement in the quality of emissions factors. For example, emissions factors that previously had a lower rating received a higher rating, and some factors previously not rated have been rated. Additionally, since our 1996 report the Agency has taken the following steps to improve the Emissions Factors Program: • Surveyed emissions factor users, including States and industries, on their emissions factors needs. • Initiated the development of an automated database to store test data results for future emissions factors development. • Worked with an industrial association to develop new emissions factors. • Began a pilot project to adjust emissions factors for non-inventory use. An EFPAG pilot project is assessing two industry sectors - hot mix asphalt plants, and pulp and paper plants. The goal of the pilot is to evaluate the impact of adjusting excellent- and above average-rated emissions factors for uncertainty. Emissions factors are used widely for hot mix asphalt plants by source owners and regulatory authorities, ------- particularly in calculating production limits below thresholds that would trigger aspects of the air operating permits program. Pulp and paper plants generally rely on emissions factors to calculate fugitive hazardous air pollutants and VOCs from these plants. In some instances, these emissions can account for 60 to 80 percent of a site's emissions. EFPAG's study found that if emissions factors were adjusted to better account for uncertainty, almost all of the 3,600 hot mix asphalt plants in the United States likely would have to recalculate production limits. As a result, some plants would be unable to retain their synthetic minor source status, thus making them subject to the Title V air operating permits program and, as a result, potentially subject to stricter regulations and State or local air toxics rules. EFPAG also found the use of adjusted emissions factors may cause some revisions to State Implementation Plan model projections, causing some plants to reduce emissions below current levels. Misuse of Emissions Factors Resulted in Significant Unidentified and Uncontrolled Emissions To date, EFPAG's pilot study has identified the potential emissions impact and possible regulatory consequences of using emissions factors to estimate emissions. There are several instances where the actual emissions impact from misuse of emissions factors has been identified, and this impact has been substantial. For example, we examined three industries where EPA officials indicated emissions factors were not acceptable for the decisions being made - petroleum refineries, wood products, and ethanol production. For these three large industries, EPA had indicated emissions were significantly underestimated due to the emissions tactors used. As a result of pollutants not being previously identified or controlled, EPA, through separate enforcement actions, had required companies to install additional emissions controls, resulting in the reduction of over one million tons of pollutants for the three industries combined. According to EPA enforcement records, for years the three industries operated with insufficient control equipment because the emissions limits in permits and the annual emissions reported by the individual industry facilities significantly understated the actual amount of emissions released into the atmosphere. EPA decided to examine the reliability of the emissions factors measures for each industry sector and, using new, more accurate measures, found actual emissions to be much higher. Table 2.2 illustrates emissions reductions, civil penalties, and control investments by each industry. 10 ------- Table 2.2: Summary of Air Violations for Three Industries Related to Emissions Factors Use Industry Petroleum Refineries Wood Products Ethanol Production Total Companies 57 5 14 **76 Sites 57 *21 91 169 Civil Penalties (millions) $45.0 $289.0 $6.6 $340.6 Control Investment (millions) $2,600.0 * $175.0 $240.5 $3,015.5 Pollutants Removed (tons) 765,000 177,000 116,750 1,058,750 " This Information available for only three wood products companies. " These companies represent a portion of the Industry, not the entire Industry. Through consent agreements reached with facilities in these sectors, EPA and the States required more stringent permits and installation of additional pollution controls to lower emissions. EPA staff indicated the problems noted for these three industries regarding poor quality emissions factors are occurring in many other industries. For the three industries, emissions estimating techniques were improved as a result of EPA enforcement actions. However, correcting such problems through enforcement actions is both costly and time-consuming. Increased scrutiny of the use of emissions factors is needed if expected environmental benefits are to be realized. Details on each of the three industries follow. Petroleum Refineries Air quality problems in the Houston-Galveston area demonstrate the serious consequences in terms of cost and effectiveness of air pollution control strategies related to the petroleum industry. Because EPA declared the Houston-Galveston area in severe noncompliance with Federal air quality standards for ozone,5 the metropolitan area was required to develop a control strategy to reduce Nitrogen Oxides (NOx) and VOC emissions. Based on modeled ozone predictions using emissions inventory data, Texas devised a strategy to reduce ozone precursor emissions that called for the reduction of NOx emissions by 90 percent. In August 2000, Texas conducted a comprehensive research project assessing source contributions to the State's air quality problems. Based on the comparison of ambient measurements of VOC concentrations to the reported emissions inventory estimates, the Texas 2000 Air Quality Study found that VOC emissions from petroleum refineries were significantly under reported in the emissions inventory. This primarily involved under 5 Ground-level ozone is not emitted directly into the air, but is created by chemical reactions between NOx and VOC in the presence of heat and sunlight. 11 ------- reporting of emissions from flares,6 process vents, and cooling towers, as well as from fugitive emissions (leaks). The under-reporting was caused largely due to the use of poor quality emissions factors. The quality of one emissions factor was so poor that for short durations, actual emissions from the flaring process could be as much as 50 times higher than emissions calculated using the emissions factor. This was because the factor did not account for variables such as the composition of the fuel being burned and the efficiency of the flaring equipment. This emissions factor was developed in the early 1980's and had not been updated at the time of the 2000 Texas review. Texas revised its emissions estimates based on its 2000 review, using improved emissions estimating techniques and new modeling. The revised estimates showed that additional VOC reductions would be needed to meet national air quality standards, while required NOx reductions under the State Implementation Plan could be eased from 90 to 80 percent. As a result of what happened in Houston-Galveston, the regional planning authority in the Philadelphia area decided to more closely study refinery VOC emissions. The study disclosed many of the same problems noted for the Houston-Galveston area, and recommended that Texas' guidelines be used when estimating emissions. The California Bay Area Air Quality Management District also closely examined emissions estimates, found similar results, and issued a new rule to obtain more accurate emissions estimates to ultimately reduce emissions. Wood Products7 In 1988, EPA began investigating a suspected nationwide pattern of Clean Air Act violations by a prominent wood products company. EPA found that the company had failed to obtain required permits for new construction and other modifications at some of its facilities. The company had used a poor quality emissions factor for estimating VOC emissions and, as a result, claimed it was not subject to the permitting requirements. This emissions factor underestimated VOC emissions because it was derived from a test method that substantially understated VOC emissions containing oxygen. The industry subsequently developed a new emissions factor that adequately accounted for VOC emissions containing oxygen. Flaring is an engineering practice that provides for process equipment to immediately release gases to a device (a flare) where they can be quickly and safely incinerated. The proper use of flaring is a good engineering practice because flares can prevent damages, fires, explosions, and injuries to employees. 7 The wood products industry sector includes manufacturers of plywood, panelboard, medium density fiberboard, and oriented strand board. The drying and pressing processes involved in this sector are a large source of emissions. 12 ------- EPA used this information as the basis for successfully negotiating the largest Clean Air Act civil penalty ever collected by EPA (up to that time), and the second largest under any environmental statute. Under the terms of a consent decree, the company was required to pay $11.1 million in civil penalties and install state-of-the-art pollution control equipment valued at $70 million. In the 1990's, EPA reached consent agreements with three other wood product companies after identifying similar violations. In addition to civil penalties, the consent agreements instructed the companies to install the Best Available Control Technology to reduce VOC emissions, conduct compliance audits, and obtain appropriate permits. Ethanol Production In 2002, based on recent success in the wood products industry, EPA began investigating Clean Air Act violations by ethanol production companies. These companies use corn to manufacture ethanol for blending with automobile fuel, and during processing burn off gases that emit VOCs into the atmosphere. The ethanol emissions factor was developed using the same test method as wood products and again underestimated the amount of VOCs being emitted into the atmosphere. EPA alleged that these companies knowingly used a faulty emissions factor for permitting. In October 2002, EPA announced consent agreements with 12 ethanol plants to install air pollution control equipment. EPA estimated that VOC emissions will be reduced by 2,400 to 4,000 tons per year and CO by 2,000 tons per year. The settlement also wilt result in estimated annual reductions of NOx by 180 tons, Paniculate Matter by 450 tons, and hazardous air pollutants by 250 tons. Increasing Demand for New Emissions Factors Illustrates Continued Importance of Emissions Factors Program As EPA and the States move forward with efforts to identify and regulate sources emitting excess levels of air pollution, there will be increased demand for new emissions factors, especially for sources of fine Paniculate Matter (PM2.5). By April 2008, EPA and States will need to identify sources of PM2.s, determine the amount of emissions from these sources, and ensure that sufficient emissions control equipment is installed at sources located in non-attainment areas. In 1997, EPA established the PM2.5 standard and in December 2004 designated areas as being in nonattainment with the standard. In April 2005, these designations became effective for 208 U.S. counties impacting a total population of 88 million people. By April 2008, 13 ------- States are required to submit a plan that identifies the sources of PM2.5, how much PMz.s each source emits, and the actions planned for adequately regulating these sources. The quality of emissions factors will directly impact the effectiveness of the plans. An implementation plan must show how an area in nonattainment will reduce emissions to meet the standards as soon as possible, but no later than 2015, and include supporting technical analyses based on emissions factor-developed emissions estimates. The Concentrated Animal Feeding Operations industry provides another example of an area that currently does not have factors but needs them. Appendix B provides details on EPA's efforts to address emissions factors in this area. Conclusion Although EPA has made progress in emissions factors development since our 1996 review, the need for better quality emissions factors has outpaced the Agency's efforts to improve existing factors and develop new ones. EPA's use of poor quality emissions factors information has hampered environmental decisions, resulting in more than one million tons of uncontrolled emissions spanning years, and an increased risk of adverse health effects. This also places a disproportionate emissions reduction burden on those facilities that use good quality emissions factors. Although our evaluation did not address the cost impacts of rectifying the inappropriate use of emissions factors, we believe these costs are substantial to both EPA and the States. For each of the three industries we reviewed (petroleum refineries, wood products, and ethanol production), there are, at a minimum, the following costs for addressing each industry: rewriting permits and determining the proper emissions limits, issuing regulations to require increased air pollution controls, and the legal actions (settlement or litigation) necessary to ensure industry complies with the new regulations. The three industries represent a very small portion of the universe of emissions estimates derived through the use of emissions factors. More effort in examining other key emissions factors may identify significant amounts of additional unregulated pollutants. Incomplete or unreliable emissions information can have serious consequences in terms of the effectiveness and cost of air pollution control strategies. The public will have little confidence in either the success or equity of EPA's decisions if those decisions are based on questionable emissions data. 14 ------- Chapter 3 jnteftt of-Efilis Program Needs Improvement EPA did not have a sufficient process for developing, improving, and rating emissions factors. While the Agency has taken various steps to improve the Emissions Factors Program, we noted deficiencies in four key areas that resulted in the use of poor and low quality emissions factors: • Lack of consistent emissions factors guidance. • Continued use of a rating system that only provides subjective information. • Insufficient funding for the program. • Not having a clear strategic plan. Given the vast number of emissions sources using factors and the differing uses of emissions factors for making significant environmental decisions, addressing these deficiencies will require a long-term, multi-year, coordinated effort among EPA, State and local agencies, industry, and others. EPA will need to assert its leadership to ensure that, in the future, emissions factors are only allowed to be used in accordance with yet to be issued EPA guidance on their proper use. In fiscal year 2003, OAQPS began a reevaluation of the Emissions Factors Program to standardize and streamline the emissions data collection and reporting process, establish procedures for defining data uncertainty and using emissions factors in non-inventory applications, and establish an outreach program to communicate changes to emissions factors stakeholders. However, to make the needed improvements to the Emissions Factors Program, EPA will need increased focus and direction, including fully developed goals and objectives. Conflicting Guidance Issued for Emissions Factors While introductory text to AP-42 states that emissions factors may be appropriate for situations such as making source-specific emissions estimates for area-wide emissions inventories, the text does not recommend emissions factor use except for inventory purposes. However, the text acknowledges that emissions factors may be used for site-specific purposes as a last resort (emphasis added) provided appropriate caveats concerning their limitations are in place. We noted three occasions where EPA has issued guidance on the use of emissions factors for source- specific purposes that conflicted with the intent of AP-42 emissions 15 ------- guidance - that is, that emissions factors not be used at individual sources. Caveats on emissions factors use in these programs varied and may have resulted in data being used for purposes outside EPA's intended use of emissions factors. For example: • Even though AP-42 specifically states that the use of emissions factors for site-specific permit limits is not recommended, EPA issued air permit guidance documents approving the use of emissions factors to set permit limits at individual facilities. In response to concerns that Title V operating permits were too costly and burdensome to implement, in July 1995, EPA issued guidance, White Paper for Streamlined Development of Part 70 Permit Applications, stating that facilities could use emissions factors estimates to determine emissions limits in the permit applications. Similarly, in August 2000, EPA issued draft guidance that provided flexibility and allowed State and local permitting agencies to use site-specific emissions factors as well as other relevant emissions factors. EPA eventually rescinded the 2000 guidance but plans to re-introduce parts of it. • EPA's New Source Review Program applies to sources undertaking major plant modifications, and involves determining the source's potential to emit. The New Source Review Workshop allows the use of AP-42 emissions factors in estimating emissions to determine a facility's potential to emit. EPA's flexible permitting approach for New Source Review, known as the Plant-wide Applicability Limitation, provides greater flexibility by allowing some emissions points within a facility to increase emissions provided the overall emissions remain below the plant-wide limit. Current rules for permit limitations provide wide latitude in allowing the use of emissions factors. • EPA allowed the use of emissions factors in a recent reconsideration of a Maximum Achievable Control Technology (MACT) standard,8 even though AP-42 specifies that emissions factors should not be used for such source-specific purposes. MACT standards specify the emissions control standards that must be achieved by an affected industry. However, if the emissions factor understates actual emissions, the facility may be required to install controls that do not effectively reduce emissions to an acceptable level; conversely, an overstatement may cause the industry to unnecessarily and unfairly be required to install more costly controls. In a July 2005 reconsideration of a rule, EPA made monitoring requirements less stringent by allowing the use of emissions factors. However, almost all of the s July 30, 2004 Environmental Protection Agency, 40 CFR Part 63: [OAR-2003-0048; FRL-XXXX- X][RIN 2060-AM78] National Emission Standards for Hazardous Air Pollutants: Plywood and Composite Wood Products; Reconsideration of original rule dated July 29, 2005. 16 ------- 66 emissions factors EPA allowed for this MACT rulemaking were either unrated or received a rating of poor or below average. EFPAG officials said that the currently planned guidance for non- inventory uses of emissions factors should address the more appropriate applications of emissions factors in these programs. Rating System Does Not Define Appropriate Uses The current rating system for emissions factors does not provide the user with a tool to adjust the emissions factor based on use. This system provides a subjective A through E rating system that is of minimal value to the user because the system does not quantify the level of uncertainty used in the ratings. EPA places too much emphasis on the amount of test data used to develop the factor, and not on the quality of that data. If there is a significant amount of test data, the factor is rated high, even though the data may be of poor quality. Further, even if a factor is rated "poor," it will still be used. An emissions factors tool that quantifies uncertainty would provide users with valuable information for adjusting the emissions factor as appropriate; taking into account the level of uncertainty during calculations can give the user a better understanding of the variations between actual emissions and emissions factors calculations. Quantifying the uncertainty of an emissions factor requires a more rigorous analysis of the test data to establish a range of uncertainty for emissions estimates. The Agency has been aware of such techniques, and over the past 10 years has funded several studies exploring the use of analytical techniques for quantifying uncertainty. In addition, the National Research Council and the NARSTO9 organization reported that EPA should increase the use of quantifying uncertainty in the development of emissions data. Also, EPA Order 5360.1 requires EPA to assess the quality of its data, and EPA's current emissions factors rating system is not consistent with the Order's data quality requirements. EPA officials told us that the majority of emissions factors are developed using 10 points of data or less, which is substantially less than the 30 to 50 data points recommended for the development of a valid statistical analysis. Without incurring the additional cost of obtaining more data, the Agency can perform EPA-accepted statistical analysts on existing data to provide users with an uncertainty rating. This will allow users to quantify the uncertainty of emissions data developed from emissions factors. ' Formerly an acronym for "North American Research Strategy for Tropospheric Ozone," NARSTO is a public/private partnership, whose membership spans government, utilities, industry, and academia throughout Mexico, the United States, and Canada. Its primary mission is to coordinate and enhance policy-relevant scientific research. 17 ------- EPA and interested stakeholders agree that the benefits of conducting additional analysis outweigh the additional resources needed to quantify the uncertainty. A range for the emissions estimate will allow the user to understand how much lower or higher actual emissions vary when compared to the emissions factor calculated estimate. For example, instead of an emissions factor being 30 pounds of emissions per 1,000 gallons of fuel consumed, the factor would have a quantifiable range - between 20 and 40 pounds - depending on the variables and uncertainties. An uncertainty expert we contacted quantified the uncertainty of two key A-rated emissions factors in EPA's AP-42 database. Both factors are rated "Excellent" - the highest level. Further, the two emissions factors represented a significant amount of emissions nationwide - estimated emissions from a total of 32,569 coal-fired boilers. The potential variances when uncertainty is considered are shown in Table 3.1. Table 3.1: Effect of Emissions Factors Uncertainty on Estimates for Coal-fired Boilers* Boiler Type Wall-fired Tangential-fired Emissions Estimate (NOX Emissions) 1,336,190 751,581 Emissions Change One to Factor Uncertainty Factor Uncertainty -41 .4% to +33.2% -31% to +27% Emissions Range (tons) 996,798 435,917 Estimate of Probable Emissions {tons} Low High 783,007 518,591 1,779,805 954,508 * Emissions factors ranges: Wall-fired (12.9 - 29.3) Ib/ton; Tangential-fired (10-4 -19.0) Ib/ton As Table 3.1 illustrates, EPA's best-rated NOx emissions factor for one type of coal-fired boiler (wall-fired) has an uncertainty range of plus 33.2 to minus 41.4 percent, meaning actual emissions nationwide could range from 783,000 tons to 1.8 million tons. Thus, without knowing the uncertainty associated with an emissions factor, the approximately 1.3 million tons reported for wall-fired boilers may be nearly as high as 1.8 million tons. Quantified uncertainty information provides a tool to the user to make more informed decisions for defining the appropriate uses of the emissions factor. Depending on the situation, the user can adjust the emissions factor based on this uncertainty. The following three uses of emissions factors demonstrate how factors can be adjusted: • If the factor is being used for a national estimate, the user would most likely choose an estimate toward the middle of the range (this is because of the law of large numbers and the likelihood that over- and under-estimates may tend to cancel each other out). 18 ------- • If the factor is being used to establish a permit limit at a specific facility, the user may need to select an estimate on the lower or higher end of the range, depending on variables and uncertainties. • If the factor is being used for emissions trading or offsetting, an appropriate use may be the low end of the emissions factor range. The uncertainty tool could allow the user to select an appropriate adjustment based on its use. Improving emissions factors will largely depend on the extent to which EPA can minimize the limitations associated with uncertainty. Without knowing the possible range of the estimate, users are not adequately informed of the risks associated with using the emissions factor. Inadequate Funding Provided and Used Currently, EPA's emissions factors workload is largely dictated by stakeholder needs, and EPA does not have a well-documented plan for prioritizing its work. Understandably, EPA officials have had to operate in a reactive mode, focusing on State and industry requests involving specific emissions factors. However, managing emissions factors development in this manner does not allow a systematic approach to ensuring the most critical emissions factors are receiving the appropriate priority. EPA stated in its 1997 Federal Managers' Financial Integrity Act (FMFIA) Assurance letter that it would request a substantial increase in funding for the development of emissions factors. As shown in Table 3.2, although EPA did request significant amounts of funding for the Emissions Factors Program since our 1996 report, it received about 25 percent or less of the amount requested for 3 of the 7 years (1999, 2000, and 2002). However, EPA actually spent significantly less on the Emissions Factors Program than had been appropriated, because EPA officials said emissions factors funds had been reprogrammed to other air program activities considered to be higher priority. As shown in Table 3.2, over the past 7 years, EPA spent between 29 percent and 72 percent of the money it received, with about half of the funds received being reprogrammed. From 1999 to 2005, the program received $10,657,000, and spent a total of $5,301,000. It should be noted that there has been renewed emphasis on emissions factors since the program was reorganized in 2003 (see next page). For example, the amount of funding spent in Fiscal Years 2004 and 2005 increased substantially from the two previous years, although 2005 expenditures were still 59 percent lower than 2001 expenditures. 19 ------- Table 3.2: Funding of the Emissions Factors Program - by Fiscal Year Budgetary Activities Requested Amount Amount Received Amount Spent Amount Reprogrammed 1999 $4,100,000 1,100,000 323,500 (776,500) 2000 $4,950,000 1 ,200,000 490,000 (710,000) 2001 $5,096,000 3,215,600 2,326,500 (889,100) 2002 $3,075,700 873,900 310,000 (563,900) 2003 $2,590,400 1,029,700 320.000 (709,700) 2004 $1,539,500 1,595,600 573,000 (1,022,600) 2005 - $4,652,600 1,642,200 958,000 (684,200) Over the past 5 years, there have also been several changes in the types of activities performed by the Emissions Factors Program, as shown in Table 3.3. With the exception of 2005, consistent funding has been devoted to maintaining the databases. Program improvements were non- existent in 2002 and 2003; however, they have been a primary emphasis during the past 2 years. With the exception of spending $200,000 to help develop one emissions factor in 2005, the Emissions Factors Program has largely not been involved in developing specific emissions factors since 2001, when over $1 million was spent. Table 3.3: Emissions Factors Program Expenditure Analysis - by Fiscal Year * •;• •-•;:MtivKte*::;; . ; :•.' Maintenance of Emissions Factors Databases Development of Specific Emissions Factors Emissions Factors Program Improvements Total 1999 $180,800 71,600 71,100 $323,500 2000 $221 ,000 269,000 0 $490,000 2001 $225,000 1,132,000 969,500 $2,326,500 2002 $245,000 65,000 0 $310,000 2003 $275,000 45,000 0 $320,000 2004 $234,500 43,000 295,500 $573,000 2005: $75,000 260,000 623,000 $958,000 ' Does not Include Administrative and Training Expenditures Given the increasing need for emissions factors and a relatively small budget, EPA will need to better leverage industry resources to obtain better emissions factors data. Although EPA recognizes the need to prioritize emissions factors work and has made efforts to do so, a well- documented set of short- and long-term priorities does not yet exist. In its 1997 FMFIA assurance letter, OAQPS stated it would refocus the Emissions Factors Program by prioritizing the most critical emissions factors, and hired a contractor to publish a document prioritizing emissions factors development. In 2002, the contractor published a document, Recommended Source Categories for AP-42 Chapter Update and Emission Test Program, also called the Scoping Study. One OAQPS official said the Scoping Study was intended to be the blueprint for future work, resource allocation, and other key program decisions. To date, EPA has not used the study to prioritize its emissions factors work. EPA officials said that they do not believe they can properly prioritize the competing needs of all of the emissions factors stakeholders to arrive at a common priority list to improve emissions factors. 20 ------- We agree that while developing a priority list of emissions factors is a challenge, it is critical for effectively managing the program. Emissions factors users at both the State and Federal levels said that EPA needs to prioritize its emissions factors work and fill the gaps that currently exist. Also, the Air Quality Management Work Group to the Clean Air Act Advisory Committee recommended that EPA review existing emissions factors to identify the most significant needs. Recent EPA Efforts Made to Revamp Emissions Factors Program EPA had recognized it did not have a sufficient process for developing, improving, and rating emissions factors. Therefore, in fiscal year 2003, OAQPS began a reevaluation of the Emissions Factors Program to: • Identify ways to make the program more responsive to the broad and diverse range of emissions factors users; • Identify methods that would expand the capabilities for improving the number and quality of available emissions factors; • Identify and implement ways to improve and expedite the emissions factors development process; • Characterize the deficiencies of using emissions factors by quantifying the uncertainties associated with the varied uses; and • Provide users with alternative methods of quantifying emissions to reduce the levels of uncertainty and to increase the accountability of stakeholders. Upon conclusion of the re-evaluation, EFPAG decided to revamp the Emissions Factors Program with the primary goal of improving emissions quantification through the use of better tools and knowledge of uncertainty. The revamping plan includes three specific tasks: (1) standardizing and streamlining the emissions data collection and reporting process, (2) establishing procedures for defining data uncertainty and using emissions factors in non-inventory applications, and (3) establishing an outreach program to communicate changes to emissions factors stakeholders. Specifically, EFPAG officials said they developed an electronic reporting tool that establishes a standardized emissions test report format with built- in quality assurance checks. They said use of this format will enable State and local air pollution control offices to readily assess the quality of submitted emissions test reports and to share those report results with others. As the tool is used, results from routine emissions tests should be easier to gather and those results should already be quality assured. 21 ------- Consistent use of the tool should increase both the quantity and quality of emissions factors. For non-inventory programs such as New Source Review and Title V permitting, EFPAG conducted case studies identifying the impact of over- or under-estimating source emissions through use of emissions factors. Acting on the results of those studies, statistical analyses of excellent and above average emissions factors were performed to determine the range of uncertainty associated with these factors. EFPAG indicated it is preparing those analyses for external peer review, and at the same time, is assessing the impact of adjusting emissions factors for non-inventory programs. Once the analyses are validated, EFPAG plans to develop guidance for using emissions factors for non-inventory programs, and where applicable, the adjustments needed for emissions factors use for each program. EFPAG officials said that this guidance will improve emissions estimates and decisions based on those estimates by reducing the uncertainty inherent with emissions factors use. Finally, EFPAG is upgrading the existing Internet Web site, named WebFIRE, from a static to an interactive mode. Future plans for this newly-designed Web site include the ability to collect, screen, and adapt emissions data from the electronic reporting tool; calculate average emissions factors; and provide individual users with site-specific values that incorporate associated uncertainties tailored to their specific program application in accordance with future guidance. WebFIRE is also to provide the means to distribute new emissions factors, guidance, and procedures, as well as other means to better provide emissions information. EPA should be commended for its efforts, but should place greater emphasis on improving the quality of these factors. EFPAG agreed that it should expand the electronic rating tool to include pollutants other than Particulate Matter, ensure the guidance specifies how program specific adjustments are to be made, and program the interactive portion of WebFIRE to accept new data and to calculate specific adjustments. Comprehensive Plan Needed to Improve Data Collection and Set Priorities The lack of a comprehensive strategic plan hinders EPA's ability to ensure the program is moving in the right direction, meeting its goals and objectives, and achieving the desired results. Some key areas the plan should include are short- and long-term goals and objectives; steps and measures to gauge progress in meeting the goals; timeframes for meeting milestones; and a process for reassessing and, when appropriate, revising the plan. A coordinated and well-thought-out plan will enable EPA to 22 ------- identify cost-efficient methods of obtaining more and better data, establish a communication strategy, and establish a system for prioritizing emissions factors development. EPA has made efforts to address these key areas, although it is not clear how these efforts fit into EPA's broader, long-term plans. Over the past 30 years, EPA's Emissions Factors Program has relied largely on scavenging for source data to develop emissions factors and often uses source data originally gathered for other purposes. For example, in the 1970's, the program benefited from the source testing conducted by EPA for the development of new emissions standards. Throughout the 1980's and 1990's, the amount of EPA-funded source testing steadily declined, with a corresponding increase in industry-funded source testing. Through its own initiative, industry will sometimes conduct source testing to test emissions, but because the industry is not required to submit the data, EPA often is not aware of the data. EPA needs to develop a system where it can gain access to the millions of dollars worth of industry-generated source testing data. Currently, EPA is working on an information system that it believes will provide the much- needed data to the stakeholders who need to develop emissions factors. If successful, this will be an important step toward sharing valuable data. However, EPA will also need to develop incentives for industry to share this information with the Agency. One way would be for EPA to issue guidance in accordance with EPA Order 5360.1 requiring and allowing only appropriate use of emissions factors for the environmental decisions to be made. Recognizing that implementation of this criteria may take some years, in the interim EPA could allow facilities to use the upper bounds of an emissions factor's uncertainty range. Also, if a facility believes the higher emissions rate stipulated by EPA's new guidance does not reflect its actual emissions, the facility could conduct stack or other testing and share this information with EPA in an effort to decrease the uncertainty of the emissions factor. The facility may be encouraged to do this in an effort to lower the upper bounds of the emissions factor uncertainty range and its resulting emissions estimate. Over time, as more facilities submit test data, EPA may have sufficient information to decrease the uncertainty of poor quality emissions factors and thus improve the estimates for an entire source category. EPA also needs to establish a communications strategy. Emissions Factors Program officials said that there are many air quality stakeholders with useful information, and accessing this knowledge pool can improve the development of emissions factors. EPA has conducted some outreach, through workshops and surveys, but the absence of a formalized 23 ------- communication strategy has resulted in valuable information not being identified. A more formal communication strategy will allow greater feedback and strengthen information sharing. EPA specifically needs to share information with stakeholders from four key areas: • Emissions inventories • Permitting • Industry • Enforcement In 2004, EPA implemented the Emissions Factors Improvement Project survey to solicit stakeholder feedback about the Emissions Factors Program and what improvements were needed. Of the 58 respondents, 37, or about 64 percent, said they did not submit data to EPA. About one- third of the 37 said they did not provide data to EPA because they did not know they should or how they should go about it. In addition, officials we interviewed from Regional Planning Organizations, the State and Territorial Air Pollution Program Administrators and the Association of Local Air Pollution Control Officials, State and Federal emissions inventory and permitting officials, and academia stated that they have limited or no contact with EPA on emissions factors. Instead of relying on stakeholders to contact them, EPA should systematically contact a broad range of stakeholders to gain access to information and knowledge that could improve the development of emissions factors. The information system that EPA is developing should increase stakeholders' ability to provide additional data and access other useful emissions factors data. However, the system will not be effectively used unless EPA takes steps to ensure that stakeholders are aware of the information system and its advantages. Procedures should be included in a communications strategy to regularly solicit key information from stakeholders. Conclusions EPA officials describe the emissions inventory as the foundation for the air program, upon which everything else is built. Emissions factors estimates are used to develop much of this inventory and, as such, are critical measures woven into the fabric of many air quality managers' most important decisions. Equally important, the models used to forecast changes in air quality under alternative reduction scenarios are also heavily reliant on accurate emissions factors and subsequent pollutant estimates. One common use of these models provides decision-makers with the information needed to develop control strategies to lower air emissions. Other types of emissions factors driven models are those used to identify the sources of pollution and link those sources to specific 24 ------- pollutants, selecting sites for increased monitoring, testing emission reduction scenarios to predict what impact they would have on pollutants of concern, and predicting changes in future concentrations. The results of these analyses shape EPA's air quality planning. Use of inaccurate or incomplete data such as emissions factors can lead to costly, unjustified controls and/or exposures to pollutants that could negatively affect human health and welfare. Further, these analyses help define which industries need to be regulated, what level of controls are needed to achieve the desired results, which sources should be targeted for enforcement initiatives, and which pollutants are of the utmost concern. Emissions data also influences EPA's decisions in studying health effects, performing risk and exposure assessments, and identifying safe levels of air pollution. The reliability and accuracy of the emissions estimates has far-reaching implications for how EPA sets its priorities and allocates the scarce resources available to meet the increasingly complex and daunting challenges of identifying the most cost-effective approaches to making the air cleaner. Emissions factors estimates, for all the uses noted above, are at the core of this process. Our findings verified that these factors remain significantly uncertain or unknown, despite their critical role. In fact, these factors are drawn upon by scientists, industry, States, and others and the rating system guiding their use does not meet the requirements of the Agency's data quality order. While more resources have been requested to improve these factors, EFPAG has continued to receive significantly less money than needed over 6 of the last 7 years. Also, for the money it did receive for emissions factors, OAQPS redirected over half of these funds to other air program activities for 5 of those 7 years. If EPA continues to use insufficient measures, such as poor and unknown quality emissions factors, to determine program results, the Agency may be overstating its progress to Congress and the public. That is, EPA may not be reaching the goals it has claimed to reach and the air may not be as clean as the Agency claims. If progress is overstated, it may also result in EPA and States making misinformed decisions on selecting the most promising future actions for improving the quality of the air. Recommendations We recommend that the Acting Assistant Administrator for Air and Radiation: 3-1 Establish a workgroup with representatives from emissions inventory, permitting, industry, and enforcement, to develop an emissions factors guidance document that addresses: (a) The appropriate and prohibited uses of emissions factors. 25 ------- (b) The intended use of the emissions factors estimates and how to adjust the estimates to more accurately reflect the use. (c) How to develop localized and site-specific emissions factors. (d) How to account for emissions factors uncertainty in newly developed factors and key existing factors. (e) Mechanisms for facilities and industry sectors to follow in developing and providing to EPA emissions factors that meet EPA Order 5360.1 data requirements. 3-2 Develop and implement a comprehensive strategic plan that focuses on addressing future challenges expected in the Emissions Factors Program. The plan should address development of: (a) Criteria for prioritizing emissions factors development, for both new factors and selected existing factors that have the most environmental impact. (b) A communications and partnering strategy that results in sustained feedback from key emissions factors users, including EPA and State permit and enforcement officials, and industry. (c) An information system that streamlines the collection of source test data for the development of emissions factors. (d) Steps to ensure emissions factors uncertainty analysis is included in the development, rating, and intended uses of emissions factors. (e) A Quality Management Plan that ensures data used for the development of emissions factors meet data quality requirements. 3-3 Once a comprehensive strategic plan is completed, have the Director for OAQPS ensure that all funds received for the Emissions Factors Program are actually spent on the program and not reprogrammed to other air activities. Agency Comments and OIG Evaluation The Agency concurred with our recommendations and stated that they generally align with its current improvement efforts. The Agency agreed that developing more emissions factors with less uncertainty is important in advancing the air program's inventory tools, but also noted that there is a need for even more accurate and representative emissions data for non- inventory uses - data obtained through direct emissions measurements. We agree with the need for direct measurements of emissions, and believe 26 ------- that appropriate guidance from EPA on the proper uses of emissions factors would provide substantial incentives for industries and State, local, and tribal agencies to work together to obtain such direct measurements. However, we also recognize that it may be years before such data are obtained and provided to EPA. The full Agency response is in Appendix C. A more detailed analysis of Agency comments and our evaluation of those comments are in Appendix D. The Agency will need to address each recommendation and provide details and milestones on its plans to address the OIG recommendations within 90 days, including: • The status of EPA's efforts to establish a workgroup to develop guidance for recommendation 3-1, which addresses the appropriate and prohibited uses of emission factors. EPA should also explain the intended use of emissions factors estimates and how to adjust the estimates to more accurately reflect the use. • Actions taken or planned to develop and implement a comprehensive strategic plan, or submit a completed plan that includes the five elements listed for recommendation 3-2. • Specific steps taken or planned, as per recommendation 3-3, to ensure that funds received for the Emissions Factors Program are spent on the program and not reprogrammed to other air activities. 27 ------- Appendix A Details on Scope and Methodology We conducted interviews with officials from: EPA's OAQPS and Office Enforcement and Compliance Assurance; air permitting officials from EPA Regions 4 and 6; North Carolina and Indiana air pollution control agencies; State and Territorial Air Pollution Program Administrators and the Association of Local Air Pollution Control Officials (STAPPA/ALAPCO); and the Western Regional Air Partnership and Midwest Regional planning organizations. We reviewed and discussed with these officials selected reports related to emissions factors, including: • An April 2005 NARSTO report, Improving Emission Inventories for Effective Air Quality Management Across North America: A NARSTO Assessment • Two National Academy of Sciences reports: The Scientific Basis for Estimating Air Emissions from Animal Feeding Operations: Interim Report (2002); and Air Emissions from Animal Feeding Operations: Current Knowledge, Future Needs (2003) • EPA 's Proposed Amendments to A ir Toxics Regulations for the Plywood and Composite Wood Products Industry, July 18, 2005 • EPA's Compilation of Air Pollutant Emission Factors, Volume 1: Stationary Point and Area Sources, AP-42, Fifth Edition, issued January 1995 • A June 2004 EFPAG survey report, Summary of Emissions Factors Improvement Project Fact Finding Survey • EPA's Procedures for Preparing Emission Factor Documents, November 1997 • Minutes from four Emissions Factors Development Workshops (Florida Conference -June 2004; Research Triangle Park (North Carolina) Conference - November 2004: and two Washington, DC Conferences - August 2004) • WHO'S COUNTING? The Systematic Underreporting of Toxic Air Emissions, June 2004, a joint study by the Environmental Integrity Project and the Galveston-Houston Association for Smog Prevention • Evaluating Petroleum Industry VOC Emissions in Delaware, New Jersey and Southeastern Pennsylvania Final Report, October 2003, Mid-Atlantic Regional Air Management Association To assess management controls, we reviewed the Office of Air and Radiation's and OAQPS's fiscal year 2004 Integrity Act Annual Assurance Letters. To obtain an understanding of the quantification of uncertainty of emissions factors, we interviewed an emissions factors uncertainty expert from North Carolina State University and reviewed technical papers associated with the subject. 28 ------- Methodology Used to Calculate Emissions Factors Uncertainty for Coal-fired Boilers in Table 3.1 Emissions estimates were obtained from the draft 2002 EPA National Emissions Inventory (NEI). Factor uncertainty ratings were obtained for each source type from an emissions factors uncertainty expert. Probable emissions estimates were calculated by applying the uncertainty rating to emissions estimates, which results in a probable low and high range of emissions. This is known as the probabilistic emissions inventory, or PEL Using the PEI allowed us to illustrate uncertainty in the NEI. PEI is a function of the reported emissions and the emissions factor uncertainty (WEF) and activity data uncertainty (UFA?). It is the probable range of emissions values in which the reported individual emissions would be found. To calculate the PEI associated with a portion of the NEI, we obtained the reported emissions value for two Source Categorization Codes. The UFEF for each source category was multiplied by the point estimate of the emissions for each category (from 2002 NEI) and summed over the two categories to arrive at a PEI: PEI = I [(UP,*) (UFAp) (El)] For this formula, PEI is summed over all source categories that have reported emissions (Ib/yr). We reviewed the following documents related to this analysis: • Report 99-267 ( 1 999), North Carolina State University, Raleigh, North Carolina • 2002 Emissions (NEI); Air Pollution Control Lecture Notes, North Carolina State University • Emission Inventory: Planning for the Future, October 28-30, 1997, Research Triangle Park, North Carolina • Journal of the Air & Waste Management Association, 2003 • Environmental Science and Technology, 2004 • Quantitative Analysis of Uncertainty and Variability in Environmental Policy Making, 1992 • Risk Analysis, 2004 Prior Coverage and Followup We followed up on EPA's actions on previous recommendations from EPA OIG Report No. 61 00306, Emission Factor Development, September 30, 1 996. As part of this followup work, we reviewed OAQPS's fiscal years 1997 and 1998 Integrity Act Annual Assurance Letters. We had found that EPA was providing poor and unreliable emissions factors to the user community. We recommended that the development of emissions factors be included as an Agency material weakness in FMFIA reporting. In its response to the draft report, EPA recognized the significant role of emissions factors, and as such, indicated it would take the following corrective action: 29 ------- * Refocus the program to address the most critical factors needed. • Request nearly four times the amount of contract resources in the fiscal year 1999 budget request to the Office of Management and Budget. • Request $5 million of additional funding from Congress for Particulate Matter research in fiscal year 1998 to be used for developing Particulate Matter emissions factors. Details on what we found as a result of our followup, as well as recommendations, are in Chapters 2 and 3 of this current report. We also reviewed two other reports that addressed emissions factors development, although we did not do followup. These were a GAO report, EPA Should Improve Oversight of Emissions Reporting by Large Facilities (GAO-01-46), April 2001; and an EPA OIG report, Substantial Changes Needed in Implementation and Oversight of Title V Permits If Program Goals Are To Be Fully Realized (2005-P-00010), March 9, 2005. Limitations Our work contained the following limitations: • We did not review emissions factors for all pollutants from all sources. • We did not review any emissions factors for mobile sources. • The quantified uncertainty information in Table 3.1 for two NOx emissions factors is unpublished data provided by the North Carolina State University emissions factors uncertainty expert, who was contracted by EPA to perform the uncertainty analysis. 30 ------- Appendix B Details on Concentrated Animal Feeding Operations Air Emissions Concentrated Animal Feeding Operations (CAFOs) are agricultural entities that raise animals in confined areas and pose a potential environmental risk as a result of the high concentrations of animals and their waste. Air emissions from agriculture sources generally have characteristics that make them difficult to control through the more conventional control technologies used at industrial sources. The difficulty and cost of monitoring agricultural pollution sources is a reason that CAFOs are largely unregulated regarding air emissions. EPA asked the National Academy of Sciences to evaluate the scientific information needed to address CAFO air emissions issues.10 The Academy found that the basic data needed for effective regulation and management of CAFO emissions do not exist. Reasonably accurate estimates of air emissions from CAFOs at the individual farm level will require defined relationships between air emissions and various factors. The Academy also found that directly measuring emissions from CAFOs is not feasible and existing emissions factors are generally inadequate. A major effort will be required to develop useful CAFO emissions factors. EPA's need for better data has led to an agreement between EPA and some sectors of the animal industry to monitor air quality on farms. This voluntary agreement calls for a 2-year national air monitoring study on emissions. Data developed from this study will be used to develop emissions factors. As part of the agreement, EPA indicated it will provide certain legal protections for past and current emissions violations for farms that participate. Environmental advocates have criticized EPA for providing this protection, and a former EPA staff attorney said EPA should not suspend its enforcement authority when the Clean Air Act already requires facilities to provide this data. However, according to EPA, this settlement will result in CAFO operators funding scientifically credible methodologies for estimating emissions as recommended by the National Academy of Sciences in its 2003 report. 10 Air Emissions from Animal Feeding Operations: Current Knowledge, Future Needs (2003) 31 ------- Appendix C Agency Response to Draft Report February 23, 2006 MEMORANDUM SUBJECT: Draft Evaluation Report: EPA Can Improve Emissions Factors Development and Management, Assignment No. 2005-00279 FROM: William L. Wehrum (Elizabeth Craig for) Acting Assistant Administrator TO: J. Rick Beusse Director for Program Evaluation, Air Issues Office of Inspector General Thank you for the opportunity to comment on the draft evaluation report, "EPA Can Improve Emissions Factors Development and Management," dated December 21,2005. My staff also appreciated the opportunity to work with you to incorporate comments on the prior versions of the draft report. The emissions factor (EF) concept was developed in 1970 by the Office of Air Quality Planning and Standards National Air Data Branch to estimate emissions from the millions of emitters in the U.S. for which measurements were either impossible to make or had not yet been made. EFs were, and still are, the first step toward quantifying emissions in a basic way for estimating annual emissions. EFs represented a significant advance over previous estimating techniques or policies and were necessary and adequate for their time. See Appendix D Notel While we agree that EFs are inherently uncertain and imperfect, developing more EFs or more certain EFs is important in advancing the air program's inventory tools. However, we believe that focusing our efforts only on EF development obscures the real need for more accurate and representative emissions data for non-inventory uses. We believe the air program needs more direct measurements of emissions for our advanced models, strategies, and national programs. This need was emphasized strongly by the National Research Council of the National Academies in its report, "Air Quality Management in the United States" (National Academies Press, 2004). Currently, we are moving toward better emissions monitoring requirements for all major polluters that will enhance our response to environmental challenges. 32 ------- See Appendix D Note 2 The recommendations provided by the Office of the Inspector General generally align with our current EF program improvement efforts. However, the report could better emphasize that: (1) EFs are just the first step toward quantifying emissions in a basic way; and (2) EPA is shifting its efforts toward more direct, continuous monitoring and measurement of emissions from all major emissions sources. For example, EPA is currently: (1) working on a process to quantify the uncertainties associated with the applications of EFs, and (2) establishing procedures to incorporate additional data into calculating EFs to reduce the uncertainty associated with the EF. Even with the ability to incorporate uncertainty into the application of an EF, EFs remain estimates and not direct emissions measurements. More advanced, accurate, continuous, and short-term determinations of emissions are needed for major emitters, and these goals can only be reached by shifting to direct emissions measurement and monitoring systems. Additional editorial comments are provided in the attachment. If you have any questions or need clarification, please contact Peter Tsirigotis of my staff at (919)541-9411. 33 ------- See Appendix D Note3 Attachment (1) The draft evaluation report makes several references to the misuse or inappropriate use of EFs. These references should be linked to the misuse or inappropriate use of EFs in non-inventory programs or applications. We note the following places in the draft evaluation report where clarification would be appropriate: • At a Glance, What We Recommend, 1st paragraph - We are making a number of recommendations to EPA to, among other things, develop emissions factors guidance that addresses the development and appropriate use of factors in non-emissions inventory programs; • At a Glance, What We Found, 2nd paragraph — Emissions factors have been inappropriately used for key environmental decisions of non-emissions inventory programs, such as setting permit limits and reporting the level of air pollution control at specific facilities. • Chapter !, Purpose, 2nd paragraph, 1st bullet - Emissions factors are of an acceptable quality for use in key environmental decisions made by EPA and State and local agencies for non-emissions inventory programs; (2) In addition, clarification is appropriate in the following sections: See Appendix D Note 4 Chapter 1, Background- In the paragraph on Source Testing and Material Balance, the cost figures are stated as absolute, when in reality they vary from pollutant to pollutant. The cost information would be more accurately expressed in ranges: • Source Testing - $ 10,000 to $50,000 annual cost; • Material Balance - $2,000 to $ 10,000 per year per process. See Appendix D NoteS Chapter 3, Rating System Does Not Define Appropriate Uses and At a Glance, What We Found- The information in the bullets below Table 3.1 in Chapter 3, Rating System Does Not Define Appropriate Uses more correctly characterizes the adjustments to applying EFs based on data uncertainty in inventory applications. It would be appropriate to express this smaller range of potential errors in the example cited at the bottom of paragraph three in the What We Found section. > Chapter 3, Comprehensive Plan Needed to Improve Data Collection and Set Priorities - The second paragraph in this section includes some clarifying historical context. Placing this text in the introduction (Chapter See Appendix D Note 6 I, Purpose section) would provide this historical context at the outset of the report. 34 ------- Appendix D Note 1 - Note 2 - Note 3 - Note 4 - Note 5 - OIG Evaluation of Agency Response We agree that emissions factors are the first step in quantifying estimates of actual emissions and that direct measures are the preferred emissions measure. We also agree with the Agency's increased efforts to obtain more direct emissions measures, such as monitoring data, for non-inventory uses. We have revised the report to reflect these points. However, considering the extensive use of emissions factors today for both inventory and non-inventory uses, we believe that emissions factors may continue to be used extensively for the foreseeable future. For example, in responding to a March 2004 EPA OIG report/' EPA informed us that it did not have the statutory authority to require the submission of hazardous air pollutant monitoring data from State and local agencies. Emissions factor-developed emissions inventory data will continue to be used frequently as a cost-effective method of estimating emissions, as compared to more expensive emissions measurements such as direct emissions monitoring. In the future, increased monitoring data may reduce the use of emissions factors for major sources; however, mobile and area sources are more likely to rely on emissions factors for a longer period of time. Therefore, we continue to believe that emissions factors may continue to be an integral part of the Air program for a significant length of time, and as such, should receive the resources needed to improve factor quality and the reliability using these estimates. We are encouraged that EPA believes our recommendations generally align with EPA's current Emissions Factors Program improvement efforts, and based on our work, we would agree. We do, however, differ somewhat on our views of the projected use of emissions factors in the future. Please see Note 1 for details. We revised the report to reflect the use of emissions factors in "non-inventory programs or allocations," where appropriate. For example, in the At a Glance, What We Recommend, first paragraph, we added that we believe guidance is needed for the non-inventory uses of emissions factors. However, in Chapter 1, Purpose, second paragraph, first bullet, we did not change this because our objectives were to address both inventory and non-inventory uses of emissions factors. We agree with this comment and have made these changes to our draft report. Due to space limitations in the "At a Glance" section of the final report, we could not accommodate the Agency's desire that we more fully characterize 11EPA 's Method for Calculating Air Toxics Emissions for Reporting Results Needs Improvement, Report No. 2004- P-00012, March 31, 2004 35 ------- an illustration of the impact of emissions factor uncertainty on estimates of emissions nationwide; however, we have added such an illustration on page 18 below Table 3.1, and removed our discussion of this information in the "At a Glance." Note 6 - We agree with this comment and added a paragraph to the background section in Chapter 1 to provide the requested historical context to the Emissions Factors Program. 36 ------- Appendix E Distribution EPA Headquarters Office of the Administrator Acting Assistant Administrator for Air and Radiation Assistant Administrator for Research and Development Agency Followup Official Agency Followup Coordinator Audit Followup Coordinator, Office of Air and Radiation Audit Followup Coordinator, Office of Research and Development General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for Public Affairs Director, Office of Air Quality Planning and Standards Deputy Director, Office of Air Quality Planning and Standards Director, Emissions Standards Division Acting Director, Emissions, Monitoring and Analysis Division Director, National Exposure Research Laboratory Audit Liaison, Office of Air Quality Planning and Standards EPA Regions Regional Air Program Directors EPA Office of Inspector General Acting Inspector General 37 ------- |