OFFICE OF INSPECTOR GENERAL
Evaluation Report
EPA Can Improve Emissions
Factors Development and
Management
Report No. 2006-P-00017
March 22, 2006
-------
Report Contributors: Patrick Milligan
Frank Martinsky
Kevin Good
Bill Nelson
Abbreviations
CAFO Concentrated Animal Feeding Operation
CO Carbon Monoxide
EFPAG Emissions Factors and Policy Applications Group (within EPA)
EPA Environmental Protection Agency
FIRE Factor Information Retrieval
FMFIA Federal Managers' Financial Integrity Act
GAO Government Accountability Office
kg kilograms
MACT Maximum Achievable Control Technology
NEI National Emissions Inventory
PEI Probabilistic Emissions Inventory
NOx Nitrogen Oxides
OAQPS Office of Air Quality Planning and Standards
OIG Office of Inspector General
PMa.5 Fine Particulate Matter
VOC Volatile Organic Compound
-------
f'Stf'-
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
2006-P-00017
March 22, 2006
C#fafyiirfttr fmptwhtg
Why We Did This Review
Wt: sought to determine
whether the air emissions
factors used by the
Environmental Protection
Agency (EPA) are of
acceptable quality for making
key environmental decisions,
an<3 whether EPA's process
foj developing, improving,
and rating emissions factors
is sufficient to meet users'
nesds.
Background
Emissions factors are broad
es.imates of the emissions
generated from a source, such
as a factory. Nationally,
emissions factors are used for
about 80 percent of emissions
reporting. An emissions
factor is a representative
v* lue that attempts to relate
the quantity of a pollutant
released with an activity rate
associated witfa the release.
Emissions factors underlie
marty environmental
decisions. Recently, States
and industry have been
developing emissions factors
attd submitting them to EPA.
For further trtfiuinatior*,
contact our Office of
Congressional and Public
liaison at (202) 566-2391.
T > view the full report,
click on the following link:
www.epa.aov/oig/reports/20C6
/;0060322-2006-P-Q0017.pdf
EPA Can Improve Emissions Factors
Development and Management
What We Found
EPA has made progress in emissions factors development since our review of the
program in 1996, but a large number of factors continue to be rated low. The
number of EPA-rated factors increased by nearly 94 percent, from 8,838 in 1996 to
17,110 in 2004. However, the percentage of emissions factors rated below average
or poor increased from 56 percent in 1996 to 62 percent in 2004.
Emissions factors, intended for use in developing emissions inventories, have been
inappropriately used for key environmental decisions beyond their intended use.
For example, emissions factors have been used for non-inventory purposes, such as
setting permit limits and reporting the level of air pollution control at specific
facilities. For three industry sectors EPA examined, inappropriate use of emissions
factors contributed to more than one million tons of pollutants not being controlled.
Demand for emissions factors is increasing, and will continue for a broad array of
environmental decisions, including measuring and reporting environmental
progress. This pertains not only to existing factors but to those that still need to be
developed, especially emissions factors for sources of fine particulate matter. If
EPA can improve the quality of its factors, this should improve environmental
decision-making for reducing air pollution. Improving the quality of emissions
factors is an extremely challenging task that may take EPA years to address.
The quality of many emissions factors remains low in part because EPA did not
have a sufficient process for developing, improving, and rating emissions factors,
nor did EPA have a comprehensive strategic plan. We found inconsistent
emissions factors guidance, continuing reliance on a qualitative rating system when
a quantitative range of uncertainty is needed, and insufficient program funding
when needs are increasing.
What We Recommend
We are making a number of recommendations to EPA to, among other things,
develop emissions factors guidance that addresses the development and appropriate
use of emissions factors for non-inventory purposes; establish a rating system that
provides the quantitative range of uncertainty for emissions factors for both
inventory and non-inventory purposes; work with industry, State and local agencies,
and others to leverage available resources for meeting increasing demands for new
factors; and establish a workgroup to develop a comprehensive strategic plan for the
Emissions Factors Program, and ensure that requested resources are used to achieve
program goals. In response to the draft report, the Agency stated that our
recommendations generally align with its current improvement efforts.
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
MEMORANDUM
SUBJECT:
FROM:
TO:
March 22, 2006
EPA Can Improve Emissions Factors Development and Management
Report No. 2006-P-00017
J. Rick Beusse /s/
Director for Program Evaluation, Air Issues
William L. Wehrum
Acting Assistant Administrator for Air and Radiation
This memorandum transmits the results of an Office of Inspector General (OIG) evaluation of
emissions factors development and management. This report contains findings that describe how
the U.S. Environmental Protection Agency (EPA) can improve emissions factors development
and management, as well as corrective actions the OIG recommends. This report represents the
opinion of the OIG and the findings contained in this report do not necessarily represent the final
EPA position. Final determinations on matters in the report will be made by EPA managers in
accordance with established procedures.
Action Required
In accordance with EPA Manual 2750, as the action official, you are required to provide a
written response within 90 days of the final report date. The response should address all
recommendations. For the corrective actions planned but not completed by the response date,
please describe the actions that are ongoing and provide a timetable for completion. Where you
disagree with a recommendation, please provide alternative actions for addressing the findings
reported.
We appreciate the efforts of EPA officials and staff in working with us to develop this report. If
you or your staff have any questions regarding this report, please contact me at 919-541-5747
or Pat Milligan at 215-814-2326.
-------
Table of Contents
At a Glance
Chapters
Introduction
Purpose
Background
Scope and Methodology.
1
1
6
Use of Unreliable Emissions Factors Adversely Impacts
Key Environmental Decisions
EPA Rated More Factors But Quality of Many Factors Remains Low..
Misuse of Emissions Factors Resulted in Significant Unidentified and
Uncontrolled Emissions
Increasing Demand for New Emissions Factors Illustrates Continued
Importance of Emissions Factors Program
Conclusion
3 EPA's Management of Emissions Factors Program Needs Improvement
Conflicting Guidance Issued for Emissions Factors
Rating System Does Not Define Appropriate Uses
Inadequate Funding Provided and Used
Recent EPA Efforts Made to Revamp Emissions Factors Program
Comprehensive Plan Needed to Improve Data Collection and
Set Priorities
Conclusions
Recommendations
Agency Comments and OIG Evaluation
10
13
14
15
15
17
19
21
22
24
25
26
A Details on Scope and Methodology
B Details on Concentrated Animal Feeding Operations Air Emissions.
C Agency Response to Draft Report
D OIG Evaluation of Agency Response
E Distribution
28
31
32
35
37
-------
Chapter 1
Introduction
Purpose
Quantifying air emissions is a vital aspect of air pollution programs.
Regulatory authorities and others use emissions values in: (1) developing
emissions inventories, (2) identifying and evaluating control strategies,
(3) determining applicability of permit and regulatory requirements, and
(4) assessing risks. Emissions factors are broad estimates of the emissions
generated from a source, such as a factory. These factors are the most
commonly used estimate for establishing emission values, and are used
nationally for about 80 percent of emissions reporting.
The Environmental Protection Agency (EPA) has long recognized the
importance of emissions factors, particularly for developing emissions
inventories. Over the last 10 years, permitting authorities, source owners
and operators, and a few Agency programs have begun using emissions
factors for purposes other than generating a national emissions inventory.
For example, emissions factors have been used to develop emissions
control strategies, determine applicability of permitting and regulatory
requirements, establish permit limits, ascertain the effects of sources, and
develop emissions reduction strategies. Therefore, the impact of the
quality of the factors is far greater than it would be if they were used only
for the inventory. Because emissions factors underlie so many
environmental decisions, the objectives of our evaluation were to
determine whether:
• Emissions factors are of an acceptable quality for use in key
environmental decisions made by EPA and State and local
agencies; and
• The Agency's process for developing, improving, and rating
emissions factors is sufficient to meet key users' needs.
Background
In the 1960's and early 1970's, emissions estimation methods came about
due to the need to estimate air pollution emissions. EPA first developed
emissions factors from source test data used to develop new emissions
standards in the 1970's. At that time, factors were mostly used to develop
emissions inventories. In the 1980's and 1990's, EPA expenditures on
source testing declined, yet during the same time period the demand for
emissions data expanded as emissions factors were increasingly used for
-------
non-inventory decisions. As noted in our prior report, EPA's emissions
factor program resources were surpassed by the need for more and
improved emissions data.1 Recently, EPA reorganized the Agency's
emissions factors program in an effort to address this challenge.
What Are the Ways of Obtaining Air Emissions Information?
Generally, air emissions information can be obtained through direct
measures of emissions or by estimating emissions. Under ideal
circumstances, all emissions data users would derive values from ongoing
emissions testing, continuous emissions monitoring, or frequent
calculation using well-accepted engineering principles. However, these
methods are time and resource intensive. Continuous Emissions
Monitoring and Source Testing represent ways to directly measure
emissions, while the others involve estimating. The five basic means of
obtaining emissions information are discussed below.
• Continuous Emissions Monitoring: This involves continuously
measuring pollutants emitted into the atmosphere from a single
source, such as a smokestack, by placing a monitor at the source.
This is one of the most reliable methods for measuring emissions,
but has annual costs ranging from about $ 10,000 to $50,000 and is
only required at the largest sources of air pollution.
• Source Testing: Like Continuous Emissions Monitoring, source
testing data are generated by placing a monitor at a source, but in
this case only measures for a limited number of hours. The facility
uses the monitoring data to calculate an annual emissions total.
This is also generally more reliable than emissions factors but has
annual costs of $20,000 for conducting a test every 5 years. While
some facilities are required to periodically conduct source tests,
many are not.
• Material Balance: For some sources with Volatile Organic
Compound (VOC) emissions, material balance (also known as
mass balance) assumes that a percentage of the materials used in a
process will evaporate to become air emissions. Therefore, the
amount of emissions resulting from evaporation is based on the
amount of evaporative material used. Recordkeeping to calculate
material balance costs about $2,000 to $10,000 per year, per
process.
• Emissions Calculating Tools: These estimating methods
represent a more advanced and complicated use of emissions
1 Emission Factor Development, EPA Office of Inspector General (OIG) Report No. 6100306, September
30, 1996.
-------
factors and other data, and include models, databases, and other
pollutant estimating software.
• Emissions Factors: An emissions factor relates the emissions
amount of a pollutant with an activity rate associated with its
release. Emissions factors are essentially averages of available
data from various sources, such as source tests. The cost for using
an emissions factor depends on the time to locate the appropriate
factor. The ability to rapidly locate emissions factors makes this
one of the least expensive methods for estimating emissions. The
general equation for emissions estimates is:
Activity Rate x Emissions Factor = Emissions
The following example illustrates one way that an emissions factor
is used. Burning distillate oil in industrial boilers produces carbon
monoxide (CO) and other emissions. The CO emissions factor for
this process is 0.6 kilograms (kg) of CO emitted per 1,000 liters of
oil burned. Assuming the boiler burns 90,000 liters of oil per day,
the following calculation shows the estimated amount of CO
emitted per day:
90,000 liters per day x 0.6 kg CO/1,000 liters = 54 kg ofCO/day
In the absence of direct measures, emissions factors are frequently
used as a quick, low cost way to estimate emissions.
How Are Emissions Factors Developed?
EPA's Emissions Factors and Policy Applications Group (EFPAG)
oversees the Emissions Factors Program. EFPAG is part of the Office of
Air Quality Planning and Standards (OAQPS) within EPA's Office of Air
and Radiation. EPA issues the guidance for developing the factors, and
initially was responsible for developing many of the factors. However,
industry and States increasingly have been developing emissions factors
and submitting them to EPA for inclusion in AP-42.2
EPA guidance states that the following five steps are to be carried out to
develop emissions factors:
1. Data Collection - For a particular industry sector, EPA collects
information related to facility process descriptions and source
emissions test data, if known to exist and available. Sources of
this information include existing data in EPA databases and
2 The Compilation of Air Pollutant Emission Factors, commonly referred to as the "AP-42 series" of
emissions factors, is the primary guidance and source of rated emissions factors used by EPA.
-------
2.
external requests to Federal, State, and local agencies and industry
trade organizations.
Data Evaluation - The data are evaluated to determine which data
should be used to develop the emissions factor. The data selected
are known as "test data" and are given test quality ratings.
3. Data Classification - The data are then grouped into clusters of
similar processes, which will eventually be averaged into an
emissions factor.
4. Data Calculation - EPA averages the groups and develops a
numerical emissions factor. The factor is given an overall
qualitative rating of "A" (excellent) through "E" (poor) based on
the ratings assigned to the test data and production quality.
5. Published Emissions Factor - The group of processes represented
by the emissions factor is assigned an existing or new source
classification code, and the emissions factor is published in
AP-42.
How Are Emissions Factors Used?
Emissions factors are used to develop the emissions data that are the
cornerstone of a host of important environmental decisions made by EPA;
State, local, and Tribal agencies; industries; environmental groups; and
others. These decisions include facility permitting, development of
control strategies, and compliance and enforcement decisions. According
to a 2001 Government Accountability Office (GAO) report,3 EPA's data
show that, nationally, emissions factors are used for about 80 percent of
emissions determinations. Emissions factors data are also used to measure
environmental progress and demonstrate program results under the
Government Performance and Results Act of 1993.4 Table 1.1 provides
some of the key uses of emissions factors.
3 April 200! GAO Report, EPA Should Improve Oversight of Emissions Reporting by Large Facilities
(GAO-01-46)
4 The Government Performance and Results Act holds Federal agencies accountable for measuring program
outcomes and reporting results annually to Congress and the public.
-------
Table 1.1: Key Uses of Emissions Factors
Emissions Factors
Uses
Determining Source
Classification
Establishing and
Enforcing Permit Limits
Permit Fees
Issuing Maximum
Achievable Control
Technology Standards
Plant-wide Applicability
Limit and Emissions
Trading
Measuring
Environmental
Progress
Description
EPA classifies a facility to be a minor or major source by
determining the facility's potential to emit; this impacts the level
of air pollution control equipment needed.
Based on emissions measures or estimates, EPA and States set
emissions limits via operating permits.
Using the emissions estimates, States calculate annual fees a
facility must pay for the emissions released.
EPA writes standards requiring major sources of hazardous air
pollutants to install the best control technology available for an
industry sector.
Both of these regulatory approaches measure emissions from a
more holistic viewpoint. For plant-wide applicability limits, some
point sources that increase in emissions are offset by
corresponding decreases from other point sources at the same
facility. This same type of offsetting method is used when
facilities trade emissions credits to other facilities emitting over
the limit.
The amount of emissions reduced remains a key measure of
environmental progress. One of the most important databases
for tracking the amount or emissions reduced is EPA's National
Emissions Inventory (NEI).
Emissions factors play a pivotal role in many decisions, especially in
measuring environmental progress. For example, every 3 years, EPA
prepares a national database of air emissions information - the NEI -
based on emissions factors and other input from States, industry, and other
stakeholders. This inventory depends heavily on emissions factors for
stationary sources (such as factories) and mobile sources (such as trucks
and automobiles). The NEI database contains information on sources that
emit criteria air pollutants (six common air pollutants that harm human
health and the environment for which specific standards are set), as well as
hazardous air pollutants, also known as air toxics. Key uses of the
inventory include:
• Conducting air dispersion modeling and analysis;
• Developing control strategies to reduce pollution levels;
• Issuing air regulations;
• Performing risk assessments;
• Screening sources for compliance investigations;
• Tracking short- and long-term trends in emissions; and
• Measuring program results in EPA's annual performance plan.
When an area of the country does not meet the National Ambient Air
Quality Standards, EPA and the States gather information about the
-------
emissions rates of various sources in that area (often derived from
emissions factors) to target important sources for control. Through its
annual Trends Report, EPA uses the emissions inventory data to gauge
progress in meeting its goals of reducing air pollution. Emissions factors
play a key role in assessing such annual progress.
How Are Emissions Factors Maintained and Accessed?
In the AP-42 series of emissions factors, EPA assigns a qualitative rating
to the emissions factor based on the quantity and quality of the data used
to develop the factor. The AP42 series is EPA's recommended source of
air pollutant emissions factors for both criteria and hazardous air pollutant
emissions, and contains over 17,000 rated emissions factors for more than
200 air pollution source categories.
The Factor Information Retrieval (FIRE) Data System is a database
containing both rated and unrated emissions factors. FIRE contains all
AP-42-rated factors, as well as approximately 4,400 unrated emissions
factors that EPA recommends for use. FIRE also contains a list of source
classification codes, and information about industries' operating systems,
processes, and chemicals emitted. FIRE incorporates new or revised
emissions factors from AP-42.
The Clearinghouse for Inventories and Emissions Factors is EPA's
electronic repository of the most up-to-date information on inventories and
emissions factors, including AP-42 and FIRE. The clearinghouse
facilitates the exchange of emissions factors and information on emissions
inventories between Federal, State, and local agencies; industry; private
citizens; universities; contractors; and foreign governments. The
clearinghouse also provides historical inventory information and
emissions estimation guidance.
Scope and Methodology
To assess the adequacy of EPA's Emissions Factors Program, we
reviewed documentation related to AP-42 emissions factors, including:
the development and prioritization of emissions factors, the process used
for rating emissions factors, key uses of emissions factors, and policies
and procedures for implementing the Emissions Factors Program. We
also conducted interviews with officials from EPA at both the
Headquarters and regional levels; officials from several States and an air
pollution association, and emissions factors experts. We conducted our
field work from March 2005 to October 2005, in accordance with
Government Auditing Standards, issued by the Comptroller General of the
United States.
-------
As part of our evaluation, we considered the results of a prior EPA O1G
report, Emission Factor Development (Report No. 6100306), dated
September 30,1996.
Additional details on our scope and methodology are in Appendix A.
-------
Use of Unreliable Emissions Factors
Adversely Impacts Key Environmental Decisions
EPA has made progress in emissions factors development since our 1996
review, primarily in the increased number of factors receiving ratings on
quality. However, the large percentage of low quality factors has not
diminished. EPA officials have identified the inappropriate use of
emissions factors for key environmental decisions, such as permit limits
and the level of air pollution control equipment installed at specific
facilities, resulting in the release of significant amounts of unidentified
and uncontrolled emissions. For example, according to EPA enforcement
records, three industries - petroleum refineries, wood products, and
ethanol production - operated with insufficient control equipment
primarily because emissions limits were significantly underestimated due
to the emissions factors used. EPA, through separate enforcement actions,
required companies in these industries to install additional emissions
controls, resulting in the combined reduction of over one million tons of
pollutants.
Absent EPA intervention through enforcement actions or stringent
guidance, industries will have little incentive to ensure that the emissions
factors used are of known and acceptable quality. This pertains not only
to existing factors but to ones that still need to be developed, especially
quality factors for sources of fine Particulate Matter. For example, factors
will be used by EPA and States to develop fine Particulate Matter control
strategies. As EPA continues to measure and report progress in reducing
harmful air pollutants, it needs to improve the quality of emissions factors
estimates. This will improve environmental decision-making related to air
pollution and human health.
EPA Rated More Factors But Quality of Many Factors
Remains Low
In our September 1996 OIG report, we noted that emissions factors were
unavailable for many sources of air pollution and, when available, many
were unreliable. This was attributed to significant funding cuts that
materially affected EPA's ability to meet an increased demand for quality
emissions factors. Table 2.1 shows that, as of March 1996, EPA rated
56 percent of the emissions factors as either below average or poor
(24 percent plus 32 percent, respectively). As of September 2004, that
number had increased to 62 percent (28 percent plus 34 percent,
respectively). EPA based its ratings on test methods, quantity of data, and
-------
whether the facilities tested represented a sufficient random sample of the
industry.
Table 2.1: Comparison of Emissions Factors Ratings (March 1996 and September 2004)
Factor
: Rating
A
B
C
D
E
Qualitative
; Description
Excellent
Above Average
Average
Below Average
Poor
Total
As of March 1996
Number of
Factors
1.270
1,190
1,513
2,077
2,788
8,838
Percent
of Total
14%
13%
17%
24%
32%
As of September 2004
Number of
Factors
2,135
1,829
2,619
4,740
5,787
17,110
Percent
of Total
12%
11%
15%
28%
34%
Our 1996 OIG report emphasized the need for EPA to invest more
resources in the emissions factors program and to develop alternative
approaches to factor development by increasing industry and State
involvement. The Agency did increase its emissions factors efforts, as
evidenced by the increased number of factors it had rated since the 1996
OIG review. Since March 1996, EPA nearly doubled the number of rated
emissions factors, from 8,838 to 17,110. As of September 2004, 4,409
emissions factors, or about 20 percent of the total of 21,519, were still
listed as unrated. Overall, the number of rated emissions factors has
increased, which reflects some improvement in the quality of emissions
factors. For example, emissions factors that previously had a lower rating
received a higher rating, and some factors previously not rated have been
rated.
Additionally, since our 1996 report the Agency has taken the following
steps to improve the Emissions Factors Program:
• Surveyed emissions factor users, including States and industries,
on their emissions factors needs.
• Initiated the development of an automated database to store test
data results for future emissions factors development.
• Worked with an industrial association to develop new emissions
factors.
• Began a pilot project to adjust emissions factors for non-inventory
use.
An EFPAG pilot project is assessing two industry sectors - hot mix
asphalt plants, and pulp and paper plants. The goal of the pilot is to
evaluate the impact of adjusting excellent- and above average-rated
emissions factors for uncertainty. Emissions factors are used widely for
hot mix asphalt plants by source owners and regulatory authorities,
-------
particularly in calculating production limits below thresholds that would
trigger aspects of the air operating permits program. Pulp and paper plants
generally rely on emissions factors to calculate fugitive hazardous air
pollutants and VOCs from these plants. In some instances, these
emissions can account for 60 to 80 percent of a site's emissions.
EFPAG's study found that if emissions factors were adjusted to better
account for uncertainty, almost all of the 3,600 hot mix asphalt plants in
the United States likely would have to recalculate production limits. As a
result, some plants would be unable to retain their synthetic minor source
status, thus making them subject to the Title V air operating permits
program and, as a result, potentially subject to stricter regulations and
State or local air toxics rules. EFPAG also found the use of adjusted
emissions factors may cause some revisions to State Implementation Plan
model projections, causing some plants to reduce emissions below current
levels.
Misuse of Emissions Factors Resulted in Significant
Unidentified and Uncontrolled Emissions
To date, EFPAG's pilot study has identified the potential emissions impact
and possible regulatory consequences of using emissions factors to
estimate emissions. There are several instances where the actual
emissions impact from misuse of emissions factors has been identified,
and this impact has been substantial. For example, we examined three
industries where EPA officials indicated emissions factors were not
acceptable for the decisions being made - petroleum refineries, wood
products, and ethanol production. For these three large industries, EPA
had indicated emissions were significantly underestimated due to the
emissions tactors used. As a result of pollutants not being previously
identified or controlled, EPA, through separate enforcement actions, had
required companies to install additional emissions controls, resulting in the
reduction of over one million tons of pollutants for the three industries
combined.
According to EPA enforcement records, for years the three industries
operated with insufficient control equipment because the emissions limits
in permits and the annual emissions reported by the individual industry
facilities significantly understated the actual amount of emissions released
into the atmosphere. EPA decided to examine the reliability of the
emissions factors measures for each industry sector and, using new, more
accurate measures, found actual emissions to be much higher. Table 2.2
illustrates emissions reductions, civil penalties, and control investments by
each industry.
10
-------
Table 2.2: Summary of Air Violations for Three Industries Related to
Emissions Factors Use
Industry
Petroleum Refineries
Wood Products
Ethanol Production
Total
Companies
57
5
14
**76
Sites
57
*21
91
169
Civil
Penalties
(millions)
$45.0
$289.0
$6.6
$340.6
Control
Investment
(millions)
$2,600.0
* $175.0
$240.5
$3,015.5
Pollutants
Removed
(tons)
765,000
177,000
116,750
1,058,750
" This Information available for only three wood products companies.
" These companies represent a portion of the Industry, not the entire Industry.
Through consent agreements reached with facilities in these sectors, EPA
and the States required more stringent permits and installation of
additional pollution controls to lower emissions. EPA staff indicated the
problems noted for these three industries regarding poor quality emissions
factors are occurring in many other industries. For the three industries,
emissions estimating techniques were improved as a result of EPA
enforcement actions. However, correcting such problems through
enforcement actions is both costly and time-consuming. Increased
scrutiny of the use of emissions factors is needed if expected
environmental benefits are to be realized. Details on each of the three
industries follow.
Petroleum Refineries
Air quality problems in the Houston-Galveston area demonstrate the
serious consequences in terms of cost and effectiveness of air pollution
control strategies related to the petroleum industry. Because EPA
declared the Houston-Galveston area in severe noncompliance with
Federal air quality standards for ozone,5 the metropolitan area was
required to develop a control strategy to reduce Nitrogen Oxides (NOx)
and VOC emissions. Based on modeled ozone predictions using
emissions inventory data, Texas devised a strategy to reduce ozone
precursor emissions that called for the reduction of NOx emissions by
90 percent.
In August 2000, Texas conducted a comprehensive research project
assessing source contributions to the State's air quality problems. Based
on the comparison of ambient measurements of VOC concentrations to the
reported emissions inventory estimates, the Texas 2000 Air Quality Study
found that VOC emissions from petroleum refineries were significantly
under reported in the emissions inventory. This primarily involved under
5 Ground-level ozone is not emitted directly into the air, but is created by chemical reactions between NOx
and VOC in the presence of heat and sunlight.
11
-------
reporting of emissions from flares,6 process vents, and cooling towers, as
well as from fugitive emissions (leaks). The under-reporting was caused
largely due to the use of poor quality emissions factors. The quality of
one emissions factor was so poor that for short durations, actual emissions
from the flaring process could be as much as 50 times higher than
emissions calculated using the emissions factor. This was because the
factor did not account for variables such as the composition of the fuel
being burned and the efficiency of the flaring equipment. This emissions
factor was developed in the early 1980's and had not been updated at the
time of the 2000 Texas review.
Texas revised its emissions estimates based on its 2000 review, using
improved emissions estimating techniques and new modeling. The
revised estimates showed that additional VOC reductions would be needed
to meet national air quality standards, while required NOx reductions
under the State Implementation Plan could be eased from 90 to 80 percent.
As a result of what happened in Houston-Galveston, the regional planning
authority in the Philadelphia area decided to more closely study refinery
VOC emissions. The study disclosed many of the same problems noted
for the Houston-Galveston area, and recommended that Texas' guidelines
be used when estimating emissions. The California Bay Area Air Quality
Management District also closely examined emissions estimates, found
similar results, and issued a new rule to obtain more accurate emissions
estimates to ultimately reduce emissions.
Wood Products7
In 1988, EPA began investigating a suspected nationwide pattern of Clean
Air Act violations by a prominent wood products company. EPA found
that the company had failed to obtain required permits for new
construction and other modifications at some of its facilities. The
company had used a poor quality emissions factor for estimating VOC
emissions and, as a result, claimed it was not subject to the permitting
requirements. This emissions factor underestimated VOC emissions
because it was derived from a test method that substantially understated
VOC emissions containing oxygen. The industry subsequently developed
a new emissions factor that adequately accounted for VOC emissions
containing oxygen.
Flaring is an engineering practice that provides for process equipment to immediately release gases to a
device (a flare) where they can be quickly and safely incinerated. The proper use of flaring is a good
engineering practice because flares can prevent damages, fires, explosions, and injuries to employees.
7 The wood products industry sector includes manufacturers of plywood, panelboard, medium density
fiberboard, and oriented strand board. The drying and pressing processes involved in this sector are a large
source of emissions.
12
-------
EPA used this information as the basis for successfully negotiating the
largest Clean Air Act civil penalty ever collected by EPA (up to that time),
and the second largest under any environmental statute. Under the terms
of a consent decree, the company was required to pay $11.1 million in
civil penalties and install state-of-the-art pollution control equipment
valued at $70 million. In the 1990's, EPA reached consent agreements
with three other wood product companies after identifying similar
violations. In addition to civil penalties, the consent agreements instructed
the companies to install the Best Available Control Technology to reduce
VOC emissions, conduct compliance audits, and obtain appropriate
permits.
Ethanol Production
In 2002, based on recent success in the wood products industry, EPA
began investigating Clean Air Act violations by ethanol production
companies. These companies use corn to manufacture ethanol for
blending with automobile fuel, and during processing burn off gases that
emit VOCs into the atmosphere. The ethanol emissions factor was
developed using the same test method as wood products and again
underestimated the amount of VOCs being emitted into the atmosphere.
EPA alleged that these companies knowingly used a faulty emissions
factor for permitting.
In October 2002, EPA announced consent agreements with 12 ethanol
plants to install air pollution control equipment. EPA estimated that VOC
emissions will be reduced by 2,400 to 4,000 tons per year and CO by
2,000 tons per year. The settlement also wilt result in estimated annual
reductions of NOx by 180 tons, Paniculate Matter by 450 tons, and
hazardous air pollutants by 250 tons.
Increasing Demand for New Emissions Factors Illustrates
Continued Importance of Emissions Factors Program
As EPA and the States move forward with efforts to identify and regulate
sources emitting excess levels of air pollution, there will be increased
demand for new emissions factors, especially for sources of fine
Paniculate Matter (PM2.5).
By April 2008, EPA and States will need to identify sources of PM2.s,
determine the amount of emissions from these sources, and ensure that
sufficient emissions control equipment is installed at sources located in
non-attainment areas. In 1997, EPA established the PM2.5 standard and in
December 2004 designated areas as being in nonattainment with the
standard. In April 2005, these designations became effective for 208 U.S.
counties impacting a total population of 88 million people. By April 2008,
13
-------
States are required to submit a plan that identifies the sources of PM2.5,
how much PMz.s each source emits, and the actions planned for adequately
regulating these sources. The quality of emissions factors will directly
impact the effectiveness of the plans. An implementation plan must show
how an area in nonattainment will reduce emissions to meet the standards
as soon as possible, but no later than 2015, and include supporting
technical analyses based on emissions factor-developed emissions
estimates.
The Concentrated Animal Feeding Operations industry provides another
example of an area that currently does not have factors but needs them.
Appendix B provides details on EPA's efforts to address emissions factors
in this area.
Conclusion
Although EPA has made progress in emissions factors development since
our 1996 review, the need for better quality emissions factors has outpaced
the Agency's efforts to improve existing factors and develop new ones.
EPA's use of poor quality emissions factors information has hampered
environmental decisions, resulting in more than one million tons of
uncontrolled emissions spanning years, and an increased risk of adverse
health effects. This also places a disproportionate emissions reduction
burden on those facilities that use good quality emissions factors.
Although our evaluation did not address the cost impacts of rectifying the
inappropriate use of emissions factors, we believe these costs are
substantial to both EPA and the States. For each of the three industries we
reviewed (petroleum refineries, wood products, and ethanol production),
there are, at a minimum, the following costs for addressing each industry:
rewriting permits and determining the proper emissions limits, issuing
regulations to require increased air pollution controls, and the legal actions
(settlement or litigation) necessary to ensure industry complies with the
new regulations.
The three industries represent a very small portion of the universe of
emissions estimates derived through the use of emissions factors. More
effort in examining other key emissions factors may identify significant
amounts of additional unregulated pollutants. Incomplete or unreliable
emissions information can have serious consequences in terms of the
effectiveness and cost of air pollution control strategies. The public will
have little confidence in either the success or equity of EPA's decisions if
those decisions are based on questionable emissions data.
14
-------
Chapter 3
jnteftt of-Efilis
Program Needs Improvement
EPA did not have a sufficient process for developing, improving, and
rating emissions factors. While the Agency has taken various steps to
improve the Emissions Factors Program, we noted deficiencies in four key
areas that resulted in the use of poor and low quality emissions factors:
• Lack of consistent emissions factors guidance.
• Continued use of a rating system that only provides subjective
information.
• Insufficient funding for the program.
• Not having a clear strategic plan.
Given the vast number of emissions sources using factors and the differing
uses of emissions factors for making significant environmental decisions,
addressing these deficiencies will require a long-term, multi-year,
coordinated effort among EPA, State and local agencies, industry, and
others. EPA will need to assert its leadership to ensure that, in the future,
emissions factors are only allowed to be used in accordance with yet to be
issued EPA guidance on their proper use. In fiscal year 2003, OAQPS
began a reevaluation of the Emissions Factors Program to standardize and
streamline the emissions data collection and reporting process, establish
procedures for defining data uncertainty and using emissions factors in
non-inventory applications, and establish an outreach program to
communicate changes to emissions factors stakeholders. However, to
make the needed improvements to the Emissions Factors Program, EPA
will need increased focus and direction, including fully developed goals
and objectives.
Conflicting Guidance Issued for Emissions Factors
While introductory text to AP-42 states that emissions factors may be
appropriate for situations such as making source-specific emissions
estimates for area-wide emissions inventories, the text does not
recommend emissions factor use except for inventory purposes. However,
the text acknowledges that emissions factors may be used for site-specific
purposes as a last resort (emphasis added) provided appropriate caveats
concerning their limitations are in place. We noted three occasions where
EPA has issued guidance on the use of emissions factors for source-
specific purposes that conflicted with the intent of AP-42 emissions
15
-------
guidance - that is, that emissions factors not be used at individual sources.
Caveats on emissions factors use in these programs varied and may have
resulted in data being used for purposes outside EPA's intended use of
emissions factors. For example:
• Even though AP-42 specifically states that the use of emissions factors
for site-specific permit limits is not recommended, EPA issued air
permit guidance documents approving the use of emissions factors to
set permit limits at individual facilities. In response to concerns that
Title V operating permits were too costly and burdensome to
implement, in July 1995, EPA issued guidance, White Paper for
Streamlined Development of Part 70 Permit Applications, stating that
facilities could use emissions factors estimates to determine emissions
limits in the permit applications. Similarly, in August 2000, EPA
issued draft guidance that provided flexibility and allowed State and
local permitting agencies to use site-specific emissions factors as well
as other relevant emissions factors. EPA eventually rescinded the
2000 guidance but plans to re-introduce parts of it.
• EPA's New Source Review Program applies to sources undertaking
major plant modifications, and involves determining the source's
potential to emit. The New Source Review Workshop allows the use
of AP-42 emissions factors in estimating emissions to determine a
facility's potential to emit. EPA's flexible permitting approach for
New Source Review, known as the Plant-wide Applicability
Limitation, provides greater flexibility by allowing some emissions
points within a facility to increase emissions provided the overall
emissions remain below the plant-wide limit. Current rules for permit
limitations provide wide latitude in allowing the use of emissions
factors.
• EPA allowed the use of emissions factors in a recent reconsideration
of a Maximum Achievable Control Technology (MACT) standard,8
even though AP-42 specifies that emissions factors should not be used
for such source-specific purposes. MACT standards specify the
emissions control standards that must be achieved by an affected
industry. However, if the emissions factor understates actual
emissions, the facility may be required to install controls that do not
effectively reduce emissions to an acceptable level; conversely, an
overstatement may cause the industry to unnecessarily and unfairly be
required to install more costly controls. In a July 2005 reconsideration
of a rule, EPA made monitoring requirements less stringent by
allowing the use of emissions factors. However, almost all of the
s July 30, 2004 Environmental Protection Agency, 40 CFR Part 63: [OAR-2003-0048; FRL-XXXX-
X][RIN 2060-AM78] National Emission Standards for Hazardous Air Pollutants: Plywood and Composite
Wood Products; Reconsideration of original rule dated July 29, 2005.
16
-------
66 emissions factors EPA allowed for this MACT rulemaking were
either unrated or received a rating of poor or below average.
EFPAG officials said that the currently planned guidance for non-
inventory uses of emissions factors should address the more appropriate
applications of emissions factors in these programs.
Rating System Does Not Define Appropriate Uses
The current rating system for emissions factors does not provide the user
with a tool to adjust the emissions factor based on use. This system
provides a subjective A through E rating system that is of minimal value
to the user because the system does not quantify the level of uncertainty
used in the ratings. EPA places too much emphasis on the amount of test
data used to develop the factor, and not on the quality of that data. If there
is a significant amount of test data, the factor is rated high, even though
the data may be of poor quality. Further, even if a factor is rated "poor," it
will still be used. An emissions factors tool that quantifies uncertainty
would provide users with valuable information for adjusting the emissions
factor as appropriate; taking into account the level of uncertainty during
calculations can give the user a better understanding of the variations
between actual emissions and emissions factors calculations.
Quantifying the uncertainty of an emissions factor requires a more
rigorous analysis of the test data to establish a range of uncertainty for
emissions estimates. The Agency has been aware of such techniques, and
over the past 10 years has funded several studies exploring the use of
analytical techniques for quantifying uncertainty. In addition, the National
Research Council and the NARSTO9 organization reported that EPA
should increase the use of quantifying uncertainty in the development of
emissions data. Also, EPA Order 5360.1 requires EPA to assess the
quality of its data, and EPA's current emissions factors rating system is
not consistent with the Order's data quality requirements.
EPA officials told us that the majority of emissions factors are developed
using 10 points of data or less, which is substantially less than the 30 to 50
data points recommended for the development of a valid statistical
analysis. Without incurring the additional cost of obtaining more data, the
Agency can perform EPA-accepted statistical analysts on existing data to
provide users with an uncertainty rating. This will allow users to quantify
the uncertainty of emissions data developed from emissions factors.
' Formerly an acronym for "North American Research Strategy for Tropospheric Ozone," NARSTO is a
public/private partnership, whose membership spans government, utilities, industry, and academia
throughout Mexico, the United States, and Canada. Its primary mission is to coordinate and enhance
policy-relevant scientific research.
17
-------
EPA and interested stakeholders agree that the benefits of conducting
additional analysis outweigh the additional resources needed to quantify
the uncertainty. A range for the emissions estimate will allow the user to
understand how much lower or higher actual emissions vary when
compared to the emissions factor calculated estimate. For example,
instead of an emissions factor being 30 pounds of emissions per 1,000
gallons of fuel consumed, the factor would have a quantifiable range -
between 20 and 40 pounds - depending on the variables and uncertainties.
An uncertainty expert we contacted quantified the uncertainty of two key
A-rated emissions factors in EPA's AP-42 database. Both factors are
rated "Excellent" - the highest level. Further, the two emissions factors
represented a significant amount of emissions nationwide - estimated
emissions from a total of 32,569 coal-fired boilers. The potential
variances when uncertainty is considered are shown in Table 3.1.
Table 3.1: Effect of Emissions Factors Uncertainty on Estimates for Coal-fired Boilers*
Boiler Type
Wall-fired
Tangential-fired
Emissions
Estimate
(NOX
Emissions)
1,336,190
751,581
Emissions Change One to
Factor Uncertainty
Factor
Uncertainty
-41 .4% to
+33.2%
-31% to
+27%
Emissions
Range (tons)
996,798
435,917
Estimate of Probable
Emissions {tons}
Low High
783,007
518,591
1,779,805
954,508
* Emissions factors ranges: Wall-fired (12.9 - 29.3) Ib/ton; Tangential-fired (10-4 -19.0) Ib/ton
As Table 3.1 illustrates, EPA's best-rated NOx emissions factor for one
type of coal-fired boiler (wall-fired) has an uncertainty range of plus 33.2
to minus 41.4 percent, meaning actual emissions nationwide could range
from 783,000 tons to 1.8 million tons. Thus, without knowing the
uncertainty associated with an emissions factor, the approximately
1.3 million tons reported for wall-fired boilers may be nearly as high as
1.8 million tons. Quantified uncertainty information provides a tool to the
user to make more informed decisions for defining the appropriate uses of
the emissions factor. Depending on the situation, the user can adjust the
emissions factor based on this uncertainty. The following three uses of
emissions factors demonstrate how factors can be adjusted:
• If the factor is being used for a national estimate, the user would
most likely choose an estimate toward the middle of the range (this
is because of the law of large numbers and the likelihood that over-
and under-estimates may tend to cancel each other out).
18
-------
• If the factor is being used to establish a permit limit at a specific
facility, the user may need to select an estimate on the lower or
higher end of the range, depending on variables and uncertainties.
• If the factor is being used for emissions trading or offsetting, an
appropriate use may be the low end of the emissions factor range.
The uncertainty tool could allow the user to select an appropriate
adjustment based on its use. Improving emissions factors will largely
depend on the extent to which EPA can minimize the limitations
associated with uncertainty. Without knowing the possible range of the
estimate, users are not adequately informed of the risks associated with
using the emissions factor.
Inadequate Funding Provided and Used
Currently, EPA's emissions factors workload is largely dictated by
stakeholder needs, and EPA does not have a well-documented plan for
prioritizing its work. Understandably, EPA officials have had to operate
in a reactive mode, focusing on State and industry requests involving
specific emissions factors. However, managing emissions factors
development in this manner does not allow a systematic approach to
ensuring the most critical emissions factors are receiving the appropriate
priority.
EPA stated in its 1997 Federal Managers' Financial Integrity Act
(FMFIA) Assurance letter that it would request a substantial increase in
funding for the development of emissions factors. As shown in Table 3.2,
although EPA did request significant amounts of funding for the
Emissions Factors Program since our 1996 report, it received about
25 percent or less of the amount requested for 3 of the 7 years (1999,
2000, and 2002). However, EPA actually spent significantly less on the
Emissions Factors Program than had been appropriated, because EPA
officials said emissions factors funds had been reprogrammed to other air
program activities considered to be higher priority. As shown in
Table 3.2, over the past 7 years, EPA spent between 29 percent and
72 percent of the money it received, with about half of the funds received
being reprogrammed. From 1999 to 2005, the program received
$10,657,000, and spent a total of $5,301,000. It should be noted that there
has been renewed emphasis on emissions factors since the program was
reorganized in 2003 (see next page). For example, the amount of funding
spent in Fiscal Years 2004 and 2005 increased substantially from the two
previous years, although 2005 expenditures were still 59 percent lower
than 2001 expenditures.
19
-------
Table 3.2: Funding of the Emissions Factors Program - by Fiscal Year
Budgetary Activities
Requested Amount
Amount Received
Amount Spent
Amount Reprogrammed
1999
$4,100,000
1,100,000
323,500
(776,500)
2000
$4,950,000
1 ,200,000
490,000
(710,000)
2001
$5,096,000
3,215,600
2,326,500
(889,100)
2002
$3,075,700
873,900
310,000
(563,900)
2003
$2,590,400
1,029,700
320.000
(709,700)
2004
$1,539,500
1,595,600
573,000
(1,022,600)
2005 -
$4,652,600
1,642,200
958,000
(684,200)
Over the past 5 years, there have also been several changes in the types of
activities performed by the Emissions Factors Program, as shown in
Table 3.3. With the exception of 2005, consistent funding has been
devoted to maintaining the databases. Program improvements were non-
existent in 2002 and 2003; however, they have been a primary emphasis
during the past 2 years. With the exception of spending $200,000 to help
develop one emissions factor in 2005, the Emissions Factors Program has
largely not been involved in developing specific emissions factors since
2001, when over $1 million was spent.
Table 3.3: Emissions Factors Program Expenditure Analysis - by Fiscal Year *
•;• •-•;:MtivKte*::;; . ; :•.'
Maintenance of Emissions
Factors Databases
Development of Specific
Emissions Factors
Emissions Factors
Program Improvements
Total
1999
$180,800
71,600
71,100
$323,500
2000
$221 ,000
269,000
0
$490,000
2001
$225,000
1,132,000
969,500
$2,326,500
2002
$245,000
65,000
0
$310,000
2003
$275,000
45,000
0
$320,000
2004
$234,500
43,000
295,500
$573,000
2005:
$75,000
260,000
623,000
$958,000
' Does not Include Administrative and Training Expenditures
Given the increasing need for emissions factors and a relatively small
budget, EPA will need to better leverage industry resources to obtain
better emissions factors data. Although EPA recognizes the need to
prioritize emissions factors work and has made efforts to do so, a well-
documented set of short- and long-term priorities does not yet exist. In its
1997 FMFIA assurance letter, OAQPS stated it would refocus the
Emissions Factors Program by prioritizing the most critical emissions
factors, and hired a contractor to publish a document prioritizing
emissions factors development. In 2002, the contractor published a
document, Recommended Source Categories for AP-42 Chapter Update
and Emission Test Program, also called the Scoping Study. One OAQPS
official said the Scoping Study was intended to be the blueprint for future
work, resource allocation, and other key program decisions. To date, EPA
has not used the study to prioritize its emissions factors work. EPA
officials said that they do not believe they can properly prioritize the
competing needs of all of the emissions factors stakeholders to arrive at a
common priority list to improve emissions factors.
20
-------
We agree that while developing a priority list of emissions factors is a
challenge, it is critical for effectively managing the program. Emissions
factors users at both the State and Federal levels said that EPA needs to
prioritize its emissions factors work and fill the gaps that currently exist.
Also, the Air Quality Management Work Group to the Clean Air Act
Advisory Committee recommended that EPA review existing emissions
factors to identify the most significant needs.
Recent EPA Efforts Made to Revamp Emissions Factors
Program
EPA had recognized it did not have a sufficient process for developing,
improving, and rating emissions factors. Therefore, in fiscal year 2003,
OAQPS began a reevaluation of the Emissions Factors Program to:
• Identify ways to make the program more responsive to the broad
and diverse range of emissions factors users;
• Identify methods that would expand the capabilities for improving
the number and quality of available emissions factors;
• Identify and implement ways to improve and expedite the
emissions factors development process;
• Characterize the deficiencies of using emissions factors by
quantifying the uncertainties associated with the varied uses; and
• Provide users with alternative methods of quantifying emissions to
reduce the levels of uncertainty and to increase the accountability
of stakeholders.
Upon conclusion of the re-evaluation, EFPAG decided to revamp the
Emissions Factors Program with the primary goal of improving emissions
quantification through the use of better tools and knowledge of
uncertainty. The revamping plan includes three specific tasks:
(1) standardizing and streamlining the emissions data collection and
reporting process, (2) establishing procedures for defining data uncertainty
and using emissions factors in non-inventory applications, and
(3) establishing an outreach program to communicate changes to
emissions factors stakeholders.
Specifically, EFPAG officials said they developed an electronic reporting
tool that establishes a standardized emissions test report format with built-
in quality assurance checks. They said use of this format will enable State
and local air pollution control offices to readily assess the quality of
submitted emissions test reports and to share those report results with
others. As the tool is used, results from routine emissions tests should be
easier to gather and those results should already be quality assured.
21
-------
Consistent use of the tool should increase both the quantity and quality of
emissions factors.
For non-inventory programs such as New Source Review and Title V
permitting, EFPAG conducted case studies identifying the impact of over-
or under-estimating source emissions through use of emissions factors.
Acting on the results of those studies, statistical analyses of excellent and
above average emissions factors were performed to determine the range of
uncertainty associated with these factors. EFPAG indicated it is preparing
those analyses for external peer review, and at the same time, is assessing
the impact of adjusting emissions factors for non-inventory programs.
Once the analyses are validated, EFPAG plans to develop guidance for
using emissions factors for non-inventory programs, and where applicable,
the adjustments needed for emissions factors use for each program.
EFPAG officials said that this guidance will improve emissions estimates
and decisions based on those estimates by reducing the uncertainty
inherent with emissions factors use.
Finally, EFPAG is upgrading the existing Internet Web site, named
WebFIRE, from a static to an interactive mode. Future plans for this
newly-designed Web site include the ability to collect, screen, and adapt
emissions data from the electronic reporting tool; calculate average
emissions factors; and provide individual users with site-specific values
that incorporate associated uncertainties tailored to their specific program
application in accordance with future guidance. WebFIRE is also to
provide the means to distribute new emissions factors, guidance, and
procedures, as well as other means to better provide emissions
information.
EPA should be commended for its efforts, but should place greater
emphasis on improving the quality of these factors. EFPAG agreed that it
should expand the electronic rating tool to include pollutants other than
Particulate Matter, ensure the guidance specifies how program specific
adjustments are to be made, and program the interactive portion of
WebFIRE to accept new data and to calculate specific adjustments.
Comprehensive Plan Needed to Improve Data Collection and
Set Priorities
The lack of a comprehensive strategic plan hinders EPA's ability to ensure
the program is moving in the right direction, meeting its goals and
objectives, and achieving the desired results. Some key areas the plan
should include are short- and long-term goals and objectives; steps and
measures to gauge progress in meeting the goals; timeframes for meeting
milestones; and a process for reassessing and, when appropriate, revising
the plan. A coordinated and well-thought-out plan will enable EPA to
22
-------
identify cost-efficient methods of obtaining more and better data, establish
a communication strategy, and establish a system for prioritizing
emissions factors development. EPA has made efforts to address these
key areas, although it is not clear how these efforts fit into EPA's broader,
long-term plans.
Over the past 30 years, EPA's Emissions Factors Program has relied
largely on scavenging for source data to develop emissions factors and
often uses source data originally gathered for other purposes. For
example, in the 1970's, the program benefited from the source testing
conducted by EPA for the development of new emissions standards.
Throughout the 1980's and 1990's, the amount of EPA-funded source
testing steadily declined, with a corresponding increase in industry-funded
source testing. Through its own initiative, industry will sometimes
conduct source testing to test emissions, but because the industry is not
required to submit the data, EPA often is not aware of the data.
EPA needs to develop a system where it can gain access to the millions of
dollars worth of industry-generated source testing data. Currently, EPA is
working on an information system that it believes will provide the much-
needed data to the stakeholders who need to develop emissions factors. If
successful, this will be an important step toward sharing valuable data.
However, EPA will also need to develop incentives for industry to share
this information with the Agency. One way would be for EPA to issue
guidance in accordance with EPA Order 5360.1 requiring and allowing
only appropriate use of emissions factors for the environmental decisions
to be made.
Recognizing that implementation of this criteria may take some years, in
the interim EPA could allow facilities to use the upper bounds of an
emissions factor's uncertainty range. Also, if a facility believes the higher
emissions rate stipulated by EPA's new guidance does not reflect its actual
emissions, the facility could conduct stack or other testing and share this
information with EPA in an effort to decrease the uncertainty of the
emissions factor. The facility may be encouraged to do this in an effort to
lower the upper bounds of the emissions factor uncertainty range and its
resulting emissions estimate. Over time, as more facilities submit test
data, EPA may have sufficient information to decrease the uncertainty of
poor quality emissions factors and thus improve the estimates for an entire
source category.
EPA also needs to establish a communications strategy. Emissions
Factors Program officials said that there are many air quality stakeholders
with useful information, and accessing this knowledge pool can improve
the development of emissions factors. EPA has conducted some outreach,
through workshops and surveys, but the absence of a formalized
23
-------
communication strategy has resulted in valuable information not being
identified. A more formal communication strategy will allow greater
feedback and strengthen information sharing. EPA specifically needs to
share information with stakeholders from four key areas:
• Emissions inventories
• Permitting
• Industry
• Enforcement
In 2004, EPA implemented the Emissions Factors Improvement Project
survey to solicit stakeholder feedback about the Emissions Factors
Program and what improvements were needed. Of the 58 respondents, 37,
or about 64 percent, said they did not submit data to EPA. About one-
third of the 37 said they did not provide data to EPA because they did not
know they should or how they should go about it. In addition, officials we
interviewed from Regional Planning Organizations, the State and
Territorial Air Pollution Program Administrators and the Association of
Local Air Pollution Control Officials, State and Federal emissions
inventory and permitting officials, and academia stated that they have
limited or no contact with EPA on emissions factors. Instead of relying on
stakeholders to contact them, EPA should systematically contact a broad
range of stakeholders to gain access to information and knowledge that
could improve the development of emissions factors.
The information system that EPA is developing should increase
stakeholders' ability to provide additional data and access other useful
emissions factors data. However, the system will not be effectively used
unless EPA takes steps to ensure that stakeholders are aware of the
information system and its advantages. Procedures should be included in
a communications strategy to regularly solicit key information from
stakeholders.
Conclusions
EPA officials describe the emissions inventory as the foundation for the
air program, upon which everything else is built. Emissions factors
estimates are used to develop much of this inventory and, as such, are
critical measures woven into the fabric of many air quality managers'
most important decisions. Equally important, the models used to forecast
changes in air quality under alternative reduction scenarios are also
heavily reliant on accurate emissions factors and subsequent pollutant
estimates. One common use of these models provides decision-makers
with the information needed to develop control strategies to lower air
emissions. Other types of emissions factors driven models are those used
to identify the sources of pollution and link those sources to specific
24
-------
pollutants, selecting sites for increased monitoring, testing emission
reduction scenarios to predict what impact they would have on pollutants
of concern, and predicting changes in future concentrations. The results of
these analyses shape EPA's air quality planning. Use of inaccurate or
incomplete data such as emissions factors can lead to costly, unjustified
controls and/or exposures to pollutants that could negatively affect human
health and welfare. Further, these analyses help define which industries
need to be regulated, what level of controls are needed to achieve the
desired results, which sources should be targeted for enforcement
initiatives, and which pollutants are of the utmost concern. Emissions data
also influences EPA's decisions in studying health effects, performing risk
and exposure assessments, and identifying safe levels of air pollution.
The reliability and accuracy of the emissions estimates has far-reaching
implications for how EPA sets its priorities and allocates the scarce
resources available to meet the increasingly complex and daunting
challenges of identifying the most cost-effective approaches to making the
air cleaner. Emissions factors estimates, for all the uses noted above, are
at the core of this process. Our findings verified that these factors remain
significantly uncertain or unknown, despite their critical role. In fact,
these factors are drawn upon by scientists, industry, States, and others and
the rating system guiding their use does not meet the requirements of the
Agency's data quality order. While more resources have been requested
to improve these factors, EFPAG has continued to receive significantly
less money than needed over 6 of the last 7 years. Also, for the money it
did receive for emissions factors, OAQPS redirected over half of these
funds to other air program activities for 5 of those 7 years.
If EPA continues to use insufficient measures, such as poor and unknown
quality emissions factors, to determine program results, the Agency may
be overstating its progress to Congress and the public. That is, EPA may
not be reaching the goals it has claimed to reach and the air may not be as
clean as the Agency claims. If progress is overstated, it may also result in
EPA and States making misinformed decisions on selecting the most
promising future actions for improving the quality of the air.
Recommendations
We recommend that the Acting Assistant Administrator for Air and
Radiation:
3-1 Establish a workgroup with representatives from emissions
inventory, permitting, industry, and enforcement, to develop an
emissions factors guidance document that addresses:
(a) The appropriate and prohibited uses of emissions factors.
25
-------
(b) The intended use of the emissions factors estimates and how to
adjust the estimates to more accurately reflect the use.
(c) How to develop localized and site-specific emissions factors.
(d) How to account for emissions factors uncertainty in newly
developed factors and key existing factors.
(e) Mechanisms for facilities and industry sectors to follow in
developing and providing to EPA emissions factors that meet
EPA Order 5360.1 data requirements.
3-2 Develop and implement a comprehensive strategic plan that
focuses on addressing future challenges expected in the Emissions
Factors Program. The plan should address development of:
(a) Criteria for prioritizing emissions factors development, for
both new factors and selected existing factors that have the
most environmental impact.
(b) A communications and partnering strategy that results in
sustained feedback from key emissions factors users, including
EPA and State permit and enforcement officials, and industry.
(c) An information system that streamlines the collection of source
test data for the development of emissions factors.
(d) Steps to ensure emissions factors uncertainty analysis is
included in the development, rating, and intended uses of
emissions factors.
(e) A Quality Management Plan that ensures data used for the
development of emissions factors meet data quality
requirements.
3-3 Once a comprehensive strategic plan is completed, have the
Director for OAQPS ensure that all funds received for the
Emissions Factors Program are actually spent on the program and
not reprogrammed to other air activities.
Agency Comments and OIG Evaluation
The Agency concurred with our recommendations and stated that they
generally align with its current improvement efforts. The Agency agreed
that developing more emissions factors with less uncertainty is important
in advancing the air program's inventory tools, but also noted that there is
a need for even more accurate and representative emissions data for non-
inventory uses - data obtained through direct emissions measurements.
We agree with the need for direct measurements of emissions, and believe
26
-------
that appropriate guidance from EPA on the proper uses of emissions
factors would provide substantial incentives for industries and State, local,
and tribal agencies to work together to obtain such direct measurements.
However, we also recognize that it may be years before such data are
obtained and provided to EPA.
The full Agency response is in Appendix C. A more detailed analysis of
Agency comments and our evaluation of those comments are in Appendix
D. The Agency will need to address each recommendation and provide
details and milestones on its plans to address the OIG recommendations
within 90 days, including:
• The status of EPA's efforts to establish a workgroup to develop
guidance for recommendation 3-1, which addresses the appropriate
and prohibited uses of emission factors. EPA should also explain
the intended use of emissions factors estimates and how to adjust
the estimates to more accurately reflect the use.
• Actions taken or planned to develop and implement a
comprehensive strategic plan, or submit a completed plan that
includes the five elements listed for recommendation 3-2.
• Specific steps taken or planned, as per recommendation 3-3, to
ensure that funds received for the Emissions Factors Program are
spent on the program and not reprogrammed to other air activities.
27
-------
Appendix A
Details on Scope and Methodology
We conducted interviews with officials from: EPA's OAQPS and Office Enforcement
and Compliance Assurance; air permitting officials from EPA Regions 4 and 6; North
Carolina and Indiana air pollution control agencies; State and Territorial Air Pollution
Program Administrators and the Association of Local Air Pollution Control Officials
(STAPPA/ALAPCO); and the Western Regional Air Partnership and Midwest Regional
planning organizations. We reviewed and discussed with these officials selected reports
related to emissions factors, including:
• An April 2005 NARSTO report, Improving Emission Inventories for Effective Air
Quality Management Across North America: A NARSTO Assessment
• Two National Academy of Sciences reports: The Scientific Basis for Estimating
Air Emissions from Animal Feeding Operations: Interim Report (2002); and Air
Emissions from Animal Feeding Operations: Current Knowledge, Future Needs
(2003)
• EPA 's Proposed Amendments to A ir Toxics Regulations for the Plywood and
Composite Wood Products Industry, July 18, 2005
• EPA's Compilation of Air Pollutant Emission Factors, Volume 1: Stationary
Point and Area Sources, AP-42, Fifth Edition, issued January 1995
• A June 2004 EFPAG survey report, Summary of Emissions Factors Improvement
Project Fact Finding Survey
• EPA's Procedures for Preparing Emission Factor Documents, November 1997
• Minutes from four Emissions Factors Development Workshops (Florida
Conference -June 2004; Research Triangle Park (North Carolina) Conference -
November 2004: and two Washington, DC Conferences - August 2004)
• WHO'S COUNTING? The Systematic Underreporting of Toxic Air Emissions,
June 2004, a joint study by the Environmental Integrity Project and the
Galveston-Houston Association for Smog Prevention
• Evaluating Petroleum Industry VOC Emissions in Delaware, New Jersey and
Southeastern Pennsylvania Final Report, October 2003, Mid-Atlantic Regional
Air Management Association
To assess management controls, we reviewed the Office of Air and Radiation's and
OAQPS's fiscal year 2004 Integrity Act Annual Assurance Letters.
To obtain an understanding of the quantification of uncertainty of emissions factors, we
interviewed an emissions factors uncertainty expert from North Carolina State University
and reviewed technical papers associated with the subject.
28
-------
Methodology Used to Calculate Emissions Factors Uncertainty for
Coal-fired Boilers in Table 3.1
Emissions estimates were obtained from the draft 2002 EPA National Emissions
Inventory (NEI). Factor uncertainty ratings were obtained for each source type from an
emissions factors uncertainty expert. Probable emissions estimates were calculated by
applying the uncertainty rating to emissions estimates, which results in a probable low
and high range of emissions. This is known as the probabilistic emissions inventory, or
PEL Using the PEI allowed us to illustrate uncertainty in the NEI. PEI is a function of
the reported emissions and the emissions factor uncertainty (WEF) and activity data
uncertainty (UFA?). It is the probable range of emissions values in which the reported
individual emissions would be found. To calculate the PEI associated with a portion of
the NEI, we obtained the reported emissions value for two Source Categorization Codes.
The UFEF for each source category was multiplied by the point estimate of the emissions
for each category (from 2002 NEI) and summed over the two categories to arrive at a
PEI:
PEI = I [(UP,*) (UFAp) (El)]
For this formula, PEI is summed over all source categories that have reported emissions
(Ib/yr).
We reviewed the following documents related to this analysis:
• Report 99-267 ( 1 999), North Carolina State University, Raleigh, North Carolina
• 2002 Emissions (NEI); Air Pollution Control Lecture Notes, North Carolina State
University
• Emission Inventory: Planning for the Future, October 28-30, 1997, Research
Triangle Park, North Carolina
• Journal of the Air & Waste Management Association, 2003
• Environmental Science and Technology, 2004
• Quantitative Analysis of Uncertainty and Variability in Environmental Policy
Making, 1992
• Risk Analysis, 2004
Prior Coverage and Followup
We followed up on EPA's actions on previous recommendations from EPA OIG Report
No. 61 00306, Emission Factor Development, September 30, 1 996. As part of this
followup work, we reviewed OAQPS's fiscal years 1997 and 1998 Integrity Act Annual
Assurance Letters. We had found that EPA was providing poor and unreliable emissions
factors to the user community. We recommended that the development of emissions
factors be included as an Agency material weakness in FMFIA reporting. In its response
to the draft report, EPA recognized the significant role of emissions factors, and as such,
indicated it would take the following corrective action:
29
-------
* Refocus the program to address the most critical factors needed.
• Request nearly four times the amount of contract resources in the fiscal year 1999
budget request to the Office of Management and Budget.
• Request $5 million of additional funding from Congress for Particulate Matter
research in fiscal year 1998 to be used for developing Particulate Matter
emissions factors.
Details on what we found as a result of our followup, as well as recommendations, are in
Chapters 2 and 3 of this current report.
We also reviewed two other reports that addressed emissions factors development,
although we did not do followup. These were a GAO report, EPA Should Improve
Oversight of Emissions Reporting by Large Facilities (GAO-01-46), April 2001; and an
EPA OIG report, Substantial Changes Needed in Implementation and Oversight of
Title V Permits If Program Goals Are To Be Fully Realized (2005-P-00010), March 9,
2005.
Limitations
Our work contained the following limitations:
• We did not review emissions factors for all pollutants from all sources.
• We did not review any emissions factors for mobile sources.
• The quantified uncertainty information in Table 3.1 for two NOx emissions
factors is unpublished data provided by the North Carolina State University
emissions factors uncertainty expert, who was contracted by EPA to perform the
uncertainty analysis.
30
-------
Appendix B
Details on Concentrated Animal
Feeding Operations Air Emissions
Concentrated Animal Feeding Operations (CAFOs) are agricultural entities that raise
animals in confined areas and pose a potential environmental risk as a result of the high
concentrations of animals and their waste. Air emissions from agriculture sources
generally have characteristics that make them difficult to control through the more
conventional control technologies used at industrial sources. The difficulty and cost of
monitoring agricultural pollution sources is a reason that CAFOs are largely unregulated
regarding air emissions.
EPA asked the National Academy of Sciences to evaluate the scientific information
needed to address CAFO air emissions issues.10 The Academy found that the basic data
needed for effective regulation and management of CAFO emissions do not exist.
Reasonably accurate estimates of air emissions from CAFOs at the individual farm level
will require defined relationships between air emissions and various factors. The
Academy also found that directly measuring emissions from CAFOs is not feasible and
existing emissions factors are generally inadequate. A major effort will be required to
develop useful CAFO emissions factors.
EPA's need for better data has led to an agreement between EPA and some sectors of the
animal industry to monitor air quality on farms. This voluntary agreement calls for a
2-year national air monitoring study on emissions. Data developed from this study will
be used to develop emissions factors. As part of the agreement, EPA indicated it will
provide certain legal protections for past and current emissions violations for farms that
participate. Environmental advocates have criticized EPA for providing this protection,
and a former EPA staff attorney said EPA should not suspend its enforcement authority
when the Clean Air Act already requires facilities to provide this data. However,
according to EPA, this settlement will result in CAFO operators funding scientifically
credible methodologies for estimating emissions as recommended by the National
Academy of Sciences in its 2003 report.
10 Air Emissions from Animal Feeding Operations: Current Knowledge, Future Needs (2003)
31
-------
Appendix C
Agency Response to Draft Report
February 23, 2006
MEMORANDUM
SUBJECT: Draft Evaluation Report: EPA Can Improve Emissions Factors
Development and Management, Assignment No. 2005-00279
FROM: William L. Wehrum (Elizabeth Craig for)
Acting Assistant Administrator
TO: J. Rick Beusse
Director for Program Evaluation, Air Issues
Office of Inspector General
Thank you for the opportunity to comment on the draft evaluation report, "EPA Can
Improve Emissions Factors Development and Management," dated December 21,2005.
My staff also appreciated the opportunity to work with you to incorporate comments on
the prior versions of the draft report.
The emissions factor (EF) concept was developed in 1970 by the Office of Air
Quality Planning and Standards National Air Data Branch to estimate emissions from the
millions of emitters in the U.S. for which measurements were either impossible to make
or had not yet been made. EFs were, and still are, the first step toward quantifying
emissions in a basic way for estimating annual emissions. EFs represented a significant
advance over previous estimating techniques or policies and were necessary and adequate
for their time.
See Appendix D
Notel
While we agree that EFs are inherently uncertain and imperfect,
developing more EFs or more certain EFs is important in advancing the air
program's inventory tools. However, we believe that focusing our efforts
only on EF development obscures the real need for more accurate and representative
emissions data for non-inventory uses. We believe the air program needs more direct
measurements of emissions for our advanced models, strategies, and national programs.
This need was emphasized strongly by the National Research Council of the National
Academies in its report, "Air Quality Management in the United States" (National
Academies Press, 2004). Currently, we are moving toward better emissions monitoring
requirements for all major polluters that will enhance our response to environmental
challenges.
32
-------
See Appendix D
Note 2
The recommendations provided by the Office of the Inspector
General generally align with our current EF program improvement
efforts. However, the report could better emphasize that: (1) EFs are
just the first step toward quantifying emissions in a basic way; and (2)
EPA is shifting its efforts toward more direct, continuous monitoring and measurement of
emissions from all major emissions sources.
For example, EPA is currently: (1) working on a process to quantify the uncertainties
associated with the applications of EFs, and (2) establishing procedures to incorporate
additional data into calculating EFs to reduce the uncertainty associated with the EF.
Even with the ability to incorporate uncertainty into the application of an EF, EFs remain
estimates and not direct emissions measurements. More advanced, accurate, continuous,
and short-term determinations of emissions are needed for major emitters, and these goals
can only be reached by shifting to direct emissions measurement and monitoring systems.
Additional editorial comments are provided in the attachment. If you have any
questions or need clarification, please contact Peter Tsirigotis of my staff at
(919)541-9411.
33
-------
See Appendix D
Note3
Attachment
(1) The draft evaluation report makes several references to the misuse or
inappropriate use of EFs. These references should be linked to the misuse or
inappropriate use of EFs in non-inventory programs or applications. We note the
following places in the draft evaluation report where clarification would be
appropriate:
• At a Glance, What We Recommend, 1st paragraph - We are making a number of
recommendations to EPA to, among other things, develop emissions factors guidance that
addresses the development and appropriate use of factors in non-emissions inventory programs;
• At a Glance, What We Found, 2nd paragraph — Emissions factors have been inappropriately
used for key environmental decisions of non-emissions inventory programs, such as setting
permit limits and reporting the level of air pollution control at specific facilities.
• Chapter !, Purpose, 2nd paragraph, 1st bullet - Emissions factors are of an acceptable
quality for use in key environmental decisions made by EPA and State and local agencies
for non-emissions inventory programs;
(2) In addition, clarification is appropriate in the following sections:
See Appendix D
Note 4
Chapter 1, Background- In the paragraph on Source Testing and
Material Balance, the cost figures are stated as absolute, when in reality
they vary from pollutant to pollutant. The cost information would be more accurately
expressed in ranges:
• Source Testing - $ 10,000 to $50,000 annual cost;
• Material Balance - $2,000 to $ 10,000 per year per process.
See Appendix D
NoteS
Chapter 3, Rating System Does Not Define Appropriate Uses and At a
Glance, What We Found- The information in the bullets below Table 3.1
in Chapter 3, Rating System Does Not Define Appropriate Uses more
correctly characterizes the adjustments to applying EFs based on data
uncertainty in inventory applications. It would be appropriate to express this smaller
range of potential errors in the example cited at the bottom of paragraph three in the What
We Found section.
> Chapter 3, Comprehensive Plan Needed to Improve Data Collection and
Set Priorities - The second paragraph in this section includes some
clarifying historical context. Placing this text in the introduction (Chapter
See Appendix D
Note 6
I, Purpose section) would provide this historical context at the outset of the report.
34
-------
Appendix D
Note 1 -
Note 2 -
Note 3 -
Note 4 -
Note 5 -
OIG Evaluation of Agency Response
We agree that emissions factors are the first step in quantifying estimates of
actual emissions and that direct measures are the preferred emissions measure.
We also agree with the Agency's increased efforts to obtain more direct
emissions measures, such as monitoring data, for non-inventory uses. We
have revised the report to reflect these points. However, considering the
extensive use of emissions factors today for both inventory and non-inventory
uses, we believe that emissions factors may continue to be used extensively
for the foreseeable future. For example, in responding to a March 2004 EPA
OIG report/' EPA informed us that it did not have the statutory authority to
require the submission of hazardous air pollutant monitoring data from State
and local agencies. Emissions factor-developed emissions inventory data will
continue to be used frequently as a cost-effective method of estimating
emissions, as compared to more expensive emissions measurements such as
direct emissions monitoring. In the future, increased monitoring data may
reduce the use of emissions factors for major sources; however, mobile and
area sources are more likely to rely on emissions factors for a longer period of
time. Therefore, we continue to believe that emissions factors may continue
to be an integral part of the Air program for a significant length of time, and
as such, should receive the resources needed to improve factor quality and the
reliability using these estimates.
We are encouraged that EPA believes our recommendations generally align
with EPA's current Emissions Factors Program improvement efforts, and
based on our work, we would agree. We do, however, differ somewhat on our
views of the projected use of emissions factors in the future. Please see
Note 1 for details.
We revised the report to reflect the use of emissions factors in "non-inventory
programs or allocations," where appropriate. For example, in the At a Glance,
What We Recommend, first paragraph, we added that we believe guidance is
needed for the non-inventory uses of emissions factors. However, in
Chapter 1, Purpose, second paragraph, first bullet, we did not change this
because our objectives were to address both inventory and non-inventory uses
of emissions factors.
We agree with this comment and have made these changes to our draft report.
Due to space limitations in the "At a Glance" section of the final report, we
could not accommodate the Agency's desire that we more fully characterize
11EPA 's Method for Calculating Air Toxics Emissions for Reporting Results Needs Improvement, Report No. 2004-
P-00012, March 31, 2004
35
-------
an illustration of the impact of emissions factor uncertainty on estimates of
emissions nationwide; however, we have added such an illustration on
page 18 below Table 3.1, and removed our discussion of this information in
the "At a Glance."
Note 6 - We agree with this comment and added a paragraph to the background section
in Chapter 1 to provide the requested historical context to the Emissions
Factors Program.
36
-------
Appendix E
Distribution
EPA Headquarters
Office of the Administrator
Acting Assistant Administrator for Air and Radiation
Assistant Administrator for Research and Development
Agency Followup Official
Agency Followup Coordinator
Audit Followup Coordinator, Office of Air and Radiation
Audit Followup Coordinator, Office of Research and Development
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Director, Office of Air Quality Planning and Standards
Deputy Director, Office of Air Quality Planning and Standards
Director, Emissions Standards Division
Acting Director, Emissions, Monitoring and Analysis Division
Director, National Exposure Research Laboratory
Audit Liaison, Office of Air Quality Planning and Standards
EPA Regions
Regional Air Program Directors
EPA Office of Inspector General
Acting Inspector General
37
------- |