^tD sr-^
* v0
wi
PRO^
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Improving air quality
Eleven Years After Agreement,
EPA Has Not Developed Reliable
Emission Estimation Methods to
Determine Whether Animal
Feeding Operations Comply With
Clean Air Act and Other Statutes
Report No. 17-P-0396
September 19, 2017
-------
Report Contributors: Richard Jones
Erica Hauck
Jim Hatfield
Kevin Good
Julie Narimatsu
Abbreviations
AFO
Animal Feeding Operation
CAA
Clean Air Act
CAFO
Concentrated Animal Feeding Operation
CERCLA
Comprehensive Environmental Response, Compensation, and Liability Act
DQO
Data Quality Objective
EEM
Emissions Estimating Methodology
EPA
U.S. Environmental Protection Agency
EPCRA
Emergency Planning and Community Right-to-Know Act
GAO
U.S. Government Accountability Office
NAEMS
National Air Emissions Monitoring Study
NAS
National Academy of Sciences
OAQPS
Office of Air Quality Planning and Standards
OIG
Office of Inspector General
PM
Particulate Matter
SAB
Science Advisory Board
USDA
U.S. Department of Agriculture
VOC
Volatile Organic Compound
Cover photos: Hogs (left) and chickens (right) in confined spaces at animal feeding
operations. (EPA photos)
Are you aware of fraud, waste or abuse in an
EPA program?
EPA Inspector General Hotline
1200 Pennsylvania Avenue, NW (2431T)
Washington, DC 20460
(888) 546-8740
(202) 566-2599 (fax)
OIG Hotiine@epa.aov
Learn more about our OiG Hotline.
EPA Office of Inspector General
1200 Pennsylvania Avenue, NW (2410T)
Washington, DC 20460
(202) 566-2391
www.epa.aov/oia
Subscribe to our Email Updates
Follow us on Twitter @EPAoia
Send us your Project Suggestions
-------
to sr^ 17-P-0396
° - * U.S. Environmental Protection Agency September 19,2017
i tkM \ Office of Inspector General
At a Glance
*/L. *,
4L prO^^
Why We Did This Review
We conducted this review to
determine what actions the
U.S. Environmental Protection
Agency (EPA) has taken to
evaluate air emissions from
animal feeding operations.
The EPA estimates there are
about 18,000 large animal
feeding operations nationwide,
which can potentially emit air
pollutants in high-enough
quantities to subject these
facilities to Clean Air Act and
other statutory requirements.
A lack of reliable methods for
estimating these emissions
prevented the EPA and state
and local agencies from
determining whether these
operations are subject to
statutory requirements.
In 2005, the EPA and the
animal feeding operations
industry entered into a
compliance agreement to
address this challenge. As part
of this agreement, the industry
agreed to fund an air emissions
monitoring study that the
EPA would use to develop
improved emission estimating
methodologies for the industry.
This report addresses the
following:
• Improving air quality.
Send all inquiries to our public
affairs office at (202) 566-2391
or visit www.epa.aov/oia.
Listing of OIG reports.
Eleven Years After Agreement, EPA Has Not
Developed Reliable Emission Estimation Methods to
Determine Whether Animal Feeding Operations
Comply With Clean Air Act and Other Statutes
Until the EPA develops
sound methods to
estimate emissions,
the agency cannot
reliably determine
whether animal feeding
operations comply with
applicable Clean Air
Act requirements.
What We Found
The industry-funded National Air Emissions
Monitoring Study (NAEMS) and the EPA's analyses
of the study's results comprised the agency's primary
actions to evaluate air emissions from animal feeding
operations over the past decade. The NAEMS
monitoring was completed more than 7 years ago at a
cost of about $15 million, but the EPA had not
finalized any emission estimating methodologies for
animal feeding operations. In addition, the EPA had
only drafted methodologies for about one-fourth of the
emission source and pollutant combinations studied in the NAEMS. The EPA
expected to develop and begin publishing emission estimating methodologies by
2009, so the methodologies could be used by the EPA, state and local agencies,
and industry operators to determine the applicability of Clean Air Act and other
statutory requirements.
Delays in developing the emission estimating methodologies stemmed from
limitations with NAEMS data, uncertainty about how to address significant
feedback from the EPA's Science Advisory Board, and a lack of EPA agricultural
air expertise and committed resources. The EPA had not finalized its work plan
or established timeframes to finish the methodologies. As a result, the
applicability of requirements to control emissions from individual animal feeding
operations remained undetermined, enforcement protections for consent
agreement participants remained in effect longer than anticipated, and a number
of agency actions on animal feeding operation emissions continued to be on
hold. Further, because the EPA had not conducted systematic planning, the
agency was at risk of developing emission estimating methodologies that cannot
be widely applied to animal feeding operations.
Recommendations and Planned Corrective Actions
We recommend that the EPA conduct systematic planning for future
development of emission estimating methodologies. Based on the results of this
planning, the EPA should determine whether it can develop emission estimating
methodologies of appropriate quality for each of the emission source and
pollutant combinations studied. If the EPA determines that it cannot develop
certain emission estimating methodologies, it should notify agreement
participants and end civil enforcement protections. For the emission estimating
methodologies that can be developed, the EPA should establish public
milestones for issuing the draft methodologies. The EPA agreed with our
recommendations, and we accepted the agency's planned corrective actions.
-------
,*^DSX
i \ UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
Q
|
-------
Eleven Years After Agreement, EPA Has Not 17-P-0396
Developed Reliable Emission Estimation Methods to
Determine Whether Animal Feeding Operations
Comply With Clean Air Act and Other Statutes
Table of Contents
Chapters
1 Introduction 1
Purpose 1
Background 1
Responsible Offices 8
Scope and Methodology 8
Prior Report 9
2 EPA Plans for Finalizing EEMs Were Not Accomplished
and Potential Air Quality Impacts Continue 10
Development of EEMs Is Years Behind Schedule 10
Responding to SAB Concerns and a Lack of Resources
Slowed Development of EEMs 12
AFO Air Emissions Remain Largely Uncharacterized and
Important Agency Actions Are on Hold 16
Conclusion 18
3 EPA Needs to Implement Systematic Planning to
Assure That EEMs Have Sufficient Quality 19
EPA Quality System 19
EPA Has Not Fully Implemented a Systematic Planning Process
to Assure a Desired Level of Quality for EEMs 21
Conclusion 22
Recommendations 23
Agency Response and OIG Evaluation 24
4 EPA Has Not Updated Some Stakeholders and Public on
Current Status of EEM Efforts 25
EPA Provided Extensive Public Outreach During Early Stages 25
EPA Has Not Publicly Communicated on EEM Development
Efforts Since 2013 25
Conclusion 26
Recommendation 26
Agency Response and OIG Evaluation 26
Status of Recommendations and Potential Monetary Benefits 27
- continued -
-------
Eleven Years After Agreement, EPA Has Not 17-P-0396
Developed Reliable Emission Estimation Methods to
Determine Whether Animal Feeding Operations
Comply With Clean Air Act and Other Statutes
Appendices
A Office of Air and Radiation Response to Draft Report 28
B Office of Enforcement and Compliance Assurance
Response to Draft Report 31
C Distribution 34
-------
Chapter 1
Introduction
Purpose
We conducted this evaluation to determine what actions the U.S. Environmental
Protection Agency (EPA) has taken to evaluate air emissions from animal feeding
operations (AFOs), including the status of the National Air Emissions Monitoring
Study (NAEMS).
Background
AFOs are agriculture operations where animals are kept and raised in confined
areas. The U.S. Department of Agriculture (USDA) has estimated that there are
about 450,000 AFOs nationwide. While the majority of these are small operations
with fewer than 300 animals, the EPA has estimated there are more than 18,000
large AFOs1 that may raise thousands of animals. For more than two decades,
movements to improve profitability within the agriculture industry have resulted
in larger AFO facilities that often are geographically concentrated. As facility size
has increased and greater numbers of animals are housed in confined spaces,
concerns have arisen regarding these facilities' impacts on the environment and
public health.
The EPA regulates certain larger AFOs under the Clean Water Act's National
Pollutant Discharge Elimination System permit program, which regulates the
discharge of pollutants to the waters of the United States. AFO air emissions are
not regulated by any AFO-specific standards under the Clean Air Act (CAA), but
AFOs that emit air pollutants in sufficient quantities can trigger CAA permit
requirements. In the late 1990s, the EPA recognized that it did not have sufficient
AFO air emissions data to develop reliable emission estimating methodologies
(EEMs) for determining whether individual AFOs are subject to CAA permit
requirements or emission reporting requirements under two other statutes: the
Comprehensive Environmental Response, Compensation, and Liability Act
(CERCLA) and the Emergency Planning and Community Right-to-Know Act
(EPCRA).2 Both CAA permitting requirements and CERCLA/EPCRA release
1 EPA water regulations define AFOs and a subset of larger AFOs called concentrated animal feeding operations
(CAFOs), and the Clean Water Act includes CAFOs as a type of point source. The CAA does not define or
reference these terms, and the EPA's Office of Air and Radiation does not distinguish between an AFO and a
CAFO. Thus, we use the term "AFO" throughout our report, even when referring to a facility that would meet the
definition of a CAFO under the Clean Water Act.
2 EPCRA and CERCLA require facilities to report emissions of certain hazardous substances if they are released in
quantities at or above certain thresholds. This includes two hazardous substances commonly released by AFOs:
ammonia and hydrogen sulfide.
17-P-0396
1
-------
reporting requirements are triggered only if a facility emits certain pollutants at or
above specific regulatory thresholds.
The agency began discussions with representatives of the AFO industry in 2001 to
address uncertainty in determining the applicability of statutory requirements for
air emissions. As a result, the EPA and certain sectors of the AFO industry3
(e.g., pork and broiler producers, egg layers, and dairy) negotiated a consent
agreement, which was published in 20054 and entered into by AFO
owners/operators who elected to participate. Under this agreement, participating
AFO owners/operators agreed to pay a civil penalty, comply with all applicable
requirements of the agreement, and participate (if selected) in a national
monitoring study. The AFO sectors agreed to fund the monitoring study to
provide data the EPA would use to develop EEMs for various AFO pollutants and
emission sources.
Air Emissions From AFOs
AFOs can release several pollutants, including but not limited to: ammonia,
hydrogen sulfide, particulate matter (PM), volatile organic compounds (VOCs)
and hazardous air pollutants. AFO air emissions come from lagoons, barns and
other structures, and manure spread on fields. Table 1 lists the key pollutants
emitted from AFOs, along with their common emission sources and associated
health and air quality effects.
Table 1: Emission sources and health effects of key pollutants from AFOs
Pollutant
Common emission sources
Health and air quality effects
Ammonia
(NHs)
Decomposition of animal
manure.
Can cause severe cough and chronic lung
disease. It also contributes directly to the
formation of PM2 5, and deposition can impact
sensitive ecosystems.
Volatile
organic
compounds
(VOCs)
Animal feed and waste.
Can cause eye, nose and throat irritation;
damage to liver, kidney and central nervous
system; and cancer. VOCs also contribute to
the formation of ground-level ozone.
Particulate
matter (PM)'
Dry manure, bedding and feed
materials, and dirt feed lots.
Exposure is linked to a variety of problems,
including decreased lung function, increased
respiratory symptoms, and premature death
in people with heart or lung disease.
Hydrogen
Sulfide (H2S)
Decomposition of animal
manure stored in wet conditions
such as lagoons.
Can cause eye and respiratory irritation at
lower concentrations. At higher
concentrations, paralysis of the respiratory
center can lead to rapid death. Excess
emissions can contribute to the formation of
PM2 5 and acid rain.
Source: EPA Office of Inspector General (OIG) analysis.
* PM includes both fine particles (PM2.5,) and coarser particles (PM10).
3 According to the EPA, state and local agencies, and an environmental organization also participated in initial
discussions on the agreement.
4 Animal Feeding Operations Consent Agreement and Final Order, 70 Fed. Reg. 4958-4977 (Jan. 31, 2005).
17-P-0396
2
-------
AFOs can be located near
residences, and some communities
have multiple AFOs nearby. For
example, several counties in eastern
North Carolina have the highest
concentration of swine AFOs in the
United States. Some studies have
raised concerns that lower-income
and minority communities are
disproportionately impacted by air
emissions from AFOs. Studies
conducted in North Carolina found
that residents living near swine
AFOs were disproportionately low-
income people of color. Air
pollution from these AFOs is
associated with the potential health
impacts listed in Table 1 above, as
well as a reduced quality of life due
to persistent odors5 and declining
property values.6
Characterizing air emissions from
AFOs is difficult due to a number of
factors. AFOs can have many and varied sources of air emissions, including barns,
houses, feedlots, pits, lagoons, basins and manure spray fields. Each of these
emission sources can emit a variety of air pollutants, and emission rates can
fluctuate depending on climate and geographical conditions, among other factors.
Further, characterizing AFO air emissions requires expertise in multiple scientific
disciplines, including animal nutrition, AFO practices and atmospheric chemistry.
The EPA and the USD A have been collaborating on a manual of voluntary best
management practices to provide AFO owner/operators and state and local
governments with options to reduce AFO air emissions. The manual contains best
management practices for reducing particulate matter, ammonia, hydrogen
sulfide, and other air emissions through various aspects of AFO management,
including feed management, manure management, land application, and other
areas. The EPA plans to publish the manual before the end of 2017, pending
agency administration approval.
Highlights from external studies on impacts
from AFO air emissions:
> Residential property values were
reduced by an average of almost
23 percent within 1.25 miles of a large
swine AFO.a
> The closer children go to school near a
large AFO, the greater the risk of asthma
symptom s.b
> Living in close proximity to large swine
AFOs may result in impaired mental
health and negative mood states, such
as tension, depression or anger.c d
a Simons, R.A. et al., 2014. The Effect of a Large Hog
Barn Operation on Residential Sales Prices in Marshall
County, KY. JOSRE. 6(1).
b Mirabelli, M. C. et al., 2006. Asthma Symptoms
Among Adolescents Who Attend Public Schools That
Are Located Near Confined Swine Feeding Operations.
Pediatrics. 118;66-75.
cBullers, S., 2005. Environmental Stressors, Perceived
Control, and Health: The Case of Residents Near
Large-Scale Hog Farms in Eastern North Carolina.
Human Ecology. 33(1).
dSchiffman, S. S. et al., 1995. The Effect of
Environmental Odors Emanating From Commercial
Swine Operations on the Mood of Nearby Residents.
Brain Research Bulletin. 37(4): 369-375.
5 Odors are not regulated by the EPA, but may be addressed under some state and local laws.
6 Simons, R.A. et al., 2014. The Effect of a Large Hog Barn Operation on Residential Sales Prices in Marshall
County, KY. JOSRE. 6(1).
Kim, J. et al., 2009. A Spatial Hedonic Approach to Assess the Impact of Swine Production on Residential Property
Values. Environ Resource Econ. 42: 509-534.
17-P-0396
3
-------
National Academy of Sciences Report on AFO Air Emissions
In 2001, the EPA and USDA jointly requested that the National Academy of
Sciences (NAS) evaluate the body of scientific information used for estimating
various kinds of air emissions from AFOs. In 2003, the NAS reported7 that
accurate emissions estimates were needed to determine AFOs' potential impacts
and to assess the implementation of measures to control emissions. The NAS also
reported that the EPA had not dedicated the necessary resources to estimate AFO
air emissions, and that the agency's approach to estimating emissions was
inadequate. That approach involved deriving emission factors from published
emissions data, as well as gathering emission factors from existing literature.
These emission factors were then applied to representative farms to estimate
annual mass emissions. The NAS reported that this approach did not account for
the variability among AFOs (e.g., differences in geography and climate) and thus
cannot adequately estimate air emissions from an individual AFO.
The NAS recommended that the EPA develop a "process-based" approach to
estimate AFO air emissions. The NAS favored such an approach for most types of
emissions as the primary focus for both short- and long-term research,8 but also
stated that short-term research should focus on providing "defensible estimates of
air emissions that could be used to support responsible regulation."9 The NAS
described process-based models as mathematical models "that describe the
movement of various substances of interest at each major stage of the process of
producing livestock products: movement into the next stage, movement in various
forms to the environment, and ultimately movement into products used by
humans."10
Air Compliance Agreement With AFO Industries
In 2002, spurred in part by uncertainty about emission levels from AFOs and
concerns about applicability of CAA requirements, representatives of the pork,
egg producers, and other AFO sectors proposed a plan to EPA officials to produce
air emissions monitoring data from AFOs. Negotiations between the EPA and
AFO sectors11 lasted for more than 2 years before an agreement was finalized in
2005. As a condition of the 2005 Air Compliance Agreement (henceforth, the
"Agreement"), the industry agreed to fund a large-scale emissions monitoring
study. The EPA was to use the emissions monitoring data to develop EEMs that
7 Air Emissions from Animal Feeding Operations: Current Knowledge, Future Needs, NAS National Research
Council (2003).
8 2003 NAS report, pp. 152-153.
9 2003 NAS report, p. 25.
10 2003 NAS report, p. 9.
11 Participating AFO sectors included egg layers, broiler chickens, dairy cattle and swine. The turkey sector was a
part of the negotiations as well, but not enough turkey AFO owners/operators signed up to fund monitoring. The
Agreement did not cover beef cattle.
17-P-0396
4
-------
AFOs could apply to estimate their emissions and determine the applicability of
CAA permitting and CERCLA/EPCRA release reporting requirements. Once a
facility applied the EEMs to determine its emissions, the facility was to submit all
required CAA permit applications and/or report any hazardous substance releases
requiring notice under CERCLA/EPCRA.12
The Federal Register Notice (henceforth, the "Notice") that published the
Agreement included the EPA's expectation that the emissions monitoring study
would begin in 2005 and last 2 years. The Notice also described the EPA's
expected timeframes for completing the tasks subsequent to the study. Based on
these original expectations, the EPA would begin publishing final EEMs in 2009,
and AFOs would have obtained any necessary permits and installed emission
controls by 2010. Figure 1 shows the timing for these different activities.
Figure 1: Expected timeframes for monitoring study and EEM development
AFOs Assess Emissions and Apply for Permits
EPA Publishes EEMs
End Monitoring
Start Monitoring
_
AFOs install emission Controls
~ ~ +
2005 2006 2007 2008 2009 2010 2011 2012 2013
Source: OIG analysis of the Notice publishing the Agreement. 70 Fed. Reg. 4958-4977
(Jan. 31, 2005).
12 In a 2008 rule, the EPA exempted from CERCLA Section 103 reporting requirements all releases of hazardous
substances to the air from animal waste at AFOs. The rule also exempted such releases from EPCRA Section 304
reporting requirements, except when AFOs confine a number of animals at or above the large CAFO threshold, as
defined under Clean Water Act regulations. However, on April 11, 2017, the U.S. Court of Appeals for the District
of Columbia Circuit ruled in favor of a group of environmental organizations that challenged the exemption and
ordered that the 2008 rule be vacated (Waterkeeper Alliance etal. v. EPA). On July 17, 2017, the EPA filed a
motion requesting the Court grant a stay of the ruling for six months to allow the EPA time to develop guidance for
farms on reporting requirements. On August 16, 2017, the Court ordered a stay of the ruling through November 14,
2017. The EPA has 75 days from August 16, 2017, to request an extension of the stay if needed.
17-P-0396
5
-------
Primary provisions for AFOs participating in the Air
Compliance Agreement include:
> Pay up to $2,500 per farm to fund a 2-year emissions
study.
> Agree to make their property available for emissions
monitoring if selected as a monitoring site for the
study.
> Pay a civil penalty ranging from $200 to $1,000,
depending on the size and number of AFOs covered
by the participant's Air Compliance Agreement.
> Receive protection from enforcement actions for civil
violations of the CAA, CERCLA and EPCRA, to last
until either (1) the EPA finalizes EEMs, or (2) the EPA
notifies the facility that it was unable to finalize EEMs.
The EPA entered into 2,568 separate
agreements with AFO owners and
operators, which covered about 13,900
AFOs in 42 states. According to the EPA,
these 13,900 AFOs comprise more than
90 percent of the largest AFOs in the
United States. Figure 2 illustrates the
percentage of all Agreement participants
by type of animal raised.
Under the Agreement, participating
AFOs were granted a release and
covenant not to sue for potential CAA,
CERCLA and EPCRA violations
alleged in the Agreement (henceforth,
"civil enforcement protections") until
the EEMs are developed and AFOs
apply for applicable CAA permits and
report qualifying releases under
CERCLA and EPCRA, or the EPA
determines it cannot develop EEMs and
notifies Agreement participants
accordingly.
Figure 2: Agreement participants
by type of animal raised
Dairies, 4%
Broiler
Chickens.
35%
tgg
Layers,
19%
Source: EPA
Monitoring Study Methodoiogy
About $15 million was collected from the AFO sectors participating in the
Agreement to fund the NAEMS emissions study. The NAEMS protocol provided
the framework for the field sampling plan, and was developed through a
collaborative effort of industry experts, university scientists, EPA and other
government scientists, and other stakeholders knowledgeable in the field. The
Agricultural Air Research Council—a nonprofit organization established by
industry—was responsible for managing and disbursing funds for the study.
The Agricultural Air Research Council was also responsible for selecting a
Science Advisor to develop a detailed study design and quality assurance plan,
and to oversee the emissions monitoring work, including work conducted by the
contracted principal investigators. The principal investigators—most of whom
were researchers at land grant universities with expertise in animal agriculture
and/or emissions measurement—carried out the monitoring at selected sites. EPA
staff did not collect monitoring data, but conducted audits at monitoring sites to
ensure that proper techniques and protocols were followed.
17-P-0396
6
-------
Monitoring was conducted at 27 total sites (i.e., specific sources of emissions
such as abam or a lagoon).13 Measurements of ammonia, particulate matter (PMio
and PM2.5),14 total suspended particulates, VOCs, hydrogen sulfide, and carbon
dioxide15 were taken at broiler chicken, egg layer, swine, and daily confinement
sites (e.g., houses and bams). Measurements of ammonia, hydrogen sulfide, and
VOCs were taken at swine and dairy open-source sites (e.g., lagoons and basins).
Figure 3 shows the location of monitoring sites across the country.
Figure 3: NAEMS monitoring site locations
Source: 01G analysis of NAEMS site reports.
Other types of measurements were also taken at monitoring sites to help
characterize emissions. These measurements included meteorological data (such
as temperature and wind speed), and information on the number of animals at
AFO monitoring locations, how the animals were housed, and how their waste
was managed The Agreement stated that the EPA would use data from the
NAEMS and any other relevant data to develop EEMs.
13 The 27 monitoring sites were located at 23 AFOs. Monitoring was conducted at two sites (emission sources) for
four of the 23 participating AFOs.
14 PM10 describes inhalable particles with diameters that are generally 10 micrometers and smaller. PM2.5 describes
fine inhalable particles with diameters that are generally 2.5 micrometers and smaller.
15 While carbon dioxide was measured at confinement sites as part of the NAEMS, the EPA never intended to create
EEMs for carbon dioxide emissions.
Dairy (2)
Swine (2)
Dairy (2)
Layer (2)
Dairy (2)
Dairy (1)*
Layer(1)
Broiler (2)
Dairy (1)
17-P-0396
7
-------
Responsible Offices
The EPA office primarily responsible for development of the Agreement was the
Office of Enforcement and Compliance Assurance. The EPA office responsible
for developing EEMs from the NAEMS data is the Office of Air Quality Planning
and Standards within the EPA's Office of Air and Radiation, while the Office of
Research and Development plays a supporting role.
Scope and Methodology
We conducted our performance audit from April 2016 through May 2017,
in accordance with generally accepted government auditing standards. Those
standards require that we obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our objective. We
believe the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objective.
To address our objective, we identified and reviewed applicable statutes,
regulations, policies and guidance, including sections of the CAA and the Clean
Water Act, CAA permitting requirements and thresholds, and the Agreement and
associated monitoring protocol. To help us determine the status of the EPA's
NAEMS, as well as other efforts to evaluate AFO air emissions, we obtained and
reviewed EPA emission reports and analyses, NAEMS-related reports and
studies, an EPA Science Advisory Board (SAB) report, and documents related to
EPA legal proceedings.
To determine state efforts to address AFO air emissions, we reviewed state
regulations and programs for a selected number of states. We also reviewed petitions
requesting that the EPA regulate AFO air emissions, and an administrative complaint
alleging discrimination against minorities in North Carolina in permitting AFOs. In
addition, we reviewed academic studies and reports to determine AFO air emissions
and health impacts, and potential disparate impacts in overburdened communities.
We interviewed EPA staff and managers in the Office of Air Quality Planning and
Standards, the Office of Enforcement and Compliance Assurance, the Office of
Research and Development, the Office of Civil Rights, the Office of Water, and
EPA Region 4 (which covers North Carolina), to gain an understanding of EPA
actions to evaluate and address AFO air emissions. We also interviewed the
following stakeholders to discuss the Agreement and the history and status of the
NAEMS:
• USDA's Natural Resources Conservation Service staff.
• SAB members who reviewed the EPA's draft EEMs.
• An AFO industry advisor.
• AFO academic researchers at Purdue University, North Carolina State
University, and University of North Carolina-Chapel Hill.
17-P-0396
8
-------
In addition, we interviewed organizations (Sierra Club, Food & Water Watch,
EarthJustice, Waterkeeper Alliance) that submitted CAA petitions to regulate AFO
emissions. We also interviewed organizations that submitted a Title VI
administrative complaint (the North Carolina Environmental Justice Network and
the Rural Empowerment Association for Community Help) alleging discrimination
in AFO permitting in North Carolina.
To assess internal controls, we reviewed EPA policies and guidance on quality
assurance, including the following:
• The EPA's Quality Policy.
• The EPA's Procedure for Quality Policy.
• The EPA's Guidance on Systematic Planning Using the Data Quality
Objectives Process.
• The EPA's Office of Air Quality Planning and Standards' Quality
Management Plan.
We also reviewed the quality assurance project plans developed for the NAEMS
and early draft EEM development.
Prior Report
In September 2008, the U.S. Government Accountability Office (GAO) issued a
report on AFOs titled Concentrated Animal Feeding Operations: EPA Needs More
Information and a Clearly Defined Strategy to Protect Air and Water Quality from
Pollutants of Concern ("GAO-08-944"). GAO reported that the EPA did not have
the data needed to effectively regulate CAFO air emissions; specifically, the EPA
lacked data on air emission from CAFOs, which the EPA is trying to address
through the NAEMS. GAO found that the EPA lacked consistent and accurate data
for CAFOs regulated under the Clean Water Act, and that such data—like the
locations of the CAFOs—could assist with an assessment of CAFO air emissions.
GAO reported that two, then-recent decisions by the EPA suggest that the agency
had not yet determined how it intended to regulate air emissions from CAFOs:
• The EPA proposed to exempt releases to the air of hazardous substances
from farm manure from both CERCLA and EPCRA notification
requirements.
• The EPA stated it will not make key regulatory decisions on how federal
air regulations apply to CAFOs until after the NAEMS is completed.
GAO recommended that the EPA (1) reassess the data collection efforts of the
NAEMS, and (2) establish a strategy and timetable for developing process-based
emission estimating protocols for CAFOs. GAO determined that the EPA has
implemented the first recommendation but has not completed the second one.
17-P-0396
9
-------
Chapter 2
EPA Plans for Finalizing EEMs
Were Not Accomplished and
Potential Air Quality Impacts Continue
The EPA had not published any final EEMs for AFOs, and had not finalized its
workplan or established timeframes for completing them. Moreover, progress had
been limited since 2013, when the EPA's SAB concluded that draft EEMs
developed by the EPA should not be applied on a national scale as intended, and
made several recommendations to improve the EPA's statistical analyses. At the
time of the Agreement in 2005, the EPA expected that it would begin publishing
final EEMs in 2009. Further, the EPA expected that by 2010 the AFO industry
would have used the EEMs to assess their emissions, apply for any applicable
CAA permits, and install any necessary emission reduction controls.
The EPA collaborated with a committee of external stakeholders to develop a
protocol they believed would provide sufficient, representative data for the EPA's
EEM development efforts. However, public comments submitted to the EPA on
the planned NAEMS protocol, and the 2008 GAO report, questioned whether the
NAEMS would provide enough data to produce scientifically and statistically
valid EEMs. As a result of the delays, individual AFOs have not applied EEMs to
determine whether their air emissions were significant enough to require CAA
permits and related emissions controls, while civil enforcement protections for
Agreement participants remained in effect.
Development of EEMs Is Years Behind Schedule
Based on the original expectations for completion of the tasks in the Notice, the
NAEMS monitoring would have been completed in 2007, and the EPA would have
begun publishing EEMs in 2009. By 2010 all facilities would have done the
following:
1. Applied the EEMs to determine whether they met or exceeded CAA
permitting and/or CERCLA/EPCRA release reporting thresholds, and
whether permitting and reporting were required.
2. Submitted any required CAA permit applications and CERCLA/EPCRA
release notifications.
3. Implemented the mitigation and emission control requirements described
in their permits. At this point, the protections from civil enforcement
actions under the Agreement would have ended for participating AFOs.
17-P-0396
10
-------
However, EPA staff told us that this timeline did not account for time required for
the EPA's Environmental Appeals Board to approve individual agreements,
which took longer than anticipated and was not completed until December 2006.
Further, it did not account for monitoring that occurred on a rolling basis, and thus
took more than 2 years to complete.
The NAEMS monitoring was completed in early 2010, about 2 years later than
originally expected. The EPA began developing draft EEMs after monitoring was
completed. In 2012, the EPA placed its draft EEMs on its public website for
public comment. Draft EEMs covered eight16 of the 3617 emission source and
pollutant combinations described in the Agreement. The EPA's Office of Air and
Radiation also submitted the draft EEMs to the SAB to obtain feedback on EEM
development and related questions. The SAB conducted its review of draft EEMs
in 2012 and issued its final report18 on April 19, 2013.
At the time we finished our review in May 2017, the EPA had not finalized any
draft EEMs, or developed any additional draft EEMs. According to the 2005
Agreement, the EPA expected to begin publishing final EEMs within 18 months
after completion of the NAEMS monitoring.
Figure 4 shows a timeline of expected and actual NAEMS and EEM development
activities up to the 2013 SAB final report.
16 These included EEMs to estimate six different types of emissions from broiler chicken houses, and EEMs to
estimate ammonia emissions from dairy and swine lagoons/basins. Also, see Table 2.
17 According to the Office of Air and Radiation, the number of EEMs that will ultimately be developed will be
influenced by factors such as differences in production, management and building conditions, as well as availability
of sufficient data.
18 SAB Review of Emissions-Estimating Methodologies for Broiler Animal Feeding Operations and for Lagoons and
Basins at Swine andDairy Animal Feeding Operations, EPA-SAB-13-003 (2013).
17-P-0396
11
-------
Figure 4: Expected and actual NAEMS/EEM development timeline
AFOs install emission Controls
End Monitoring
Start Monitoring
%T
2005
2011 2012 20121
2007
Monitoring Ended
Monitoring Started
EPA Publishes EEMs
EPA Published Draft EEMS
SAB Final Report on Draft EEMs
AFOs Assess Emissions and Apply for Permits
Source: OIG analysis of EPA documents.
Responding to SAB Concerns and a Lack of Resources Slowed
Development of EEMs
The SAB identified several concerns with the draft EEMs, and the Office of Air
and Radiation did not agree with some of the concerns. Since that time, EEM
development slowed considerably, as the EPA decided how to address the SAB's
concerns. The EPA also encountered resource constraints and a lack of available
technical expertise.
17-P-0396
12
-------
Table 2 shows all emission source and pollutant combinations from the
Agreement,19 and the draft EEMs that were developed and submitted to the SAB
for review.
Table 2: Status of EEM development
"O
o
E
sr
3
PM2.5
PM-io
TSP
H2S
VOC
nh3
ijilijllilllililij
jlgljljljljlgl
illllilllilllilili!
~
= Planned,
not developed
= Planned,
draft developed
Broiler Dairy
Chicken Barns
Houses (NV)
Dairy Laying
Barns Hen
(MV) Houses
AFO Type/Emission Source
Swine
Barns
Swine Dairy
Lagoons Lagoons
/Basins /Basins
Source: OIG analysis.
PM2.5: Particulate matter <2.5 micrometers
PM10: Particulate matter <10 micrometers
TSP: Total suspended particulates
H2S: Hydrogen Sulfide
VOC: Volatile organic compounds
NH3: Ammonia
SAB Review of Draft EEMs and EPA Response
The SAB concluded that the data and methodology used to develop the draft
EEMs limited the ability of the models to estimate emissions beyond the small
number of AFOs in the NAEMS data set. Specifically, the SAB concluded that
the number of sites monitored was too small relative to the size of the industry;
the models were based on variables that did not accurately predict emissions; the
EPA should not have combined swine and dairy lagoon/basin data; and there were
significant limitations with the VOC data for broiler houses. Thus, the SAB
recommended that the EPA not apply the current version of the EEMs beyond the
AFOs in the EPA's dataset.
19 This included EEMs for both naturally ventilated (NV) and mechanically ventilated (MV) dairy barns, as
discussed in the Agreement.
17-P-0396
13
-------
The SAB made a number of other recommendations, including having the EPA
do the following:
• Expand its dataset by collecting data from monitoring efforts outside of
the NAEMS, and using NAEMS data that were initially excluded due to
the EPA's data completeness criteria.
• Not generate an EEM for VOC emissions from broiler operations based on
current data limitations.
• Separate swine and dairy lagoon/basin data that had been combined for
EEM development.
The SAB also advocated a process-based modeling approach to EEM
development. The NAS had advocated a process-based modeling approach to
estimating emissions in its 2003 report. Further, in its 2008 report, GAO
recommended that the EPA establish a strategy and timetable for developing
process-based emission estimating protocols for CAFOs. The SAB noted the
following:
Process-based models would be more likely to be successful in
representing a broad range of conditions than the current models
because process-based models represent the chemical, biological
and physical processes and constraints associated with emissions.
According to the Notice publishing the Agreement, the EPA believed process-
based modeling to be a large and complex, multiyear research effort. Therefore,
the EPA planned to develop an interim modeling approach, which would be a
critical first step to developing a process-based modeling approach. The modeling
approach the EPA ultimately selected for the draft EEMs used a statistical
software program to analyze the various measurements taken during the NAEMS
and identify those variables that predict emissions. The SAB recognized that
the EPA may need to apply statistical approaches to assess emissions while it
was developing and evaluating process-based models, and thus made
recommendations to improve the EPA's chosen approach, as discussed above.
Prior Stakeholder Feedback Questioned the NAEMS Monitoring
Approach
The SAB's concerns about the number of monitoring sites being able to support
statistically based EEMs was raised in public comments on the Agreement and
protocol before the EPA began developing EEMs, and was also raised by GAO in
its 2008 report on the EPA's efforts to characterize AFO pollution.
After the NAEMS protocol was made available for public comment in 2005, a
number of external groups expressed concerns about the study design and whether
it would lead to credible scientific data. Some commenters noted that the number of
17-P-0396
14
-------
sites was too limited to account for all the differences in types of manure
management systems, building types, ventilation rates, feeding practices, animal
type/age, animal management practices, geography and climate. The commenters
noted that even for the types of AFOs monitored, there may not be a sufficient
number of samples to establish statistically valid EEMs. Similarly, in its 2008
report, GAO cautioned that the NAEMS may not supply the data needed for the
EPA to develop comprehensive EEMs. Further, the GAO report stated that
members of the USDA Agricultural Air Quality Task Force had raised concerns
about the quality and quantity of data collected, and had pushed for the EPA to
review the first 6 months of monitoring data to determine whether the study needed
to be revised to yield more useful information.
According to the NAEMS Science Advisor, the NAEMS protocol could be viewed
as a compromise between compliance-minded EPA, budget-minded industry, and
publication-minded universities. The protocol developers decided on an approach
that focused on collecting a comprehensive set of monitoring data (i.e., 2 years of
monitoring many different AFO conditions and parameters) at a smaller number of
sites, as opposed to collecting a smaller set of data at more sites. According to the
EPA, costs were a factor in this decision because mobilizing and demobilizing
equipment and then re-deploying at new sites would have depleted funds that could
be used for monitoring. The protocol developers believed the chosen monitoring
plan would produce sufficient data for EEM development if the selected monitoring
sites represented how the majority of animals are raised in the different AFO
sectors.
Although the monitoring protocol was developed as a joint effort of researchers
knowledgeable about AFO operations and/or monitoring techniques, there was no
comprehensive internal or external assessment to determine the amount of data
needed to produce scientifically and statistically sound EEMs that could be
extrapolated nationwide. The EPA did not perform such an assessment prior to
the NAEMS, in part, because it did not know which variables would most impact
air emissions at AFOs, and the agency wanted to see the data before selecting a
modeling approach for EEM development. Also, the NAEMS protocol and
detailed monitoring plans were not peer reviewed to ensure that the NAEMS
would provide sufficient data for the EPA to produce a comprehensive suite of
EEMs.
EPA's EEM Development Activities Since 2013 Have Been Limited
The EPA planned to continue EEM development using its statistically based
approach, and had addressed some of the SAB's recommendations by acquiring
additional data sets from other external studies, and reassessing data completeness
criteria for the NAEMS. However, the draft EEMs that were submitted to the
SAB for review had not been revised, and the EPA had not begun developing
EEMs for the remaining 28 emission source and pollutant combinations.
17-P-0396
15
-------
A lack of expertise and resources slowed the agency's work on the EEMs in
recent years. According to EPA managers, the agency in recent years did not have
staff with combined expertise in agricultural emissions, air quality and statistical
analysis. At the time the NAEMS protocol was developed, the EPA had more
applicable expertise, but the key staff involved in the NAEMS protocol
development retired. Further, competing priorities resulted in the EPA's Office of
Air and Radiation putting the EEM effort largely on hold. The EPA had dedicated
few agency resources to develop EEMs since the SAB's 2013 final report. The
few remaining agency staff who worked on the NAEMS and subsequent data
analysis were reassigned to other work, and the EPA stopped funding the contract
for NAEMS analysis.
The EPA's most recent draft EEM development work plan, dated March 2016,
provided a general framework for how the EPA intended to finish all planned
EEMs. The draft plan stated that a new staff person with appropriate expertise,
along with student contractor support, would complete the EEMs. The EPA hired
the new staff person and a student contractor in January 2017 but had not yet
finalized timeframes for completing EEM development.
AFO Air Emissions Remain Largely Uncharacterized and
Important Agency Actions Are on Hold
Eleven years after the Agreement was entered, and 7 years after NAEMS
monitoring was completed, the EPA, state, local and tribal permitting authorities,
and AFO owners/operators, did not have scientifically defensible EEMs needed to
make CAA and CERCLA/EPCRA compliance determinations. In addition, the
civil enforcement protections for the approximately 14,000 AFOs that participated
in the Agreement remained in effect more than 6 years after intended expiration,
and several important EPA actions were on hold pending development of the
EEMs.
CAA Permit and CERCLA/EPCRA Reporting Determinations Have
Not Been Made
Per the Agreement, facilities were not required to determine whether CAA
permitting and CERCLA/EPCRA reporting requirements apply to them until the
EPA publishes final EEMs. However, once final EEMs are published,
participating AFOs are required to use the EEMs to estimate their emissions and
come into compliance with applicable CAA and CERCLA/EPCRA requirements.
17-P-0396
16
-------
The Agreement states that a source with emissions exceeding CAA major source
permitting thresholds20 would have to do one of the following:
1. Apply for and obtain a permit that contains a federally enforceable limitation
or condition that limits the potential emissions to less than the applicable
major source threshold for the area where the source is located.
2. Install either best available control technology in attainment areas,21 or
lowest achievable emission rate technology in nonattainment areas;22 and
then obtain a federally enforceable permit that incorporates the appropriate
best available control technology or lowest achievable emission rate limit.
Delays in issuing the EEMs resulted in facilities continuing to have civil
enforcement protections even if their emissions were exceeding CAA permit or
CERCLA/EPCRA reporting thresholds. Given the lack of reliable EEMs, it was
difficult to estimate how many facilities could be exceeding these thresholds.
However, monitoring conducted as part of an EPA enforcement case in 2003
demonstrated that some large AFOs can exceed the 250-tons-per-year permitting
threshold for PM emissions. That monitoring showed total PM emissions of 550
and 700 tons per year at two large egg-layer AFOs.
The NAEMS Science Advisor analyzed NAEMS data for the pork and egg-layer
industries, which indicated that pork and egg-layer AFOs could frequently exceed
the EPCRA reporting threshold for ammonia of 100 pounds per day. This analysis
indicated that pork and egg layer AFOs were unlikely to exceed 250 tons per year
of PMio or VOC emissions. However, the Science Advisor's analysis did not
address whether pork or egg-layer AFOs would trigger permitting requirements in
poor air quality areas where regulatory thresholds are lower.
Paragraph 38 of the Agreement required the EPA to end civil enforcement
protections for those emission sources/types for which the EPA determined it was
unable to develop EEMs. As described earlier, the SAB concluded in its 2013 report
that the EPA did not have sufficient data to develop an EEM for VOC emissions
from broiler houses. Further, more than 7 years since completion of the NAEMS, the
EPA had only developed draft EEMs for eight of a possible 36 emission source and
pollutant combinations. However, the EPA had not yet determined that it could not
develop any of the EEMs, and thus has not waived enforcement protections for any
of the emissions sources covered under the 2005 Agreement.
20 Applicable regulatory thresholds range from 10 tons per year in areas with very poor air quality (called extreme
nonattainment areas) to 250 tons per year in areas with adequate air quality (called attainment areas).
21 A geographic area is generally designated as being in attainment for a particular criteria air pollutant if the
concentration of that pollutant is found to be at or below the regulated or "threshold" level for the associated
National Ambient Air Quality Standard.
22 A geographic area is generally designated as being in nonattainment for a particular criteria air pollutant if the
concentration of that pollutant is found to exceed the regulated or "threshold" level for the associated National
Ambient Air Quality Standard.
17-P-0396
17
-------
Agency Actions on Hold
Delays in completing EEMs have also caused important agency efforts to address
or mitigate AFO air emissions to remain on hold. The EPA stated it would not
take the following actions until the EEMs are finalized because they are needed to
inform the agency's decision-making:
Responding to citizen petitions to regulate AFOs. The EPA has received
petitions to address AFO emissions in regulations beyond the current
permitting CAA provisions, which include a 2009 petition to list and regulate
AFOs as a source category under CAA Section 111, and a 2011 petition to
regulate ammonia as a criteria pollutant under CAA Sections 108 and 109.
EPA staff told us they did not plan to evaluate the need for additional
regulations as laid out in these petitions until the EEMs are finalized.
Defining "source" for aggregation purposes. The aggregation of sources
pertains to how many individual emission sources are counted together to
determine whether a facility exceeds CAA major source status, and thus
impacts how many facilities could exceed permitting thresholds. For example,
if a barn at an AFO rather than the entire AFO is a "source," fewer AFOs
could be impacted by CAA permitting requirements. The EPA had not issued
guidance on this issue, and said it planned to do so after developing the EEMs.
In our view, final EEMs are also necessary for the EPA to develop compliance
and enforcement strategies for Agreement non-participants, and to assess whether
AFO emissions may contribute to disproportionate health risks to certain
communities.
Conclusion
The EPA's ability to characterize and address AFO air emissions is unchanged
since its 2005 Agreement with the AFO industry intended to produce reliable
emissions estimation methods. As a result, individual AFOs have not estimated
their emissions to determine whether they are required to implement controls to
reduce emissions and/or report their emissions to the appropriate emergency
responders. Additionally, other important agency actions pertaining to AFO air
emission estimates continue to be on hold.
Timeframes for completing EEM development were uncertain, as staffing and
contract support needed to finish EEMs only recently became available and the
EPA had not yet finalized its work plan at the time we completed our review.
Further, SAB concerns about the EPA's EEM development methodology have not
been resolved. Despite these uncertainties, parties to the 2005 Agreement
continue to receive protections from civil enforcement actions. We make
recommendations in Chapters 3 and 4 of this report.
17-P-0396
18
-------
Chapter 3
EPA Needs to Implement Systematic Planning to
Assure That EEMs Have Sufficient Quality
The EPA's planning for EEM development did not describe the desired level of
quality needed for the EEMs' intended purpose of estimating individual AFO air
emissions nationwide. The establishment of such criteria is a key component of
systematic planning for agency projects. In accordance with the agency's data
quality policies, EPA organizations should conduct systematic planning to ensure
that projects will result in scientific products that are defensible and useful for
their intended purpose. The agency's most recent EEM development draft work
plan used the terms "appropriate" and "meaningful" to describe final EEM
products, but did not explain how those terms would be used to evaluate the
quality or acceptability of the final EEMs.
As noted in Chapter 2, the agency's SAB concluded that the EPA's 2012 draft
EEMs were not suitable for their intended purpose. Consequently, if the agency
does not fully implement systematic planning for future EEM development, the
EPA is at risk of producing additional draft EEMs that are not sufficient for
estimating air emissions at individual AFOs across the United States.
EPA Quality System
The EPA's Procedure for its Quality Policy23 establishes management principles
and responsibilities for ensuring that EPA products and services meet agency
quality-related requirements, and are of sufficient quality for their intended use and
support the EPA's mission to protect human health and the environment. The
policy applies to agency products and services developed for external distribution
or dissemination. Each EPA organization is responsible for implementing the EPA
Quality Policy and Program within its organization. Requirements for
implementing the program include conforming to the minimum specifications of
the American National Standards Institute and the American Society for Quality
Control standard, ANSI/ASQC E4-1994.24
23 EPA Chief Information Officer's CIO Order 2106-P-01.0 (October 20, 2008).
24 Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental
Technology Programs, the American National Standards Institute and the American Society for Quality Control
(1994). This standard is the basis for the EPA's Quality System.
17-P-0396
19
-------
At the project level, these minimum specifications include the following:
• Using a systematic planning approach (e.g., the data quality objectives
process) to develop acceptance or performance criteria covered by the
EPA Quality Policy.
• Having approved quality assurance project plans, or equivalent
documents, for all applicable tasks involving environmental data.
To implement the EPA's Quality Policy, each EPA organization must develop a
quality management plan that describes its quality system, documents its quality
policies, and identifies the environmental programs to which the quality system
applies. The EPA's Office of Air Quality Planning and Standards (OAQPS)
developed a quality management plan that describes options for ensuring
that OAQPS projects are of appropriate quality for their intended purpose.
These options include elements of systematic planning to ensure that quality
considerations are built into a product at the beginning, and consist of
(1) developing a quality assurance project plan or similar document, and/or
(2) conducting pre-dissemination review (e.g., peer review) of information.
According to the OAQPS quality management plan, quality documentation
describes in detail the activities that must be implemented to assure that the
results of work will satisfy the stated performance criteria. The performance
criteria may be stated in the form of data
quality objectives (DQOs). DQOs are
qualitative or quantitative statements that
clarify project technical and quality
objectives, define the appropriate type of
data, and specify tolerable levels of potential
decision errors (e.g., uncertainty) that will be
used as the basis for identifying the data
needed to support decisions. EPA quality
assurance guidance25 recommends that
systematic planning include DQOs when
data are to be used to make a regulatory
decision or emission estimations.
Further, DQOs should be specified for a project before the agency develops its
plan for collecting the data, since the DQOs will drive key data collection
decisions. For estimation, the guidance states that DQOs are typically expressed
in terms of acceptable uncertainty (e.g., width of an uncertainty band or interval)
associated with a point estimate at a desired level of statistical confidence.
The DQO process is the agency's
recommendation when data are to
be used to make some type of
decision (e.g., compliance or
noncompliance with a standard) or
estimation (e.g., ascertain the
mean concentration level of a
contaminant).
Guidance on Systematic Planning
Using the Data Quality Objectives
Process, EPA QA/G-4, February 2006
25 The EPA's Guidance on Systematic Planning Using the Data Quality Objectives Process (2006).
17-P-0396
20
-------
The OAQPS quality management plan also provides for the pre-dissemination
review of OAQPS information as a way to provide assurance that quality has been
built into the information that the office disseminates. The quality management
plan cites peer review as an example of pre-dissemination review, and notes that it
can be appropriate to incorporate the pre-dissemination review for project
planning documents, such as the quality assurance project plan, prior to beginning
the project.
EPA Has Not Fully Implemented a Systematic Planning Process to
Assure a Desired Level of Quality for EEMs
The EPA's planning process for EEM development had yet to establish data
quality objectives describing the performance or acceptance criteria for the final
EEMs. While extensive planning went into assuring the quality of the monitoring
data collected during the NAEMS, this planning did not describe the desired
quality of the end products resulting from EPA analysis of the NAEMS data
(i.e., the EEMs), or the type and extent of emissions monitoring data needed to
produce EEMs of desired quality.
Planning for Draft Development of EEMs Was Not Systematic
Ideally, under a systematic planning process, a methodology for producing a final
product at the desired quality is determined up front. This methodology then
drives the data collection efforts. When data are
to be used to make some type of decision or
estimation, the EPA recommends that the
desired level of quality be expressed in the form
of DQOs. As noted in Chapters 1 and 2, the
EPA collaborated with external scientists to
develop the monitoring protocol. However,
several factors influenced the scope of the
NAEMS, and that effort was not specifically
designed to produce data to satisfy acceptance
criteria for the EEMs. Among these factors was
that, prior to the study, the EPA did not know which variables most impact air
emissions at AFOs. Thus, the EPA tried to create an EEM development
methodology using the data that was available from the NAEMS.
The NAEMS protocol stated that the NAEMS and subsequent data analyses and
interpretation would allow the EPA and livestock and poultry producers to
"reasonably determine" which AFOs were subject to CAA regulatory provisions
and CERCLA/EPCRA reporting requirements. However, as part of its planning,
the EPA did not define what was meant by "reasonably determine." The EPA
developed a quality assurance project plan for its efforts to develop the draft
EEMs that were published in 2012, but it focused on assessing the quality of
incoming data from the NAEMS and other sources. The quality assurance project
Unless some form of planning
is conducted prior to investing
the necessary time and
resources to collect data, the
chances can be unacceptably
high that these data will not
meet specific project needs.
Guidance on Systematic
Planning Using the Data Quality
Objectives Process, EPA QA/G-4,
February 2006
17-P-0396
21
-------
plan did not include DQOs or other performance criteria defining the acceptable
level of uncertainty for EEM predictions, or the quality control measures the EPA
would use to assure its statistical models were scientifically and statistically
sound.
The EPA had its draft EEMs peer reviewed by the SAB, but the agency did not
involve the SAB in its planning process to ensure that the NAEMS would provide
sufficient data for EEM development. As discussed in Chapter 2, the SAB
concluded that the EPA's draft EEMs were not useful for making compliance
determinations nationwide due to problems with the underlying data and analysis.
Plans for Completing Development of EEMs Can Be Strengthened
The EPA had not yet conducted systematic planning for the EEM completion
effort, but had developed a draft work plan. That draft work plan contained little
information about systematic planning to assure the quality of future EEMs. The
plan did not address whether a quality assurance project plan would be developed,
or commit to peer review of the planned methodology or the draft or final
EEMs.26
The draft work plan described a future scoping study that would allow the EPA to
plan activities and resources for developing "appropriate" EEMs, and stated that
EEMs developed in the future would be tested to determine whether they can
reproduce "meaningful" emissions estimates. However, the work plan did not
define or establish acceptance criteria for "appropriate" or "meaningful" EEMs.
Staff from OAQPS stated that they planned to make quality planning decisions
once the new staff person had been hired to conduct the scoping study and
subsequent EEM development.
Conclusion
As explained in the EPA's quality assurance guidance, systematic planning that
defines the level of quality required for an end product should be conducted prior
to data collection efforts, to reduce the risk that the data collected is not sufficient.
Such planning for the EEMs was not conducted prior to the NAEMS or draft
EEM development efforts, in part, because the EPA did not have a full
understanding of the factors that influence AFO air emissions. Further, the
NAEMS protocol and monitoring plans were not developed exclusively to
provide data needed for EEM development. Based on its experience and peer
review feedback in developing the initial set of draft EEMs, the EPA should be in
a better position to conduct systematic planning for the EEM completion effort.
26 In the draft plan, the EPA stated it will provide developed EEMs to "appropriate stakeholders and possibly the
Science Advisory Board" for review, and then modify the EEMs based on comments received. However, the plan
does not commit to obtaining independent, external peer review of the EEMs or the planned methodology that will
be used to develop the EEMs.
17-P-0396
22
-------
Without adequate systematic planning, the EPA is at risk of spending additional
time and resources to develop EEMs that still are not sufficient for estimating
AFO emissions nationwide.
Recommendations
We recommend that the Assistant Administrator for Air and Radiation:
1. In accordance with EPA quality assurance guidance, conduct
comprehensive systematic planning for future emission estimating
methodology development through either the quality assurance project
plan or pre-dissemination review processes.
• If the EPA chooses to develop a quality assurance project plan, it
should first develop data quality objectives for the emission
estimating methodologies.
• If the EPA chooses to conduct a pre-dissemination review, it
should obtain independent, external feedback on the adequacy of
its emission estimating methodologies development and plans prior
to beginning the project.
2. Based on the results of systematic planning, determine and document the
decision as to whether the EPA is able to develop scientifically and
statistically sound emission estimating methodologies for each originally
planned emission source and pollutant combination.
3. For the emission source and pollutant combinations for which the Office
of Air and Radiation determines it can develop scientifically and
statistically sound emission estimating methodologies, establish public
milestone dates for issuing each draft emission estimating methodology.
For any emission source and pollutant combinations for which the Office
of Air and Radiation determines that it cannot develop scientifically and
statistically sound emission estimating methodologies, notify the Office of
Enforcement and Compliance Assurance of that determination.
We recommend that the Assistant Administrator for Enforcement Compliance and
Assurance:
4. For any emission source and pollutant combinations for which the Office
of Air and Radiation determines it cannot develop emission estimating
methodologies, notify Air Compliance Agreement participants of this
determination, and that the release and covenant not to sue for those
emission sources and pollutant types will expire in accordance with
paragraph 38 of the 2005 Air Compliance Agreement.
17-P-0396
23
-------
Agency Response and OIG Evaluation
The Office of Air and Radiation agreed with Recommendations 1, 2 and 3,
and provided acceptable planned corrective actions and completion dates.
The Office of Enforcement and Compliance Assurance agreed with
Recommendation 4 and provided an acceptable corrective action plan.
The agency also provided technical comments that were incorporated into our
final report as appropriate. Appendices A and B contain the responses to our
report from the Office of Air and Radiation, and the Office of Enforcement and
Compliance Assurance, respectively.
17-P-0396
24
-------
Chapter 4
EPA Has Not Updated Some Stakeholders and
Public on Current Status of EEM Efforts
The 2005 Air Compliance Agreement between the AFO industry and the EPA
generated significant stakeholder and public interest in AFO air emissions, and any
actions the agency would take to address those emissions. Leading up to the
monitoring study, and for 2 years after monitoring data was available, the EPA
provided frequent public updates related to the NAEMS and EEMs. However, since
the SAB's 2013 final report, the agency had provided only high-level updates to
selected stakeholders. This left many stakeholders and the public uninformed about
the current status of the work, the reasons for delays, and current timelines for
finalizing the EEMs. The EPA should resume providing public updates on the
status of EEM development through its website or other public means, to ensure the
transparency of its process and accountability in setting completion dates.
EPA Provided Extensive Public Outreach During Early Stages
The EPA issued four press releases in 2006 announcing individual agreements
entered into between the EPA and AFOs. Further, in the years after it received all
monitoring data in 2010, the EPA provided frequent updates on EEM
development efforts and the SAB's review of draft EEMs. In 2011, the EPA
published data from the NAEMS monitoring, issued a Call for Information to
collect information to supplement the NAEMS data, and updated the public on
processes related to the planned SAB review. In 2012, the EPA released its draft
EEMs for public comment.
EPA Has Not Publicly Communicated on EEM Development Efforts
Since 2013
Since the EPA posted the SAB's 2013 final report on its public website, the EPA
had not updated some stakeholders and the public on recent aspects of its
NAEMS data analysis and EEM development efforts. An OAQPS manager told
us that the agency planned to post final EEMs on its public webpage, but used
other mechanisms to provide updates on the status of EEM development. Such
updates were provided only upon request, and typically to groups with which the
agency had regular contact, such as the USDA's Agricultural Air Quality Task
Force. Numerous interested parties—including the SAB Chair, a SAB panel
member, and three external groups—told us that they had no information about
the ongoing NAEMS data analysis, the reasons for delays, or how long it might
take the EPA to publish final EEMs.
Further, staff at the USD A told us that while they periodically received high-level
updates from the EPA at Agricultural Air Quality Task Force and intra-agency
17-P-0396
25
-------
workgroup meetings, they were not aware of the EPA's current plans for
completing EEM development. The EPA's 2016 update to the Agricultural Air
Quality Task Force provided the SAB's recommendations regarding the draft
EEMs, as previous updates had done, and stated that the EPA will continue
developing EEMs to account for air emissions from AFOs.
Conclusion
Despite being years behind schedule in finalizing the EEMs, the EPA has not
provided public updates since 2013 on the NAEMS data analysis and the agency's
current efforts to finalize the EEMs. Thus, stakeholders and the public do not
know where the EPA currently stands with respect to EEM development. To
ensure transparency and accountability in completing EEMs for the $15 million
investment in the NAEMS study, the EPA should provide public updates on the
status of EEM development and establish public milestones for completion of
each draft EEM.
Recommendation
We recommend that the Assistant Administrator for Air and Radiation:
5. Provide the public with the status of emission estimating methodology
development and the agency's planned next steps for analyzing the
National Air Emissions Monitoring Study data and finalizing the emission
estimating methodologies, including the completion of milestone dates for
each draft emission estimating methodology it plans to develop.
Agency Response and OIG Evaluation
The Office of Air and Radiation agreed with Recommendation 5, and provided an
acceptable corrective action plan and completion date. The Office of Air and
Radiation also provided technical comments that were incorporated into our final
report as appropriate. Appendix A contains the Office of Air and Radiation's
response to our report.
17-P-0396
26
-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
Rec.
No.
Page
No.
Subject
Status1
Action Official
Planned
Completion
Date
Potential
Monetary
Benefits
(in $000s)
23 In accordance with EPA quality assurance guidance, conduct
comprehensive systematic planning for future emission
estimating methodology development through either the quality
assurance project plan or pre-dissemination review processes.
o If the EPA chooses to develop a quality assurance project
plan, it should first develop data quality objectives for the
emission estimating methodologies,
o If the EPA chooses to conduct a pre-dissemination review,
it should obtain independent, external feedback on the
adequacy of its emission estimating methodologies
development and plans prior to beginning the project.
23 Based on the results of systematic planning, determine and
document the decision as to whether the EPA is able to develop
scientifically and statistically sound emission estimating
methodologies for each originally planned emission source and
pollutant combination.
23 For the emission source and pollutant combinations for which the
Office of Air and Radiation determines it can develop
scientifically and statistically sound emission estimating
methodologies, establish public milestone dates for issuing each
draft emission estimating methodology. For any emission source
and pollutant combinations for which the Office of Air and
Radiation determines that it cannot develop scientifically and
statistically sound emission estimating methodologies, notify the
Office of Enforcement and Compliance Assurance of that
determination.
Assistant Administrator for
Air and Radiation
3/31/18
Assistant Administrator for
Air and Radiation
Assistant Administrator for
Air and Radiation
6/30/18
6/30/18
23 For any emission source and pollutant combinations for which
the Office of Air and Radiation determines it cannot develop
emission estimating methodologies, notify Air Compliance
Agreement participants of this determination, and that the
release and covenant not to sue for those emission sources and
pollutant types will expire in accordance with paragraph 38 of the
2005 Air Compliance Agreement.
26 Provide the public with the status of emission estimating
methodology development and the agency's planned next steps
for analyzing the National Air Emissions Monitoring Study data
and finalizing the emission estimating methodologies, including
the completion of milestone dates for each draft emission
estimating methodology it plans to develop.
Assistant Administrator for 9/30/182
Enforcement and
Compliance Assurance
Assistant Administrator for 6/30/18
Air and Radiation
1 C = Corrective action completed.
R = Recommendation resolved with corrective action pending.
U = Recommendation unresolved with resolution efforts in progress.
2 If applicable, based on the Office of Air and Radiation's deteimination in response to Recommendation 3.
17-P-0396 27
-------
Appendix A
Office of Air and Radiation
Response to Draft Report
^ pro\&
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D C. 20460
JUN 2 3 2017
OFFICE OF
AIR AND RADIATION
MEMORANDUM
SUBJECT: Response to the Office of Inspector General's Draft Report, Emissions From
Animal Feeding Operations Remain Largely Uncharacterized More Than 7 Years
After Study Completed (Project No. OPE-FY16-0018)
The EPA's Office of Air and Radiation (OAR) appreciates the opportunity to review and comment
on the Office of Inspector General (OIG) draft report titled "Emissions From Animal Feeding
Operations Remain Largely Uncharacterized More Than 7 Years After Study Completed" OAR
agrees in general with the OIG's recommendations.
OAR's current task is the development of Emissions Estimating Methodologies (EEMs) for
animal feeding operations (AFOs), using statistically-based methodologies to develop
emissions factors for select types of AFOs from data collected through the National Air
Emissions Monitoring Study (NAEMS). In partnership with the Office of Research and
Development (ORD), we are undertaking this effort and incorporating a National Academy
of Sciences (NAS) recommendation that the EPA develop an interim method for estimating
emissions while we participate in a longer-term effort to develop process-based EEMs. In
addition, our work will include objectives outlined in the 2005 Air Compliance Agreement
(Agreement) the EPA entered into with participating AFOs. The AFO sectors represented in
the Agreement covered the monitoring study costs. Individual participating AFOs did not
directly pay monitoring study funds. The EPA remains committed to fulfilling this goal of
developing EEMs for AFOs based on scientifically and statistically sound methods. The
FROM: Sarah Dunham
Acting Assistant Adminisirator
TO:
Carolyn Copper
Assistant Inspector General
Office of Program Evaluation
Office of Inspector General
17-P-0396
28
-------
statistically-based EEMs must also be easily implemented by the agricultural community and
other users, and be based on non-proprietary inputs.
While we generally agree with your characterizations of the Agreement and the associated
NAEMS, there are a few places where information in the draft report is slightly unclear where the
information differs from our understanding of specific facts. Please refer to the attached list of
these instances and suggested revisions intended to help clarify and improve the draft report's
accuracy.
Below are OAR's responses to the OIG's specific recommendations (recommendation numbers 1,
2, 3 and 5), which we developed in consultation with ORD. On June 9, 2017, OECA provided a
separate response to recommendation number 4 as it is assigned to their office. In the attached
technical comments, we provide suggested additional detailed changes in the form of a markup.
Recommendation 1: In accordance with EPA quality assurance guidance, conduct
comprehensive systematic planning for future emission estimating methodology
development through either the quality assurance project plan or pre-dissemination review
processes.
• If the EPA chooses to develop a quality assurance project plan, it should first
develop data quality objectives for the emission estimating methodologies.
• If the EPA chooses to conduct a pre-dissemination review, it should obtain
independent, external feedback on the adequacy of its emission estimating
methodologies development and plans prior to beginning the project.
Response 1: OAR and ORD agree with this recommendation and have initiated development of a
quality assurance project plan (QAPP) for evaluation of the data and completion of the EEMs. As
part of the QAPP development, appropriate data quality objectives will be defined. We intend to
make this document publicly available on our website (see below).
Planned completion date: FY 2018, Q2 (March).
Recommendation 2: Based on the results of systematic planning, determine and document
the decision as to whether the EPA is able to develop scientifically and statistically sound
emission estimating methodologies for each originally planned emission source and pollutant
combination.
Response 2: OAR agrees with this recommendation. As noted, completion of this task is
contingent upon the results and decisions made during the QAPP development. Upon completion
of the QAPP, OAR and ORD will determine which EEMs can be completed and the appropriate
schedules for their completion. We intend to make the schedules publicly available on our website
(see below).
Planned Completion Date: As stated above, development of the QAPP is ongoing with
completion anticipated in the second quarter of FY 2018. Upon completion of the QAPP, decisions
17-P-0396
29
-------
on EEM development and schedules will be determined and transmitted to the Office of
Enforcement and Compliance Assurance (OECA). We anticipate that the schedules will be
established in third quarter of FY 2018.
Recommendation 3: For the emission source and pollutant combinations for which the Office
of Air and Radiation determines it can develop scientifically and statistically sound emission
estimating methodologies, establish public milestone dates for issuing each draft emission
estimating methodology. For any emission source and pollutant combinations for which the
Office of Air and Radiation determines that it cannot develop scientifically and statistically
sound emission estimating methodologies, notify the Office of Enforcement and Compliance
Assurance of that determination.
Response 3: OAR agrees with this recommendation and will develop a schedule for completion
of the EEMs after completion of data review and QAPP development, which is currently planned
for completion in the second quarter of FY 2018.
Planned Completion Date: As stated above, development of the QAPP is ongoing with
completion anticipated in the second quarter of FY 2018. Upon completion of the QAPP, decisions
on EEM development and schedules will be determined and transmitted to OECA and made
available to the public. We anticipate that the schedules will be established in the third quarter of
FY 2018.
Recommendation 5: Provide the public with the status of emission estimating methodology
development and the agency's planned next steps for analyzing the National Air Emissions
Monitoring Study data and finalizing the emission estimating methodologies, including the
completion milestone dates for each draft emission estimating methodology it plans to
develop.
Response 5: OAR agrees with this recommendation and will post the schedule on our website for
completion of the EEMs after completion of data review and QAPP development, which is
currently planned for completion in the second quarter of FY 2018. We anticipate providing
updates on our progress with subsequent website postings.
Planned Completion Date: As stated above, development of the QAPP is ongoing with
completion anticipated in the second quarter of FY 2018. Upon completion of the QAPP, decisions
on EEM development and schedules will be determined and milestones will be made available to
the public. We anticipate that the schedules will be established in the third quarter of FY 2018.
If you have any questions regarding this response, please contact Mike Jones, Office of Air Quality
Planning and Standards (OAQPS) Audit Liaison, at (919) 541-0528.
Attachment
17-P-0396
30
-------
Appendix B
Office of Enforcement and Compliance Assurance
Response to Draft Report
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON. DC 20460
JUN - 9 7017
OFFICE OF
ENFORCEMENT AND
COMPLIANCE ASSURANCE
VlEMOKANDl M
SUB.IEC I: Response lo lhe ()lfice of Inspector General Draft Report: "Emissions from Animal
Feeding Operations Remain Largely Uneharaeterized More Than 7 Years After Study
Completed." Project No. OPE-FYI6-OOI8 (May 12. 2017)
FROM: Lawrence I'. Star lipid I
Acting Assistant/Administrator
Ol'llee of Enforcqucnljrid Compliance Assurance
TO: Carolyn Copper
Assistant Inspector General
Office of Program Evaluation
Ofllee of Inspector < ieneral
Thank you for the opportunity to respond to the Office of Inspector General (01G) Draft Report,
"Emissions from Animal Feeding Operations Remain Largely Uneharaeterized More Than 7
Years After Study Completed" (Draft Report). The Office of Enforcement and Compliance
Assurance (OECA) appreciates OIG's careful examination of this issue, and we are committed to
following the terms of the Animal Feeding Operations (AFO) Air Compliance Agreement
(Agreement) and OIG's recommendation for OECA - Recommendation Number 4. We concur
with Recommendation Number 4, and we provide a high-level intended corrective action with an
estimated completion date below.
While we generally agree with your characterizations of the Agreement and its associated
National Air Emissions Monitoring Study (NAEMS), there are a few places where the Draft
Report is slightly unclear or where the information differs from our understanding of specific
facts. Enclosed for your consideration, we include a list of these instances and suggested
revisions intended to help clarify and improve the Draft Report's accuracy.
17-P-0396
31
-------
OECA has discussed the Draft Report with the Office of Air and Radiation (OAR) and we
understand that OAR will be providing a separate response addressing the Draft Report's
findings and recommendations for OAR - Recommendation Numbers 1, 2, 3, and 5.
OECA Response to Recommendation Number 4 — Concur
No.
Recommendation
High-Level Intended
Corrective Action
Planned Completion
Date
4
For any emission source and
pollutant combinations for
which the Office of Air and
Radiation determines it
cannot develop emission
estimating methodologies,
notify Air Compliance
Agreement participants of
this determination and that
the release and covenant not
to sue for those emission
sources and pollutant types
will expire in accordance
with paragraph 38 of the
2005 Air Compliance
Agreement.
If the EPA determines it
cannot develop emission
estimating methodologies for
any emission source and
pollutant combinations, OECA
will notify Agreement
participants in writing that the
EPA has made such a
determination and that the
release and covenant not to sue
will expire in accordance with
paragraph 38 of the
Agreement.
If necessary, OECA
will complete the
intended corrective
action within 60 days
of OAR finalizing its
determination.
We concur with OIG's recommendation that OECA notify Agreement participants if OAR
determines that it cannot develop emission estimating methodologies for any emission source
and pollutant combinations. OECA notes that this recommendation will only require a corrective
action if OAR determines it cannot develop emission estimating methodologies for any source
and pollutant combinations. Paragraph 38 of the Agreement requires the EPA to notify
Agreement participants in writing if the Agency makes such a determination. OECA intends to
continue abiding by the Agreement's terms, and we will notify Agreement participants if the
Agency determines it cannot develop emission estimating methodologies for any emission
source and pollutant combinations.
If you have any questions regarding this response, please contact OECA Audit Liaison,
Gwendolyn Spriggs, at 202.564.2439.
Attachment
cc: Susan Shinkman, OECA/OCE
Rosemarie Kelley, OECA/OCE
Lauren Kabler, OECA/OCE
Apple Chapman, OECA/OCE
Tim Sullivan, OECA/OCE
17-P-0396
32
-------
Gwendolyn Spriggs, OECA/OAP
Sarah Dunham, OAR
Robin Dunkins, OAR/OAQPS
Mike Jones, OAR/OAQPS
17-P-0396
-------
Appendix C
Distribution
The Administrator
Chief of Staff
Chief of Staff for Operations
Deputy Chief of Staff for Operations
Assistant Administrator for Air and Radiation
Assistant Administrator for Enforcement and Compliance Assurance
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Career Deputy Assistant Administrator, Office of Air and Radiation
Deputy Assistant Administrator, Office of Enforcement and Compliance Assurance
Audit Follow-Up Coordinator, Office of the Administrator
Audit Follow-Up Coordinator, Office of Air and Radiation
Audit Follow-Up Coordinator, Office of Enforcement and Compliance Assurance
17-P-0396
34
------- |