UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
April 19, 2013
EPA-SAB-13-003
The Honorable Bob Perciasepe
Acting Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: SAB Review of Emissions-Estimating Methodologies for Broiler Animal Feeding
Operations and for Lagoons and Basins at Swine and Dairy Animal Feeding Operations
Dear Acting Administrator Perciasepe:
This Science Advisory Board (SAB) report responds to a request from the EPA's Office of Air and
Radiation (OAR) to review and provide advice on scientific issues associated with development of
Emissions-Estimating Methodologies (EEMs) at two types of animal feeding operations (AFOs): EEMs
for barns or buildings at confined broiler AFO facilities and an EEM for open lagoons and basins at
swine and dairy AFO facilities. EEMs are tools for estimating air pollutant emissions from industries
where site-specific emissions data are not available. The SAB was asked to comment on various aspects
of two EPA draft reports, including the overall approach for developing the EEMs, combination of
lagoon and basin data, use of static predictor variables within the EEMs, specific approaches for
development of the ammonia (lagoon NHa) and broiler volatile organic compound EEMs, and handling
of negative and zero data results.
The EPA developed these EEMs to address requirements of a 2005 voluntary air compliance consent
agreement between the EPA and nearly 14,000 broiler, dairy, egg layer and swine AFOs. Under the
agreement, the EPA also will develop EEMs for egg-layers, and swine and dairy confinement facilities.
The EEMs will be used by the AFO industry to estimate daily and annual air emissions for use in
determining regulatory responsibilities under the Clean Air Act, the Comprehensive Environmental
Response, Compensation, and Liability Act and the Emergency Planning and Community Right-to-
Know Act. The pollutants monitored under the agreement include: ammonia, hydrogen sulfide,
paniculate matter and volatile organic compounds. The 2005 consent agreement also provides that, if the
SAB decides that the available data are not adequate to support development of the EEMs, then the EPA
can delay development of the EEMs until adequate data are available.
The EPA developed the broiler and lagoon EEMs after reviewing emissions data from two key sources:
(a) data received in response to an agency 2011 Call for Information seeking additional data on AFOs
and emissions to ensure a review of the broadest range of available scientific data and (b) the National
Air Emissions Monitoring Study (NAEMS). The NAEMS is a two-year study of emissions from AFOs
that produce pigs, broiler chickens, egg, and milk. The study was funded by the AFO industry as part of
the 2005 voluntary air compliance agreement with the EPA.
-------
The EPA's draft EEMs are described in two February 2012 draft documents: "Development of
Emissions-Estimating Methodologies for Broiler Animal Feeding Operations" (Broiler Report), and
"Development of Emissions-Estimating Methodologies for Lagoons and Basins at Swine and Dairy
Animal Feeding Operations" (Lagoon Report). The documents describe the sites monitored and the data
submitted to the EPA. They provide a detailed discussion of the statistical methodology used to develop
the draft EEMs for AFOs, which are to be applied throughout the country.
The EPA developed broiler EEMs for ammonia, hydrogen sulfide, paniculate matter and volatile
organic compounds using NAEMS emissions and process information collected from two confinement
facilities on one broiler operation in California and from two broiler operations in Kentucky. The EPA
developed swine and dairy lagoon EEMs for ammonia by combining NAEMS emissions and process
information collected from three dairies located in Indiana, Washington and Wisconsin, three swine
breeding and gestation farms located in Indiana, North Carolina and Oklahoma, and three swine growing
and finishing farms located in Iowa, North Carolina and Oklahoma.
In summary, the SAB concludes that the EPA has developed statistical models based on combined data
sets and predictor variables which have limited the ability of the models to predict emissions beyond the
small number of farms in the dataset. While basing the EEMs on data from a small number of farms
does not necessarily limit the applicability of the EEMs to national populations, the assumptions and
forms of the statistical models used in the current EEMs are not suitable for use outside the range of
parameter values in the current data. The SAB recommends that the EPA not apply the current versions
of the statistical and modeling tools for estimating emissions beyond the farms in EPA's data set. Within
the report, SAB provides recommendations for how the agency may expand the data set and the
applicability of the models.
In addition, the SAB does not support the combination of swine and dairy lagoon/basin datasets to
develop swine and dairy EEMs and finds significant problems with the EPA's approach of using static
predictor variables as surrogates for data on dynamic lagoon/basin conditions. The SAB finds significant
uncertainties associated with the broiler volatile organic compounds data used in the EPA's analysis and
concludes that these data are insufficient to support development of a broiler EEM for volatile organic
compounds at this time.
The SAB strongly recommends that the EPA develop a process-based modeling approach to predict air
emissions from broiler confinement facilities and swine and dairy lagoons/basins. Process-based models
would be more likely to be successful in representing a broad range of conditions than the current
models because process-based models represent the chemical, biological and physical processes and
constraints associated with emissions. This recommendation is consistent with recommendations
provided to the EPA in the 2003 National Research Council report, Air Emissions from Animal Feeding
Operations: Current Knowledge, Future Needs. The EPA should consider developing EEMs at a variety
of levels of complexity to provide options for producers with different levels of data availability and
data and model uncertainty.
While the NAEMS does not provide sufficient data to implement a completely rigorous process-based
modeling approach, it is sufficient to start the development and evaluation of simplified process-based
modeling approaches that would reflect the heterogeneity of AFOs. The EPA should identify critical
data gaps and begin the process of identifying key parameters to include within process-based models.
The EPA also should consider conducting a full mass balance analysis to help identify key parameters to
-------
be used in a process-based modeling approach. The SAB has identified in this report several key factors
and parameters affecting emissions that the EPA should consider to help develop process-based
modeling. The report recommends several alternative approaches for developing a draft process-based
lagoon/basin EEM for ammonia emission. The SAB also makes several recommendations regarding the
EPA's handling of negative and zero values for both direct concentration measurement and calculated
emission values.
The SAB recognizes that the EPA may need to apply statistical approaches to assess emissions while it
is developing and evaluating process-based models. The SAB provides suggestions in this report to
improve the agency's statistical approach for developing EEMs. Also, the SAB provides a number of
general and specific recommendations to improve the clarity and scientific basis of EPA's analyses
within EPA's draft Broiler Report and Lagoon Report.
The SAB appreciates the opportunity to provide the EPA with advice on this important subject. We look
forward to receiving the agency's response and to providing future advice on this topic.
Sincerely,
Dr. David T. Allen, Chair
Science Advisory Board
Enclosures
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board, a public
advisory group providing extramural scientific information and advice to the Administrator and other
officials of the Environmental Protection Agency. The Board is structured to provide balanced, expert
assessment of scientific matters related to the problems facing the agency. This report has not been
reviewed for approval by the agency and, hence, the contents of this report do not necessarily represent
the views and policies of the Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government, nor does mention of trade names or commercial products constitute a
recommendation for use. Reports of the EPA Science Advisory Board are posted on the EPA website at
http ://www. epa.gov/sab.
-------
U.S. Environmental Protection Agency
Science Advisory Board
Animal Feeding Operations Emission Review Panel
CHAIR
Dr. David T. Allen, Gertz Regents Professor of Chemical Engineering and the Director of the Center
for Energy and Environmental Resources, The University of Texas, Austin, TX
MEMBERS
Dr. Viney Aneja, Professor, Department of Marine, Earth, and Atmospheric Sciences, North Carolina
State University, Raleigh, NC
Dr. Brent Auvermann, Professor of Biological and Agricultural Engineering, Texas A&M AgriLife
Extension Service, Amarillo, TX
Dr. Peter Bloomfield, Professor, Statistics Department, North Carolina State University, Raleigh, NC
Dr. Alicia Carriquiry, Distinguished Professor and Associate Chair, Statistics Department, Iowa State
University, Ames, IA
Dr. Nichole Embertson, Nutrient Management and Air Quality Specialist, Whatcom Conservation
District, Lynden, WA
Dr. William Brock Faulkner, Assistant Professor, Department of Biological and Agricultural
Engineering, Texas A&M University, College Station, TX
Dr. Robert Hagevoort, Assistant Professor and Extension Dairy Specialist, New Mexico State
University Agricultural Science Center, Clovis, NM
Dr. Richard Kohn, Professor, Animal and Avian Sciences Department, University of Maryland,
College Park, MD
Dr. April Leytem, Research Soil Scientist, Northwest Irrigation and Soils Research Laboratory,
U.S. Department of Agriculture-Agricultural Research Service, Kimberly, ID
Dr. Ronaldo Maghirang, Professor, Biological and Agricultural Engineering Department,
Kansas State University, Manhattan, KS
Dr. Deanne Meyer, Livestock Waste Management Specialist, Department of Animal Science,
University of California, Davis, Davis, CA
Dr. Wendy Powers-Schilling, Director of the Institute for Agriculture and Agribusiness, Director of
Environmental Stewardship for Animal Agriculture, and Professor in the Departments of Animal
Science and Biosystems and Agricultural Engineering, Michigan State University, East Lansing, MI
ii
-------
Dr. C. Alan Rotz, Agricultural Engineer, Pasture Systems and Watershed Management Research Unit,
U.S. Department of Agriculture-Agriculture Research Service, University Park, PA
Dr. Paul D. Sampson, Research Professor and Director of Statistical Consulting Programs, Department
of Statistics, University of Washington, Seattle, WA
Dr. Eric P. Smith, Professor and Head, Department of Statistics, Virginia Polytechnic Institute and
State University, Blacksburg, VA
Dr. John Smith, Dairy Specialist and Professor, Department of Animal Sciences, The University of
Arizona, Tucson, AZ
Dr. Eileen Fabian Wheeler, Professor, Department of Agricultural and Biological Engineering, The
Pennsylvania State University, University Park, PA
Dr. Lingying Zhao, Associate Professor, Department of Food, Agricultural and Biological Engineering,
The Ohio State University, Columbus, OH
SCIENCE ADVISORY BOARD STAFF
Mr. Edward Hanlon, Designated Federal Officer, U.S. Environmental Protection Agency, Washington,
D.C.
in
-------
U.S. Environmental Protection Agency
Science Advisory Board
CHAIR
Dr. David T. Allen, Gertz Regents Professor of Chemical Engineering and the Director of the Center
for Energy and Environmental Resources, The University of Texas, Austin, TX
MEMBERS
Dr. George Alexeeff, Director, Office of Environmental Health Hazard Assessment, California
Environmental Protection Agency, Oakland, CA
Dr. Pedro Alvarez, Department Chair and George R. Brown Professor of Engineering, Department of
Civil & Environmental Engineering, Rice University, Houston, TX
Dr. Joseph Arvai, Svare Chair in Applied Decision Research, Institute for Sustainable Energy,
Environment, & Economy, Haskayne School of Business, University of Calgary, Calgary, Alberta,
Canada
Dr. Thomas Burbacher, Professor, Department of Environmental and Occupational Health Sciences,
School of Public Health, University of Washington, Seattle, WA
Dr. Ingrid Burke, Director, Haub School and Ruckelshaus Institute of Environment and Natural
Resources, University of Wyoming, Laramie, WY
Dr. Thomas Burke, Professor and Jacob I. and Irene B. Fabrikant Chair in Health, Risk and Society
Associate Dean for Public Health Practice, Johns Hopkins Bloomberg School of Public Health, Johns
Hopkins University, Baltimore, MD
Dr. Edward T. Carney, Departmental Senior Science Leader and Director of Predictive Toxicology
Center, Toxicology & Environmental Research and Consulting, The Dow Chemical Company, Midland,
MI
Dr. Terry Daniel, Professor of Psychology and Natural Resources, Department of Psychology, School
of Natural Resources, University of Arizona, Tucson, AZ
Dr. George Daston, Victor Mills Society Research Fellow, Global Product Stewardship, The Procter &
Gamble Company, Mason, OH
Dr. Costel Denson, Managing Member, Costech Technologies, LLC, Newark, DE
Dr. Otto C. Doering III, Professor, Department of Agricultural Economics, Purdue University, W.
Lafayette, IN
Dr. Michael Dourson, President, Toxicology Excellence for Risk Assessment, Cincinnati, OH
iv
-------
Dr. Joel Ducoste, Professor, Department of Civil, Construction, and Environmental Engineering,
College of Engineering, North Carolina State University, Raleigh, NC
Dr. David A. Dzombak, Walter J. Blenko, Sr. University Professor of Environmental Engineering,
Department of Civil and Environmental Engineering, College of Engineering, Carnegie Mellon
University, Pittsburgh, PA
Dr. T. Taylor Eighmy, Vice Chancellor for Research and Engagement, University of Tennessee,
Knoxville, TN
Dr. Elaine Faustman, Professor and Director, Environmental and Occupational Health Sciences,
University of Washington, Seattle, WA
Dr. R. William Field, Professor, Department of Occupational and Environmental Health, and
Department of Epidemiology, College of Public Health, University of Iowa, Iowa City, IA
Dr. H. Christopher Frey, Distinguished University Professor, Department of Civil, Construction and
Environmental Engineering, College of Engineering, North Carolina State University, Raleigh, NC
Dr. John P. Giesy, Professor and Canada Research Chair, Veterinary Biomedical Sciences and
Toxicology Centre, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
Dr. Barbara L. Harper, Risk Assessor and Environmental-Public Health Toxicologist, and Division
Leader, Hanford Projects, and Program Manager, Environmental Health, Department of Science and
Engineering, Confederated Tribes of the Umatilla Indian Reservation (CTUIR), West Richland, WA
Dr. Cynthia M. Harris, Director and Professor, Institute of Public Health, Florida A&M University,
Tallahassee, FL
Dr. Robert Johnston, Director of the George Perkins Marsh Institute and Professor, Economics, Clark
University, Worcester, MA
Dr. Kimberly L. Jones, Professor and Chair, Department of Civil Engineering, Howard University,
Washington, DC
Dr. Bernd Kahn, Professor Emeritus and Associate Director, Environmental Radiation Center, Georgia
Institute of Technology, Atlanta, GA
Catherine Karr, Associate Professor - Pediatrics and Environmental and Occupational Health Sciences
and Director - NW Pediatric Environmental Health Specialty Unit, University of Washington, Seattle,
WA
Dr. Madhu Khanna, Professor, Department of Agricultural and Consumer Economics, University of
Illinois at Urbana-Champaign, Urbana, IL
Dr. Nancy K. Kim, Senior Executive, Health Research, Inc., Albany, NY
-------
Dr. Francine Laden, Mark and Catherine Winkler Associate Professor of Environmental
Epidemiology, Harvard School of Public Health, and Channing Division of Network Medicine, Brigham
and Women's Hospital and Harvard Medical School, Boston, MA
Dr. Cecil Lue-Hing, President, Cecil Lue-Hing & Assoc. Inc., Burr Ridge, IL
Dr. Elizabeth Matsui, Associate Professor, Pediatrics, School of Medicine, Johns Hopkins University,
Baltimore, MD
Dr. Surabi Menon, Director of Research, ClimateWorks Foundation, San Francisco, CA
Dr. James R. Mihelcic, Professor, Civil and Environmental Engineering, University of South Florida,
Tampa, FL
Dr. Christine Moe, Eugene J. Gangarosa Professor, Hubert Department of Global Health, Rollins
School of Public Health, Emory University, Atlanta, GA
Dr. Horace Moo-Young, Dean and Professor, College of Engineering, Computer Science, and
Technology, California State University, Los Angeles, CA
Dr. Eileen Murphy, Director of Research and Grants, Ernest Mario School of Pharmacy, Rutgers
University, Piscataway, NJ
Dr. James Opaluch, Professor and Chair, Department of Environmental and Natural Resource
Economics, College of the Environment and Life Sciences, University of Rhode Island, Kingston, RI
Dr. Duncan Patten, Director, Montana Water Center, and Research Professor, Hydroecology Research
Program, Department of Land Resources and Environmental Sciences, Montana State University,
Bozeman, MT
Dr. Martin Philbert, Dean and Professor, Environmental Health Sciences, School of Public Health,
University of Michigan, Ann Arbor, MI
Dr. Stephen Polasky, Fesler-Lampert Professor of Ecological/Environmental Economics, Department
of Applied Economics, University of Minnesota, St. Paul, MN
Dr. C. Arden Pope, III, Professor, Department of Economics, Brigham Young University, Provo, UT
Dr. Stephen M. Roberts, Professor, Center for Environmental and Human Toxicology, University of
Florida, Gainesville, FL
Dr. Amanda Rodewald, Professor of Wildlife Ecology, School of Environment and Natural Resources,
The Ohio State University, Columbus, OH
Dr. James Sanders, Director and Professor, Skidaway Institute of Oceanography, Savannah, GA
Dr. William Schlesinger, President, Gary Institute of Ecosystem Studies, Millbrook, NY
VI
-------
Dr. Gina Solomon, Deputy Secretary for Science and Health, Office of the Secretary, California
Environmental Protection Agency, Sacramento, CA
Dr. Daniel O. Stram, Professor, Department of Preventive Medicine, Division of Biostatistics,
University of Southern California, Los Angeles, CA
Dr. Peter S. Thorne, Director, Environmental Health Sciences Research Center and Professor and
Head, Departmment of Occupational and Environmental Health, College of Public Health, University of
Iowa, Iowa City, IA
Dr. Paige Tolbert, Professor and Chair, Department of Environmental Health, Rollins School of Public
Health, Emory University, Atlanta, GA
Dr. Jeanne VanBriesen, Professor, Department of Civil and Environmental Engineering, Carnegie
Mellon University, Pittsburgh, PA
Dr. John Vena, University of Georgia Foundation Professor in Public Health and
Head, Department of Epidemiology and Biostatistics, Georgia Cancer Coalition Distinguished Scholar,
College of Public Health , University of Georgia, Athens, GA
Dr. R. Thomas Zoeller, Professor, Department of Biology, University of Massachusetts, Amherst, MA
SCIENCE ADVISORY BOARD STAFF
Dr. Angela Nugent, Designated Federal Officer, U.S. Environmental Protection Agency, Washington,
D.C.
vn
-------
TABLE OF CONTENTS
Acronyms and Abbreviations ix
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 9
2.1. BACKGROUND 9
2.2. SAB REVIEW 10
3. RESPONSES TO EPA's CHARGE QUESTIONS 11
3.1. THE EPA's APPROACH FOR DEVELOPING THE EEMs 11
3.2. COMBINATION OF LAGOON AND BASIN DATA 16
3.3. USE OF STATIC PREDICTOR VARIABLES 17
3.4. ALTERNATIVE APPROACHES FOR AMMONIA EMISSIONS-ESTIMATION METHODOLOGIES 20
3.5. COMMENTS ON APPROACH FOR HANDLING NEGATIVE AND ZERO DATA 23
3.6. ALTERNATIVE APPROACHES FOR NEGATIVE AND ZERO DATA 26
3.7. BROILER VOLATILE ORGANIC COMPOUND (VOC) EMISSIONS-ESTIMATION METHODOLOGIES 27
4. SPECIFIC RECOMMENDATIONS FOR the DRAFT BROILER AND LAGOON
REPORTS 29
4.1. RECOMMENDATIONS FOR REVISING THE DRAFT EPA BROILER REPORT 29
4.2. RECOMMENDATIONS FOR REVISING THE DRAFT EPA LAGOON REPORT 32
BIBLIOGRAPHY 34
APPENDIX A-EPA'S CHARGE QUESTIONS A-l
APPENDIX B-ADDITIONAL RESPONSE TO CHARGE QUESTION 1 B-l
Vlll
-------
Acronyms and Abbreviations
AFOs
AFO Panel
ASAE
bLS
CAA
CERCLA
CH4
EEMs
EPA
EPCRA
ER
H2S
MDL
NAEMS
N
NH3
NH4
OAR
PM
QA/QC
RPM
S
SAB
SAS
SPY
THM
TSP
VOCs
VR
Animal Feeding Operations
SAB Animal Feeding Operations Emissions Review Panel
American Society of Agricultural Engineers
Backward Lagrangian Stochastic Model
Clean Air Act
Comprehensive Environmental Response, Compensation, and Liability Act
Methane
Emissions-Estimating Methodologies
U.S. Environmental Protection Agency
Emergency Planning and Community Right-to-Know Act
Emission Rate
Hydrogen Sulfide
Minimum Detection Level
National Air Emissions Monitoring Study
Nitrogen
Ammonia
Ammonium
Office of Air and Radiation
Particulate Matter
Quality Assurance/Quality Control
Radial Plume Mapping
Sulfur
EPA Science Advisory Board
Statistical Analysis Software
Static Predictor Variables
Trihalomethanes
Total Suspended Particulates
Volatile Organic Compounds
Ventilation Rate
IX
-------
1. EXECUTIVE SUMMARY
Overview
The EPA's Office of Air and Radiation (OAR) requested that the Science Advisory Board (SAB) review
two draft documents related to air emissions from animal feeding operations (AFOs): "Development of
Emissions-Estimating Methodologies for Broiler Animal Feeding Operations"(hereafter, the "Broiler
Report") and "Development of Emissions-Estimating Methodologies for Lagoons and Basins at Swine
and Dairy Animal Feeding Operations'' (hereafter, the "Lagoon Report"). In these documents, EPA
described draft emissions-estimating methodologies (EEMs) for broiler AFOs and for lagoons and
basins at swine and dairy AFOs to address requirements of a 2005 voluntary air compliance consent
agreement between the EPA and nearly 14,000 broiler, dairy, egg layer, and swine AFOs. The EPA
requested that the SAB provide advice on scientific issues associated with development of the EEMs.
The SAB was asked to comment on various aspects of the EPA's draft reports, including the overall
approach for developing the EEMs, combination of lagoon and basin data, use of static predictor
variables within the EEMs, specific approaches for development of the ammonia (lagoon NH3) and
broiler volatile organic compound (VOC) EEMs and handling of negative and zero data results.
The EPA developed draft EEMs for broiler confinement facilities and for open lagoons and basins at
swine and dairy AFOs after reviewing data on emissions from two key sources: (a) data that the EPA
received in response to a 2011 Call for Information seeking additional data on AFOs and emissions, and
(b) the National Air Emissions Monitoring Study (NAEMS). The NAEMS was a two-year study of
emissions from AFOs that raise pigs and broiler chickens, and from egg-laying operations and dairies.
The study was funded by the AFO industry as part of the 2005 voluntary air compliance agreement with
the EPA.
At a series of public meetings, the SAB Animal Feeding Operations Emission Review Panel (AFO
Panel) reviewed the draft EPA documents, considered public comments, and requested and considered
additional data and information from the EPA to develop advice on the scientific adequacy, suitability
and appropriateness of the EPA's EEMs and draft reports. The chartered SAB deliberated on the panel
draft report in March 2013 and approved the report with clarifying edits. The body of this report
provides the advice and recommendations of the SAB.
In its review of the EEMs, the SAB finds that the EPA used a small number of broiler, swine and dairy
facilities to develop draft EEMs, and the EEMs developed from this limited sample are intended to be
applied to AFOs throughout the country. The methods used in developing the EEMs are not well suited
for extrapolation to conditions beyond those represented in the data set. Therefore the EEMs may not be
assumed to accurately predict emissions from other farms in the United States. The SAB advises the
EPA not to apply the current versions of the models for estimating emissions beyond those covered in
the data set.
There is a provision in the 2005 Consent Agreement that, if the SAB decides that the available data are
not adequate to support development of the EEMs, the EPA can delay development of the EEMs until
adequate data are available. As outlined in responses to specific charge questions below, the EPA should
consider using data collected through mechanisms outside of the consent agreement, including data
published in peer-reviewed literature, to expand the data set. SAB strongly recommends that the EPA
not combine the swine and dairy datasets. A combination of these two datasets would overlook the basic
1
-------
differences in microbial processes and waste characteristics and undermine the credibility of conclusions
drawn from such analyses. The SAB finds significant limitations inherent in the EPA's approach of
using static predictor variables as surrogates for data on dynamic lagoon/basin conditions because such
an approach obscures key emission processes and variable interactions. The approach fails to account
for regional and inter-species variability among the fundamental drivers of emission processes. In
addition, there are significant uncertainties associated with the broiler VOC data used in the EPA's
analysis, and the SAB finds that these data are insufficient to support development of a broiler EEM for
VOCs at this time.
The SAB strongly recommends that the EPA use a process-based modeling approach to predict air
emissions from broiler confinement facilities and swine and dairy lagoons/basins. This recommendation
is consistent with recommendations provided to EPA in the 2003 National Research Council report Air
Emissions from Animal Feeding Operations: Current Knowledge, Future Needs. Process-based models
are more likely to be successful in representing the broad range of AFO conditions than the statistical
models used in the draft Broiler and Lagoon Reports because process-based models represent the
chemical, biological and physical processes and constraints to be addressed by EEMs.
A rigorous process-based model would quantify the flows of materials from one process on a farm to the
next; more simplified process-based models which incorporate chemical, biological and physical
constraints can also be developed. The EPA should develop a modeling approach that allows
opportunity to add data as additional information becomes available. The SAB also encourages EPA to
estimate uncertainty associated with predictions from the modeling approaches that are developed.
In addition, the SAB recommends that after the EPA updates its approaches for developing EEMs for
broiler confinement facilities and swine and dairy lagoons/basins consistent with SAB's advice, the
agency should use these updated approaches to develop draft EEMs for egg-layers, swine and dairy
confinement facilities. The EPA should develop a process-based modeling approach to make predictions
of air emissions from these sectors.
The SAB recognizes that there are potential drawbacks with developing and applying process-based
models to assess emissions at AFO facilities. Since a single set of processes may not control emissions
at all farms across the nation in a particular AFO sector, a large number of parameters and static
variables may be required to address the variety of factors that affect emissions within a sector. Also,
interactions among the parameters may need to be assessed and incorporated into the modeling
approach. Since different farms may have different processes that control emissions, process-based
models should be robust enough so that input variables would discriminate between these different
conditions. The EPA should estimate and evaluate uncertainty associated with different modeling
approaches during the model building exercise, to determine the degree to which different models might
be required.
In summary, EPA has developed statistical models based on combined data sets and predictor variables
that have limited the ability of the models to predict emissions beyond the small number of farms in the
dataset. While basing the EEMs on data from a small number of farms does not necessarily limit the
applicability of the EEMs to national populations, the assumptions and forms of the statistical models
used in the current EEMs are not suitable for use outside the domain of the current data. The SAB
recommends that the EPA not apply the current versions of the statistical and modeling tools for
estimating emissions beyond the farms in EPA's data set. SAB recommends that EPA use process-based
-------
models that can be applied, tested, and adapted outside the domain of the current data. SAB also
recommends not combining the swine and dairy datasets.
A more detailed description of the technical recommendations is included in this SAB report, and the
responses to specific charge questions are highlighted below.
EPA'S Approach for Developing the EEMS (Charge Question 1)
Please comment on the statistical approach used by the EPA for developing the draft EEMsfor broiler
confinement houses and swine and dairy lagoons/basins. In addition please comment on the approach
for developing draft EEMsfor egg-layers, swine and dairy confinement houses.
The EPA developed separate broiler confinement facility EEMs for ammonia (NHs), paniculate matter
(PMio and PM^.s), total suspended particulates (TSP), volatile organic compounds (VOCs), and
hydrogen sulfide (E^S) using emissions and process information collected from one broiler operation in
California and from two broiler operations in Kentucky. The EPA developed a swine and dairy lagoon
open source EEM for NHs using emissions and process information collected from three dairies located
in Indiana, Washington and Wisconsin, three swine breeding and gestation farms located in Indiana, North
Carolina and Oklahoma, and three swine growing and finishing farms located in Iowa, North Carolina and
Oklahoma. EPA used Statistical Analysis Software (SAS) to evaluate parameters statistically to
determine if predictor variables could be used by the EPA to develop these EEMs. Based on the results
of the EPA's predictor analysis, broiler EEMs were developed using the following input parameters:
bird inventory; ambient meteorological parameters (i.e., temperature, relative humidity, and barometric
pressure), and confinement parameters (i.e., house temperature and relative humidity). EPA's swine and
dairy lagoon NHs EEM was developed using the following input parameters: ambient temperature,
relative humidity, solar radiation, and wind speed.
The SAB has a number of suggestions for improving the modeling approach used by the EPA for
developing the draft EEMs for broiler confinement facilities and swine and dairy lagoons/basins. The
EEMs developed from the limited data are intended to be applied to AFOs throughout the country. The
SAB finds that the EPA's EEMs in both reports are based on statistical analyses of datasets that use a
small number of input parameters. They are dependent mathematically on key variables (e.g., bird
weight) that cannot be confidently extrapolated beyond the range of values in the data set. The data are
not well suited for extrapolation to conditions beyond those represented in the data set and therefore the
EEMs derived from them may not be assumed to accurately predict emissions from other farms in the
United States.
The SAB recommends that the EPA should not apply the current versions of the statistical and modeling
tools for estimating emissions beyond the range of values in the data set. The EPA should consider using
data collected through mechanisms outside the consent agreement, including data published in or that
support literature, raw data from key studies, and additional data that the EPA has collected since
receiving data in response to the Call for Information that the EPA released that sought additional data
on AFOs and emissions. Literature that should be considered is included in the bibliography of this SAB
report. The Broiler and Lagoon Reports should include model uncertainty analysis that recognizes the
limitations of using a small number of locations. The EPA should estimate and evaluate uncertainty
associated with different modeling approaches during the model building exercise to determine the
degree to which different models might be required. The EPA should consider approaches in addition to
the cross-validation method used to evaluate the EEMs.
-------
In addition, application of polynomial regression to model nonlinear relationships (e.g., the use of cubic
functions to represent nonlinear dependence in average mass of animals) leads to poor predictions near
the extremes of the experimental conditions and when the models are extrapolated outside of the data set
range, as would be likely in application of the EEMs to AFOs nationwide. The EPA should restrict the
range of mass that should be reported if the cubic model is used and orthogonal polynomials should be
used if a polynomial approach is taken. The EPA should also provide more information on the merits of
applying such regression analysis within this project. The EPA should develop a modeling approach that
allows opportunity to add data if data are available that would reflect the heterogeneity of AFOs.
In light of the limitations of the statistical models, the SAB strongly recommends that the EPA should
develop a process-based modeling approach to predict air emissions from broiler confinement facilities
and swine and dairy lagoons/basins. A rigorous process-based model would quantify the flows of
materials from one process on a farm to the next (e.g., flows from feed through the animal housing to
manure); more simplified process-based models which incorporate chemical, biological and physical
constraints can also be developed. Because process-based models represent the chemical and physical
processes in an EEM they are more likely than the current models to be successful in representing a
broad range of conditions. The EPA should consider developing EEMs at a variety of levels of
complexity to provide options for producers with different levels of data availability. A simple approach
might use a small number of variables to place constraints on predicted emissions, such as limiting total
predicted ammonia emissions based on the nitrogen available in feed. A more complex approach to the
same emissions might attempt to perform a mass balance on nitrogen. The EPA should also identify
critical data gaps associated with development of such modeling approaches and begin the process for
identifying the key parameters to be included within the process-based models. The EPA should
consider conducting a full mass balance analysis to help in the assessment of key parameters that would
be used in a process-based modeling approach.
The SAB has identified in this report key factors and parameters that the EPA should consider within
process-based modeling approaches. The NAEMS does not provide sufficient data to evaluate and
estimate coefficients for a modeling approach for estimating emissions that incorporates all of the key
factors and parameters. In particular, the NAEMS data set does not include sufficient information for the
steps from feed development to manure collection. Also, the NAEMS swine and dairy lagoons/basins
data are particularly limited regarding feed input data, nutrient and chemical loading inputs into lagoons,
and the chemical and physical composition and pH of lagoons.
Combination of Lagoon and Basin Data (Charge Question 2)
Please comment on the agency's decision to combine the swine and dairy dataset to ensure that all
seasonal meteorological conditions are represented. In addition, the agency also seeks the SAB's
comments on whether the agency should combine lagoon and basin data.
After conducting an initial analysis of the NAEMS data submitted for swine and dairy lagoons/basins,
the EPA began developing a draft EEM for NHa. The EPA's review of the literature indicated that
lagoon/basin emissions were influenced by several factors, including lagoon/basin pH and temperature.
To enable the dataset used to develop the draft EEM to represent all seasonal meteorological conditions
for the entire two-year monitoring period, the EPA decided to combine the swine and dairy data to
develop the draft NHa EEM, and is considering whether to combine the swine and dairy data to develop
the draft H2S EEM. Although this combination of data sets attempts to resolve problems associated with
-------
inadequate sample design by combining data from separate species, the SAB strongly recommends that
the EPA not combine the swine and dairy datasets. EPA's combination of data from these two sources
does not account for the differences in chemical composition and concentration between swine and dairy
lagoons. Lagoons and basins are not the same and operate very differently; a lagoon is used to provide
biological treatment and long term storage, and a basin is used for short term storage and may not
provide biological treatment. Lagoon decomposition of manure is much greater than in a basin, since
lagoons maintain bacterial populations to aid in the digestion of newly added manure while basins do
not. In addition, characteristics of swine and dairy manure are significantly different. A combination of
these two datasets would overlook the basic differences in microbial processes and waste characteristics
and undermine the credibility of conclusions drawn from such analyses.
Furthermore, it is not appropriate to combine the data from different lagoons/basins within species if
there are no predictor variables describing the chemical, physical, and biological characteristics of the
lagoons in the model. For example, variations in the chemical composition of dairy lagoons across the
country, driven by differences in manure handling systems, lead to differences in the processes that
control ammonia (or other compound) emissions. Separating the swine and dairy lagoon data while still
using the predictor variables selected in the current EEMs (i.e., ambient temperature, relative humidity,
solar radiation and wind speed) will only provide an estimate for the specific lagoons included in the
dataset.
Use of Static Predictor Variables (Charge Question 3)
Please comment on the agency's decision to use static predictor variables as surrogates for data on
lagoon/basin conditions. Given the uncertainties in that approach, does the SAB recommend that EPA
consider specific alternative approaches for statistically analyzing the data that would allow for the site-
specific lagoon liquid characteristics to be used as predictor variables?
To maximize the number of NFL? emissions measurements used to develop the draft EEM, the EPA used
static predictor variables as surrogates for data on lagoon/basin conditions (i.e., nitrogen content of
lagoon liquid, lagoon pH, oxidation reduction potential and temperature). The EPA used the static
variables of animal type, total live mass of animal capacity on the farm, and the surface area of the
lagoon to represent total nitrogen loading rates and the potential for release to the air. There are
significant problems with using static predictor variables as surrogates for data on lagoon/basin
conditions. Such an approach obscures key emission processes and variable interactions and does not
account for regional and inter-species variability among the fundamental drivers of emission processes.
It would be inappropriate to extrapolate this approach to operations not represented by the study
locations.
Several of EPA's static predictor variables are also individually deficient. For example, the lagoon/basin
surface area is generally highly variable at swine and dairy facilities, particularly in situations where
lagoons/basins have sloping sides, where small changes in water depth can translate into large changes
in surface area. Also, animal numbers represent a fundamental variable that drives nitrogen loading and,
subsequently, NFb emissions. In addition, using the current modeling approach, the range of climatic,
management, feeding, and animal-performance conditions represented by the AFOs in the NAEMS is
too narrow to provide reliable emission estimates across the full range of conditions in which dairy and
swine producers operate in the United States; for example, moderate winters or extended, hot summers
are not represented.
-------
As discussed in more detail under the response to Charge Question 1, the SAB recommends that the
EPA develop a process-based approach that uses appropriate biological, physical, and chemical
variables that are region- and species-specific. Functional relationships in any statistical model should be
based on the key drivers of emission processes.
Alternative Statistical Approaches for Developing the NHjEEM (Charge Question 4)
Does the SAB recommend that EPA consider alternative approaches for developing the draft NHs EEM
that balances the competing needs for a large dataset (to reflect seasonal meteorological conditions)
versus incorporating additional site-specific factors that directly affect lagoon emissions. If so, what
specific alternative approaches would be appropriate to consider?
The SAB concludes that the EPA should consider the following alternative statistical approaches for
developing a draft lagoon NHa EEM, since there are limited data and the EEM needs to be broadly
applicable across the United States for determining emissions from lagoons:
Expand Data Completeness Methodology: The EPA's data completeness methodology
assumes that a valid monitoring hour is one in which 75 percent of the data recorded during
that hour were valid. EPA should expand its data completeness criteria in order to increase
the amount of data available to develop an NHs EEM. SAB finds that the EPA should
include data with less than 75 percent completeness for any given hour, since there are
already many gaps in the data used for the development of these EEMs. In addition, the
EPA should examine the 75 percent completeness criterion for daily averages; currently,
EPA considers a valid monitoring day to be one in which 75 percent of the 60 second
average data values used were valid. EPA should consider whether the missing values are
random or whether they occurred in some discernible pattern, and consider using methods
to "gap fill" missing data.
Use Backward Lagrangian Stochastic (bLS) Data: EPA's calculated daily lagoon emissions
were developed based on measurements obtained using the Radial Plume Mapping (RPM)
model rather than the bLS model. The EPA should consider using the emissions estimated
with the bLS method instead of or in conjunction with the RPM data, since there is such a
paucity of data in the current RPM dataset. Since the drivers of emissions (i.e., lagoon
chemistry and biology) are changing slowly (more in terms of weeks or months, not
minutes), it may be preferable to use daily average data values rather than hourly values. If
daily values are used, the bLS dataset has 285 valid days as opposed to only 69 valid days
using the RPM model. These daily averages could be used in conjunction with measured
lagoon characteristics in order to develop a more robust model. In addition, published
validation studies indicate that the bLS model has performed very well for open area
sources.
Revise Units for Emissions Estimates: The EPA's unit for emissions is kg/30-min. The
SAB finds that EEMs that use emissions rate/ha or emissions rate/live wt or some other
denominator that captures the physical differences of the operations would more
appropriately account for actual emissions that are released at dairy and swine facilities.
Use Appropriate Predictor Variables to Estimate Emissions: The EPA should apply both
the environmental factors (manure temperature, air temperature, wind speed, and solar
-------
radiation) and predictor factors/variables that actually drive emissions. These variables
include available lagoon chemistry data such as nitrogen content and pH of the lagoon, and
the manure management system. The potential effects of surface crust on reducing
emissions should also be considered. The EPA's predictor factors/variables should have
realistic biological thresholds and boundaries to ensure that the methodology does not
result in an estimated emission rate that is not feasible. The EPA should compare the
results of the EEMs that it develops with emissions reported in the literature.
Approaches for Handling Negative and Zero Data (Charge Questions 5 and 6)
Please comment on the EPA 's approach for handling negative or zero emission measurements.
In the interest of maximizing the number of available data values for development of the draft H2S
EEMs for swine and dairy lagoons/basins, does SAB recommend any alternative approaches for
handling negative and zero data other than the approach used by the agency.
Some NAEMS emissions measurements were reported as either negative or zero emissions values. The
EPA considered whether to include these negative and zero emissions values in the data used to develop
the EEMs. The agency evaluated whether the negative or zero values represented variability in
emissions measurements due to instrument/equipment performance and concluded that all negative
values should not be considered in the development of the EEMs. The EPA also reviewed the data to see
if the data quality measures were properly performed according to the Quality Assurance Project Plan.
The SAB has several recommendations regarding the EPA's handling of negative and zero values for
both direct concentration measurement and calculated emission values. In general, a zero or negative
direct concentration measurement value can occur due to a true value that is at or below the Minimum
Detection Level (MDL), instrument measurement error, a measurement value that is adjusted by the
equipment calibration offset procedure, and instrument fluctuation due to influence by ambient
conditions. Each of these cases is considered individually and recommendations are provided in the full
report. In some cases the SAB recommends that zero and negative direct concentration values be
included in the development of EEMs.
Negative and zero calculated emission data generally should be included when calculating EEMs. If the
measured concentration data are considered valid and included in the dataset, then the emission value
calculated from that dataset also should be considered valid, whether it is negative, zero or positive. If
the calculated value is negative, the EPA should consult the raw data to assess whether the value was
due to calculation, instrument results, ambient conditions, or some other effect.
Outliers (observations that appear to be different from the other observations in the sample set) should
be first treated per the quality assurance/quality control process to determine (if possible) their origin
and then included or not in EPA's analyses with a clear explanation for the decisions made.
Broiler VOC EEM (Charge Question 7)
Please comment on the approach EPA used to develop the draft broiler VOC EEM.
The EPA reviewed the VOC data submitted for the California and Kentucky broiler sites. The two sites
used different VOC measurement techniques. Based on analysis of the measurement and analytical
techniques and the VOC data, the EPA used only the VOC data from the Kentucky sites when
-------
developing the draft VOC EEM.
There are significant uncertainties associated with the broiler VOC data collected as part of the NAEMS,
and the SAB therefore concludes that the broiler VOC data cannot support the development of a broiler
VOC EEM at this time. Although the NAEMS dataset is too limited to produce an EEM, there are
valuable components of the VOC data that should be reported. The KY1B VOC data may generally be
valid and usable if the EPA extensively and clearly documents the methods that were used to collect
VOC data. The EPA should also provide information on the total and speciated VOC concentrations at
the sites where data were collected. The SAB recommends that the EPA investigate the factors that drive
generation of VOC emissions from broiler facilities and develop a process-based modeling approach to
estimate VOC emissions from these operations.
Comments on the Draft Broiler and Lagoon Reports
In addition to evaluating the technical content of the reports, the SAB considered whether the draft
Broiler and Lagoon Reports were presented in a clear, comprehensive, and scientifically sound manner.
This SAB report suggests alternative analyses or presentation that should be conducted. Overall, many
areas of the draft reports should be enhanced to strengthen the clarity and scientific basis of the EPA's
analyses. Both reports should be updated to set a long-term goal for producing process-based models
and to indicate additional data received by the agency from Dr. Al Heber of Purdue University, the
NAEMS science advisor, since the time of the initial publication of the NAEMS dataset. The SAB also
concludes that the reports should more comprehensively describe data completeness, representativeness,
and limitations, and whether there are sufficient data to begin a process-based modeling approach.
Various suggestions are included for improving the EPA's statistical approach. Furthermore, SAB
recommends that the reports more fully explain why any of the NAEMS data were excluded from EEM
development. Since NAEMS data have significant limitations, the reports should include an assessment
that considers use of data that were not collected as part of the NAEMS data collection effort.
-------
2. INTRODUCTION
2.1. Background
In 2011, the EPA's Office of Air and Radiation (OAR) initiated development of draft emissions-
estimating methodologies (EEMs) for animal feeding operations (AFOs) at broiler confinement facilities
and for open lagoons and basins at swine and dairy AFOs. EEMs are tools for estimating emissions from
AFOs and are commonly used to estimate emissions from industries where site-specific emissions data
are not available because of costs or other factors. The EPA developed EEMs for confinement structures
(e.g., barns or buildings at broiler facilities) and for open area sources (i.e., lagoons and basins at swine
and dairy facilities).
The EPA developed the EEMs for broiler confinement facilities and for open lagoons and basins at
swine and dairy AFOs to address requirements of a voluntary air compliance consent agreement (U.S.
EPA 2005) signed in 2005 between the EPA and nearly 14,000 broiler, dairy, egg layer and swine
AFOs. The goals of the agreement are to reduce air pollution, monitor AFO emissions, promote a
national consensus on methodologies for estimating emissions from AFOs, and ensure compliance with
the requirements of the Clean Air Act (CAA), the Comprehensive Environmental Response,
Compensation and Liability Act (CERCLA) and the Emergency Planning and Community Right-to-
Know Act (EPCRA). The EEMs will be used by the AFO industry to estimate daily and annual
emissions for use in determining their responsibilities under these regulatory programs. The pollutants
monitored under the agreement include: ammonia, hydrogen sulfide, particulate matter, and VOCs. As
part of the agreement, EPA is charged with developing EEMs for broiler, dairy, egg layer and swine
AFO sectors. There is a provision in the Consent Agreement that, if the SAB decides that the available
data are not adequate to support development of the EEMs, the EPA can delay development of the
EEMs until adequate data are available.
At broiler confinement facilities, young chickens between 28 and 63 days old are raised for meat. The
most common type of housing for broilers is enclosed housing with a compacted soil floor covered with
dry bedding such as sawdust, wood shavings, or chopped straw. Mechanical ventilation is typically
provided using a negative-pressure system, with exhaust fans drawing air out of the house, and fresh air
returning through ducts around the perimeter of the roof.
Swine AFOs involve the breeding and growth of pigs for meat. Dairy AFOs produce milk. At many
swine and dairy AFOs, manure handled as a slurry or liquid is stored in external earthen impoundments
such as anaerobic lagoons. Lagoons are designed to hold the total volume of manure and process
wastewater generated in addition to precipitation runoff. In the dairy industry, liquid-solid separation
may be used to remove solids collected from runoff from dry lots and/or flushed manure from barns and
milking centers. The liquid wastes separated from solid wastes are sent to an external storage pond or
anaerobic lagoon, usually constructed as an earthen basin.
The EPA developed EEMs for broiler confinement facilities and for open lagoons and basins at swine
and dairy AFOs after reviewing data on emissions from two key sources: (a) data that the EPA received
in response to its Call for Information (U.S. EPA 2011) seeking additional data on AFOs and emissions
to ensure that the agency reviewed the broadest range of available scientific data, and (b) the National
Air Emissions Monitoring Study (NAEMS).l The NAEMS was a two-year study of emissions from
1 described at http://www.epa.gov/oecaagct/airmonitoringstudy.html
9
-------
AFOs that raise pigs and broiler chickens, and from egg-laying operations and dairies. The study was
funded by the AFO industry as part of the 2005 voluntary air compliance agreement with the EPA.
2.2. SAB Review
During the summer of 2011, the EPA requested that the Science Advisory Board (SAB) provide advice
on scientific issues associated with the EPA's development of the EEMs. In February 2012, the EPA
developed two draft documents ("Development of Emissions-Estimating Methodologies for Broiler
Animal Feeding Operations " and "Development of Emissions-Estimating Methodologies for Lagoons
and Basins at Swine and Dairy Animal Feeding Operations"). The documents provided to the SAB
describe the sites monitored, the data submitted to the EPA, and a detailed discussion of the statistical
methodology used to develop the draft EEMs. After addressing SAB advice in the Broiler and Lagoon
Reports, the EPA intends to use the updated overall approach to develop draft EEMs for egg-layer AFO
facilities and swine and dairy AFO confinement facilities.
The EPA asked the SAB to provide advice on the agency's overall approach for developing the EEMs
(see Charge Questions provided as Appendix A to this report). The EPA also requested advice on
whether it should combine lagoon and basin data, whether it should use static or dynamic predictor
variables for its model and how to handle data that were reported as negative or zero results. In addition,
the EPA requested advice on alternative approaches for developing the NHs EEM for swine and dairy
facilities and on whether it should develop an EEM for VOCs from broiler AFOs.
The SAB Animal Feeding Operations Emission Review Panel (AFO Panel) reviewed the draft EPA
documents, considered public comments, and held a public meeting on March 14-16, 2012, to develop
advice on the scientific adequacy, suitability and appropriateness of EPA's draft documents. The AFO
Panel considered oral statements that were received from the public during the public meeting and
written public comments that were received on the draft EPA documents. At the March 2012 public
meeting, the AFO Panel raised several questions and requested additional data. The EPA responded to
these requests and provided supplemental information to the AFO Panel in July and August 2012 . The
SAB AFO Panel held a follow-up public teleconference call on August 13, 2012 to review the agency's
additional information and to consider whether the EPA's supplemental responses changed any of the
AFO Panel's preliminary findings and recommendations identified at the March 2012 public meeting.
The AFO Panel held a public teleconference on October 24, 2012, to discuss substantive comments from
Panel members on this draft SAB report. On a public teleconference on March 7, 2013, the chartered
SAB deliberated on and approved the panel's draft report subject to clarifying edits.
The EPA plans to consider SAB advice on the draft Broiler and Lagoons Reports as it finalizes those
documents. SAB recommends that after the EPA updates its approaches for developing EEMs for
broiler confinement facilities and swine and dairy lagoons/basins consistent with SAB's advice, the
agency should use these updated approaches to develop draft EEMs for egg-layers, swine and dairy
confinement facilities.
The Executive Summary highlights the SAB's major findings and recommendations. The SAB's full
responses to the charge questions are detailed in Section 3. Section 4 provides recommendations to
guide the EPA in revising the Broiler and Lagoon Reports.
2 This information, and all other materials considered by the SAB during the review, is available on the SAB website at
http://yosemite.epa.gov/sab/sabproduct. nsf/fedrgstr_activites/AFO-AEEM?OpenDocument
10
-------
3. RESPONSES TO EPA'S CHARGE QUESTIONS
3.1. The EPA's Approach for Developing the EEMs
Question 1: Please comment on the statistical approach used by the EPA for developing the draft EEMs
for broiler confinement houses and swine and dairy lagoons/basins. In addition please comment on the
approach for developing draft EEMs for egg-layers, swine and dairy confinement houses.
3.1.1. Background
The EPA developed separate broiler confinement facility EEMs for NHs, PMio, PM2.5, TSP, VOC and
H2S using NAEMS emissions and process information collected from one broiler operation in California
and from two broiler operations in Kentucky. EPA applied Statistical Analysis Software (SAS) to
evaluate parameters statistically to determine if they were predictor variables appropriate to use to
develop the EEMs. Based on the results of the predictor analysis, EPA developed broiler EEMs using
the following input parameters: bird inventory; ambient meteorological parameters (i.e., temperature,
relative humidity, and barometric pressure) and confinement parameters (i.e., house temperature and
relative humidity).
The EPA developed a swine and dairy lagoon open source EEM for NHa using NAEMS emissions and
process information collected from three dairies, three breeding and gestation swine farms and three
swine growing and finishing farms. The EPA applied SAS to evaluate the parameters statistically and
determined input parameters in a manner similar to that used to develop the broiler EEMs. The EPA
developed its swine and dairy lagoon NH3 EEM using the following input parameters: ambient
temperature, relative humidity, solar radiation and wind speed.
The EPA evaluated the parameters statistically using a mean trend function that provided a point
prediction of emissions under a given set of conditions. The agency chose a mean trend function to
quantify the relationship between predictor variables and pollutant emissions by analyzing the emissions
data. The EPA also chose a probability distribution and covariance function to quantify other
contributions to variability in emissions and to provide estimates of uncertainty.
3.1.2. Response
A small number of broiler, swine and dairy facilities were used to develop the EEMs, and the EEMs
developed from this limited sample are intended to be applied to AFOs throughout the country. The
methods used in developing the EEMs are not well suited for extrapolation to conditions beyond those
represented in the data set and therefore the EEMs may not be assumed to be accurate predictors of
emissions from other farms in the United States. The SAB concludes that the EPA should not apply the
current versions of the models for estimating emissions beyond the farms in the data set.
The SAB strongly recommends that the EPA develop process-based models of air emissions from AFOs
of all types (e.g., broiler, dairy, egg layers and swine). This approach was recommended previously and
described in detail by the National Research Council (NRC 2003). A rigorous process-based model
would quantify the flows of materials from one process on a farm to the next (e.g., flows from feed
through the animal housing to manure storage to field application and crop production). Rigorous
process-based models would require consideration of emissions from each component of the farm
system based on the concentrations and amount of reactants that lead to the emission from that
component. More simplified process-based models which incorporate chemical, biological and physical
constraints can also be developed.
11
-------
Statistical Approach
The SAB reviewed the statistical approach taken by the EPA for estimation of air emissions from broiler
confinement operations and dairy and swine lagoons.
The SAB recommends that the EPA should not apply the current versions of the statistical and modeling
tools for estimating emissions beyond the farms in the data set, since it may not be possible to use the
EEM broiler and lagoon/basin models developed by the EPA to extrapolate to other farms with
reasonable accuracy. While the statistical approach to analysis of the data may be acceptable for the
small number of locations and limited range of conditions represented in the dataset, the EEMs are not
well suited for extrapolation to conditions beyond those represented in that dataset. Such extrapolations
will be necessary if the EEMs are applied nationally. Further, some of the variables used for the model
predictions do not make mechanistic sense. It would be more plausible and more credible to use
variables known to be logically or experimentally linked to emissions (e.g., nitrogen content of litter to
predict ammonia emissions). Such variables would be more likely to perform well across a broader
collection of facilities than the variables used by the EPA in the draft EEMs.
To make accurate predictions across farms, it is desirable to have measurements from a larger number of
farms that adequately represent conditions at farms across the United States. Only two sites were
evaluated for broiler operations, and this limited number of sites is unlikely to represent the industry as a
whole. Only one site was used to estimate VOC emissions from broiler houses, and this was clearly not
adequate to derive meaningful conclusions for the entire industry nationwide. In addition, the six swine
and three dairy lagoons sampled cannot represent all lagoons across both industries. The SAB cautions
against the use of polynomial models when the use of the model is likely to extend beyond the range of
data measured to develop the relationships since such models can lead to clearly erroneous predictions
(e.g., negative emissions or "near zero" emissions from large birds) under certain production regimes
employed in the United States.
The SAB finds that most emission measures were over-weighted for periods of higher emissions such as
during warmer weather, and the range in weather parameters for the dataset may not reflect the range in
measurements across the country. The SAB recommends that the EPA evaluate the effects of weather on
emissions and evaluate the ranges in weather patterns within the dataset, relative to the industry across
the United States, to determine how much of the data collected can be extrapolated to farms in different
climatic regions. In general, ranges of data should be explained in the reports and extrapolation beyond
those limits should be counter indicated.
In addition, the EPA should create a modeling approach that relies on default parameters that can be
attained by most farms within a reasonable budget. The modeling approach should allow opportunity to
add data if new data are available that would reflect the heterogeneity of AFOs. The EPA should
estimate and evaluate uncertainty associated with different modeling approaches during the model
building exercise to determine the degree to which different models might be required.
In descriptions of the methodology supporting the EEM, the EPA should describe methods for
calculating confidence values to present variability of data, include quantitative statistical analyses that
compare the confinement buildings (i.e., house) for the animals, consider additional approaches besides
the cross-validation method used to evaluate the model, and more comprehensively describe data
completeness, representativeness, and limitations.
12
-------
The SAB has a number of additional, specific suggestions for improving the statistical modeling
approach used by the EPA for developing the draft EEMs for broiler confinement facilities and swine
and dairy lagoons/basins. These suggestions are provided as Appendix B to this report.
Process-Based Models
The SAB strongly recommends that the EPA develop process-based models of air emissions from AFOs
of all types (e.g., broiler, dairy, egg layers and swine). This approach was recommended previously and
described in detail by the National Research Council (NRC 2003). The NRC Report (in finding 9 and in
Chapter 5} suggested that a process-based modeling approach can provide more useful estimates
for air emissions than an "emission factors" approach. The NRC panel concluded that existing
emission factors for AFOs are generally inadequate because of limited numbers of measurements,
as well as the limited generality of the models, on which the emission factors are based. The NRC
panel noted that improving existing emission factors to the point where they could provide
scientifically credible estimates of either emission rates or concentrations would require a large
number of observations to characterize the variability among and within AFOs. NRC noted that an
emission factor approach has particular difficulty if a small set of AFOs is used to represent the
broad range of AFOs for varied livestock industries in various geographic regions. The NRC panel
concluded that a process-based approach would ensure more accurate accounting of the flow of
chemicals that influence air emissions, and provide a "mass balance" control for the total flow of
inputs to and outputs from the operation. The NRC panel also concluded that a process-based
approach does not obviate the need for data collection, but it enables the use of data in
conjunction with chemical, physical and biological constraints.
A rigorous process-based model would quantify the flows of materials from one process on a farm to the
next (e.g., flows from feed through the animal housing to manure storage to field application and crop
production). Rigorous process-based models would require consideration of emissions from each
component of the farm system based on the concentrations and amount of reactants that lead to the
emission from that component. More simplified process-based models which incorporate chemical,
biological and physical constraints can also be developed.
In process-based emission modeling, system processes are mathematically represented at an appropriate
level of detail to capture the important dynamics and interactions among components. In the most
rigorous form, a process-based model is developed from the scientific understanding of the physical,
chemical, biological, and other processes that control emissions. Although empirical data may be used to
help establish certain model coefficients3 or relationships, the primary need for empirical data is for
evaluation or verification of the mechanistic models used to describe system processes. This is different
from an empirical approach where regression techniques are used to formulate models from
experimental data and independent datasets are needed for validation. Process-based modeling provides
a robust emission estimation approach, since the mechanistic models are designed to be valid beyond the
datasets used to establish model coefficients.
By representing the chemical, biological and physical processes and constraints in an EEM, the SAB
concludes that process-based models are more likely than the current statistical models to be successful
in representing a broad range of conditions. In their most rigorous forms, process-based models are data
intensive; however, process considerations can be incorporated into models at a variety of levels of
3 For purposes of this report, the term "coefficient" refers to unknown constants (regression coefficients, the variance, and the
auto-correlation coefficient) whose values give the EEMs their shape.
13
-------
complexity. The EPA should consider developing EEMs at a variety of levels of complexity to provide
options for producers with different levels of data availability. While the NAEMS does not provide
sufficient data to implement a rigorous process-based modeling approach, it is sufficient to start the
development of a modeling approach for estimating emissions. The EPA should create a modeling
approach that can be defined using default parameters4 that can be simply attained and that would reflect
the heterogeneity of AFOs.
For example, emissions from manure lagoons would be based on the composition of manure, which
would in turn depend on flows into and out of the manure lagoon. The flows into the manure lagoon
would be derived from the manure production from the animal housing in the form of excreted feces and
urine and bedding. Flows into a lagoon would need to consider inputs from the milking parlors and
account for clean water collection from slabs and surfaces that may change the volume and solids ratios.
Flows out of the lagoon would be equivalent to the flows into the lagoon minus compounds emitted into
the air, leached, or mineralized in the soil. Furthermore, mass flows in the manure lagoon would be
quantified for each air species of interest (e.g., NHa, CH/i) based on the nutrient loading rates and
concentrations of the nutrients that lead to those species (e.g., urea, NH4, organic nitrogen and organic
carbon).
Developing a rigorous process-based EEM will require extensive data beyond the range of values,
conditions, and types of farms available in the NAEMS data set. To address this data gap the EPA
should consider using data collected through mechanisms outside the consent agreement, including data
published in peer-reviewed literature, raw data from key studies, data that support key literature, and
additional data that the EPA has collected since receiving data in response to the Call for Information
(U.S. EPA 2011) on AFOs and emissions. These data would need to measure the emissions from various
components for the farm enterprise as a function of variables that should matter based on a mechanistic
understanding of the emissions. For example, nutrients in animal manure could be estimated based on
nutrient intake and production rates or at least expected intake for a level of production. Nitrogen flows
would be especially relevant to ammonia emissions. The amount of urine and fecal nitrogen could be
used to estimate emissions from the barn floor or subsequent manure storage and application. The
NAEMS data could be used to some extent to evaluate the accuracy of process-based models.
Rigorous process-based models are data intensive, but process concepts, such as limiting predicted
releases of nitrogen in emissions to be less than nitrogen inputs, can be used in simplified models.
Models of varying complexity should be developed based on the level of input provided by a given
producer (e.g., one model may be developed considering the composition of a feed ration, while a less
complex model using default industry values could be used if a producer does not wish to or cannot
disclose information regarding feed rations).
The advantages of using a process-based model include the following:
More existing data could be used, such as data for estimated emissions from a certain
component of the farm under certain conditions.
Estimates derived would be more robust across different farm types.
Control strategies could be developed for reducing emissions from farms based on
implementing technology standards or performance standards, wherein the standards
would predict specific impacts using the process-based models.
4 For purposes of this report, the term "parameter" refers to the data and data collection methods used to support the modeling
approach.
14
-------
These advantages would enhance the robustness of EEMs and ensure their applicability into the future
rather than representing only a snapshot in time. Regardless of the approach that is used, uncertainty
associated with the prediction at new farms should be evaluated.
The SAB has identified several key factors and parameters that the EPA should consider within process-
based modeling approaches. Key factors and parameters that impact broiler emissions may include, but
are not limited to: animal activity (perhaps assessed through lighting program hours for light and dark
periods); key diet ingredients (that result in releases of gaseous pollutants, such as total nitrogen); water
management; manure composition (moisture, mass, and nitrogen); total number of animal units;
temperature in the house; and ventilation rate. Key factors and parameters that affect dairy and swine
lagoon emissions may include but are not limited to: sulfur, nitrogen and carbon content of feed;
conversion of feed nutrients to animal product (milk and meat); nutrients fed; climate variables such as
temperature and wind speed; the lagoon sulfur, nitrogen and carbon content; surface area; depth; manure
residence time; volume; temperature; pH; oxidation-reduction potential; and presence or absence of a
surface crust. The NAEMS does not provide sufficient data to evaluate and estimate parameters for a
modeling approach for estimating emissions incorporating all of these key factors and parameters. In
particular, the NAEMS data set does not include sufficient information for the steps from feed
development to manure collection. The NAEMS swine and dairy lagoons/basins data are particularly
limited regarding feed input data, nutrient and chemical loading inputs into lagoons, and the chemical
and physical composition and pH of lagoons. The bibliography to this SAB report includes citations for
process-based models that the EPA should consider in its development of process-based models.
The SAB recognizes that there are potential drawbacks with developing and applying process-based
models to assess emissions at AFO facilities. Since a single set of processes may not determine
emissions for all farms across the nation in a particular AFO sector, a large number of parameters and
static variables may be required to address the variety of factors that affect emissions on a number of
farms within a sector. Also, interactions among the parameters may need to be assessed and
incorporated into the modeling approach. Since different farms may have different processes that control
emissions, process-based models should be robust enough so that input variables would discriminate
between these different regimes of estimation. The EPA should estimate and evaluate uncertainty
associated with different modeling approaches during the model building exercise to determine the
degree to which different models might be required.
In addition, the SAB recommends that after the EPA updates its approaches for developing EEMs for
broiler confinement facilities and swine and dairy lagoons/basins consistent with SAB's advice, the EPA
should use these updated approaches to develop draft EEMs for egg-layers, swine and dairy confinement
facilities. The SAB also recommends that the EPA should develop a process-based modeling approach
to predict air emissions from these sectors. The EPA should consider developing EEMs at a variety of
levels of complexity to provide options for producers with different levels of data availability. The EPA
should also identify critical data gaps associated with development of such modeling approaches and
begin the process for identifying which key parameters should be included within the process-based
models. The EPA should consider conducting a full mass balance analysis to help in the assessment of
key parameters that would be used in a process-based modeling approach.
15
-------
3.2. Combination of Lagoon and Basin Data
Question 2: Please comment on the agency's decision to combine the swine and dairy dataset to ensure
that all seasonal meteorological conditions are represented. In addition, the agency also seeks the
SAB's comments on whether the agency should combine lagoon and basin data.
3.2.1. Background
After conducting an initial analysis of the NAEMS data submitted for swine and dairy lagoons/basins,
the EPA began developing a draft EEM for NHa. The EPA's review of literature indicated that
lagoon/basin emissions were influenced by several factors, including lagoon/basin temperature. To
ensure that the dataset used to develop the draft EEM represented all seasonal meteorological conditions
for the entire two-year monitoring period, EPA decided to combine the swine and dairy data that the
EPA relied on to develop the draft NH3 EEM.
3.2.2. Response
The SAB recommends against combining swine and dairy datasets. The EPA justifies combining the
swine and dairy data to ensure that multiple seasonal meteorological conditions are represented and a
sufficiently large data set is available for analysis. Initial site selection for dairy lagoons in this study did
not provide representation for measurements of all seasonal meteorological conditions. Neither moderate
winters nor extended hot conditions in summer were represented. Although combining datasets attempts
to resolve problems associated with inadequate sample design by combining data from separate species,
it should not be done, and it is not clear what inferences could be made from any models resulting from
combined datasets.
Lagoons and basins are not the same and operate very differently. Treatment lagoons rely upon
microbial populations to digest organic fractions of manure. Intermediary compounds are consumed by
other populations of microbes. The net result is digestion and decomposition of organic matter. This
process occurs more rapidly in lagoons than in basins. EPA's combination of data from these two
sources does not account for the differences in chemical composition and concentration between swine
and dairy lagoons. Combining species data without correcting for nutrient loading rates and chemical
differences overlooks the basic differences in microbial processes and waste characteristics and
undermines the credibility of conclusions drawn from such analyses.
Although the current EEM approach represents multiple seasons, little attention is paid to many relevant
factors including: chemical, physical, and biological differences in the contents and functionality of the
various lagoons and basins; difference in species; production efficiency; diets; feed intake; animal
stocking densities; injection of fresh water; and lagoon loading. Inputs into lagoons/basins (loading rates
for nutrients and chemical constituents) vary by facility and must be considered as such inputs are
feedstocks for microbial populations present in containment structures. More rapidly fermentable
carbohydrates will be present in the swine manure. Different compositions of nitrogen and sulfur are
also expected. Combined, these differences in influent concentrations translate to differences in
microbial decomposition activities, rates and intermediary compounds, all influencing potential
conversion to methane or non-conversion and potential release of emissions to the atmosphere. Nitrogen
quantity and composition in waste streams, pH, temperature at the interface between the water surface
and the atmosphere and wind speed are known to play key roles in volatilization of nitrogen as
ammonia, yet none of these factors is considered in EPA's EEM.
16
-------
It is not appropriate to combine both datasets to compensate for flaws in the study design. The EPA
informed the SAB that the EPA combined the basin and lagoon data collected through the NAEMS
effort to allow the estimation of basin NHa emissions in high temperature ranges only measured in
lagoons. Extrapolating basin NH3 emissions to higher temperatures based upon lagoon NH3 emissions
measured at higher temperatures is an example of erroneous analytical practice. This extrapolation
assumes that basin and lagoon NHa emission dependency on temperature is the same. Such an
assumption is not known to be true. The EPA should identify any other modeling assumptions or data
used to estimate NH3 emissions that might differ for lagoons and basins. The SAB requests, for
example, that the EPA clarify:
Whether the basins developed any crusts or other solids on the surface which might
obstruct diffusion of NH3 across the liquid/atmosphere interface;
Dimensions of the basins and lagoons;
Whether there are significant differences between lagoons and basins that would affect
the wind fetch and hence gas stripping effects of flow across the liquid/atmosphere
interface;
Whether there are pH differences;
Whether redox potentials are similar; and
Whether any basins have anoxic surface layers.
The NRC report on AFO emissions concluded that emissions should be estimated based upon a process-
based model. A process-based approach will require special attention if different treatment systems are
to be combined. The microbial processes must be shown to be sufficiently similar. Once this is
established, then it might be possible for the EPA to identify lagoon and basin characteristics such as:
nitrogen, sulfur, and carbon concentrations; residence time; temperature; pH; and other characteristics,
and identify the range of data needed to develop a nationally applicable process-based emission model.
Such an approach would require taking into account how the microbial processes and the chemical and
physical processes are controlled by dominant characteristics in each system. As discussed in more
detail in section 3.1.2 of this report, the SAB notes that developing a rigorous process-based EEM will
require extensive data beyond those available in the NAEMS data set.
3.3. Use of Static Predictor Variables
Question 3: Please comment on the agency's decision to use static predictor variables as surrogates for
data on lagoon/basin conditions. Given the uncertainties in that approach, does the SAB recommend
that EPA consider specific alternative approaches for statistically analyzing the data that would allow
for the site-specific lagoon liquid characteristics to be used as predictor variables?
3.3.1. Background
To maximize the number of NH3 emissions measurements used to develop the draft EEM, the EPA used
static predictor variables (SPVs) as surrogates for data on lagoon/basin conditions (i.e., nitrogen content
of lagoon liquid, lagoon pH, oxidation reduction potential and temperature). The EPA used the static
variables of animal type, total live mass of animal capacity on the farm and the surface area of the
lagoon to represent NH3 precursor loading and the potential for release to the air.
17
-------
3.3.2. Response
There are significant problems in using SPVs as surrogates for data on lagoon/basin conditions. Such an
approach obscures key emission processes and variable interactions and does not account for regional
and inter-species variability among the fundamental drivers of emission processes. It would be
inappropriate to extrapolate this approach to types of operations not represented by the study locations.
The SAB recommends that the functional relationships in any EEM should be based on the key drivers
of emission processes.
Use of static predictor variables as surrogates for data on lagoon/basin conditions
To develop an EEM for NHs emissions from lagoons and basins at dairy and swine operations, the EPA
proposes to use SPVs such as total animal live weight and lagoon surface area, in lieu of time-varying,
lagoon or basin characterization data. Examples of the latter would include lagoon nitrogen loading,
feed-to-gain performance (for feeder pigs), and milk production (for milking herds).
The EPA model uses a combination of static (e.g., farm characteristics) and dynamic (e.g.,
meteorological) variables and interactions. In the EPA's formulation, the SPVs may be either raw or
transformed measurement data, depending on the individual variables' distributions. Because of the
small number of farms relative to the number of static variables, all SPVs could not be considered.
Instead, the EPA developed several models using subsets of potential static variables. Implicit in the
modeling approach is the assumption that processes associated with NHa generation can be adequately
modeled through linear statistical models.
As noted in the response to Charge Question 1, the SAB has identified several key factors and
parameters that the EPA should consider within process-based modeling approaches. For a discussion on
key factors and parameters that impact broiler emissions and dairy and swine lagoon emissions, see
section 3.1.2.
As presented in the draft EPA document, the SPV approach is problematic for a number of interrelated
reasons:
If a given SPV is not clearly, unambiguously and fundamentally related to the emission
rate through a well-established emissions mechanism, then the resulting EEM cannot
reasonably be extrapolated to other AFOs. Given the EPA's desire to use the EEM for a
number of U.S. facilities, the model should account for the wide variation in design,
climate, and management factors across the country.
Several of the SPVs that the EPA selected for its EEM are individually deficient. For
example:
o Lagoon surface area
In the case of storages that are managed as anaerobic lagoons and that therefore
maintain a relatively constant depth over time, liquid surface area would be a
reasonable SPV. However, design and management factors, both of which are site-
specific, determine whether or not a given storage actually maintains a constant
depth. In the general case, particularly where storages have sloping sides, small
changes in depth can translate into large changes in surface area, even within a
span of hours to days.
o Animal numbers
It is reasonable to suppose that nitrogen loading to a basin scales by animal
numbers, provided that all other feed-intake, retention/milk production, and
18
-------
management variables remain static. But that (highly contingent) scalability ought
not to be taken to mean that animal numbers represent a fundamental variable
driving NHa emissions. In the case of dairies, for example, milking herds may be
managed according to productivity, feeding higher-energy, higher-protein diets to
higher-producing cows, and vice-versa. Simply doubling herd size, without
knowledge of the feed intake, performance, management factors associated with
the additional animals, and the degree of solids separation does not necessarily
double the emissions attributable to the per-animal emissions processes; but that is
what the SPY approach implicitly assumes.
Dairies and swine operations differ substantially and in ways that cannot reasonably be
collapsed into a single pseudo-species. Because nitrogen loading to a lagoon or basin, a
key driver of NHs emissions, is driven in large measure by feed composition, feed intake,
nitrogen retention (for swine operations), and milk production (for dairies), among other
key variables, inter-species effects on diet and the manure produced must be taken into
account in SPY evaluation. Swine and dairy EEMs should be individually formulated.
The range of climatic, management, feeding, and animal-performance conditions
represented by the livestock operations in the NAEMS study is too narrow to provide
reliable emissions estimates across the full range of conditions in which dairy and swine
producers operate in the United States. For example, the datasets used in the NAEMS
study do not represent moderate winters or extended, hot summers.
In summary, the EPA has applied a statistical analysis that obscures key emission processes and variable
interactions, that fails to account for regional and inter-species variability among the fundamental
drivers of emission processes, and that cannot reasonably be extrapolated to types of operations not
represented by the study locations. The SAB recommends that the EPA consider a process-based
approach that uses appropriate, physically based, region- and species-specific variables.
Alternative approach for statistically analyzing the data
A statistical model developed from limited data, and not grounded in the chemical, physical and
biological processes that control emissions, will not provide a satisfactory EEM for use beyond the
range of values, conditions, and types of farms in the data set from which it was created. An alternative
to the statistical approach proposed by the EPA is to develop functional relationships based upon a
scientific understanding of the principles involved in the emission process and use a statistical procedure
to quantify the required parameters. This process-based approach must begin by identifying the
appropriate dependent and independent variables. For ammonia emission from a manure lagoon or
basin, for example, the predicted variable should be the emission per unit surface area of the lagoon or
basin. The independent variables must include both weather conditions and manure characteristics.
Important weather variables that must be included are ambient temperature and wind speed. Solar
radiation and precipitation may also contribute and should be used if the data are available. Important
manure characteristics include dry matter and nitrogen concentrations. The organic and inorganic
nitrogen contents would also be helpful if that information is available. Other important manure
characteristics include pH and temperature (if it is different from ambient temperature). Management
can affect the amount of crusting that occurs on the manure surface, and a surface crust can reduce
emissions from 20 to 80 percent depending upon the thickness and uniformity of the crust across the
surface. If the appropriate manure characteristics are defined and used, the manure source (e.g., dairy,
swine, or poultry) would not be important. For all of these variables, the temporal resolution of the data
should be consistent with the time scales on which the variables are changing. For example, manure
characteristics will not change rapidly, so hourly or daily data are not needed for these variables.
19
-------
The functional form of the predictive relationship must be established based upon the biological,
chemical and physical processes driving emissions; most often this will require nonlinear relationships.
As the independent variables approach maximum and minimum potential values, predicted emissions
must also approach appropriate values (i.e., emission predictions must approach zero under the
appropriate conditions and approach some maximum value at the outer extremes). Unreasonable
predictions such as negative or infinite values should not occur. The functional relationship must allow
an appropriate prediction across the full possible range of each independent variable and combination of
variables that might be used. Only this type of relationship can be used to extrapolate to conditions
outside the original dataset. An EEM that is applied to all manure storages throughout the country must
be satisfactorily applied to conditions beyond the limited data from which it was developed.
After the functional form of the relationship is established and the appropriate independent and
dependent variables are included in that function, a statistical approach can be used to help quantify
parameters along with scientific understanding. Somewhat limited data can be used to determine
parameters that should be appropriate beyond the bounds of the original data. Extensive verification is
required across the full range of possible conditions and some parameter adjustment may be needed to
avoid inappropriate predictions outside the bounds of the original data. Therefore, statistical accuracy
relative to the original data may be sacrificed to assure a full range of appropriate predictions. The
NAEMS data should provide an appropriate dataset for model parameterization, but other data and
published information should be used for establishing the structure and parameters of the EEM and
evaluating that EEM for more diverse conditions. This level of rigor in EEM development and
evaluation is necessary to develop a nationally applicable EEM.
3.4. Alternative Approaches for Ammonia Emissions-Estimation Methodologies
Question 4: Does the SAB recommend that EPA consider alternative approaches for developing the
draft NHs EEM that balances the competing needs for a large dataset (to reflect seasonal
meteorological conditions) versus incorporating additional site-specific factors that directly affect
lagoon emissions. If so, what specific alternative approaches would be appropriate to consider?
3.4.1. Background
The EPA requested SAB advice on alternative approaches for developing an NHs EEM that would
balance the competing needs for a large dataset (to reflect seasonal meteorological conditions) versus
incorporating additional site-specific factors that directly affect lagoon emissions.
3.4.2. Response
The SAB recommends that the EPA consider alternative approaches for developing a NH3 EEM, since
the NAEMS data are limited, and since EPA's goal is to develop an EEM that would be broadly
applicable across the United States for determining emissions from lagoons. The SAB has several
recommendations that the agency should consider to enhance its ability to develop a better EEM.
Completeness goals for data
Section 3.1.2 of EPA's draft Lagoon Report notes that: "A valid monitoring day is one in which 75
percent of the hourly average data values used to calculate the daily value were valid measurements. An
hourly average is considered valid if 75 percent of the data recorded during that hour were valid." This
statement is incorrect, and should state that: "A valid monitoring day is one in which 75 percent of the
60-s records used to calculate the daily value were valid measurements."
20
-------
The EPA should clarify why the completeness goal of 75 percent was deemed critical for determining an
hourly average and whether it limited this criterion to 75 percent of the raw data or to some other
criterion. The EPA should consider whether or not this criterion is too stringent, given the data
limitations. If collected data were of good quality during a particular hour interval, EPA should include
these data as there are already many gaps in the data used for the development of these EEMs.
The requirement for valid monitoring days to have 75 percent of the daily data may bias and limit the
dataset. A 75 percent completeness goal means that as many as six hours of data could be missing in a
day, and it is important to know when data are missing and whether the missing data would bias the
daily average. For example, if data were consistently missing at a time period when the emissions might
be high or low, then the overall average may be biased in one direction or the other. It is important to
note if the missing hourly values were random or if they occurred in some discernible pattern. In
addition, the EPA should consider using methods to fill missing data gaps. In many cases, emissions
follow very distinct patterns and it is possible to fill in missing data using interpolation or other
algorithms that would increase the number of valid days available for analysis.
Emissions estimated using the Backward Lagrangian Stochastic Model (bLS) method
As discussed in Section 5.1 of the Lagoon Report, EPA's calculated daily lagoon emissions were
developed based on measurements obtained using the Radial Plume Mapping (RPM) model rather than
the Backward Lagrangian Stochastic (bLS) model. The SAB recommends that the EPA consider using
the bLS data either instead of the RPM data or in conjunction with the RPM data, since there is a paucity
of data in the current dataset. There are two points to consider here. The first point is the decision to use
30-minute emission values, as opposed to using daily values. While doing this does result in a greater
number of data points, the use of daily averages may better capture emission trends. As there are large
diurnal emission patterns in any given day, this may overshadow predictor variable effects or add more
"noise" in the analysis. As stated above, if the 30-minute averages are from time periods when the
lagoon emissions are typically high or low, this could affect the overall EEM estimate, whereas using a
daily emission value may eliminate that potential problem. Additionally, the real drivers of emissions
(i.e., lagoon chemistry and biology) change slowly (more in terms of weeks or months, not minutes),
therefore it might be better to use daily values in conjunction with the available lagoon chemistry data to
build more powerful models (more on this point below).
As stated in Section 5.1 of the Lagoon Report:
The EPA used the RPM data because these measurements were obtained using instrumentation
and procedures that were similar to EPA's developmental test method OTM-10. The EPA did
not use the bLS emissions measurements because these data were collected under the NAEMS to
conduct a validation study of the bLS model performance relative to the RPM model.
Furthermore, because the RPM emissions dataset is much larger than the bLS dataset, including
the bLS measurement in the EEM development dataset would not provide any additional
information on lagoon emissions.
If daily values are used, then the bLS dataset has 285 valid days as opposed to only 69 valid days using
the RPM model. To conduct a validation study, the true emission values from the source should be
known. Because the true emissions are not known from any of the open area sources, it would not be
possible to establish which model performed better and which model produced an emission rate closest
to the true rate. Therefore, one cannot draw conclusions as to which model more closely estimated the
21
-------
true emissions from the source. Based on the few published validation studies available, the bLS model
has performed very well for open area sources. Ro et al. (2011, 2012) found that the bLS model more
accurately predicted emissions from open sources than the RPM model. The RPM and bLS emissions
estimates were very close in several of the datasets collected in the NAEMS study. It might therefore be
possible to fill in missing days by combining the two datasets and eliminating the overlap. This would
result in more available days for use in the development of the model.
Units of emissions estimate
Use of proper units to express the emissions estimates is also a concern. The draft EEMs use kg/30-
minutes as the unit of emissions, but perhaps better relationships could be developed if the EPA used
kilograms per time per hectare (kg/ha-sec), kilograms per time per kilogram of live weight of animal
(kg/live wt-sec), or some other denominator that captured the physical differences between operations.
These variables (lagoon size and animal weight) were included as predictor variables, but it would
potentially be better to account for these in the emission unit, therefore eliminating the need to have
them as a predictor variable.
Use of available lagoon chemistry data
In the draft EEMs, the predictor variables chosen to estimate emissions are inadequate. The factors that
actually drive the emissions (i.e., lagoon characteristics) were not included in any of the analyses. It
seems highly unlikely that a suitable methodology could be developed to predict NHs emissions across
the country when (at a minimum) the nitrogen content and pH of the lagoon have not been included as
variables in the model. The model should also consider the potential effects of surface crust on
emissions. Some of the predictors chosen (such as temperature, day of year, and wind speed) would
certainly have an impact on emissions, but due to differences in lagoon composition and chemistry, the
effects would be farm-specific and would not translate to other farms. For instance, it is possible for two
farms in the same area, with the same number of animals and same meteorological conditions to have
greatly different emissions due to differences in the pH and nitrogen content of their lagoons. There does
seem to be both nitrogen and pH data available for four of the farms, representing approximately 46
percent of the 30-minute emissions estimates used in the models. If daily emissions estimates were used
and the lagoon chemistry data were extrapolated to other days, there may be a suitable dataset that could
be used to develop a more robust EEM using both the lagoon characteristics as well as meteorological
data. The SAB finds that developing an EEM that incorporates lagoon chemistry, meteorological and
farm data would be much more valid than relying on weather data and static predictor variables alone,
even though the dataset would be smaller.
Biological thresholds
One other concern related to the development of the EEMs using the current technique is that there is no
recognition of realistic biological thresholds. Estimates from any model should not violate biological
boundaries (e.g., one cannot emit more nitrogen than is present). There should be some upper and lower
threshold limits to ensure that the methodology would not result in an estimated emission rate that is not
realistic. SAB also recommends that the EPA compare the results of the EEMs that it develops with
emissions documented in available literature. There are a number of models available that are used to
estimate NHa emissions. One could use the nitrogen and weather information available for the lagoons,
attempt to calculate emission rates and compare those with published emission estimates from the
literature.
22
-------
Primary and secondary units
Selection of appropriate units to express emissions is influenced by the type of facility and final use of
the data. Primary emission units are directly from on-farm measurements whereas secondary units are
based on parameters collected to allow conversion from one emission expression to another. The
uncertainty associated with the measurements needs to be reported (Wheeler et al. 2010; Xin et al.
2010). The following are five potential expressions of Emission Rate (ER), defined as contaminant mass
per unit time for types of source. Some examples are provided for situations in which they are most
useful.
Per Farm (e.g., ER/500-cow-dairy): Not commonly used due to complexity of accounting for
all emission sources under various management options, weather, and geographical
differences.
9
Per Unit of Area (e.g., ER/m ) for animal housing, open lots, manure storage, and feed
storage. Most common for emissions that do not originate from a fully enclosed building.
Per Animal Unit (e.g., ER/bird) for animal, place (i.e., # stalls), body weight, productive
animal ["per milking cow" = lactating/dry cow + her replacements]: Very commonly used
for enclosed buildings or where the animal population is relatively stable in both number and
body weight.
Per Unit of Food Product (e.g., ER/lb pork, gallon of milk, dozen eggs, or weaned piglet) for
final food product or animals marketed: Increasing in use as animal agriculture has become
more efficient in product produced with reduced animal population.
Per Inputs (e.g., ER/kg nitrogen fed): Best use in models and pollutant mitigation where the
biological, chemical, and management influences can be fairly evaluated.
3.5. Comments on Approach for Handling Negative and Zero Data
Question 5: Please comment on the EPA 's approach for handling negative or zero emission
measurements.
3.5.1. Background
Some NAEMS emissions measurements were reported as either negative or zero emissions values. The
EPA considered whether to include these negative and zero emissions values in the data used to develop
the EEMs. The agency evaluated whether the negative or zero values represented variability in
emissions measurements due to instrument/equipment performance. The EPA also reviewed the data to
see if the data quality measures were properly performed according to the Quality Assurance Project
Plan. The EPA concluded that all negative values should not be considered in the development of the
EEMs.
3.5.2. Response
Overview
There are two types of data assessed in the EPA's documents: directly measured air pollutant
concentrations and calculated air emission rate values. In both cases, the EPA must address negative and
zero values. In the draft EEMs, the EPA has not included negative values in the EEM development
process and kept the zero values. The SAB has reviewed the EPA's treatment of these values and
provides the following suggestions for the handling of negative and zero data for both direct
concentration measurement and calculated emission values.
23
-------
Negative values
There was a relatively small number (<1.7 percent for broiler and <2 percent for swine and dairy lagoon
data) of negative data points, but their inclusion in the model is important. Negative values appear in
both direct concentration measurements and calculated air emission rates. The SAB suggests several
approaches for handling the negative values.
Direct Air Pollutant Concentration Measurement Values
Except in a few possible situations, negative measures of concentrations are problematic. Since a
rigorous Quality Assurance/Quality Control (QA/QC) protocol was implemented for the NAEMS data,
and the raw data subjected to a flagging/validation process based on the QA/QC, EPA should remove
negative concentration values due to instrument malfunction or any other obvious errors. Therefore, in
the submitted and updated dataset, a negative concentration measurement value would occur due to a
true value that is at or below the minimum detection level (MDL), a measurement value that is adjusted
by the equipment calibration offset procedure, or instrument fluctuation due to influence by ambient
conditions. Each of these cases is considered individually.
Minimum Detection Level. From a statistical point of view, the correct approach for dealing
with negative values due to MDL is to recognize that those values are censored. That is, it is
known that the measured value is below the instrument's minimum limits of detection, but
above zero (a true concentration can never be below zero). These censored values should be
included in all statistical analyses. Suggestions for the treatment of this type of negative
value are as follows:
o Use the negative value produced as it is.
o Employ the EPA procedure of using half of the MDL when the observed value is
below the MDL. (Theoretically this method is better, but it is also very difficult to
differentiate the negative values that are due to MDL).
Calibration Offset. Negative values can arise due to instrument "noise" or adjustment of
calibration offset, which is calculated based on the average zero and span values over a
period of time. The negative gas concentration values attained during offset correction should
generally be very small in comparison with the mean measurement values. Due to the nature
of equipment noise, the resulting measurement values can be positive, zero or negative. Since
there is no way to identify and adjust for positive noise, the negative noise measurement
should be kept to ensure unbiased statistical analysis.
Ambient Influence. Variability in instrument measurements can result from variations in
ambient conditions (e.g., atmospheric stability) resulting in overestimated positive or
negative values. The bias, either positive or negative, will depend on the instrument type
(paniculate matter or gas) and ambient condition. For example, in the measurement of PM
from broiler confinement housing, negative PM concentrations can occur due to short term
fluctuations in relative humidity which causes fluctuation in the real-time Tapered Element
Oscillating MicroBalance (TEOM) PM concentration measurement process. When the air
humidity increases, the TEOM measurement will have an increased bias. If the air humidity
decreases, then the TEOM measurement bias will decrease, and a negative PM concentration
can possibly occur. Since it is very difficult to identify and quantify the positive bias, the
negative bias measurement should be kept for non-biased statistical analysis. In cases such as
these, the negative values that are produced from these situations will introduce a bias (that is
likely small) to the data. If excluded from the dataset, standard errors of estimated model
coefficients will be underestimated and, consequently, confidence intervals around, predicted
24
-------
concentrations, for example, will be too narrow, indicating a precision that is higher than
what it should be.
Overall, it is important to qualify unexpected observations individually and to understand and document
why an observation is negative. In some cases, it will be decided that the measurement is the result of
operator error, instrument failure, instrument drift or some other factor. In these cases, and absent
additional information that might permit correcting the measurement, observations should be discarded.
Calculated Emission Rate Data
Air emission rates were calculated by subtracting the measured background concentration value from
the directly measured concentration value, and multiplying by the airflow rate. Where the calculated
value was negative, the EPA decided not to include the negative value in the model because the agency
concluded that it suggested that the area in question (i.e., confinement houses, lagoon), was acting as a
sink (EPA's Lagoon Report, pg 3). The SAB, however, recommends that negative calculated emission
data be included in the model under certain conditions.
Negative calculated emission values can arise from the following scenarios:
In this study, the background and source measurements were measured either
intermittently (twice a day for gas), or continuously without correction for lag time in the
barn (PM data), thus leading to a bias either up or down, introducing the potential for
negative emission values. Because bias could occur in either the positive or negative
direction, negative calculated emission values should be retained in the dataset, as long as
their individual measured value was already validated. Omitting these data would bias the
model in the upward direction. The true estimated value is more accurate if all calculated
values are included.
A calculation bias may also occur when the measured values are at or near the detection
limit, or negative. Calculation of negative emission rates due to small or negative values
should be very small, and should be kept.
In some scenarios, outdoor events may affect the background concentration. For example,
if there was activity outside the poultry barn which resulted in increased pollutant
concentration (e.g., other barn cleanout and manure movement), the measured
background values would be biased upwards, and subsequently, the calculated emission
value may become negative. Alternatively, a positive bias could occur if meteorological
conditions caused the exhaust air to come back into the barn, thus influencing the
measured concentration. In these situations, errors caused by special abnormal outdoor
events should be identified and removed from the study results if appropriate.
Negative emission rates can be used to develop a model that never predicts negative emissions. In some
cases, these negative emission rates may be necessary to appropriately describe the uncertainty of the
model.
Overall, if the measured concentration data are validated and included in the dataset, then the emission
value calculated from that dataset is also valid, whether it is negative or positive.
Zero values
Zero values are present in the direct measurement data as well as in the calculated emissions dataset. If
during measurement of direct air pollutant concentrations, or after instrument calibration, the resulting
measurement is zero, the SAB recommends that the value ought to be used in statistical analyses.
However, few instruments have the precision needed to distinguish a true zero from a small value.
25
-------
Consequently, zero measurements will often correspond to censored observations and thus should be
treated as such. The use of zero values in the model is likely to produce small biases in both the
estimated regression coefficients and their standard errors.
After elimination of invalid data (including those data associated with zero fan flow that the NAEMS
Science Advisor recommended invalidating), if a calculated emission value is zero, it should be included
in the dataset. There are many cases in which emissions of a given pollutant may be zero from a
particular source and should be included in any analysis. Overall, if the emission value, calculated from
valid data, is zero, then that value should always be included in the model.
Outliers
The EPA did not apply formal statistical outlier tests as part of the modeling process. Instead, the EPA
applied standard procedures (control charts and custom software) to flag data believed to be outliers
(See Page 2, Attachment A, of EPA's July 2012 Supplemental Data5) as part of the data verification
process. The SAB suggests that outlier analysis procedures be conducted as part of the model building
process.
3.6. Alternative Approaches for Negative and Zero Data
Question 6: In the interest of maximizing the number of available data values for development of the
draft H2& EEMsfor swine and dairy lagoons/basins, does SAB recommend any alternative approaches
for handling negative and zero data other than the approach used by the agency.
3.6.1 Background
Some NAEMS emissions measurements were reported as either negative or zero emissions values. EPA
sought SAB advice on alternative approaches for handling this data.
3.6.2. Response
It is understood that the dataset for hydrogen sulfide (H2S) for swine and dairy lagoons/basins was small
due to data summary methods and/or instrument deficiency in being able to record
concentration/emission values. Instrument deficiency was due to changes in wind direction, inadequate
wind speeds or other unknown variables. This cannot be corrected after the fact. The Broiler and Lagoon
Reports should fully discuss the occurrence and reasons for the lack of sufficient data and large amount
of poor quality data.
The summary methods used by the EPA excluded data if the 75 percent completeness level for various
time periods (i.e., hourly, daily, and total) was not met. The 75 percent completeness criterion is too
stringent and unnecessary in this case. The SAB suggests that the criterion be re-evaluated so that more
data can be included. To maximize the dataset, it is recommended that all data meeting the criteria
outlined in Charge Question 5 be included for analysis, regardless of the 75% completeness criterion.
See the SAB response to Charge Question 5 for general recommendations for handling negative and
zero data for any dataset.
5 See EPA's July 2012 Report, "Additional Data for SAB Review: EPA's Emissions Estimating Methodologies for Animal
Feeding Operations for Broiler Sector and for Swine and Dairy Lagoons and Basins", available at on the SAB website at
http ://yo Semite, epa.gov/sab/sabproduct. nsf/fedrgstr_activite s/AFO-AEEM?OpenDocument
26
-------
3.7. Broiler Volatile Organic Compound (VOC) Emissions-Estimation Methodologies
Question 7: Please comment on the approach EPA used to develop the draft broiler VOC EEM.
3.7.1. Background
The EPA reviewed the VOC data submitted for the California and Kentucky broiler sites. The two sites
used different VOC measurement techniques. Based on analysis of the measurement and analytical
techniques and the VOC data, the EPA used only the VOC data from the Kentucky sites when
developing the draft VOC EEM.
3.7.2. Response
The SAB has identified significant limitations with the broiler VOC data and concluded that the broiler
VOC data cannot support the development of a broiler VOC EEM at this time.
Under the Consent Agreement (U.S. EPA 2005), the EPA is required to set an EEM for daily and annual
VOC emissions. However, there is a provision in the Consent Agreement that, if the SAB decides that
the available data are not adequate to support development of the EEMs, the EPA can delay
development of the EEMs until adequate data are available. Limitations of the broiler VOC data include:
The procedures used to collect VOC data at Site CA1B (i.e., Trihalomethane analyzer
with photobooster) did not produce useful data for model development and evaluation
and should not be used in development of an EEM. Therefore, data from only two farms
in one geographic region (KY1B-1 and KY1B-2) are available to the EPA through the
NAEMS study.
Canisters, which can only be used to assess a limited suite of compounds, were used to
sample VOCs. Other sampling techniques are required to gather other VOCs that cannot
be analyzed using canister analysis.
From Site KY1B, VOC recovery rates from the canister are unknown as not all
compounds are able to be extracted from electropolished canisters onto sorbent tubes, and
sorbent tubes were not utilized for direct collection of VOCs.
Sampling at Site KY1B was conducted quarterly over a 21-month period (i.e., seven
collection events), during which time two samplers were placed at the exhaust fans of
each of two facilities. However, background samples were not collected at the inlet of the
barns, so no data were available from which to determine the net increases in VOC
concentrations attributable to the housing facilities.
VOC concentration data from Site KY1B are limited to the specific climate and
management conditions of the site and cannot be applied to all production facilities
across the United States with a reasonable degree of confidence regarding their
representativeness.
Based on these concerns, the SAB recommends that the EPA not generate an EEM for VOCs from
broiler operations at this time.
Although the NAEMS dataset is too limited to produce an EEM, valuable components of the VOC data
should be reported. Based on the EPA's presentation of KY1B VOC data, those data appear generally
valid and usable if (and only if) the methods used to collect VOC data are more extensively and clearly
documented than in the EPA's first draft Broiler Report. In the draft, the agency reported in detail how
data were supposed to be collected at both sites, but details of how and what data were actually collected
27
-------
were incomplete and unclear. The EPA should state unambiguously what data were actually collected
from each site, how they were collected and analyzed and what data passed QA/QC criteria checks. Data
collected absent strict adherence to SOP and Quality Assurance Project Plans (QAPP), including
equipment calibration methods, are not valid and should be identified as such.
Data reported by the EPA should include total and speciated VOC concentrations to provide general
information on broiler emissions from the sites where data were collected. Moreover, "Total VOCs"
should be explicitly defined to clarify whether reported values represent the sum of all VOCs analyzed
or the total VOCs quantified by the analyzer, which will capture only a portion of all VOCs present in a
sample. These data may help identify important compounds emitted from broiler facilities, which can
help guide future data collection efforts. An indication of the magnitude of VOC concentrations relative
to any reports of background VOC concentrations reported for this region would help, qualitatively, to
identify those compounds that appear to be emitted in substantial quantities from the AFOs. One
challenge with the incomplete data collection is how the EPA determines if "substantial quantities of
compounds are emitted" when the entire VOC suite emitted is not quantified. When such quantification
does not occur, it is not possible to identify if one compound or another is a substantial component of the
VOCs emitted. Also, the determination of what is "substantial" is subjective without a numeric qualifier.
After reporting the available data, the EPA should defend the decision to not develop an EEM given the
limited information available and the uncertainty of the data collected in the NAEMS. To develop an
EEM for VOCs, it is essential that EPA lay the foundation for development of a process-based model for
estimating emissions from these operations. This foundation requires a comprehensive investigation of
existing scientific literature and future research regarding factors driving generation of VOC emissions
from broiler facilities.
28
-------
4. SPECIFIC RECOMMENDATIONS FOR THE DRAFT BROILER AND
LAGOON REPORTS
The SAB provides the following general comments on EPA's draft Broiler Report and Lagoon Report.
The SAB considered whether the draft Broiler and Lagoon Reports were presented in a clear,
comprehensive, and scientifically sound manner.
Overall, SAB finds that both reports should be updated to describe the importance of retaining a long-
term goal for producing process-based models. The SAB also concludes that the reports should more
comprehensively describe data completeness, representativeness, limitations and whether there are
sufficient data to begin a process-based modeling approach. The SAB recommends that the discussions
of mechanisms of data collection, including pollutant concentrations, ventilation rates within barns, and
feed composition and quantity should be enhanced in the reports. Furthermore, the reports should more
fully explain why any of the NAEMS data were excluded from EEM development. Since NAEMS data
have significant limitations, the reports should include an assessment of additional data that the EPA has
collected through EPA's Call for Information (U.S. EPA 2011).
Specific SAB recommendations for each EPA draft report, beyond those made in response to the charge
questions in section of this SAB report, are noted below. The SAB recommends that the EPA consider
the references provided in the bibliography of this SAB report to improve the literature base for the
Broiler and Lagoon Reports and to help ensure a more comprehensive understanding of AFO broiler
and/or swine and dairy lagoon/basin operations.
4.1. Recommendations for Revising the Draft EPA Broiler Report
The SAB recommends that the EPA reorganize the report and rewrite several sections to address issues
raised in the SAB report. The EPA should develop a process-based modeling approach to predict air
emissions from broiler farms and incorporate that approach into the report. The EPA should also make a
number of improvements to the statistical approach for developing EEMs. In particular, the agency
should describe methods for calculating confidence values to present variability of data, include
quantitative statistical analyses that compare houses, consider approaches in addition to the cross-
validation method used to evaluate the model, and more comprehensively describe data completeness,
representativeness, and limitations.
Section 1 should describe the importance of pursuing a long-term goal of producing process-based
models and refer to the NRC recommendations (NRC 2003) on this topic. This section should also note
that the developed models are considered short-term tools with limited application for estimating
emissions.
The limitations of the data set and the various data measurement problems that occurred as part of the
NAEMS data collection efforts should be more comprehensively described and summarized in Section
1. For example, data from poultry sites were collected for typical bird grow-out periods, but there are
birds that are grown for much shorter periods (e.g., Cornish hens) and those that are grown for much
longer periods (e.g., large roasters). These limitations should be clearly stated because the current EEMs
for ammonia would not fit some of the situations well (i.e., emissions would be estimated to go to zero
for some of the largest birds and would be negative for some of the smallest birds). The discussion of
mechanisms for data collection, ventilation rates within barns and feed nutrients consumed should also
be enhanced.
29
-------
The introduction section should clearly acknowledge that the broiler data were collected at an extremely
limited number of study sites (four broiler barns on three farms, including two farms at the Kentucky
site). The EPA should consider clarifying the text to note that the 2,600 industrial participants in the
Consent Agreement are a very small fraction of the one-half million AFOs in the country. EPA should
consider clarifying the percentage of total confinement animal production represented by these industrial
respondents.
The Report should acknowledge that similar to airflow rate and static pressure, flock mortality data is
not readily available for all four broiler houses. The Report should acknowledge that EPA removed
over 40 days of data from EEM development because of missing mortality data.
The text in Sections 1 and 2 would be strengthened by referral to the mechanistic processes that drive
the emissions that the developed EEMs are estimating. The primary physical/biological/chemical
mechanisms that lead to emissions of each regulated parameter should be described in relation to the
surrogate statistical parameter. This would strengthen the validity of the statistical model employed. For
example, the product of bird number and mass is considered a surrogate for fresh manure production
that impacts ammonia emissions.
The text should note that the EPA planned to measure several key parameters that affect emissions
generation, such as animal activity, diets, feed rate and composition, water management, and manure
composition (moisture and nitrogen), total number of animals, and ventilation rate. The text should note
that the EPA did not utilize these parameters during EEM development because the EPA judged that
data for these variables were insufficient in quantity and/or quality. EPA should describe data that were
collected but not yet transmitted to the agency as of the development of the draft EEMs documents.
The accurate determination of ventilation rate (VR) is a very important aspect of the NAEMS data
collection and is necessary to achieve representative emission data. The determination of accurate
ventilation rate should be given more prominence in the report with a concise description of how this
was achieved. The description of ventilation systems and control operations for each barn also should be
clarified, particularly regarding inlet description and function.
The EPA should clarify the range of conditions under which the NAEMS-based EEMs can be used. For
example, the EPA should describe the ambient temperature range during grow-out or litter management
period between flocks within which the EEMs can be applied. The EPA should also add cautionary
notes regarding the use of EEMs outside of the studied range.
The report should note that broiler confinement facilities are commonly managed as both bird
production facilities and as dry manure storage if litter is not completely cleaned out between flocks.
The report should discuss the importance of stockpiled litter storage emission measurements (litter being
the combination of bedding and manure) and the link of such emissions to the process-based model
development and evaluation. The microbial degradation and natural chemical interactions associated
with all the parameters measured should be described. Throughout the report, the emissions from
populated houses during grow-out and empty houses during litter management should be presented
separately since the house is managed very differently during these two time periods. In addition, the
differential in emissions observed from fully cleaned out houses versus de-caked, built-up litter houses
should be presented separately.
30
-------
The EPA should improve the clarity of the discussions on the NAEMS monitoring sites and on the data
available for EEM development. The report should discuss why the data sets used are representative of
the industry and the literature. For example, it is unclear how well California farm CA1B, a 16-house
broiler ranch in Stanislaus County, California, built in the 1960s, represents modern industry practices.
Also, pancake brooders (used in Kentucky) are primarily used by one parent company (i.e., integrator,
who typically operates or contracts every aspect of the broiler production process), so emissions from
houses employing such equipment during the brooding period are not likely to represent emissions from
facilities operated by other integrators. EPA should develop criteria for considering additional data and
how to use such data.
EPA describes many parameters that were not used in its analysis. EPA should clarify which parameters
were used for developing EEMs and discuss the reasons for, and the importance of, not including other
parameters for which data were collected in the analysis.
The EPA should take the following steps to provide additional information regarding the data used in
developing EEMs:
Identify the number of samples collected during each sampling event and the periods that
data were collected;
Clarify the VOC discussions regarding Kentucky and California VOC analyses. This
discussion is poorly written and very confusing (the EPA should note that the California
VOC data were not used and why these data were not used);
Describe fan calibration procedures and frequency;
Clarify how the change in purge time for the first four months of gas sampling in
California was addressed;
Describe the sampling schedule for PMio, PM2.5 and TSP samples;
Explain the data to be collected in the sampling plan and why data that were specified in
the sampling plan were not collected;
Describe sampler inlet systems used for measurement and address associated issues with
use of these inlets in some applications (e.g., aspiration of PM by low volume inlets);
Describe ventilation rate, which includes discussion on the FANS system and repeated
calibrations; and
Clarify a potential discrepancy associated with the KY1B dataset, available at
http://www.epa.gov/airquality/agmonitoring/data.html. The report does not indicate that
ambient weather data and confinement data are available for the Kentucky site, although
the data posted on the website spreadsheet are described as containing "daily mean
concentrations of pollutants, weather and barn conditions."
The EPA should clearly specify criteria for data completeness, use of data, eliminating data, collection
of background concentration data, and use of data available in the literature for modeling verification.
EPA should also discuss why a 75 percent completeness value was used as a threshold for using data,
why there are missing data days and why some data were collected in some seasons and not in others.
The EPA should also clarify how the agency identified outliers in the data and the reasons for their
inclusion or omission. The discussion on seasonal influences should be improved to discuss whether
such influences should be incorporated into the model. The text should also describe how anomalies are
defined and applied in the data set. Finally, the Broiler Report (Sections 4.1.2 and 5.1.2) notes that "A
valid monitoring day is one in which 75 percent of the hourly average data values used to calculate the
daily value were valid measurements." This statement is incorrect and should be changed to read: "A
31
-------
valid monitoring day is one in which 75 percent of the 60-s records used to calculate the daily value
were valid measurements."
4.2. Recommendations for Revising the Draft EPA Lagoon Report
The SAB recommends that the EPA reorganize the report and rewrite several sections to address issues
raised in the SAB report. Recommendations are provided to more comprehensively describe data
completeness, representativeness and limitations. Many comments that the SAB provides to strengthen
the Broiler Report also apply to the Lagoon Report (e.g., comments on data completeness, use of data,
and statistical and process-based model approaches). The EPA should review the Lagoon Report in light
of these comments and make revisions as appropriate.
Section 1 should describe the importance of pursuing a long-term goal for developing process-based
models, and refer to the NRC recommendations (NRC 2003) on this topic. This section should also note
that the development of empirical models is considered a short-term tool for estimating emissions.
The discussion of the U.S. dairy and swine industries should be rewritten. Additional details should be
provided on the overall operations at the dairy and swine industry facilities, particularly the facility
waste handling techniques and manure management systems. The EPA should consider conducting a
nitrogen balance analysis to better understand the mass of nitrogen that may be emitted. Additional
information on the lagoons where data were collected should be provided, as well as information on
what constitutes a standard lagoon throughout the industry. Section 1 notes that due to the limited
amount of data for nitrogen content, solid content and pH of the lagoon liquid, these data were not
included in the EEM. Section 2 also notes that data on manure residence time, amount of sulfur ingested
by an animal and amount of carbon in feed were not collected. The EPA should summarize the
limitations of the data set and the various data measurement problems that occurred as part of the
NAEMS data collection efforts.
The discussion of manure management, storage and stabilization should be revised. Discussion of the
design difference between storage and treatment ponds (i.e., basins and lagoons, respectively) should be
corrected to indicate that treatment ponds are designed specifically for biological treatment and storage
ponds are not designed for biological treatment. In addition, the report should indicate that waste
characteristics for swine and dairy animals are significantly different. Standardized definitions exist for
manure treatment/storage structures. The EPA's report should use American Society of Agricultural
Engineers/American Society of Agricultural and Biological Engineers (ASAE/ASABE) Standard:
Uniform Terminology for Rural Waste Management (ASABE S292.5). The text should describe the
processes that generate ammonia from nitrogen and cause volatilization of that nitrogen. The text should
also describe the microbial degradation and natural chemical interactions for all parameters measured.
Additional details on hydrocarbon and VOC sampling results, average dairy cow weight and manure
management systems should be provided. The EPA report should provide additional information on the
lagoons where data were collected, and a definition of a standard. In addition, it should be noted that the
EPA's analysis used data from a wash water dairy lagoon, not a manure storage lagoon, which may
affect the EEM estimation efforts. Finally, the appendices reference several pre-study validation studies.
The results from these validation studies should be included in the report so that it would be possible to
evaluate the data quality that may have been generated using these tested techniques.
32
-------
The EPA should clearly specify criteria for data completeness, use of data, eliminating data, collection
of background concentration data, and use of data available in the literature for modeling verification.
EPA should also clarify how outliers were identified and the reasons for their inclusion or omission.
33
-------
BIBLIOGRAPHY
SAB suggests that the EPA consider the following additional references to improve the literature base
for the draft EPA Reports and help ensure a more comprehensive understanding of AFO broiler and/or
swine and dairy lagoon/basin operations:
Analytical Methods Committee. 2001. Measurement of near zero concentration: recording and reporting
results that fall close to or below the detection limit. Analyst 126(2): 256-259.
Aneja, V.P., J.P. Chauhan, and J.T. Walker. 2000. Characterization of atmospheric ammonia emissions
from swine waste storage and treatment lagoons. J. of Geophysical Research 105 (No. D9):
11535-11545.
Aneja, V.P., J.M. Overton, B.P. Malik, Q. Tong, and D. Kang. 2001. Measurement and modeling of
ammonia emissions at waste treatment lagoon-atmospheric interface. Journal of Water, Air, and
Soil Pollution 1:177-188.
Aneja, V.P., S.P. Arya, 1C. Rumsey, D.S. Kim, K. Bajwa, H.L. Arkinson, H. Semunegus, D.A. Dickey,
L.A. Stefanski, L. Todd, K. Mottus, W.P. Robarge, and C.M. Williams. 2008. Characterizing
Ammonia Emissions from Swine Farms in Eastern North Carolina: Part 2Potential
Environmentally Superior Technologies for Waste Treatment. J. Air & Waste Manage. Assoc.
58:1145-1157.
Aneja, V.P., S.P. Arya, 1C. Rumsey, D.S Kim, K.S. Bajwa, and C.M. Williams. 2008. Characterizing
ammonia emissions from swine farms in eastern North Carolina: Reduction of emissions from
water-holding structures at two candidate superior technologies for waste treatment. Atmospheric
Environment 42: 3291-3300.
Aneja, V.P., W.H. Schlesinger, and J. Willem Erisman. 2009. Effects of agriculture upon the air quality
and climate: research, policy, and regulations. Environ. Sci. Technol. 43, 4234-4240.
American Society of Agricultural and Biological Engineers (AS ABE). 2005. Manure Production and
Characteristics. ASABE Standard D384.2. St. Joseph. MI. Joseph, Michigan.
Bajwa, K.S., V.P. Aneja, and S.P. Arya. 2006. Measurement and estimation of ammonia emissions from
lagoon-atmosphere interface using a coupled mass transfer and chemical reactions model, and an
equilibrium model. Atmospheric Environment 40: S275-S286.
Blunden, J. and V.P. Aneja. 2008. Characterizing ammonia and hydrogen sulfide emissions from a
swine waste treatment lagoon in North Carolina. Atmospheric Environment 42: 3277-3290.
Blunden, J., V.P. Aneja, and J.H. Overton. 2008. Modeling hydrogen sulfide emissions across the gas-
liquid interface of an anaerobic swine waste treatment storage system. Atmospheric Environment
42: 5602-5611.
Blunden, J., V.P. Aneja, and P.W. Westerman. 2008. Measurement and analysis of ammonia and
hydrogen sulfide emissions from a mechanically ventilated swine confinement building in North
Carolina. Atmospheric Environment 42: 3315-3331.
34
-------
Blunden, J., V.P. Aneja, and W.A. Lonneman. 2005. Characterization of non-methane volatile organic
compounds at swine facilities in eastern North Carolina. Atmospheric Environment 39: 6707-
6718.
Burns, R.T., H. Xin, R.S. Gates, H. Li, L.B. Moody, D.G. Overhults, J.W. Earnest, S.J. Hoff, and S.
Trabue. 2009. Final project report on Southeastern broiler gaseous and particulate matter
emissions monitoring.
Chianese, D.S., C.A. Rotz , and T.L. Richard. 2009. Simulation of methane emissions from dairy farms
to assess greenhouse gas reduction strategies. American Society of Agricultural and Biological
Engineers (ASAE): 52(4): 1313-1323.
Coufal, C.D., C. Chavez, P.R. Niemeyer, and J.B. Carey. 2006. Effects of top-dressing recycled broiler
litter on litter production, litter characteristics, and nitrogen mass balance. Poultry Sci. 85: 392-
397.
Coufal, C.D., C. Chavez, P.R. Niemeyer, and J.B. Carey. 2006. Measurement of broiler litter production
rates and nutrient content using recycled litter. Poultry Sci. 85: 398-403.
Coufal, C.D., C. Chavez, P.R. Niemeyer, and J.B. Carey. 2006. Nitrogen emissions from broilers
measured by mass balance over eighteen consecutive flocks. Poultry Sci. 85: 384-391.
Currie, L.A. 1999. Detection and Quantification Limits: Origin and Historical Overview. Analytica
ChimicaActa39\: 127-134.
Flood, C.A., J.L. Koon, R.D. Trumbull, and R.N. Brewer. 1992. Broiler growth data: 1986-1991. Trans.
American Society of Agricultural Engineers 35(2): 703-709.
Gates, R.S., K.D. Casey, E.F. Wheeler, H. Xin, and A.J. Pescatore. 2007. U.S. broiler ammonia
emissions inventory model. Atmospheric Environment: 42(14):3342-3350
Hafner, S.D., F. Montes, and C.A. Rotz. 2012. A mass transfer model for VOC emission from silage.
Atmospheric Environment: 54: 134-140.
Janiga, I, J. Mocak, and I. Garaj. 2008. Comparison of Minimum Detectable Concentration with the
IUPAC Detection Limit. Measurement Science Review. 8(1): 108-110.
Lacey, R.E., J.S. Redwine, and C.B. Parnell. 2003. Particulate matter and ammonia emission factors for
tunnel-ventilated broiler production houses in the southern U.S. Transactions of the ASAE 46(4):
1203-1214.
Li, C., W. Salas, R. Zhang, C. Krauter, C.A. Rotz , and F. Mitloehner. 2012. Manure-DNDC: a
biogeochemical process model for quantifying greenhouse gas and ammonia emissions from
livestock manure systems. Nutrient Cycling in Agroecosystems: 74(1).
35
-------
Liang, Y, H. Xin, H. Li, R.S. Gates, E.F. Wheeler, and K.D. Casey. 2005. Effect of measurement
interval on estimation of ammonia emission rates for layer houses. Transactions ofASAE:
49(1): 183-186.
Lin, X.J., E.L. Coitus, R. Zhang, S. Jiang, and AJ. Heber. 2012. Air emissions from broiler buildings in
California. Transactions of the American Society of Agricultural and Biological Engineers
(ASABE) 55(5): 1895-1908.
Mai one, G.W., T. Sims, and N. Gedamu. 1992. Quantity and quality of poultry manure produced under
current management programs. Final report to the Delaware Department of Natural Resources
and Environmental Control and Delmarva Poultry Industry, Inc., University of Delaware,
Research and Education Center, Georgetown, Delaware.
Metcalf and Eddy. 2003. Wastewater Engineering: Treatment and Reuse. 4th ed. New York: McGraw-
Hill.
Montes, F., C.A. Rotz, and H. Chaoui. 2009. Process modeling of ammonia volatilization from
ammonium solution and manure surfaces: A review with recommended models. American
Society of Agricultural and Biological Engineers: 52(5): 1707-1719.
Mosquera, J. and N.W.M. Ogink. 2004. Determination of the variation sources associated with ammonia
emission measurements of animal housings. AgEngr 2004. 12-16, September 2004, Leuven,
Belgium.
NRC (National Research Council). 2003. Air Emissions from Animal Feeding Operations: Current
Knowledge, Future Needs. Washington, DC: The National Academies Press.
Natural Resource, Agriculture, and Engineering Service (NRAES). 1999. Poultry Waste Management
Handbook. Coop. Extension, Ithaca, NY.
Ogink, N.W.M., J. Mosquera, and R.W. Melse. 2008. Standardized testing procedures for assessing
ammonia and odor emissions from animal housing systems in The Netherlands In: Proceedings
of the Mitigating Air Emissions from Animal Feeding Operations Conference, Des Moines,
Iowa, USA, 19-21 May, 2005. - Des Moines : Mitigating Air Emissions from Animal Feeding
Operations Conference, 2008-05-197 2008-05-21
Redwine, J.S., R.E. Lacey, S. Mukhtar, and J.B. Carey. 2002. Concentration and emissions of ammonia
and particulate matter in tunnel-ventilated broiler houses under summer conditions in Texas.
Transactions ofASAE 45(4): 1101-1109.
Ro, K.S., M.H. Johnson, P.G. Hunt, and T.K. Flesch. 2011. Measuring trace gas emission from multi-
distributed sources using vertical radial plume mapping (VRPM) and backward Lagrangian
stochastic (bLS) techniques. Atmosphere 2(3): 553-566.
Ro, K.S., M.H. Johnson, K.C. Stone, P.G. Hunt, T.K. Flesch, and R.W. Todd. 2012. Inverse-dispersion
technique for assessing lagoon gas emissions. American Society of Agricultural and Biological
Engineers, 2012 Conference Proceedings, Dallas, Texas, July 29 - August 1, 2012.
36
-------
Rocke, D.M., B. Durbin, M. Wilson, and H.D. Kahn. 2003. Modeling Uncertainty in the Measurement
of Low Level Analytes in Environmental Analysis. Ecotoxicology and Environmental Safety 56:
78-92.
Rotz , C.A., F. Montes, and D.S. Chianese. 2012. The carbon footprint of dairy production systems
through partial life cycle assessment. Journal of Dairy Science 93(3): 1266-1282.
Rumsey, I. C., V.P. Aneja, and W.A. Lonneman. 2012. Characterizing non-methane volatile organic
compounds emissions from a swine concentrated animal feeding operation. Atmospheric
Environmental: 348-357.
South Coast Air Management District has had emissions estimates for dairy in poultry for years.
SCAQMD Poultry and Dairy Emission factors and guidelines for using the online 'calculator'
can be found here: www.aqmd.gov/aer/Updates/GuideCalcEmisDairyPoultry.pdf (January
2009).
Summers, M.D. 2005. FINAL REPORT: Quantification of Gaseous Emissions from California Broiler
Production Houses. Available at: http://www.arb.ca.gov/ag/caf/poulemisrpt.pdf accessed 29
March 2012.
Trabue, S., K. Scoggin, H. Li, R. Burns, H. Xin, and J. Hatfield. 2010. Speciation of volatile organic
compounds from poultry production. Atmospheric Environment 44: 3538-3546.
U.S. EPA (Environmental Protection Agency). 2005. Animal Feeding Operations Consent Agreement
and Final Order. Federal Register 70(19):4958-4977.
U.S. EPA (Environmental Protection Agency). 2011. Call for Information: Information Related to the
Development of Emission-Estimating Methodologies for Animal Feeding Operations. Federal
Register 76(12):3060-3062.
Wheeler, E.F., K.D. Casey, R.S. Gates, H. Xin, J.L. Zajaczkowski, P.A. Topper, Y. Liang, and AJ.
Pescatore. 2006. Ammonia emissions from twelve USA broiler chicken houses. Transactions
ofASABE49(5):l495-l5U.
Wheeler, E.F.; D. Meyer, P. Martin, D. Schmidt, and W. Powers. 2010. Recommended Units and
Supporting Data for Standardized Reporting of Air Emissions from Animal Agriculture.White
Paper USD A NRCS Agricultural Air Quality Task Force. 21 September 2010 Livestock and
Poultry Sub-Committee.
http://vosemite.epa. gov/sab/sabproduct.nsf/C86C8E839E06C34C852579BA006D31Al/$File/Pu
blic+Comments+submitted+by+Sally+Shaver+and+Dr.+Robert+Burns,+representing+the+USD
A+Ag+Air+Quality+Task+Force-3-7-12.pdf
Wilson, M.D., D.M. Rocke, B. Durbin, and H.D. Kahn. 2004. Detection Limits and Goodness of Fit
Measures for the Two-component Model of Chemical Analytical Error. Analytica ChimicaActa
509: 197-208.
Xin, H.; H. Li, R. Gates, R. Burns, and K. Casey. 2010. Natural Resource Conservation Service (NRCS)
White Paper "Methodologies and Protocols for Analysis of Raw Data to Minimize Uncertainty
37
-------
of Resultant Emissions Estimation".
http://vosemite.epa. gov/sab/sabproduct.nsf/C86C8E839E06C34C852579BA006D31Al/$File/Pu
blic+Comments+submitted+by+Sally+Shaver+and+Dr.+Robert+Burns,+representing+the+USD
A+Ag+Air+Quality+Task+Force-3-7-12.pdf
38
-------
APPENDIX A-EPA'S CHARGE QUESTIONS
February 17, 2012
MEMORANDUM
SUBJECT: Animal Feeding Operations Air Emissions Estimating
Methodologies from the National Air Emissions Monitoring Study
FROM: Stephen D. Page, Director
Office of Air Quality Planning and Standards (C404-04)
TO: Ed Hanlon
Designated Federal Officer
Animal Feeding Operations Emission Review Panel
EPA Science Advisory Board Staff Office (1400R)
This memorandum requests that the Science Advisory Board (SAB) review and comment on the draft
emissions estimating methodologies (EEMs) for animal feeding operations (AFOs). In preparation for
this review, the SAB has formed the Animal Feeding Operations Emission Review Panel. We envision
conducting multiple meetings of this panel to cover the material we are requesting to be reviewed. This
memorandum contains background material and charge questions for review by the expert SAB Panel at
the initial meeting. We request that these materials be forwarded to the SAB Panel for their review.
As the attachment and associated documents illustrate, the EPA staff has carefully considered the data
collected as part of the National Air Emissions Monitoring Study (NAEMS) and now ask the Panel to
refine and comment upon our work thus far to create EEMs. To bound and define the discussion, the
attachment offers charge questions for the Panel to consider.
By way of background, in 2005, the EPA entered a voluntary consent agreement with the AFO industry
in which AFOs that chose to sign the Air Compliance Agreement (Agreement) shared responsibility for
funding a nationwide emissions monitoring study. The NAEMS monitoring protocol was developed
through a collaborative effort of AFO industry experts, university scientists, U.S. Department of
Agriculture and EPA scientists and other stakeholders. The monitoring study was designed to gather
data for developing methodologies for estimating emissions from AFOs and to help AFOs determine
and comply with their regulatory responsibilities under the Clean Air Act (CAA), the Comprehensive
Environmental Response, Compensation and Liability Act (CERCLA), and the Emergency Planning and
Community Right-To-Know Act (EPCRA). Once the EPA publishes the applicable EEMs, the
Agreement requires each participating AFO to certify that it is in compliance with all relevant
requirements of the CAA, CERCLA and EPCRA.
We appreciate your efforts and those of the Panel to prepare for the upcoming meeting and look forward
to discussing this project in detail. Questions regarding the attached materials should be directed to Ms.
Robin Dunkins, EPA-OAQPS (telephone: 919-541-5335; email: dunkins.robin@epa.gov).
Attachment
A-l
-------
cc: Bill Harriett
Robin Dunkins
Larry Elmore
Lawrence Elworth
Allison Mayer
Janet McCabe
Peter Tsirigotis
A-2
-------
ATTACHMENT
Regulatory Background
In 2005, the EPA entered a voluntary consent agreement with the animal feeding operations (AFO)
industry in which AFOs that chose to sign the Air Compliance Agreement (Agreement) shared
responsibility for funding the National Air Emissions Monitoring Study (NAEMS). Approximately
2,600 AFOs, representing nearly 14,000 facilities that include broiler, dairy, egg layer and swine
operations, received the EPA's approval to participate in the Agreement.
To provide a framework for the NAEMS, AFO industry experts, university and government scientists
and other stakeholders collaborated to develop a comprehensive monitoring plan. The study was
designed to generate scientifically credible data to characterize emissions from the participating animal
sectors.
Consistent with the Agreement, the Agriculture Air Research Council (AARC), a nonprofit entity
comprised of participating AFO industry representatives, administered the monitoring study. The AARC
was responsible for selecting the Independent Monitoring Contractor (IMC) and the study's Science
Advisor with EPA approval. The Agreement outlined the roles and responsibilities of the AARC, the
IMC and the Science Advisor.
The monitoring plan specified the general geographic location of the farms to be monitored, animal
production phase, ventilation type, manure management/handling system and other pertinent
information for each animal sector.
For broilers, two sites were to be monitored - one on the West Coast and the other in the
Southeast. Both were to be mechanically ventilated and have litter on the floor.
For the swine industry, the sites were to be located in the Southeast (sow and finisher), Midwest
(sow and finisher), and West (sow). Mechanically-ventilated buildings, a deep pit building,
lagoons and basin manure storage types were to be monitored.
For dairy, both naturally- and mechanically-ventilated buildings, lagoons and basins were
monitored. Five dairies were monitored, one dairy in each of the following geographical areas:
Northeast, Midwest, Northwest, West and South.
For confinement sources, the EVIC monitored for ammonia (NHa), particulate matter (PMio, PM2.5, TSP),
volatile organic compounds (VOCs) and hydrogen sulfide (H^S). For lagoons and basins, H^S, NHa and
VOC were to be monitored. Accordingly, the EPA is then responsible for developing EEMs for each of
these pollutants.
Charge to the Science Advisory Board (SAB) AFO Air Emissions Review Panel
In preparation for the first and second meeting, the EPA has analyzed the NAEMS data for two broiler
sites and nine swine and dairy lagoons/basins. For the purpose of this study, the EPA used the
description of a lagoon and basin as provided in the MidWest Plan Service "Manure Storages" (MWPS-
18 Section 2) document. According to MWPS, "A lagoon is a biological treatment system designed and
operated for biodegradation of organic matter in animal manure to a more stable end product. A basin,
while similar to but smaller than a lagoon, is designed to store manure only and is not a treatment
system."
A-3
-------
For a broiler confinement house, the EPA has developed draft EEMs for NHs, PMio, PM2.s, TSP, VOC
and H2S. For swine and dairy lagoons/basins, the EPA has only developed a draft EEM for NHa. The
documents provided to the SAB describe the sites monitored; the data submitted to the EPA; and a
detailed discussion of the statistical methodology used to develop the draft EEMs. This material is
provided to inform the SAB panel of the EEM development process used by the agency. In subsequent
meetings, the EPA will address draft EEMs for egg-layers, swine and dairy confinement houses and
other pollutants for swine and dairy lagoons/basins.
Issue 1: Statistical Methodology used to develop draft EEMs
The EPA seeks the SAB's input on the statistical methodology used by the EPA to develop the draft
EEMs. Section 7.0 and 8.0 of the broiler document and section 5.0 of the swine and dairy lagoon/basin
document provide an overview of the statistical methodology used to develop the draft EEMs. A flow
diagram of the statistical methodology is provided in Figure 7-1 in the broiler document and Figure 5-1
in the swine and dairy lagoon/basin document. The EPA considers this statistical methodology to be the
best approach for analyzing the data and intends to use this same approach to develop draft EEMs for
the egg-layers, swine and dairy confinement houses.
Using the process described in the sections listed above, we developed a mean trend function that
provides a point prediction of emissions under a given set of conditions. We chose an appropriate mean
trend function to quantify the relationship between predictor variables and pollutant emissions by
analyzing the emissions data and incorporating knowledge of the emissions generating processes. The
EEM development process also involves choosing a probability distribution and covariance function to
appropriately quantify other contributions to variability in emissions, and thereby to accurately quantify
methods at all stages. If necessary, we will adjust the statistical methodology based on our review of the
SAB's input.
Question 1: Please comment on the statistical approach used by the EPA for developing the draft EEMs
for broiler confinement houses and swine and dairy lagoons/basins. In addition, please comment on
using this approach for developing draft EEMs for egg-layers, swine and dairy confinement houses.
Issue 2: Statistical Methodology used to develop swine and dairy lagoon/basin draft EEMs
After conducting an initial analysis of the NAEMS data submitted for swine and dairy lagoons/basins,
the EPA decided to focus on developing a draft EEM for NFL?. The EPA's review of current literature
indicates that lagoon/basin emissions are influenced by several factors, one of these being lagoon/basin
temperature. To ensure that the dataset used to develop the draft EEM represented all seasonal
meteorological conditions for the entire two year monitoring period, the EPA decided to combine the
swine and dairy data. Combining the swine and dairy lagoon/basin dataset also resulted in combining
lagoon and basin emissions data.
To maximize the number of NFTj emissions measurements used to develop the draft EEM, the EPA used
static predictor variables (SPVs) as surrogates for data on lagoon/basin conditions (i.e., nitrogen content
of lagoon liquid, lagoon pH, oxidation reduction potential and temperature). The static variables of
animal type, total live mass of animal capacity on the farm and the surface area of the lagoon were used
to represent NFb precursor loading and the potential for release to the air. Consistent with operating
parameters associated with statistical degrees-of-freedom, we concluded that two degrees of freedom
A-4
-------
was the maximum that the data would credibly allow for inclusion in the developing the draft EEM. As
a result, the EPA developed three sets of draft EEMs, using the paired combinations of these static
variables (i.e., animal type, surface area, farm size) and the continuous variables representing
meteorological conditions (i.e., temperature, atmospheric pressure, humidity, wind speed, solar
radiation).
Question 2: Please comment on the agency's decision to combine the swine and dairy dataset to ensure
that all seasonal meteorological conditions are represented. In addition, the agency also seeks the SAB's
comments on whether the agency should combine lagoon and basin data.
Question 3: Please comment on the agency's decision to use SPVs as surrogates for data on
lagoon/basin conditions. Given the uncertainties in that approach, does the SAB recommend that the
EPA consider specific alternative approaches for statistically analyzing the data that would allow for the
site-specific lagoon liquid characteristics to be used as predictor variables?
Question 4: Does the SAB recommend that EPA consider alternative approaches for developing the
draft NHs EEM that balances the competing needs for a large dataset (to reflect seasonal meteorological
conditions) versus incorporating additional site-specific factors that directly affect lagoon emissions. If
so, what specific alternative approaches would be appropriate to consider?
Issue 3: Negative and Zero Data
Some emissions measurements were reported to the EPA as either negative or zero emissions values.
When developing the draft EEMs, the EPA used the following general approach regarding inclusion of
negative and zero emissions values in the data.
The EPA evaluated whether the negative or zero values represent the variability in emissions
measurements due to the means of obtaining the measurements. For example, negative values for
a pollutant concentration might result when the concentration of the pollutant falls below the
minimum detection limit of a monitor. For all EEM datasets, the EPA included zero values
because these values potentially represent instances where the emissions from the source were
zero (e.g., a frozen lagoon), or the background and pollutant concentrations from the source were
the same. Regarding negative values, in cases where the dataset available to develop draft EEMs
was relatively large and the emissions were significantly greater than zero, the EPA excluded
negative emissions values from the EEM datasets. The EPA used this approach to develop the
entire broiler confinement house draft EEMs and swine and dairy lagoon/basin NFb draft EEMs.
The EPA reviewed the data to see if the data quality measures were properly performed
according to the Quality Assurance Project Plan.
If the EPA identified data where the quality assurance measures were not followed, we contacted
the science advisor to determine if the corrected data could be submitted to the EPA.
The EPA has conducted a preliminary analysis of the swine and dairy lagoon/basin H^S emissions data.
Our analysis indicates that we may need to modify our approach for handling negative and zero data in
order to develop a draft F^S EEM for swine and dairy lagoons/basins. A modification may be needed
due to the limited number of FI^S emissions values, the presence of a greater percentage of negative
emissions values and emissions values that are closer to zero than the NFb emissions for swine and dairy
A-5
-------
lagoons/basins. The EPA's concern is that failure to include the negative measurements in the dataset, or
setting them equal to zero, would result in an EEM that fails to fully quantify uncertainty around the
point prediction of emissions attributable to measurement error.
Question 5: Please comment on the EPA's approach for handling negative or zero emission
measurements.
Question 6: In the interest of maximizing the number of available data values for development of the
draft H2S EEMs for swine and dairy lagoons/basins, does SAB recommend any alternative approaches
for handling negative and zero data other than the approach used by the agency.
Issue 4: Volatile Organic Compounds (VOC) Data
The EPA reviewed the VOC data submitted for the California and Kentucky broiler sites. The two sites
used different VOC measurement techniques. Based on our analysis of the measurement and analytical
techniques and the VOC data, the EPA decided to use only the VOC data from the Kentucky sites when
developing the draft VOC EEM.
Question 7: Please comment on the approach EPA used to develop the draft broiler VOC EEM.
A-6
-------
APPENDIX B-ADDITIONAL RESPONSE TO CHARGE QUESTION 1
Overview of Current EPA Statistical Approach
SAB understands that EPA needs a method to routinely estimate air emissions from AFOs. EPA
developed statistical models to make these estimations, and these statistical models need to contribute to
the goal of developing models to make accurate predictions on farms across the U.S.
The statistical models that EPA has developed, based on combined data sets and a small number of static
predictor variables, have limited ability to predict emissions beyond the small number of farms in the
dataset. While basing the EEMs on data from a small number of farms does not necessarily limit the
applicability of the EEMs to national populations, the assumptions and forms of the statistical models
used in the current EEMs are not suitable for use outside the domain of the current data.
The SAB has a number of specific suggestions for improving the statistical modeling approach used by
the EPA for developing the draft EEMs for broiler confinement facilities and swine and dairy
lagoons/basins. These suggestions are provided below.
1. SAB suggests that residual analyses have more importance in the report and modeling process. It is
preferable to plot residuals to look for oddities, lack of fit, serial correlation and lack of support for the
model rather than histograms of the data. EPA should assess the mean and variance specifications in an
extensive analysis of residuals. EPA should also assess the covariance structure, including the likely
contemporaneous correlation among residuals for different houses at a single site. Also, EPA should
break down residuals by farm (hence by animal). EPA should also generate, separately for each farm,
time series of measured and fitted or predicted emissions according to the model.
2. SAB recommends the EPA consider other approaches to the validation method used to evaluate
model predictions. K-fold cross-validation methods are preferable to simple data splitting. EPA should
consider splitting data based on factors related to study design (such as flock, house and location) as a
way to evaluate model predictive ability.
3. EPA's modeling approach suffers from largely ignoring the sampling/design structure of the data and
implications. The model development process, with an overemphasis on p-values of predictors, would
suggest that the primary goal might be inference rather than prediction. The sampling design determines
the ability to make statements about the collection of potential samples. In this design, there are
locations (sites), houses within locations, and flocks within houses. These design factors are not
represented in the final models and the flock factor is ignored in the modeling process. These factors are
fundamental when it comes to making inferences about what factors and interactions are important as
they affect the variance estimates and degrees of freedom for testing. While it would be useful to add
factors associated with year and season, SAB agrees that the imbalance in the data will likely cause
limitations when the model is applied to new sites. The importance of factors such as location has been
reported in the literature (Ogink et al, 2008; Mosquera and Ogink, 2004).
The report presents evidence of model heterogeneity; however the heterogeneity is ignored. Table 7-8
on page 7-36 of the EEM report indicates a significant variance component associated with houses. This
would suggest that different models would be required for accurate prediction at new locations. At a
B-l
-------
minimum, there is a need to further evaluate the predictive ability of the models for individual houses (a
hold-out approach is recommended for this verification) and at new locations. In addition, on page 7-37
of the report, the authors note that graphical displays indicated greater variability in Kentucky houses
relative to CA1B houses. However, this variation difference was ignored "[b]ecause NAEMS sites were
selected to represent emissions for the industry as a whole, and the EEM will be used to quantify all
such emissions, the EPA used a single pooled variance parameter". The evidence would suggest that the
EPA is aware of differences but chooses to ignore them. In this case, at the least, there should be some
documentation of how ignoring differences in locations affect any conclusions about emissions levels.
4. SAB has the following concerns regarding both deterministic and stochastic components of the
model. To deal with nonlinearity associated with average bird mass the EPA used polynomial
regression. Polynomial regression while beneficial for interpolation, can lead to poor predictions near
the extremes of the experimental conditions, and to disastrous extrapolations only just beyond those
extremes. The model results should provide a table of values that might occur for maximum average
bird mass and determine if model extrapolation is a problem. The restriction on the range of average
mass should be clearly reported if the cubic model is used. Some alternative strategies to polynomials
for nonlinear relationships might be considered. For example, one could use low degree of freedom
splines that are linear at the boundaries. If polynomials are to be used, SAB recommends use of
orthogonal polynomials. With these one can arguably consider simpler final models by eliminating some
interaction terms rather than keeping all polynomial terms in any interaction considered.
The EPA should further investigate the correlation structure and use of random effects. EPA should
clarify whether the very high temporal correlation structure has been adequately modeled. Common time
series tools (ACF and PACF) should be considered to assess the adequacy of the AR(1) model. ACFs
and/or PACFs of model residuals, and boxplots of emissions by the many farm level categorical factors,
perhaps separately for different seasons or levels of other factors, should be considered. The defense of
the current model seems to be based entirely on the coverage of predictive intervals. While this is
important, this does not guarantee a good model (overall coverage near 95% does not necessarily mean
that coverage conditional on other factors is also 95%). The extremely high autocorrelation suggests that
perhaps there are some other temporal trend features that could/should be identified. Time series plots of
observed and fitted (or predictions in the case of cross-validation) emissions should be separately
prepared for each of the houses at the three sites. EPA should prepare ACFs and/or PACFs of model
residuals and boxplots of emissions by the many farm level categorical factors, perhaps separately for
different seasons or levels of other factors.
EPA's analysis considers random effects associated with house and location. It is the opinion of the
panel that there are too few levels of the house and site factors to analyze them as random effects and
they should be modeled and tested as fixed effects. While a desired result would be that the house and
site factors would act like additive effects in addition to other predictors, EPA may need to consider
interaction effects that permit other predictors to have different coefficients at different sites. EPA's
approach must recognize the importance of flocks and consider adding random effects for flocks to the
model. It is possible that other factors (such as buildup) may account for most of the flock effects, but it
is still necessary to consider a flock random effect to account for what must be considered dependent
observations (beyond the temporal dependence).
In addition, the EPA should realize that the response variable is censored and this ought to be accounted
for in the methodology. Finally, some of the predictors (e.g., number of birds in the house and average
bird weight) are measured with error. If this error is not accounted for when fitting the model, then the
relationship between the response and the predictors is attenuated.
B-2
-------
5. Cross-validation is a useful tool for model selection and for evaluating predictive ability. Its value is
constrained by the method for selecting the test set for model evaluation. By selecting a random sample
of observations, as was done in this study, the results concerning predictive ability are limited. It is
unlikely the method as applied in the EPA report will give a good measure of the predictive ability for a
site in Florida, or another state or another location within Kentucky. While it is not possible to estimate
the predictive ability with the current data, it should be possible to estimate prediction error for different
flocks, for different houses and for different locations by running exercises using these factors to select
holdout samples. The cross-validation exercise could help identify the limitations to the model and to
obtain a better estimate of the prediction error at new locations or new flocks. Even with the available
data, cross-validation may not inform about the reliability of predictions for sites in other states, or even
other sites in Kentucky.
The exercise described in the Draft as "cross-validation" is not what most statisticians understand by
that description. Five-fold cross-validation would involve a similar division of the data set into fifths,
but each would be held out in turn, and predicted using a model fitted to the other four fifths. SAB
suggests that EPA consider a leave-out-one-flock-at-a-time cross-validation strategy. EPA should
provide more information on the likelihood that observations from successive flocks might be nearly
independent, and whether flock-to-flock variability vs. daily predictions is the fundamental variance
component for inferences.
EPA should consider building an EEM model on just one site (or perhaps the pair of sites in Kentucky)
and examine how well the predictions apply to another site. This approach would severely restrict the
amount of data available for modeling; however, if predictions were good in this assessment, there may
be some hope that EPA's model could actually be applied to other sites.
6. As described below, SAB has reservations about EPA's use of regression to evaluate the model
predictions for the hold-out data. The use of R2 coefficients as a measure of predictive ability of the
model should be reassessed. The R2 value measures the closeness of predicted values to observed values
and may be used with cross-validation to infer how good the model will be with future data, collected
under similar conditions. As calculated, the values in the report do not provide evidence of "external"
predictive ability but rather "internal" predictive ability. Thus, it may not give evidence of how well data
will be predicted at a new location or for another year.
The only true validation of EPA's model is to see how the model works or 'predicts' when used in
practice. Such an effort requires some validation data that played no role in developing or estimating the
model, as well as observed emissions that can be compared with predictions. EPA, however, states
(Section 7.5, page 7-37) that:
To choose final mean trend variables from these candidates, the EPA used an approach that
included simultaneous evaluation of fit statistics calculated on the base dataset with fit statistics
calculated on the cross-validation dataset.
EPA therefore used the hold-out data to play a role in model development (note: the term "hold-out
data" is used instead of "cross-validation dataset" since SAB has already provided comments on how
EPA's effort differs from conventional cross-validation). This exercise therefore cannot be viewed as a
true validation.
B-3
-------
7. SAB recommends that residual analyses be part of the report. Histograms are used to indicate that the
data are skewed; however, these plots are rather limited, as EPA's February 2012 draft Broiler report
points out. It is preferable to plot residuals in order to look for oddities, lack of fit, serial correlation and
lack of normality. The mean and variance specifications should be assessed in an extensive analysis of
residuals. Table 7-9 is not an appropriate method to assess mean-variance relationship as the constant
range of NFL? values in the rows of the table constrain the standard deviations to be similar. The
covariance structure, especially the likely contemporaneous correlation among residuals for different
houses at a single site, should also be assessed using the same residuals.
In addition to normal QQ plots and the overall plots of residual plots vs. fitted value, SAB strongly
recommends that EPA break down and examine residuals (and/or validation prediction errors) according
to spatial and temporal design factors. For example, boxplots of residuals should be made by site, house,
flock, and season. SAB also suggests that EPA prepare time series plots of observed and fitted (or
predictions in the case of cross-validation) emissions separately for each of the five houses at the 3 sites.
8. The variable selection approach in the model building is likely suboptimal with respect to the goal of
accurate prediction. SAB recommends that EPA assess a modern text focusing on prediction, such as
"The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman. Since the primary aim is
prediction, EPA should not base variable selection on backward elimination with a conservative p<.001
criterion. The apparent significance of individual predictors is not a primary concern, especially in the
context of (somewhat) correlated predictors. SAB finds no justification for making decisions about
inclusion of sets of interaction terms on the basis of a small change in R . While the final choice of
model was not completely automatic according to the backward elimination algorithm, EPA should
consider the results of an all subsets regression procedure rather than backward elimination (although
this might only be possible without all the interaction effects) using an AIC or BIC criterion.
Uncertainty in the "best" model could be assessed with cross-validation.
B-4
------- |