UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
July 6,2012
EPA-SAB-12-008
The Honorable Lisa P. Jackson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: Science Integration for Decision Making at the U.S. Environmental Protection Agency
Dear Administrator Jackson:
Today the EPA faces increasingly complex environmental and public health challenges. Effective
environmental policy making requires integration of science from many disciplines, including the social
sciences, to inform the decision process. With your support, the Science Advisory Board (SAB)
undertook a study to evaluate the practice of science integration at the EPA and recommend how the
agency might strengthen the process and capacity for integrating science into decision making.
The SAB defines science integration as a three-part process: problem formulation - asking the right
questions; assessment - combining information and analyses from different scientific fields to address
the problem; and decision making and evaluation - application of the science and ongoing evaluation of
the outcome of the decision. The concept of science integration is rooted in the origins of the agency.
The Presidential commission that recommended the establishment of the EPA in 1970 explicitly called
for the EPA to use science to evaluate and address the interrelated nature of environmental quality.
Science integration resonates with the public and is consistent with your vision of "One EPA," with the
goal of transcending historical silos of program and region to apply the highest quality scientific
information to solve environmental problems. It is also consistent with recent advice from the National
Research Council (NRC). The NRC report Science and Decisions recommended a formal problem
formulation approach to define the scientific questions and assure that the scientific assessments address
the needs of decision makers. Similarly, the NRC report Sustainability and the U.S. EPA called for
coordination across scientific disciplines and organizations to achieve the goal of sustainability.
Over 6,000 EPA employees are involved in scientific assessments, research, and related activities, with
approximately 1,300 full-time scientific staff in the Office of Research and Development (ORD) and
approximately 4,700 full-time scientific staff in program and regional offices. To understand the scope
and practice of science integration at the EPA, members of an SAB committee conducted interviews
with EPA program offices, all ten EPA regions, the ORD and other offices supporting decision making.
In all, members of the SAB committee held 72 interviews with more than 450 individuals. The goals for
-------
the interviews were defined through an interview protocol. The interviews were designed to encourage
open discussion about the actual practice of science integration at the EPA. As a result, the interview
summaries are qualitative in nature. The interviews focused on five main topics: (1) practices for
integrating science to support decision making; (2) consideration of public, stakeholder, external
scientific and other input in science assessment; (3) drivers and impediments to implementing science
integration; (4) feedback on how science is used in decision-making; and (5) the workforce to support
science integration. The interviews focused on EPA processes that promote or impede science
integration and thus the enclosed SAB report does not evaluate the quality of the EPA's decisions or the
quality of the science supporting them.
The SAB made general observations from the interviews about the EPA's science integration practices
and the most significant needs shared by managers and staff in program and regional offices across the
agency. Key findings and recommendations are summarized below and discussed in more detail in the
enclosed report:
Findings
• Science integration is an integral component of many decisions at EPA. The SAB interviews
confirmed that agency staff and managers view science as an important component of decision
making at the EPA, whether decisions involve regulatory, enforcement or voluntary programs.
• There is a critical need for more high quality assessments translating existing science on a
broad range of topics important to decision making at the EPA. Regional and program offices
emphasized the importance of science assessments that evaluate the state of existing science.
However, interviewees noted that scientific literature reviews published in peer-reviewed
journals generally do not provide assessment information that meets the EPA's regulatory needs.
The EPA has a continuing need to develop capacity for trans-disciplinary scientific assessment,
translation, and integration.
• No EPA program has fully implemented all the steps of science integration. The SAB
envisions a framework for science integration with three major components: problem
formulation; analysis and decision making; and implementation and performance evaluation. The
first step, problem formulation, may be the most important. Problem formulation is a systematic
planning step, linked to the regulatory and policy context of an environmental problem, which
identifies the major factors to be considered, developed through interactions among policy
makers, scientists and stakeholders. The analysis and decision-making step often includes the
assessment of existing science.
• Regulatory program and disciplinary "silos" remain significant barriers to science integration.
Narrow interpretations of legislative mandates and the organizational structure of the EPA's
regulatory programs often have posed barriers to innovation and cross-program problem solving.
Rigidity within scientific disciplines also can pose an obstacle to science integration.
Interdisciplinary work is difficult; experts often use different terminology and methodologies.
These differences can become intellectual silos when the science integration is not formally
facilitated.
-------
• Some managers actively promote science integration, but more could be done in most program
and regional offices. Time and resource constraints are important barriers to science integration
across the EPA, but notably some leaders and managers make science integration a priority. The
need for improving science integration is most acute in the regions and program offices on the
front line for addressing environmental issues. Currently, the EPA does not have a single entity
responsible for managing and strengthening the EPA's scientific workforce so that it functions as
a resource for the agency as a whole.
Recommendations
The SAB has three principal recommendations for strengthening science integration at the agency.
• The EPA should explicitly plan for science integration to support environmental decisions. For
each decision requiring scientific information, science integration will require an initial problem
formulation step, with the following components:
o Involvement of the responsible decision-maker to define the initial questions that will
look broadly at the physical, economic, and social context of specific environmental
problems;
o Identification of options for intervention and risk management;
o An assessment plan that discusses the appropriate level and types of science required for
the decision;
o Expectations regarding the required timeline and resources; and
o An appropriate balance of public and stakeholder engagement.
• Managers should be engaged in and accountable for integrating science into decision making,
starting with problem formulation and science assessment, in their own organizations and
across the EPA. The SAB recommends that EPA managers consistently devote attention to
implementing all the components of science integration. Management should be accountable for
problem formulation to martial integrated thinking about complex environmental problems as
they occur in the real world.
• The EPA should increase and improve support and training for scientists and managers
across the agency, especially in programs and regions, to strengthen capacity for science
integration. Traditional rewards and recognition for scientific excellence focus on discovery,
peer reviewed publication, and national and international recognition by peers. As a result there
are few professional incentives for scientists to focus on support of regulatory decision making.
The SAB recommends that scientists throughout the agency be encouraged to participate actively
in developing improved approaches to integrate science into agency decisions and be rewarded
for their valuable contributions.
Strengthening science integration at the EPA will require change: change in agency culture, change in
how the agency works, and increased support for scientists and managers in program and regional
offices responsible for science integration. The benefits will be more informed, effective decision
making and increased public understanding of and confidence in EPA decisions.
-------
We thank the many EPA personnel who participated in the SAB's science integration interviews for
their time and insights. We look forward to receiving your response regarding our recommendations.
Sincerely,
/Signed/
/Signed/
Dr. Deborah L. Swackhamer
Chair
Science Advisory Board
Dr. Thomas A. Burke
Chair
Committee on Science Integration for
Decision Making
Enclosure
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board (SAB), a public
advisory group providing extramural scientific information and advice to the Administrator and other
officials of the Environmental Protection Agency. The SAB is structured to provide balanced, expert
assessment of scientific matters related to problems facing the Agency. This report has not been
reviewed for approval by the Agency, and, hence, the contents of this report do not necessarily represent
the views and policies of the Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government. Mention of trade names of commercial products does not constitute a
recommendation for use. Reports of the SAB are posted on the EPA website at http://www.epa. gov/sab.
1
-------
U.S. Environmental Protection Agency
Science Advisory Board
Committee on Science Integration for Decision Making
CHAIR
Dr. Thomas Burke, Professor and Jacob I and Irene B. Fabrikant Chair in Health, Risk and Society
Associate Dean for Public Health Practice, Johns Hopkins University, Baltimore, MD
MEMBERS
Dr. Gregory Biddinger, Managing Director, Natural Land Management, Houston, TX
Dr. James Bus, Director of External Technology, Toxicology and Environmental Research and
Consulting, The Dow Chemical Company, Midland, MI
Dr. Deborah Cory-Slechta, Professor, Department of Environmental Medicine, School of Medicine
and Dentistry, University of Rochester, Rochester, NY
Dr. Terry Daniel, Professor of Psychology and Natural Resources, Department of Psychology, School
of Natural Resources, University of Arizona, Tucson, AZ
Dr. T. Taylor Eighmy, Senior Vice President for Research, Office of the Vice President for Research,
Texas Tech University, Lubbock, TX
Dr. Penelope Fenner-Crisp, Independent Consultant, North Garden, VA
Dr. John P. Giesy, Professor and Canada Research Chair, Veterinary Biomedical Sciences and
Toxicology Centre, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
Dr. Rogene Henderson, Senior Scientist Emeritus, Lovelace Respiratory Research Institute,
Albuquerque, NM
Dr. James H. Johnson Jr., Professor and Dean Emeritus, College of Engineering, Architecture &
Computer Sciences, Howard University, Washington, DC (Service from May 2009-March 2012)
Dr. Wayne Landis, Professor and Director, Institute of Environmental Toxicology, Huxley College,
Western Washington University, Bellingham, WA
Dr. Jill Lipoti, Director, Director of the Division of Water Monitoring and Standards, New Jersey
Department of Environmental Protection, Trenton, NJ
Dr. Gary Sayler, Beaman Distinguished Professor, Joint Institute for Biological Sciences, Oak Ridge
National Laboratory, University of Tennessee, Knoxville, TN
11
-------
Dr. Thomas L. Theis, Director, Institute for Environmental Science and Policy, University of Illinois at
Chicago, Chicago, IL
Dr. Barton H. (Buzz) Thompson, Jr., Robert E. Paradise Professor in Natural Resources Law at the
Stanford Law School and Perry L. McCarty Director, Woods Institute for the Environment, Stanford
University, Stanford, CA
Dr. Lauren Zeise, Chief, Reproductive and Cancer Hazard Assessment Branch, Office of
Environmental Health Hazard Assessment, California Environmental Protection Agency, Oakland, CA
SCIENCE ADVISORY BOARD STAFF
Dr. Angela Nugent, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board, Washington, D.C.
111
-------
U.S. Environmental Protection Agency
Science Advisory Board
CHAIR
Dr. Deborah L. Swackhamer, Professor and Charles M. Denny, Jr., Chair in Science, Technology and
Public Policy, Hubert H. Humphrey School of Public Affairs and Co-Director of the Water Resources
Center, University of Minnesota, St. Paul, MN
SAB MEMBERS
Dr. George Alexeeff, Acting Director, Office of Environmental Health Hazard Assessment, California
Environmental Protection Agency, Oakland, CA
Dr. David T. Allen, Professor, Department of Chemical Engineering, University of Texas, Austin, TX
Dr. Pedro Alvarez, Department Chair and George R. Brown Professor of Engineering, Department of
Civil & Environmental Engineering, Rice University, Houston, TX
Dr. Joseph Arvai, Svare Chair in Applied Decision Research, Institute for Sustainable Energy,
Environment, & Economy, Haskayne School of Business, University of Calgary, Calgary, Alberta,
Canada
Dr. Claudia Benitez-Nelson, Full Professor and Director of the Marine Science Program, Department
of Earth and Ocean Sciences, University of South Carolina, Columbia, SC
Dr. Patricia Buffler, Professor of Epidemiology and Dean Emerita, Department of Epidemiology,
School of Public Health, University of California, Berkeley, CA
Dr. Ingrid Burke, Director, Haub School and Ruckelshaus Institute of Environment and Natural
Resources, University of Wyoming, Laramie, WY
Dr. Thomas Burke, Professor and Jacob I. and Irene B. Fabrikant Chair in Health, Risk and Society
Associate Dean for Public Health Practice, Johns Hopkins University, Baltimore, MD
Dr. Terry Daniel, Professor of Psychology and Natural Resources, Department of Psychology, School
of Natural Resources, University of Arizona, Tucson, AZ
Dr. George Daston, Victor Mills Society Research Fellow, Product Safety and Regulatory Affairs,
Procter & Gamble, Cincinnati, OH
Dr. Costel Denson, Managing Member, Costech Technologies, LLC, Newark, DE
Dr. Otto C. Doering, III, Professor, Department of Agricultural Economics, Purdue University, W.
Lafayette, IN
Dr. Michael Dourson, President, Toxicology Excellence for Risk Assessment, Cincinnati, OH
iv
-------
Dr. David A. Dzombak, Walter J. Blenko, Sr. Professor of Environmental Engineering, Department of
Civil and Environmental Engineering, College of Engineering, Carnegie Mellon University, Pittsburgh,
PA
Dr. T. Taylor Eighmy, Senior Vice President for Research, Office of the Vice President for Research,
Texas Tech University, Lubbock, TX
Dr. Elaine Faustman, Professor and Director, Institute for Risk Analysis and Risk Communication,
School of Public Health, University of Washington, Seattle, WA
Dr. John P. Giesy, Professor and Canada Research Chair, Veterinary Biomedical Sciences and
Toxicology Centre, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
Dr. Jeffrey K. Griffiths, Professor, Department of Public Health and Community Medicine, School of
Medicine, Tufts University, Boston, MA
Dr. James K. Hammitt, Professor, Center for Risk Analysis, Harvard University, Boston, MA
Dr. Barbara L. Harper, Risk Assessor and Environmental-Public Health Toxicologist, and Division
Leader, Hanford Projects, and Program Manager, Environmental Health, Department of Science and
Engineering, Confederated Tribes of the Umatilla Indian Reservation (CTUIR), West Richland, WA
Dr. Kimberly L. Jones, Professor and Chair, Department of Civil Engineering, Howard University,
Washington, DC
Dr. Bernd Kahn, Professor Emeritus and Associate Director, Environmental Radiation Center, Georgia
Institute of Technology, Atlanta, GA
Dr. Agnes Kane, Professor and Chair, Department of Pathology and Laboratory Medicine, Brown
University, Providence, RI
Dr. Madhu Khanna, Professor, Department of Agricultural and Consumer Economics, University of
Illinois at Urbana-Champaign, Urbana, IL
Dr. Nancy K. Kim, Senior Executive, Health Research, Inc., Troy, NY
Dr. Cecil Lue-Hing, President, Cecil Lue-Hing & Assoc. Inc., Burr Ridge, IL
Dr. Judith L. Meyer, Professor Emeritus, Odum School of Ecology, University of Georgia, Lopez
Island, WA
Dr. James R. Mihelcic, Professor, Civil and Environmental Engineering, University of South Florida,
Tampa, FL
Dr. Christine Moe, Eugene J. Gangarosa Professor, Hubert Department of Global Health, Rollins
School of Public Health, Emory University, Atlanta, GA
v
-------
Dr. Horace Moo-Young, Dean and Professor, College of Engineering, Computer Science, and
Technology, California State University, Los Angeles, CA
Dr. Eileen Murphy, Director of Research and Grants, Ernest Mario School of Pharmacy, Rutgers
University, Piscataway, NJ
Dr. James Opaluch, Professor and Chair, Department of Environmental and Natural Resource
Economics, College of the Environment and Life Sciences, University of Rhode Island, Kingston, RI
Dr. Duncan Patten, Research Professor, Hydroecology Research Program, Department of Land
Resources and Environmental Sciences, Montana State University, Bozeman, MT
Dr. Stephen Polasky, Fesler-Lampert Professor of Ecological/Environmental Economics, Department
of Applied Economics, University of Minnesota, St. Paul, MN
Dr. C. Arden Pope, III, Professor, Department of Economics, Brigham Young University, Provo, UT
Dr. Stephen M. Roberts, Professor, Department of Physiological Sciences, Director, Center for
Environmental and Human Toxicology, University of Florida, Gainesville, FL
Dr. Amanda Rodewald, Professor of Wildlife Ecology, School of Environment and Natural Resources,
The Ohio State University, Columbus, OH
Dr. Jonathan M. Samet, Professor and Flora L. Thornton Chair, Department of Preventive Medicine,
Keck School of Medicine, University of Southern California, Los Angeles, CA
Dr. James Sanders, Director and Professor, Skidaway Institute of Oceanography, Savannah, GA
Dr. Jerald Schnoor, Allen S. Henry Chair Professor, Department of Civil and Environmental
Engineering, Co-Director, Center for Global and Regional Environmental Research, University of Iowa,
Iowa City, IA
Dr. Gina Solomon, Senior Scientist, Health and Environment Program, Natural Resources Defense
Council, San Francisco, CA
Dr. Daniel O. Stram, Professor, Department of Preventive Medicine, Division of Biostatistics,
University of Southern California, Los Angeles, CA
Dr. Peter Thorne, Professor and Head, Occupational and Environmental Health, College of Public
Health, University of Iowa, Iowa City, IA
Dr. Paige Tolbert, Professor and Chair, Department of Environmental Health, Rollins School of Public
Health, Emory University, Atlanta, GA
Dr. John Vena, Professor and Department Head, Department of Epidemiology and Biostatistics,
College of Public Health, University of Georgia, Athens, GA
vi
-------
Dr. Robert Watts, Professor of Mechanical Engineering Emeritus, Tulane University, Annapolis, MD
Dr. R. Thomas Zoeller, Professor, Department of Biology, University of Massachusetts, Amherst, MA
SCIENCE ADVISORY BOARD STAFF
Dr. Angela Nugent, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board, Washington, D.C.
vii
-------
Science Integration for Decision Making at the
U.S. Environmental Protection Agency (EPA)
Introduction
In October 2008, the Science Advisory Board (SAB) was asked by then-Administrator Stephen L.
Johnson to develop advice on how the EPA can strengthen scientific assessments for decision making
(Appendix A). The SAB subsequently received support from Administrator Lisa P. Jackson to undertake
this study to evaluate EPA's processes for integrating scientific assessment into environmental decision
making, as previously recommended by the National Research Council (NRC 2009) and the SAB (U.S.
EPA SAB 2000). To implement this broad charge, the SAB conducted interviews with EPA managers
and scientists to determine current practices and identify barriers to implementing NRC and SAB
recommendations. This report identifies immediate and future actions to further develop and
institutionalize integrated environmental decision making at the EPA. Areas of consideration include
scientific leadership, scientific practices, scientific collaboration across disciplines, and scientific
expertise and workforce.
Study approach
To understand how science integration is practiced, members of an SAB committee conducted
interviews between October 26, 2009, and February 4, 2010, with EPA senior leaders, managers and
scientists in program offices and ten EPA regions. They also conducted interviews with senior leaders,
managers and scientists in EPA's Office of Research and Development (ORD).
The interviews focused on five main topics: (1) practices for integrating science to support decision
making; (2) consideration of public, stakeholder, external scientific and other input in science
assessment; (3) drivers and impediments to implementing past recommendations for science integration;
(4) ways programs receive feedback on how science is used in decision-making; and (5) the workforce
to support science integration for decision making. The EPA offices identified the managers and staff to
participate in the interviews, and interviewees received a draft interview protocol and questions in
advance (Appendix B). Interviewees were asked to comment on the current and recent past practice of
science integration in their organization based on their personal experience. Interviewers stressed their
interest in whether and how the EPA actually practiced science integration, not the nominal approach.
The SAB interviewers received extensive background materials from each of the programs and regions
prior to the interviews. In all, members of the committee held 72 interviews with more than 450
individuals (Appendix C).
After the interviews, interviewees were provided a draft interview summary for review and comment. A
compilation of interview summaries1 and the background information provided by regional and program
1 Science Integration Fact-finding Discussion Summaries 2009-2010 are available at:
http://vosemite.epa.gov/sab/sabproduct.nsf/WebBOARD/SI Summaries/$File/SciIntSummaries-06.28.12 with TOCtxll'
(accessed 06/29/2012).
1
-------
2
offices are available on the SAB website. The SAB considered these materials in developing the
general observations and recommendations in this report.
Scope of study
The SAB recognizes ORD's important role in EPA research and notes that the SAB's interviews were
conducted during a time of change. In 2010, ORD initiated new approaches to integrated
transdisciplinary research with an emphasis on sustainability and innovation (Anastas 2010). The SAB
(U.S. EPA SAB 2010 and 2011) supports ORD's initial steps to implement this new direction. ORD's
new focus on systems thinking, problem solving and involving EPA programs and regions in research
planning has the potential to improve science integration practices at the EPA. Although the SAB
conducted interviews with ORD personnel and ORD research was discussed in many interviews, this
report focuses on science as a whole and not on ORD research efforts to generate new science. A full
discussion of ORD's new strategic research directions and its relevance for program and regional needs
is outside the scope of this effort and will be addressed in other SAB advisory reports.
Over 6,000 EPA employees are involved in scientific assessments, research, and related activities, with
approximately 1,300 full-time scientific staff in ORD and approximately 4,700 full-time scientific staff
"3
in program and regional offices. These program and regional offices, together with ORD, are
responsible for integrating science to inform the environmental decisions made by the EPA
Administrator. Program and regional offices also play a primary role in integrating science to inform the
environmental decisions delegated to them.
Although the goals for the interviews were defined through an interview protocol and study plan, the
interviews were not formally structured. Instead they were designed to encourage open discussion about
the actual practice of science integration at the EPA. As a result, the interview summaries are qualitative
in nature and not appropriate for a quantitative analysis. The interviews and the interview summaries
documenting them provided qualitative information that varied in detail from one interview to another
and focused on issues of particular concern to the individuals present at the time of the interviews. This
report makes general observations from the interview summaries. While the interviews disclosed that
science integration approaches at the EPA differ at every level of organization and across programs, the
interviews provided general insights about EPA science integration practices and the most significant
needs shared by managers and staff in program and regional offices across the EPA. These insights
provide the foundation for the observations, findings and recommendations in this report.
Finally, although the observations and recommendations in this report concern science integration for
decision making, this report does not assume that science is the sole input for environmental decision
making. The SAB acknowledges that other factors (such as law, politics, policy and values) play
important roles (Bipartisan Policy Center 2009) in agency decision making. Because the SAB interviews
focused on EPA processes that promote or impede science integration, the observations in this report are
not an evaluation of the quality of the EPA's decisions or the quality of the science supporting them.
2 SAB Committee on Science Integration for Decision Making Discussion of Preliminary Fact Finding, Meeting Materials,
agency-provided background materials are available at:
http://vosemite.epa.gov/sab/sabproduct.nsf/MeetingCal/266562906BCF3B0B852576DD0067F2E37QpenDocument
(accessed 11/21/11).
3 Summary Minutes of the United States Environmental Protection Agency (U.S. EPA) Science Advisory Board (SAB)
Committee on Science Integration for Decision Making March 30-31, 2010, page 13, available at
http://vosemite.epa. gov/sab/sabproduct.nsf/MeetingCal/266562906BCF3B0B852576DD0067F2E3/$File/SAB+Sci+Int+Dec
n+Mkg+Update+03 30-31 2010+Minutes+with+attachments.pdf (accessed 07/02/12).
2
-------
Definitions and assumptions
For the purpose of this report, the SAB defines "science" broadly as a process that builds and organizes
knowledge in the form of testable explanations and predictions about the world and also as the organized
knowledge that results from that process. In the context of the EPA's work, science involves knowledge
and data that help the agency address environmental and public health protection questions.
The SAB defines science integration for decision making in terms of a three-part framework that
emphasizes: (1) formulating the right questions (problem formulation); (2) assessing the science needed
to address those questions (assessment), combining information and analysis from different scientific
and technical fields as needed to fully address complex environmental problems; and (3) decision
making that uses science as a key input and that consistently evaluates the use of science (decision
making and evaluation). The SAB recommended such a three-part framework for integrated
environmental decision making in 2000 (U.S. EPA SAB 2000). The SAB provides more detail about
this three-part SAB framework for environmental decision making as shown in Figure D-l (see
Appendix D).
The EPA adopted an official three-part framework for ecological risk assessment (EPA 1998) consistent
with the SAB's 2000 recommendations (see Figure D-2 in Appendix D). Other organizations have
developed similar, formal processes to foster science integration for decision-making. The NRC
recommended a detailed three-part framework for risk-based decision making that focused on science
integration for human health protection (NRC 2009). The NRC issued a separate framework for
sustainability decisions in 2011 (NRC 201 lb). The NRC sustainability framework emphasized scoping,
integration of science across disciplines and evaluation to integrate science into sustainability decisions.
These structured frameworks emphasize the need to carefully plan the science needed by decision
makers and use information gained through the analytical and decision process to inform future
decisions. They have common goals: to provide decision makers with needed information, including
information about the novel or specific dimensions of the problem being addressed; to tailor assessments
and analyses to the scale and timetable of the decision; to consider opportunities to frame environmental
questions broadly to identify opportunities to protect public health and the environment and promote
sustainability; and to institutionalize a process for evaluating science integration for decision making.
The SAB supports the view that science integration involving problem formulation, assessment, and
decision making with evaluation is necessary to inform environmental decisions. The effectiveness of
any framework formally adopted by the EPA or recommended for its use, however, depends on the
actual practice of agency science integration, which is the focus of this report.
General observations and findings
The science integration interviews provided the basis of the following general observations and findings
about science integration practices at the EPA.
Science integration is an integral component of many decisions at EPA. The SAB interviews
confirmed that agency staff and managers view science as an important component of decision making
at the EPA, whether decisions involve regulatory, enforcement or voluntary programs. Nationally
significant regulatory decisions involving science are addressed at the level of the Administrator. For
these actions and other significant actions to be taken by senior managers, the agency's Action
3
-------
Development Process and associated Analytical Blueprint Process provide a structure to encourage
science integration.4 However, the SAB was not provided with sample analytical blueprints because
EPA considers them to be deliberative documents. The effectiveness of the analytical blueprint process
and the regulatory development process and the consistency of the use of these processes were not
demonstrated to the SAB. In addition, some regional and site-specific decisions and routine decisions
involving science are not part of the Action Development Process and are made by senior managers,
mid-level program managers, permit writers, enforcement personnel and branch chiefs with no
consistent process to encourage and document science integration.
No EPA program has fully implemented all steps and all aspects of science integration. The
framework for science integration recommended by the SAB (U.S. EPA SAB 2000) and supported by
the NRC (NRC 2009; NRC 2011) has three major components: problem formulation; analysis and
decision making; and implementation and performance evaluation. The first step, problem formulation,
may be the most important. Problem formulation is a systematic planning step, linked to the regulatory
and policy context of an environmental problem, which identifies the major factors to be considered,
developed through interactions among policy makers, scientists and stakeholders. The problem
formulation process should result in a conceptual model that identifies the sources, environmental
stressors or health hazards, exposed populations and the relationships among them. Problem formulation
also includes initial consideration of the options for interventions or risk management.
The National Ambient Air Quality Standard (NAAQS) review process for criteria air pollutants comes
closest to full implementation of the SAB's vision for science integration (see Text Box 1 on page 5). It
involves the most effective science integration process, but it still has two major limitations. The
NAAQS process focuses on single pollutants, rather than multiple pollutants and does not integrate
health and ecological assessments into the assessment of costs and benefits to help decision makers
evaluate the marginal benefits of additional health or ecological protection. The Clean Air Act mandates
that the EPA review criteria air pollutants every five years. Science integration benefits from this
iterative review and re-evaluation of science for decision making. Most other EPA programs lack such a
strong mandate for science integration and the sustained funding and institutional support needed to
make science integration happen.
In the area of ecological protection, where the EPA has adopted an explicit agency framework that
parallels the SAB's recommendations regarding science integration (U.S. EPA SAB 2000), many EPA
interviewees spoke of the need for better processes for formulating questions relating to ecological and
ecosystem protection and more engagement on the part of decision makers in these issues. These views
were consistent with a recent reassessment of the EPA's implementation of the ecological risk
framework, which emphasized a critical need for improving problem formulation processes to support
decision making (Suter and Maciorowski 2010; U.S. EPA 2010b).
4 EPA's Action Development Process (U.S. EPA 2011) requires EPA programs and regions that develop significant agency
actions (e.g., significant regulations or decisions) to generate an analytical blueprint to identify plans for data collection and
analysis. These blueprints are to describe how information from multiple disciplines will be collected, peer reviewed and
used to develop the action. The process is to be multi-disciplinary, collaborative, cross-office and cross-media to ensure that a
variety of perspectives are integrated for decision making. EPA has developed supplemental guidance on consideration of
environmental justice (U.S. EPA 2010a) and children's health protection (U.S. EPA 2006) in action development.
4
-------
Text Box 1: Science Integration and the National Ambient Air Quality Standard (NAAQS)
Review Process
The NAAQS review for a criteria air pollutant generally follows a structured process that facilitates science
integration.
At the start of a new review cycle for a criteria air pollutant, EPA's Office of Research and Development
(ORD) convenes a public workshop to identify key new science published since the last review. EPA's Office
of Air and Radiation (OAR) then prepares an Integrated Review Plan to identify how EPA will review the new
science in light of the EPA's statutory mandate to determine whether current standards protect public health
and public welfare from known or anticipated adverse effects with an adequate margin of safety. ORD prepares
an Integrated Science Assessment reviewing the new science. ORD recently developed the Health and
Environmental Research Online (HERO) database as a publicly accessible repository for peer reviewed
literature used to develop Integrated Science Assessments. OAR prepares a Risk and Exposure Assessment and
a Policy Assessment that reviews the science as it relates to key aspects of regulations. The goal of the Policy
Assessment is to provide a transparent staff analysis of the scientific basis for policy options for consideration
by senior EPA management prior to rulemaking. Such an evaluation of policy implications is intended to help
"bridge the gap" between the agency's scientific assessments and the policy judgments required of the EPA
Administrator in determining whether it is appropriate to retain or revise the NAAQS.
The reviews are iterative, since the EPA must conduct NAAQS reviews for each of the six criteria pollutants
every five years and reconsider the science supporting the NAAQS. As a result, the EPA has updated and
refined the assessment process over time and identified key remaining uncertainties for research to address.
NAAQS reviews allow for multiple opportunities for public comment and for peer review by the EPA's Clean
Air Scientific Advisory Committee, a federal advisory committee mandated to provide review and advice for
the NAAQS process.
Even the NAAQS process, however, does not fully implement the SAB's recommended approach because the
NAAQS process has been limited in two ways. First, NAAQS reviews have focused primarily on single-
pollutant air quality issues, with limited consideration of multi-pollutant impacts and impacts of criteria
pollutants on water quality-related ecological impacts. Second, NAAQS reviews by law focus on human health
and ecological impacts; EPA must set health-based standards that allow for an "adequate margin of safety,"
without consideration of costs. Health and ecological assessments are not integrated with assessments of
benefits and costs and, as a result, decision makers do not have the benefit of an integrated assessment that
would help them evaluate the marginal benefits of additional health or ecological protection.
Regulatory program and disciplinary "silos" remain significant barriers to science integration.
Narrow interpretations of legislative mandates and the organizational structure of the EPA's regulatory
programs have posed barriers, in many cases, to innovation and cross-program problem solving. EPA
managers and staff in many interviews, especially in program offices, defined success in terms of
meeting statutory requirements and court-ordered deadlines for their programs. Although meeting legal
mandates is essential, the EPA needs a broader perspective that extends beyond specific program
objectives to achieve multiple environmental protection goals, including sustainability. A narrow focus
on "program silos" and legal defensibility can be a barrier to formulating and responding to problems as
they occur in the real world. Such a limited approach can hinder integration of new scientific
information into decisions and new applications of science to develop innovative, effective solutions to
environmental problems. Rigidity within scientific disciplines also can pose an obstacle to science
integration. Interdisciplinary work is difficult; experts often use different terminology and have different
assumptions. These differences can become intellectual silos when the EPA does not implement
procedures requiring science integration.
5
-------
Science integration practices vary across the agency. Although EPA staff interviewed generally agreed
that science is critical for decision making, science integration practices vary across the EPA. Some
programs, e.g., the National Estuary Program (U.S. EPA 2005) and the Superfund Program5, appear to
have well established processes to integrate science into decision making through a problem formulation
exercise; other programs do not. Some programs, such as the Drinking Water Program or the NAAQS
program, rely on ORD science and assessments. Other programs, such as the Hazardous Waste Program,
rely on other federal agencies, such as the Department of Energy and Department of Defense, for key
science support.
To meet decision makers' needs for scientific information and assessments, program and regional
scientists turn to a variety of sources (e.g., contractors, local colleges and universities, other federal
agencies, states, Potentially Responsible Parties, and non-governmental organizations). One region has
used a highly structured approach, the Multicriteria Integrated Resource Assessment (MIRA) Process
and associated logic models,6 as a framework to help stakeholders and decision makers understand the
relationships between relevant scientific data and decision options (Stahl et al. 2002). Other programs
and regions take a less structured approach. The Office of Solid Waste and Emergency Response
maintains a website7 as a key communication tool used actively across and beyond the EPA to facilitate
science integration. The site hosts seminars, offers podcasts, provides a portal for databases and tools
related to clean-up technology, and serves as a central place to share information among EPA scientists
and scientists outside the agency. Other EPA programs lack such a resource to foster exchange and
integration of a wide variety of scientific information supporting decision making.
Because there is no single way to integrate science to support EPA's decisions, the public has a wide
variety of experiences when engaging with the EPA on science integration. There are different names
for and approaches to problem formulation, assessment, and decision making/evaluation across many
EPA programs because of the history, legislative mandate and available funding for those programs.
Interviewees in some programs, such as the Office of Chemical Safety and Pollution Prevention, and in
some regional offices reported that they did not consistently make the key step of problem formulation
part of their science integration practice.
There is a critical need for more high quality assessments of existing science on a broad range of
topics important to decision making at the EPA. EPA interviewees in regional and program offices
emphasized the importance of science assessments that evaluate the state of existing science relevant to
the agency's decision needs. Interviewees noted that scientific literature reviews published in peer-
reviewed journals generally do not provide assessment information that meets the EPA's regulatory or
environmental protection needs. As one regional interviewee noted, "Published literature is out there but
it is passive. The question is 'how do you apply it to a practical problem?"' Program and regional
decisions can be delayed or questioned when the agency lacks current and credible assessments of
available science.
The Integrated Risk Information System (IRIS) program, which provides assessments of chronic human
health effects used widely by EPA programs and regions, was discussed in many SAB interviews. This
5 For information about problem formulation in the Superfund Program, see http://www.epa.gov/ttn/naaqs/review.html
(accessed 10/26/2011) and http://www.epa.gov/oswer/riskassessment/superfund_eco_planning.htm (accessed 10/26/11).
6 For more information about MIRA and its applications, see http://www.epa.gov/reg3esdl/data/mira.htm (accessed
December 12, 2011).
7 Hazardous Waste Clean-Up Information (CLU-IN). http://www.clu-in.org/ (accessed 11/18/2011).
6
-------
SAB report does not address evaluation of the IRIS program, which has been the topic of several recent
reports (U.S. Government Accountability Office 2008, 201 la; NRC 2009, 201 la).
Program offices also general chemical assessments for decision making, but the SAB found that
available resources for chemical assessment, the number of scientific staff engaged in the work, and the
institutional and legal framework supporting these assessments differ across the agency. The EPA's
Office of Pesticide Programs provides an example of an organization with a well-defined regulatory
mission, a large scientific staff, a dedicated external peer review mechanism (the EPA's Federal
Insecticide Fungicide and Rodenticide Act Scientific Advisory Panel), legal authority to require
pesticide registrants to generate key data when needed, and an iterative process for reviewing pesticide
active ingredients. Most other EPA programs and regions lack the infrastructure required to generate all
assessments needed to support their own activities. Scientists in these offices work within statutory
constraints, often on an extremely short time-table and with limited budgets. Within those constraints
they either assess available scientific information themselves or rely on ORD, other parts of the EPA, or
other federal or state agencies for the science assessments needed to support decision making.
In addition to chemical-specific assessments, the program and regional interviewees called for science
assessments for other kinds of environmental topics. During SAB interviews, EPA scientists and
managers across the agency called for additional assessments of existing scientific knowledge to apply
to the practical problems they faced. As examples, they noted a need for assessments of existing
scientific knowledge about the environmental impacts of pharmaceuticals and personal care products;
biofuels; nutrients; underground injection; hydraulic fracturing; mountain-top mining; and climate
change at temporal and spatial scales useful for regional decisions. They spoke of the needs for
assessments of specific ecological impacts that can guide clean-up efforts and of the broad need for
assessments of sustainability science.
Some managers actively promote science integration, but more could be done in most program and
regional offices. Time and resource constraints are important barriers to science integration across the
EPA, but notably some leaders and managers make science integration a priority and implement
mechanisms within their organizations despite these constraints. As one manager noted, leadership
requires that managers be "committed to listening to staff and supporting science needs... managers
must demand such planning for high quality science. They should not expect that good science will
'happen organically'—that things will come together by themselves." Effective leadership for science
integration emphasizes the importance of problem formulation to frame environmental problems, as
needed, beyond the limits of narrow programmatic concerns. One manager explicitly noted that "There
is a need to recruit, nurture and guide people to make decisions outside the box, while still making
timely decisions." Some senior managers interviewed provide their workforce with training beyond their
own specific programs and encourage them to learn more about other EPA programs. Some managers
were willing to explore the flexibility of specific statutes, to seek opportunities to work with other
programs, and to look for novel ways to achieve environmental goals.
EPA organizations with successful science integration efforts devote attention to the institutional
mechanisms required. Some examples are: (1) explicit problem formulation activities that involve
o
managers, scientists and the public in decision science models or model building exercises; (2) a
program-office plan that identifies the research needed to achieve its goals, meet statutory obligations
8 For examples in two EPA regions, see Science Integration Fact-finding Discussion Summaries 2009-2010, pp. 9, 22-25.
7
-------
and fulfill court mandates (U.S. EPA 2009); (3) cross-media or cross-disciplinary project teams;9 (4)
effective program and regional-office science councils;10 and (5) meetings where project teams present
their integrated science to decision makers in forums open to program or regional staff.11 These
mechanisms increase the transparency of (and remove the mystery from) science integration and make it
more understandable and accessible to agency staff and managers.
An overarching barrier to consistent science integration is the lack of strong, coordinated management at
EPA to support the scientists in the regional and program offices. As noted above, the vast majority of
the EPA's scientific workforce is employed in EPA regions and program offices. These program and
regional scientists play key roles in the integration of science into decisions and in defining needs for
research, science assessment and technical assistance. Support for science in the EPA's regional and
program offices has long been recognized as a priority (U.S. EPA Expert Panel 1992) and there have
been calls for the appointment of a top science official who would have responsibility and authority for
all the research, science and technical functions at the agency (e.g., NRC 2000; U.S. Government
Accountability Office 201 lb). This specific science leadership question is a complex and important one
that is not the focus of this SAB report.
Currently, the EPA does not have a single entity responsible for managing and strengthening EPA's
scientific workforce so that it functions as a resource for the agency as a whole. SAB interviews in
programs and regions consistently highlighted the limited career tracks, limited funds for training, and
limited interaction with ORD available for scientists program and regional offices. ORD principally
focuses on supporting ORD scientists, although it supports several small but important programs in the
regions, such as the Regional Science Liaison Program, Regional Research Partnership Program and the
Regional Applied Research Effort Program. Program and regional offices manage their scientific
workforces relatively independently, with some organizations providing stronger support than others.
Effective science integration requires the recruitment, retention, and development of leading scientists
from many fields across EPA programs and regions, as well as in ORD. The EPA has not developed a
coordinated human resource strategy for building this science base within ORD and beyond.
9 Several EPA regions and programs described examples, see Science Integration Fact-finding Discussion Summaries 2009-
2010, pp. 25-26; 121-125; and 128-130.
10An effective internal EPA science council in a program or regional office would have a charter, clear responsibilities and
operating rules, and strong support from senior managers. Science councils can facilitate science integration by tracking
emerging environmental issues and the science integration actions needed; helping to prioritize research needs for
development of science to integrate into future decisions; encouraging communication and collaboration across programs and
disciplines; and facilitating training and information exchange to strengthen science integration. For two examples, see
Science Integration Fact-finding Discussion Summaries 2009-2010, pp. 1-3, 123.
11 See Science Integration Fact-finding Discussion Summaries 2009-2010, pp. 3-4.
8
-------
Recommendations
The SAB provides the recommendations below as the most significant actions for the EPA to take to
strengthen science integration.
The EPA should explicitly plan for science integration to support environmental decisions. EPA's
mission to protect public health and the environment requires a difficult balancing of science and policy
considerations. The complexity of environmental decisions and growing emphasis on sustainability as
an environmental goal requires integration of scientific input from diverse fields of the natural, public
health, social, behavioral and decision sciences. The SAB reiterates its recommendation made in 2000
(U.S. EPA SAB 2000) that the EPA implement the framework for science integration depicted in Figure
D-l (Appendix D) to support decisions agency-wide.
The SAB recommends that the EPA's senior leadership communicate that science integration is needed
and expected to support the EPA's decisions. It should be a priority for managers to plan for the research
and science assessments needed for decisions and then train, encourage, and expect staff to collaborate
so that science integration for decision making is realized.
For each decision requiring scientific information, science integration will require an initial problem
formulation step, with the following components:
• Involvement of the responsible decision-maker to define the initial questions that will look
broadly at the physical, economic, and social context of specific environmental problems to seek
a management decision with the broadest environmental benefits;
• Identification of options for intervention and risk management;
• An assessment plan that: discusses the appropriate level of science required for the decision; the
type of science; and where one might find the science that is needed, who is involved and their
roles and responsibilities;
• Expectations regarding the required timeline and resources; and
• An appropriate balance of public and stakeholder engagement, with clear expectations for the
roles of each and how the EPA will address public input, as recommended by the SAB and NRC
(EPA SAB 2001; NRC 2008).
The problem formulation step should inform the data analysis and decision-making phase, where science
assessments will provide key information. Once decisions are made, the EPA should evaluate the use of
integrated science for decision making with the goal of improving future decision making.
Text Box 2 identifies several specific suggestions for strengthening problem formulation and planning
for science integration to support environmental decisions.
9
-------
Text Box 2: Specific suggestions for strengthening problem formulation and planning
for science integration
• The EPA's Action Development and Analytical Blueprint Process should:
o Be structured to include a problem formulation step for every new action
involving science, as described in this report;
o Encourage scientists and decision makers to identify environmental management
options that can achieve multiple environmental protection goals;
o Require that analytical plans specify a strategy for public and stakeholder
involvement in science integration; and
o Require science integration evaluations for major agency actions.
• The EPA should commission case studies in each region and program office documenting a
problem formulation process, as described in this report. Reports of the case studies should be
made available for discussion at a public workshop to study the practical implications of the
problem formulation approach across the EPA.
• ORD and the EPA's National Center for Environmental Economics should collaborate to
strengthen EPA's decision science capabilities.
• Senior managers and scientists from programs and regions should continue to participate in
planning ORD research activities and ORD should regularly inform them about and involve them
in research at key stages of development. ORD should have a structured process to seek feedback
from program and regions on the use of ORD research to support decisions.
• Regions and program offices should develop regular plans to identify the science needed to
support upcoming environmental decisions. These assessments would identify needs to be met by
internal program or regional scientists and science needs to be met by sources outside the region
or program. These plans should be independently peer reviewed.
Managers should be engaged in and accountable for integrating science into decision making,
starting with problem formulation and science assessment, in their own organizations and across
EPA. The SAB recommends that EPA managers consistently devote attention to implementing all the
components of science integration. Management should be accountable for problem formulation to
martial integrated thinking about complex environmental problems as they occur in the real world. They
should also be accountable for generating the science assessments and analyses needed by EPA's
programs. Agency coordination mechanisms such as the Risk Assessment Forum, the Science and
Technology Policy Council, and the Action Development process have the potential encourage science
integration, but their effectiveness depends on sustained leadership, management involvement and
resources.
In addition to the suggestions in Text Box 2 for strengthening problem formulation and science planning,
which involve increased management engagement, Text Box 3 provides suggestions for increasing
transparency and public involvement in science integration to further engage managers and increase
public accountability for science integration. All these suggestions have a common goal: more consistent
integration of science to strengthen environmental decision making.
10
-------
Text Box 3: Specific suggestions for strengthening management engagement and
accountability through increased transparency and public involvement
• The EPA's programs and regions should identify and implement mechanisms to
strengthen transparency and documentation of how science is integrated into
decisions
• The EPA should seek opportunities to engage the SAB and NRC more actively
in fostering science integration throughout the agency, including advisory
activities at the regional and program level.
• The EPA should pilot ways to build on Superfund's public involvement
infrastructure to increase the transparency of and strengthen public involvement
in science integration across EPA programs and regional offices.
The EPA should increase and improve support and training for scientists and managers across the
agency, especially in programs and regions, to strengthen capacity for science integration. Traditional
rewards and recognition for scientific excellence focus on discovery, peer reviewed publication, and
national and international recognition by peers. As a result there are few professional incentives for
scientists to focus on support of regulatory decision making. The SAB recommends that scientists
throughout the agency be encouraged to participate actively in developing improved approaches to
integrate science into agency decisions and be rewarded for their valuable contributions.
The EPA should increase the incentives (e.g., awards, performance evaluations, developmental
assignments, and career opportunities) for scientists and managers throughout the Agency to support the
translation and integration of science into decision making and should develop metrics to evaluate the
successful use of science as an input for decision making. Such metrics might be based on established
metrics for evaluating how research generated by agricultural extension scientists at state universities
has been translated for field use. Programs for enhanced interactions among EPA scientists, including
encouraging exchanges of ORD scientists with program and regional offices, and vice versa, also would
facilitate the development of this culture. Fundamental to success will be development of an agency-
wide human resource plan to attract, develop and retain highly qualified staff scientists in the regions,
programs and ORD so that the EPA's scientific workforce will be renowned for excellence in
integrating science to support decision making. This human resource plan should receive external peer
review from an appropriate advisory group.
Text Box 4 identifies several specific suggestions for increasing and improving support and training for
scientists and managers involved in science integration across the EPA, especially in programs and
regions. These suggestions aim to strengthen EPA's capacity for science integration and recognize
scientists' contributions in this area.
11
-------
Text Box 4: Specific suggestions for increasing and improving support and training for
scientists and managers across EPA to strengthen EPA's capacity for science integration
• The EPA should build on Superfund's communication and training infrastructure to
support cross-disciplinary exchange of information across all of EPA programs and
between the EPA and external scientists.
• The EPA should review the EPA's Risk Characterization Policy (U.S. EPA 2001) and
guidance on assessment factors (U.S. EPA 2003) to identify how they might be updated in
light of current needs for science integration and provide implementation training for
managers and scientists.
• The EPA should design and deliver training to managers and scientists about the potential
of programs across the agency, in conjunction, to achieve EPA's protection environmental
goals.
• EPA should develop an awards program to recognize excellence in the area of science
integration for decision making. These awards should have a stature similar to EPA's
Scientific and Technological Achievements Awards, which recognizes excellence in peer-
reviewed publications by EPA scientists.
• EPA managers should be recognized and rewarded for the robustness, transparency and
scientific defensibility of their decisions.
• The EPA should inform and engage EPA scientists in the implementation of SAB and
NRC recommendations on topics that have a broad agency scope.
Conclusion
Effective environmental policy making requires the integration of science from many disciplines to
inform the decision process. This concept is fundamental to the identity of the EPA. The presidential
commission that called for the establishment of the EPA in 1970 noted that "The environment, despite
its infinite complexity, must be perceived as a unified, interrelated system." The Ash Commission stated
that an environmental protection agency was needed and must have capabilities "to conduct research on
the adverse effects of pollution, to gather information on environmental conditions and use it in
modifying programs or recommending policy changes" (Ash et al. 1970). The commission proposed an
organizational structure for the EPA that would:
• Recognize the interrelated nature of pollution problems;
• Address the fact that pollutants cut across media lines;
• Encourage balanced budget and priority decisions between component functions; and
• Permit more effective evaluations of total program performance.
These goals have remained relevant since the EPA's founding in 1970, and the EPA has explored a
variety of organizational and management strategies to advance them. The concept of science integration
resonates with the public and is consistent with the vision of "One EPA" (Jackson 2010), where the
agency transcends historical barriers of program and region to use relevant scientific and technical
information to help solve environmental problems. It is consistent with recent guidance on advancing
the EPA's use of science in a variety of fields (NRC 2009; U.S. EPA SAB 2009; NRC 201 la), as well
as with the NRC report on sustainability (NRC 201 lb), which observed that "Meeting the goal of
sustainable development requires an integration of social, environmental, and economic policies,
12
-------
necessitating interdisciplinary coordination among federal agencies with varying missions to address
this goal."
Science integration occurs in many ways and in many programs at the EPA, but it could occur more
consistently and more fully to help EPA better protect public health and the environment. Strengthening
science integration at the EPA will require change: change in agency culture, change in how the agency
works, and increased support for scientists and managers in program and regional offices responsible for
science integration.
13
-------
REFERENCES
Anastas, P. 2010. Memorandum entitled "The Path Forward." Original memo dated on the ORD Intranet
as 03/04/2010 and made available to the SAB Staff Office on March 31, 2010.
http://vosemite.epa.uov/sab/sabproduct.nsf/7Q6BB04146A5F 14C852576FQ004E5E6Q/$File/Ana
stas+Path+Forward+3-5-10.pdf (accessed 10/4/2011).
Ash, Roy L. et al. 1970. Memorandum to the President entitled "Federal Organization for Environmental
Protection," April 29, 1970. http://www.epa.gov/aboutepa/historv/org/origins/ash.html (accessed
11/09/2011).
Jackson, Lisa P. 2010. "Seven Priorities for EPA's Future," January 12, 2010.
http://blog.epa.gov/administrator/2010/01/12/seven-priorities-for-epas-future/ (accessed
04/18/2012).
Bipartisan Policy Center. 2009. Science for Policy Project. Improving the use of science in regulatory
policy. Final Report. Washington, DC.
National Research Council. 2000. Strengthening Science at the U.S. EPA-Management and Peer-Review
Practices. Washington, DC: National Academies Press.
National Research Council. 2008. Public Participation in Environmental Assessment and Decision
Making. Washington, DC: National Academies Press.
National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC:
National Academies Press.
National Research Council. 201 la. Review of the Environmental Protection Agency's Draft IRIS
Assessment of Formaldehyde. Washington, DC: National Academies Press.
National Research Council. 201 lb. Sustainability and the U.S. EPA. Washington, DC: National
Academies Press.
Stahl, Cynthia H., Alan J. Cimorelli and Alice H. Chow. 2002. A New Approach to Environmental
Decision Analysis: Multi-criteria Integrated Resource Assessment (MIRA). Bulletin of Science,
Technology, and Society 22: 443-459.
http://vvvvvv.ceep.udel.edu/publications/sustainabledevelopment/publications/2002 sd new appr
oach mira.pdf (accessed 11/04/11).
Suter II, Glenn W. and Maciorowski, Anthony F. 2010. The Future of Ecological Assessment at the U.S.
Environmental Protection Agency, Human and Ecological Risk Assessment: An International
Journal, 16: 6, 1221 - 1226. URL: http://dx.doi.org/10.1080/10807039.201Q.526461 (accessed
04/18/12).
U.S. Environmental Protection Agency. 1998. Guidelines for ecological risk assessment.
EPA/630/R095/002F. http://rais.ornl.gov/documents/ECOTXTBX.PDF (accessed 04/18/2012).
R-l
-------
U.S. Environmental Protection Agency. 2001. Risk Characterization Handbook. EPA 100-B-00-002.
http ://www. epa. gov/ spc/pdfs/rchandbk.pdf (accessed 11/04/11).
U.S. Environmental Protection Agency. 2003, A Summary of General Assessment Factors for
Evaluating the Quality of Scientific and Technical Information, EPA 100/B-03.001.
http ://www. epa. gov/ spc/pdfs/assess2.pdf (accessed 11/176/11).
U.S. Environmental Protection Agency. 2005. Community-Based Watershed Management Handbook.
EPA 842-B-05-003.
http://vvater.epa.gov/tvpe/oceb/nep/upload/2007 04 09 estuaries nepprimeruments NEPPrimer
.pdf (accessed 10/26/2011).
U.S. Environmental Protection Agency. 2006. EPA's Action Development Process; Guide to
Considering Children's Health When Developing EPA Actions.
http://vosemite.epa.gov/sab/sabproduct.nsf/EC 15285B0339AB84852578650059C175/$File/EPA
ADP Guide 508-children's+health.pdf (accessed 11/04/11).
U.S. Environmental Protection Agency. 2009. National Water Program Research. EPA 822-R-09-012.
http://vvater.epa.gOv/scitech/research-riskassess/researchstrategv/index.cfm#strategv (accessed
11/18/2011).
U.S. Environmental Protection Agency. 2010a. EPA 's Action Development Process: Interim Guidance
on Considering Environmental Justice During the Development of an Action,
http://vosemite.epa.gov/sab/sabproduct.nsf/4D46D5F60CC33393852578650059C7CF/$File/con
siderins-ei-in-rulemakins-suide-07-2010.pdf (accessed 11/04/11).
U.S. Environmental Protection Agency. 2010b. Integrating Ecological Assessment and Decision-
Making at EPA: A Path Forward Results of a Colloquium in Response to Science Advisory
Board and National Research Council Recommendations. EPA/100/R-10/004.
http://vvvvvv.epa.gov/raf/publications/pdfs/integrating-ecolog-assess-decision-making.pdf
(accessed 04/18/12).
U.S. Environmental Protection Agency. 2011. EPA 's Action Development Process; Guidance for EPA
Staff on Developing Quality Actions.
http://vosemite.epa. gov/sab/sabproduct.nsf/5088B3878A90053E8525788E005EC8D8/$File/adp
03-00-11.pdf (accessed 11/03/2011).
U.S. Environmental Protection Agency Expert Panel on the Role of Science at EPA. 1992. Safeguarding
the Future: Credible Science, Credible Decisions. Washington, D.C. U.S. EPA. EPA/600/9-
91/050.
U.S. Environmental Protection Agency Science Advisory Board. 2000. Toward Integrated
Environmental Decision-Making. EPA-SAB-EC-00-01.
http://vosemite.epa.gov/sab/sabproduct.nsf/D3381 1633594B9D78525719B00656478/$File/ecirp
01 l.pdf (accessed 11/18/11).
R-2
-------
U.S. Environmental Protection Agency Science Advisory Board. 2001. Improved Science-Based
Environmental Stakeholder Processes. EPA-SAB-EC-COM-01-006.
http://vosemite.epa.uov/sab/sabproduct.nsf/F3DB 1F5C6EF90EE1852575C500589157/$File/EP
A-SAB-09-012-unsi gned.pdf (accessed 02/06/12).
U.S. Environmental Protection Agency Science Advisory Board. 2009. Valuing the Protection of
Ecological Systems and Services. EPA-SAB-09-012.
http://vosemite.epa.uov/sab/sabproduct.nsf/CEE3F362F 1A1344E8525718E004EA078/$File/eec
m01006 report appna-e.pdf (accessed 02/06/12).
U.S. Environmental Protection Agency Science Advisory Board. 2010. Office of Research and
Development Strategic Research Directions and Integrated Transdisciplinary Research.
http://vosemite.epa.uov/sab/sabproduct.nsf/E989ECFC 125966428525775B0047BE1 A/$File/EP
A-SAB-10-010-unsi gned.pdf (accessed 02/22/12)
U.S. Environmental Protection Agency Science Advisory Board. 2011. Office of Research and
Development (ORD) New Strategic Research Directions: A Joint Report of the Science Advisory
Board (SAB) and ORD Board of Scientific Councilors (BOSC). EPA-SAB-12-001
http://vosemite.epa.uov/sab/sabproduct.nsf/804D 1 A3A4A393C028525793000732744/$File/EPA
-SAB-12-001 -unsiuned.pdf (accessed 10/09/11).
U.S. Government Accountability Office. 2008. EPA's New Assessment Process Will Increase
Challenges EPA Faces in Evaluating and Regulating Chemicals. GAO-08-743T.
U.S. Government Accountability Office. 2011a. Challenges Remain with EPA 's Integrated Risk
Information System Program. GAO-12-42.
U.S. Government Accountability Office. 201 lb. To Better Fulfill Its Mission, EPA Needs a More
Coordinated Approach to Managing Its Laboratories. GAO-11-347.
R-3
-------
Attachment A: Request for the SAB Study
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20480
OCT 2 0 2008
MEMORANDUM
THE AOMffflSTRATOR
SUBJECT; Request for a Science Advisory Board Study
TO:
Dr. Deborah Swaekhamer
Chair, Science Advisory Board
At the U.S. Environmental Protection Agency, sound decision-making depends on
getting the best available science. Daring its 30-year history of advising EPA Administrators,
the Science Advisory Board has emphasized the need for anticipating future environmental
threats and investing in emerging research and science critical for informing decisions. As our
understanding of complex environmental problems improves, integrated approaches for
delivering the best science need to be developed and implemented.
The SAB's 2000 report Toward Integrated Environmental Decision-Making suggested an
integrated decision-making framework for evaluating and responding to environmental
problems. 1 ask that the SAB initiate a study that builds on its 2000 study to develop
independent advice on how EPA can strengthen scientific assessments for decision making. The
SAB might consider EPA's organizational structure and functions in light of how they influence
the development and application of science assessments in different decision-making contexts. It
would also be valuable for the SAB to recommend how to strengthen EPA's approaches for
integrating traditional human health and ecological science assessments with socioeconomics
analyses, decision sciences, and technology development and assessments to better support
policy development. Finally, as EPA continues to plan for human capital needs, I would like the
SAB to provide advice on ways to attract and retain the best diverse technical workforce.
Attached is a brief description of the proposed study. Please feel free to tailor the scope
and depth of the study as appropriate. I ask the study be completed in a timely manner for the
next EPA Administrator's consideration and implementation.
Attachment
tatetnei Address (URt| •
R«cyct«tiVReeycl»bi* • Frinte
-------
Effective human health and environmental protection requires a strong foundation of
scientific knowledge. Scientific information often includes considerable uncertainly resulting m
a diversity of scientific interpretations. The development and application of scientific knowledge
in identifying potential threats, characterizing risks, formulating technological solutions, and
evaluating the benefits and costs of U.S. Environmental Protection Agency actions are major
science functions at EPA. The scope and depth of such science assessments greatly vary under
different legislation and policies.
These functions are carried out by scientists, engineers, and economists with specialized
program knowledge. They, in tum, rely on technical support by outside experts procured
through Agency's interagency agreements or contracts. In addition, EPA's National Center for
Environmental Assessment in the Office and Research Development develops technical
assessments for EPA's Integrated Risk Information System which are ised throughout the
Agency. Summaries of the potential human health effects information that may result from
exposure to chemicals in the environment, along with the supporting Toxicologic®! Reviews, are
made available electronically on IRIS for use by EPA, states, and tribal governments.
Over the years, reports From the National Research Council, the General Accountability
Office, and other organizations point out that, while EPA has knowledgeable experts, the
Agency's policies and regulations are too often perceived to lack a strong scientific foundation
and EPA's science is of uneven quality. To address these issues, EPA established several
science coordinating bodies. For instance:
« the Risk Assessment Forum consists of Agency senior scientists that develop
Agency-wide technical guidelines for human health risk assessment, ecological nsk
assessment, and exposure assessment;
• the Science Policy Council develops Agency position papers on cross-cutting and
emerging issues (e.g. peer review practices, data quality guidelines, genomics,
nanotechnol ogy); and
• the Council on Regulatory Environmental Modeling guides the development and
use of environmental models.
A-2
-------
Staff support for these coordinating bodies is now centralized in the newly created EPA
Office of the Science Advisor, In addition to these groups, the National Regional Science
Council promotes communication and collaboration of regional scientists to identify common
regional needs.
Nonetheless, scientists, engineers, economists, and other technical professionals, by
necessity, continue to be spread throughout tie Agency and have limited opportunity to interact
with their peers in other organizational units. Such segregation can result in duplication of effort,
as well as conflicting scientific approaches to the evaluation of similar environmental agents by
different offices. While the Agency has tried to minimize such occurrences through its science
and science policy coordinating bodies, existing coordination processes can be slow and tend to
occur in the later phases of assessment development and approval. Furthermore, the
environmental problems of today are more complex, often cross state and national boundaries,
and require consideration of difficult trade-offs and integration of socioeconomic and
technological solutions, EPA's existing science and science policy coordinating bodies primarily
address immediate scientific needs of the Agency and may miss a longer-term strategic
viewpoint.
Proposal
The SAB has provided scientific advice and recommendations to the Agency on a wide
variety of scientific issues for more than 30 years. Because of the SAB's unique perspective, it
would be of value for the SAB to evaluate the Agency's current organizational structures and
functions concerning the development and application of science assessments in different EPA
decision-making contexts. The evaluation would result in advice and recommendations on bow
the Agency might strengthen scientific assessments, communication of uncertainties of the
assessments, and how the results are used. Areas for consideration may include: scientific
leadership, consistent scientific practices; scientific collaboration within and between disciplines;
and multi-disciplinary approaches for integrating natural science assessments with economic and
social science assessments.
A-3
-------
1
Appendix B: Interview Protocol and Advance Questions
The SAB conducted interviews with EPA Offices and Regions that use science to support decision
making. Two or more committee members were involved in each interview. The SAB Staff Office
Director or Deputy Director provided introductions, and the Designated Federal Officer (DFO) for the
committee took notes and assisted the SAB committee in consolidating and summarizing information
gleaned from the interview sessions. The interviews were held at the designated location of EPA Offices.
The SAB requested separate interview sessions with decision makers, policy makers, and scientific and
technical staff. SAB members used the following questions as a guide for the interviews. The SAB
recognized that not all questions will be relevant and appropriate for all EPA offices. The interview
questions cover topics such as 1) practices for integrating science to support decision making; 2)
consideration of public, stakeholder, external scientific, and other governmental input in science
assessment for decision making; 3) drivers and impediments to implementing past recommendations*
for science integration; 4) ways EPA receives feedback on how science is used in decision-making; and
5) the EPA workforce related to science integration supporting decision making.
The SAB committee asked interviewees to review the questions below the interviews and to describe one
or two important and representative examples of science-based decisions specific to their organization.
The committee expressed interest in learning what interviewees viewed as what is and is not working
well, and what changes are needed to improve science integration to support environmental decision
making. The SAB DFO provided draft summaries of the interviews to the interviewees for comment.
Advance questions for Policy and Decision Makers:
1. Practices for integrating science to support decision making
1.1. What kinds of decisions does your organization make?
1.2. What is (are) your role(s) in the decision-making process?
1.3. For each type of decision please describe the process by which it is made. What types of
assessments do you include to inform your decisions?
1.4. Do the decision-making processes used by your office employ planning and scoping, and
problem formulation phases? If yes, how are planning and scoping, and problem formulation
conducted? What kinds of preliminary assessments are conducted?
1.5. Has your organization applied any of the processes and approaches recommended by the SAB
and NRC for integrating science supporting decision making? Has it used other models and
approaches? If so, has it been useful to apply these models/approaches?
1.6. As applicable, discuss a particular past recommendation that relates to the example(s) of
science-based decisions you have described for the committee. Did the recommendation affect
your decision(s)? If it affected the decisions, in what ways did this occur?
*With special consideration of decision-making processes and approaches described in the Toward Integrated Environmental
Decision-Makins. (SAB, 2000) and Science and Decisions (NRC, 2009) and recommendations related to public participation
in science and environmental protection in Improved Science-Based Environmental Stakeholder Processes (SAB, 2001) and
Public Participation in Environmental Assessment and Decision Making (NRC, 2008).
B-l
-------
1.7. How do you assess the level of analysis needed for a particular science assessment, and when is
the analysis judged to be sufficiently completed to allow decision making?
1.8. Is the science assessment and decision-making process altered to accommodate different
locations in the United States or different spatial scales? Do science assessment and decision-
making processes change to address short-term and long-term needs?
1.9. What scientific data or information do you need to support decisions? Do you have the
data/information that you need, when you need it? If not, what do you do? Are you constrained
from using all available scientific information in decisions or generating new data and
information to support decisions?
1.10 How are different assessments in different disciplines (including social and decision
sciences) integrated as part of the science decision-making process?
1.11 How do you like information about the uncertainties in scientific assessments
presented? What are some examples of presentation of uncertainties in scientific
assessments that have helped you understand the science related to a decision and had
an impact on that decision?
Consideration of public, stakeholder, external scientific, and other governmental input in
science assessment for decision making
2.1. What role do the regulated community; non-governmental organizations; and the general public
play in your organization's science assessment process? If involvement occurs, how is it
accomplished? At what steps in the process are these groups involved?
2.2. To what degree and how do you coordinate scientific assessments with international
organizations, other federal agencies, states and tribes? How does this coordination happen?
2.3. What role does the external scientific community play in integrating science to support decision-
making in your organization? How does your organization engage the external scientific
community to help your decision makers get the science needed to support decisions?
2.4. Has your organization applied any of the SAB's or NRC's recommendations relating to public
participation in science supporting environmental decision-making? Have these reports
influenced how public/stakeholder input has been used in your organization's science
assessments? If so, has it been useful to apply these models/approaches?
Drivers and impediments to implementing past recommendations for science integration
3.1. Are there perceived or actual barriers for developing and/or implementing new or existing
decision-making processes or frameworks that integrate the best available science? If yes, what
are they?
Ways EPA receives feedback on how science is used in decision-making
4.1. How does your organization determine the effectiveness of implemented decisions (whether the
decision resulted in reduced risk and improvement to public health and the environment)?
4.2. Does your organization use feedback on decisions to detect emerging science, influence future
policy, set priorities? If so, how?
EPA workforce related to science integration supporting decision making
5.1. How does your organization's scientific and technical workforce adapt to shifts in priorities and
resources?
5.2. How do scientists stay current in their areas of expertise, or expand their expertise based on
current and future scientific needs?
B-2
-------
5.3.What is the current balance between near-term program support research and longer-term
research to advance the science?
6. Are there other questions we should ask that would help us understand how science and
scientific assessments are integrated to support your decisions?
Questions for Scientific and Technical Staff:
1. Practices for integrating science to support decision making
1.1. What kinds of decisions are made in your organization and what is your role(s) in the decision-
making process?
1.2. What types of science assessments are done to support your organization's decisions (e.g.,
technology, benefits, human health, ecological, behavioral/social/economic, etc.)?
1.3. Who actually conducts science assessments (e.g., your organization's staff, contractors, other
EPA offices/personnel)?
1.4. How are assessments in different disciplines (including social and decision sciences) integrated
as part of the science decision-making process?
1.5. How do you work within your own office, and with other EPA Offices and Regions to
coordinate analyses needed for decision-making? What science data, models, analyses, etc. do
you obtain from other units to support decision making in your unit?
1.6. Do you conduct formal uncertainty analyses? How are analyses matched to the needs of
decision makers? How is uncertainty communicated to decision makers, stakeholders and the
public?
1.7. What roles do computational models have in science integration for decision making in your
organization. Do you make use of EPA's Council for Regulatory Environmental Modeling or
the Models Knowledge Base, and if so, how?
1.8. What improvements are needed to integrate science assessments to support decision-making
processes?
1.9. What are current interactions among your organization and the Agency's laboratories (e.g.,
ORD, Regional, Program-specific)?
2. Consideration of public, stakeholder, external scientific, and other governmental input in
science assessment for decision making
2.1. To what degree do you coordinate development of your organization's scientific assessments
with international organizations, other federal agencies, states and tribes? How does this
coordination happen?
2.2. What role do the regulated community, non-governmental organizations, other international,
federal, state or tribal governments and the general public play in your organization's science
assessment process? If involvement occurs, how is it accomplished? At what steps in the
process are these groups involved?
2.3. What role does the external scientific community play in integrating science to support your
organization's decision-making? How does your organization engage the external scientific
community in getting the science needed to support environmental decisions?
B-3
-------
3. Drivers and impediments to implementing past recommendations for science integration
3.1. Are there perceived or actual barriers for developing and/or implementing new or existing
decision-making processes or frameworks that integrate the best available science? If yes, what
are they?
4. Ways EPA receives feedback on how science is used in decision-making
4.1. How does your organization determine the effectiveness of implemented decisions (whether the
decision resulted in reduced risk and improvement to public health and the environment)?
4.2. Does your organization use feedback on decisions to detect emerging science, influence future
policy, set priorities? If so, how?
5. EPA workforce related to science integration supporting decision making
5.1. How do you stay current in their areas of expertise, or expand their expertise based on current
and future scientific needs?
6. Are there other questions we should ask that would help us understand how science and
scientific assessments are integrated in support of your organization's decisions?
B-4
-------
Appendix C: List of SAB "Science Integration for Decision Making" Interviews,
2009-2010
EPA Office and Location
Date
Region 1 (Boston, MA),
October 28, 2009
Region 2 (New York, NY)
December 17, 2009
Region 3 (Philadelphia, PA)
January 19, 2010
Region 4 (Atlanta, GA)
October 26, 2009
Region 5 (Chicago, IL)
January 25, 2010
Region 6 (Dallas, TX)
December 9, 2009
Region 7 (Kansas City, KS)
December 16, 2009
Region 8 (Denver, CO)
December 15, 2009
Region 9 (San Francisco, CA)
January 6, 2010
Region 10 (Seattle, WA)
December 8, 2009
Office of Air and Radiation,
November 19, 2009
January 12, 2010
Office of Prevention, Pesticides and Toxic Substances*
January 26, 2010
Office of Solid Waste and Emergency Response
December 1, 2009
November 24, 2009
Office of Water
Office of Environmental Information Toxic Release
Inventory Program
January 20, 2010
January 28, 2010
December 24, 2009
Office of Research and Development
November 30, 2009
January 29, 2010
January 25, 2010
January 28, 2010
February 3, 2010
February 4, 2010
* Currently the Office of Chemical Safety and Pollution Prevention.
C-l
-------
EPA Office and Location Date
Office of the Science Advisor January 21, 2010
Office of Children's Health Protection January 21, 2010
National Center for Environmental Economics January 21, 2010
C-2
-------
Appendix D: Science Integration Frameworks Discussed in this Report
kilos imiicn
iApert icm
Valutas
- iutojm.iTiou
- f-Apcri Judgment
- Values
lusriiimonal Milieu
VIMSUU
ANALYSIS AND DECISION-MAKING
(\\ h.u are the Ivsi nsk t eduction ^ppommmes
Options Analysis Ptrjottndiice ,\fuisttn^
Risk Asst'ssmcnl Scrveu/n^/SekvUon
> 1" low .uv v> c dcan^l5)
Jmph mtnhftMi Mittiftormg andKepoftmg
hij^umtnon Fixdiuttton
piiasl ru
IMPI 1:Vli.NTAIK>N and
P[ RR)R\1\N(.:H T VAI UATfON
li.it .lie theluoil uupvuloul eiusK'nmaiu) risk-..'
\\ hat are <« ctmivmut mil gojli.-")
RhL < t'mpamoni Goal Setting
Prvlimtna/v < Analysis
m IASE I
PROBLEM FORMULATION
Figure D-1. Framework for integrated environmental decision-making
(Source: U.S. EPA SAB. 2000. U.S. SAB. 2000. Toward Integrated Environmental Decision-Making. EPA-
SAB-EC-00-011)
C-l
-------
Discussion
Between the
Risk Assessor
and
Risk Manager
(Planning)
Ecological Risk Assessment
PliQBLF.M f'OHMUL AT:Ofi
A
N
A
L
Y
S
I
S
r
I
on
L <:x;; .jr.
Characlerizatic
of
Ecological
V
HISK CMARAClbHI/ATiOfJ
J
Discussion Between the
Risk Assessor and Risk Manager
(Results)
T
Risk Management
O
Q>
5f
>
o
-Q
C
gh
5'
3
<
------- |