UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
September 28, 2012
EPA-SAB-12-010
The Honorable Lisa P. Jackson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Dear Administrator Jackson:
The EPA Office of the Science Advisor requested that the Science Advisory Board (SAB) review the
EPA Risk Assessment Forum's (RAF) draft Ecological Assessment Action Plan titled Integrating
Ecological Assessment and Decision-Making at EPA: 2011 RAF Ecological Assessment Action Plan
(August 11, 2011). In the Plan the EPA proposes a set of science policy and technical practice initiatives
to improve the quality, scope and application of the agency's ecological assessments. The Plan, which
was developed in response to recommendations provided in 2007 by the SAB and in 2009 by the
National Research Council, is brief and does not contain detailed information indicating how it will be
implemented. The EPA requested that the SAB comment on the technical merit of the initiatives
outlined in the Plan and provide advice on how the Plan could be further developed and implemented.
The enclosed report provides the consensus advice and recommendations of the SAB Ecological
Processes and Effects Committee. Because the Plan at this stage represents a strategy prospectus, rather
than a detailed blueprint for implementation, the SAB responses to the EPA's charge questions are
intended to provide initial ideas to guide further development of the agency's Plan.
The SAB urges the EPA to invest additional resources in strengthening the agency's ecological
assessments. The Plan is a solid starting point for the EPA's effort to integrate ecological risk
assessment and decision-making. The six science policy and other technical practice initiatives in the
Plan are responsive to previous SAB and NRC recommendations with no significant omissions. While
each of science policy initiatives is critical to the long-term success of the Plan, the SAB finds that three
of them may have the greatest likelihood of achieving the agency's goals in the near term: use of weight-
of-evidence approaches in ecological risk assessments; improved communication of ecological
assessment issues and results to decision-makers; and incorporation of ecosystem services into
ecological risk assessment methods. The other three science policy initiatives in the Plan: developing a
systems approach to ecological assessment, using adaptive management, and strengthening the EPA's
ecological protection goals are equally important and represent key beneficial long-term changes in the
Subject: SAB Review of the EPA's Ecological Assessment Action Plan
-------
culture and approach to ecological risk assessment. The SAB provides the following key
recommendations for developing and refining the Plan.
• Weight-of-Evidence: The Plan calls for the development of guidance on the use of a weight-of-
evidence approach in EPA ecological assessments. The scientific merit of using a weight-of-
evidence approach in ecological risk assessment is clear, and successful implementation of an
integrated assessment framework will hinge on a weight-of-evidence determination. A scientifically
rigorous weight-of-evidence approach must rely primarily on statistically-based decision points
rather than best professional judgment. Therefore, the SAB continues to recommend that the EPA
develop program-specific guidance that provides statistically-based approaches and decision-
making frameworks for weighing and integrating multiple lines of evidence in ecological risk
assessments. Useful statistically-based approaches have been developed for weighing multiple lines
of evidence. These approaches would provide a solid basis for EPA guidance but a significant
amount of work will be required to synthesize the science and develop the guidance. The EPA
should develop case studies to illustrate the use of such new approaches to augment the ecological
risk assessment guidance documents used by practitioners.
• Communication: The Plan calls for development of methods for better communication of
ecological assessment issues and results between ecological risk assessment practitioners and
decision-makers within the agency. The SAB supports this initiative but finds that the proposed
survey methodology for exploring how ecological risk assessments are used in agency decision-
making is a preliminary approach that will not provide all of the information needed by the EPA to
develop better communication methods. Upon completion of the proposed work, the project should
be broadened to address better communication throughout all stages and among all participants in
the risk assessment/risk management process, including key stakeholders. The EPA should develop
communication guidance, supporting tools and strategies that can be adapted to the needs of specific
regulatory applications and a range of intended audiences. As the guidance and tools are developed,
the EPA should consider obtaining external technical input from social scientists, ethicists,
marketing professionals and media specialists who have a good understanding of risk
communication and broader environmental concerns.
• Ecosystem Services: The EPA is developing a white paper that interprets conventional ecological
assessment endpoints in the context of ecosystem services. The SAB encourages the agency to
complete the white paper and submit a shorter version to a peer-reviewed publication to make it
available to a wider audience. The SAB supports the Agency's proposal in the Plan to update the
current guidance on Generic Ecosystem Assessment Endpoints by including a broader range of
ecosystem services. The EPA should undertake a thorough revision of the guidance rather than
developing an addendum. The SAB also recommends that the Agency look to other federal agencies
for operating models of the integration of ecosystem services information into management decision
processes.
• Systems Approach to Ecological Assessment: The Plan calls for developing a systems approach
to ecological assessment that includes multiple media and endpoints and integration of different
types of assessments. EPA scientists have developed a good preliminary framework for integrated
environmental assessment. The SAB recommends that this preliminary framework be further
developed, that it address the cumulative effects of multiple stressors in the context of climate
change, and that it explicitly incorporate ecosystem services endpoints.
-------
• Adaptive Management: The Plan calls for the use of adaptive management to test and revise risk
management actions. The SAB supports the goal of incorporating adaptive management principles
into the agency's risk assessment framework but recognizes that implementation of this goal may be
difficult given the complexity of consistent and continuous ecosystem monitoring and evaluation
over appropriate time scales. The SAB emphasizes that principles of rigorous statistical design
should be applied in order to implement effective adaptive management approaches.
• Strengthening EPA's Ecological Protection Goals: The Plan calls for strengthening the EPA's
ecological protection goals. The EPA should articulate and elucidate its ecological protection goals.
To accomplish this, the agency's ecological scientists will need to develop information and
perspectives that will enable them to communicate more effectively with decision-makers and the
public. Clearly, ecosystem function and human health are tightly linked, and incorporation of
ecosystem services into the ecological assessment process can strengthen the EPA's ecological
protection goals. Environmental justice also is a useful platform to highlight the relationship of
ecosystem condition to the health of vulnerable human populations.
The SAB encourages the EPA to incorporate input and perspectives from other entities as it elaborates
the current brief Plan into more detailed project plans. These entities should include U.S. agencies
involved in resource management, U.S. States, other countries (including Canada, Australia, China and
the European Union), nongovernmental organizations, and social scientists. These additional
perspectives on issues such as sustainability, adaptive management, communication and environmental
justice would improve the Plan and expand its breadth appropriately. The SAB also recommends that the
Plan explicitly address the importance of the problem formulation stage of ecological risk assessments in
ensuring a systems-level approach and articulating how the assessments will inform agency decisions.
The SAB appreciates the opportunity to provide advice to the EPA on the Ecological Assessment Action
Plan. We look forward to receiving the agency's response to this report.
Sincerely,
/Signed/
/Signed/
Dr. Deborah L. Swackhamer, Chair
EPA Science Advisory Board
Dr. Ingrid Burke, Chair
SAB Ecological Processes and
Effects Committee
Enclosure
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board (SAB), a public
advisory committee providing extramural scientific information and advice to the Administrator and
other officials of the Environmental Protection Agency. The SAB is structured to provide balanced,
expert assessment of scientific matters related to problems facing the agency. This report has not been
reviewed for approval by the agency and, hence, the contents of this report do not necessarily represent
the views and policies of the Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government, nor does mention of trade names or commercial products constitute a
recommendation for use. Reports of the EPA Science Advisory Board are posted on the EPA Web site
at: http://www.epa.gov/sab.
1
-------
U.S. Environmental Protection Agency
Science Advisory Board (SAB)
Ecological Processes and Effects Committee (EPEC) Augmented for
Review of the Ecological Assessment Action Plan
CHAIR
Dr. Ingrid Burke, Director, Haub School and Ruckelshaus Institute of Environment and Natural
Resources, University of Wyoming, Laramie, WY
SAB MEMBERS
Dr. Judith L. Meyer, Professor Emeritus, Odum School of Ecology, University of Georgia, Lopez
Island, WA
Dr. Amanda Rodewald, Professor of Wildlife Ecology, School of Environment and Natural Resources,
The Ohio State University, Columbus, OH
EPEC MEMBERS
Dr. Ernest F. Benfield, Professor of Ecology, Department of Biological Sciences, Virginia Polytechnic
Institute and State University, Blacksburg, VA
Dr. Peter Chapman, Principal and Senior Environmental Scientist, Environmental Sciences Group,
Golder Associates Ltd, Burnaby, BC, Canada
Dr. Loveday Conquest, Professor, School of Aquatic and Fishery Sciences, University of Washington,
Seattle, WA
Dr. Richard Di Giulio, Professor, Nicholas School of the Environment, Duke University, Durham, NC
Dr. Robert Diaz, Professor, Department of Biological Sciences, Virginia Institute of Marine Science,
College of William and Mary, Gloucester Pt., VA
Dr. Lucinda Johnson, Center Director, Center for Water and the Environment, Natural Resources
Research Institute, University of Minnesota Duluth, Duluth, MN
Dr. Thomas W. La Point, Professor, Department of Biological Sciences, University of North Texas,
Denton, TX
Dr. William Stubblefield, Senior Research Professor, Department of Molecular and Environmental
Toxicology, Oregon State University, Corvallis, OR
11
-------
CONSULTANTS
Dr. Gregory Biddinger, Managing Director, Natural Land Management, Houston, TX
Dr. G. Allen Burton, Professor and Director, Cooperative Institute for Limnology and Ecosystems
Research, School of Natural Resources and Environment, University of Michigan, Ann Arbor, MI
Dr. Wayne Landis, Professor and Director, Department of Environmental Toxicology, Institute of
Environmental Toxicology, Huxley College of the Environment, Western Washington University,
Bellingham, WA
SCIENCE ADVISORY BOARD STAFF
Dr. Thomas Armitage, Designated Federal Officer, U.S. Environmental Protection Agency,
Washington, DC
111
-------
U.S. Environmental Protection Agency
Science Advisory Board
CHAIR
Dr. Deborah L. Swackhamer, Professor, Hubert H. Humphrey School of Public Affairs and Co-
Director of the Water Resources Center, University of Minnesota, St. Paul, MN
SAB MEMBERS
Dr. George Alexeeff, Director, Office of Environmental Health Hazard Assessment, California
Environmental Protection Agency, Oakland, CA
Dr. David T. Allen, Professor, Department of Chemical Engineering, University of Texas, Austin, TX
Dr. Pedro Alvarez, Department Chair and George R. Brown Professor of Engineering, Department of
Civil & Environmental Engineering, Rice University, Houston, TX
Dr. Joseph Arvai, Svare Chair in Applied Decision Research, Institute for Sustainable Energy,
Environment, & Economy, Haskayne School of Business, University of Calgary, Calgary, Alberta,
Canada
Dr. Claudia Benitez-Nelson, Full Professor and Director of the Marine Science Program, Department
of Earth and Ocean Sciences, University of South Carolina, Columbia, SC
Dr. Patricia Buffler, Professor of Epidemiology and Dean Emerita, Department of Epidemiology,
School of Public Health, University of California, Berkeley, CA
Dr. Ingrid Burke, Director, Haub School and Ruckelshaus Institute of Environment and Natural
Resources, University of Wyoming, Laramie, WY
Dr. Thomas Burke, Professor and Jacob I. and Irene B. Fabrikant Chair in Health, Risk and Society
Associate Dean for Public Health Practice, Johns Hopkins Bloomberg School of Public Health, Johns
Hopkins University, Baltimore, MD
Dr. Terry Daniel, Professor of Psychology and Natural Resources, Department of Psychology, School
of Natural Resources, University of Arizona, Tucson, AZ
Dr. George Daston, Victor Mills Society Research Fellow, Product Safety and Regulatory Affairs,
Procter & Gamble, Cincinnati, OH
Dr. Costel Denson, Managing Member, Costech Technologies, LLC, Newark, DE
Dr. Otto C. Doering, III, Professor, Department of Agricultural Economics, Purdue University, W.
Lafayette, IN
Dr. Michael Dourson, President, Toxicology Excellence for Risk Assessment, Cincinnati, OH
iv
-------
Dr. David A. Dzombak, Walter J. Blenko, Sr. University Professor of Environmental Engineering,
Department of Civil and Environmental Engineering, College of Engineering, Carnegie Mellon
University, Pittsburgh, PA
Dr. T. Taylor Eighmy, Senior Vice President for Research, Office of the Vice President for Research,
Texas Tech University, Lubbock, TX
Dr. Elaine Faustman, Professor and Director, Institute for Risk Analysis and Risk Communication,
School of Public Health, University of Washington, Seattle, WA
Dr. John P. Giesy, Professor and Canada Research Chair, Veterinary Biomedical Sciences and
Toxicology Centre, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
Dr. Jeffrey K. Griffiths, Professor, Department of Public Health and Community Medicine, School of
Medicine, Tufts University, Boston, MA
Dr. James K. Hammitt, Professor, Center for Risk Analysis, Harvard University, Boston, MA
Dr. Barbara L. Harper, Risk Assessor and Environmental-Public Health Toxicologist, and Division
Leader, Hanford Projects, and Program Manager, Environmental Health, Department of Science and
Engineering, Confederated Tribes of the Umatilla Indian Reservation (CTUIR), West Richland, WA
Dr. Kimberly L. Jones, Professor and Chair, Department of Civil Engineering, Howard University,
Washington, DC
Dr. Bernd Kahn, Professor Emeritus and Associate Director, Environmental Radiation Center, Georgia
Institute of Technology, Atlanta, GA
Dr. Agnes Kane, Professor and Chair, Department of Pathology and Laboratory Medicine, Brown
University, Providence, RI
Dr. Madhu Khanna, Professor, Department of Agricultural and Consumer Economics, University of
Illinois at Urbana-Champaign, Urbana, IL
Dr. Nancy K. Kim, Senior Executive, Health Research, Inc., Troy, NY
Dr. Cecil Lue-Hing, President, Cecil Lue-Hing & Assoc. Inc., Burr Ridge, IL
Dr. Judith L. Meyer, Professor Emeritus, Odum School of Ecology, University of Georgia, Lopez
Island, WA
Dr. James R. Mihelcic, Professor, Civil and Environmental Engineering, University of South Florida,
Tampa, FL
Dr. Christine Moe, Eugene J. Gangarosa Professor, Hubert Department of Global Health, Rollins
School of Public Health, Emory University, Atlanta, GA
v
-------
Dr. Horace Moo-Young, Dean and Professor, College of Engineering, Computer Science, and
Technology, California State University, Los Angeles, CA
Dr. Eileen Murphy, Director of Research and Grants, Ernest Mario School of Pharmacy, Rutgers
University, Piscataway, NJ
Dr. James Opaluch, Professor and Chair, Department of Environmental and Natural Resource
Economics, College of the Environment and Life Sciences, University of Rhode Island, Kingston, RI
Dr. Duncan Patten, Director, Montana Water Center, and Research Professor, Hydroecology Research
Program, Department of Land Resources and Environmental Sciences, Montana State University,
Bozeman, MT
Dr. Stephen Polasky, Fesler-Lampert Professor of Ecological/Environmental Economics, Department
of Applied Economics, University of Minnesota, St. Paul, MN
Dr. C. Arden Pope, III, Professor, Department of Economics, Brigham Young University, Provo, UT
Dr. Stephen M. Roberts, Professor, Department of Physiological Sciences, Director, Center for
Environmental and Human Toxicology, University of Florida, Gainesville, FL
Dr. Amanda Rodewald, Professor of Wildlife Ecology, School of Environment and Natural Resources,
The Ohio State University, Columbus, OH
Dr. Jonathan M. Samet, Professor and Flora L. Thornton Chair, Department of Preventive Medicine,
Keck School of Medicine, University of Southern California, Los Angeles, CA
Dr. James Sanders, Director and Professor, Skidaway Institute of Oceanography, Savannah, GA
Dr. Jerald Schnoor, Allen S. Henry Chair Professor, Department of Civil and Environmental
Engineering, Co-Director, Center for Global and Regional Environmental Research, University of Iowa,
Iowa City, IA
Dr. Gina Solomon, Deputy Secretary for Science and Health, Office of the Secretary, California
Environmental Protection Agency, Sacramento, CA
Dr. Daniel O. Stram, Professor, Department of Preventive Medicine, Division of Biostatistics,
University of Southern California, Los Angeles, CA
Dr. Peter S. Thorne, Director, Environmental Health Sciences Research Center and Professor and
Head, Department of Occupational and Environmental Health, College of Public Health, University of
Iowa, Iowa City, IA
Dr. Paige Tolbert, Professor and Chair, Department of Environmental Health, Rollins School of Public
Health, Emory University, Atlanta, GA
vi
-------
Dr. John Vena, University of Georgia Foundation Professor in Public Health and
Head, Department of Epidemiology and Biostatistics, Georgia Cancer Coalition Distinguished Scholar,
College of Public Health , University of Georgia, Athens, GA
Dr. Robert Watts, Professor of Mechanical Engineering Emeritus, Tulane University, Annapolis, MD
Dr. R. Thomas Zoeller, Professor, Department of Biology, University of Massachusetts, Amherst, MA
SCIENCE ADVISORY BOARD STAFF
Dr. Angela Nugent, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board, Washington, DC
vii
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 8
3. RESPONSES TO EPA'S CHARGE QUESTIONS 9
3.1. Overall Technical Merit of the Proposed Science Policy and Technical Practice
Initiatives 9
3.2. Importance of Developing an Integrated Assessment Approach 11
3.3. Use of a Weight-of-Evidence Approach in Ecological Risk Assessments 15
3.4. Communication of Ecological Assessment Issues and Results to Decision-Makers
and Stakeholders 20
3.5. Incorporation of Ecosystem Services into Ecological Risk Assessment Methods 25
3.6. Use of Adaptive Management for Testing and Revising Risk Management Actions 30
3.7. Strengthening the EPA's Ecological Protection Goals 34
REFERENCES 37
APPENDIX A: THE CHARGE TO THE SAB A-l
viii
-------
1. EXECUTIVE SUMMARY
The Risk Assessment Forum (RAF) in the EPA Office of the Science Advisor (OSA) has developed a
draft Ecological Assessment Action Plan titled Integrating Ecological Assessment and Decision-Making
at EPA: 2011 RAF Ecological Assessment Action Plan (August 11, 2011). The draft Ecological
Assessment Action Plan (hereafter referred to as the "Plan") sets forth proposed science policy and
technical practice initiatives to improve the quality, scope and application of the EPA's ecological
assessments. The Plan was developed in response to a 2007 EPA Science Advisory Board (SAB)
Report, Advice to EPA on Advancing the Science and Application of Ecological Risk Assessment in
Environmental Decision-Making and a 2009 National Research Council (NRC) report, Science and
Decisions: Advancing Risk Assessment. To address the recommendations in these reports, the RAF
convened an intra-agency colloquium to review the EPA's ecological risk assessment practices and
guidance in light of the SAB and NRC advice. The initiatives in the EPA's Plan correspond to key
recommendations in the colloquium report, Integrating Ecological Assessment and Decision-making at
EPA: A Path Forward (hereafter referred to as the Colloquium Report). The science policy initiatives in
the Plan focus on: (1) developing a systems approach to ecological assessment that integrates multiple
media, endpoints and types of assessments; (2) developing weight-of-evidence (WOE) as an option for
inference in ecological assessments; (3) improving communication of ecological assessment issues and
results; (4) incorporating ecosystem services into ecological risk assessment methods; (5) using adaptive
management for testing and revising risk management actions; and (6) strengthening the EPA's
ecological protection goals. Other specific technical practice initiatives in the Plan focus on the need for
improvements in: training; quality assurance; guidance to address multiple and specific receptors; life
cycle evaluations; uncertainty analysis; and access to information.
The EPA requested that the SAB review the agency's Plan and provide advice on the technical merit and
implementation of the proposed initiatives. This Executive Summary highlights the findings and
recommendations of the SAB in response to the charge questions provided in Appendix A.
Overall Technical Merit of the Proposed Science Policy and Technical Practice Initiatives
The EPA asked the SAB to comment on whether the initiatives in the Plan are responsive to the advice
previously provided by SAB and NRC and whether the initiatives reflect the most important set of
activities needed to advance the application of ecological risk assessment in environmental decision-
making.
The SAB urges the EPA to invest additional resources in strengthening the agency's ecological
assessments. The science policy and technical practice initiatives proposed in the Plan follow logically
from the EPA Colloquium Report and are responsive to the previous SAB and NRC recommendations,
but additional information should be included in the Plan to indicate how it will be used and
implemented by the agency. The SAB has not identified any significant omissions in the proposed set of
initiatives. The Plan is a solid starting point for the EPA's effort to integrate ecological risk assessment
and decision-making. Due to imposed page limitations, the Plan is very brief, but presentations to the
SAB by EPA staff supplied additional information about the development and implementation of the
Plan. The SAB has provided recommendations to further develop and refine the Plan, and the EPA has
indicated more detailed information will be included in individual project plans as they are developed.
Unfortunately, however, the Plan appears to be very myopic (i.e., EPA-centric) with little recognition or
inclusion of ideas from other U.S. agencies or international agencies that have worked on ecological
1
-------
problems of national and international scope. The EPA should explore relevant activities of other U.S.
agencies that are involved in resource management and have mandates related to the EPA's, including
the National Oceanic and Atmospheric Administration (NOAA) and U.S. Forest Service (USFS), and
collaborate with these agencies on the integration of ecological risk assessment and decision-making.
The EPA should also carefully review related activities occurring in other countries, including Canada,
Australia, China and the European Union as well as related work undertaken by nongovernmental
organizations and U.S. States. Although other jurisdictions look at ecological risks in different ways, the
EPA should take advantage of international advancements in the science of ecological risk assessment.
The Plan would also benefit from additional input from social scientists. Their perspectives on issues
such as sustainability, adaptive management, communication and environmental justice would be very
helpful.
In developing state of the art approaches for ecological risk assessment, the EPA should incorporate
probabilistic quantitative approaches such as Bayesian methods. In general, the SAB recommends
development of case studies to illustrate these new approaches. The SAB also finds that the results of
ecological risk assessments generally lack an interpretation of how humans will be affected. Ecological
and human health risk assessments have the potential to provide complementary information that can be
used in an integrated "holistic" approach to risk assessment.
The Plan should explicitly address the importance of the problem formulation stage of ecological risk
assessment in: (1) clearly articulating how information from ecological risk assessments will inform
agency decisions; (2) considering constraints; and (3) ensuring a systems-level approach. Prior to and
during problem formulation, an open dialogue among scientists, risk assessors, risk managers, decision-
makers, and stakeholders is essential.
In its presentation to the SAB, the EPA also requested advice concerning prioritization of the initiatives
in the Plan. The SAB finds that all six of the science policy initiatives proposed by the EPA are critical
to the long-term success of the Plan, but three of them could provide immediate benefits in a shorter
period of time than the others. The SAB recommends that the EPA address the following three
initiatives first: (1) use of weight-of-evidence approaches in ecological risk assessments; (2)
communication of ecological assessment issues and results to decision-makers; and (3) incorporation of
ecosystem services into ecological risk assessment methods. These three initiatives have not been
ranked by the SAB according to priority. The other science policy initiatives in the Plan are equally
important and represent key beneficial long-term changes in the culture and approach to ecological risk
assessment.
Importance of Developing an Integrated Assessment Approach
The Plan calls for developing a systems approach to ecological assessment. This approach would
include multiple media and endpoints and integration of different types of assessments. The EPA asked
the SAB to comment on how guidance for an integrated approach might contribute to better decision-
making.
In general, the SAB finds that EPA decision-makers would benefit from using an ecological risk
assessment approach that combines multiple assessment types and integrates multiple and varied
assessment activities across the agency. EPA scientists presented a good preliminary framework for such
an approach to the SAB. This preliminary framework should be further developed.
2
-------
A number of key issues should be addressed in implementing the integrated risk assessment approach.
First, successful implementation of the integrated assessment framework will hinge on a WOE
determination, and additional guidance on weighing and integrating multiple lines of evidence is needed
by EPA risk assessors and managers. Second, implementation of an integrated framework will hinge on
the integrity of the data and models used in each of the individual ecological assessment types, and it is
not clear whether sufficient data (both quantity and quality) are available to fully implement the
framework. Third, in developing the framework, the EPA should address the issues of appropriate
spatial and temporal scales of assessments and the cumulative effects of chemical, physical and
biological stressors. In this regard, the EPA should be mindful of multiple stressor research that is being
conducted outside of the agency. Fourth, the SAB recommends that each of the assessment processes in
the framework be considered in the context of changing climate, and that ecosystem services endpoints
be explicitly incorporated. Finally, as discussed in section 3.2.3 of this report, the SAB strongly
encourages the EPA to develop and apply adverse outcome pathway and adaptive management
approaches as part of the framework.
Use of a Weight-of-Evidence Approach in Ecological Risk Assessments
The Plan calls for developing guidance for weighing multiple lines of evidence (LOE) in ecological risk
assessments. The SAB was asked to comment on the scientific merit and limitations of using a WOE
approach in decision-making and to offer advice on weighing lines of evidence.
The SAB strongly supports development of guidance for weighing and integrating multiple LOE in
ecological risk assessments. The scientific merit of using a WOE approach is clear as evidenced in the
large number of scientific publications on this subject and in the consistent and continuing use of WOE
in ecological risk assessment. A well-developed WOE approach would enable risk assessors to assign
quantitative weights to results from different studies (with associated estimates of uncertainty) and to
combine them into an assessment of defined risk. A scientifically rigorous WOE approach must rely
primarily on statistically-based decision points rather than best professional judgment but
implementation of this approach will not be possible without EPA WOE guidance that is program-
specific and ideally provides structured decision-making frameworks. Useful statistically-based
approaches, including meta-analysis, have been developed for weighing multiple lines of evidence, and
they would provide a solid basis for EPA guidance that could be structured to address the needs of
agency programs. However, significant work will be required to synthesize the science and develop this
guidance.
There are a number of challenges inherent in using WOE in ecological risk assessments for decision-
making. The EPA has used a WOE approach to conduct human health risk assessments (e.g., to evaluate
toxicological data and assess carcinogens). However, it seems unlikely that ecological risk assessments
are as amenable to formalization as human cancer risk assessments, which have discrete or common
endpoints (i.e., mortality and morbidity) that can be compared between studies. Many ecological risk
assessments are inherently unique, and a high degree of flexibility to address the nuances associated
with a particular assessment will remain desirable for the foreseeable future. WOE approaches have
often been based on best professional judgment and have varied widely in their scientific rigor and
statistical credibility. Therefore, a consistent approach should be developed to interpret LOE and WOE
in ecological risk assessments. In particular, the SAB recommends development of guidance, with
associated case studies, on the use of statistical methods such as Bayesian analysis and causal
argumentation to develop hypotheses or risk questions focused on causal relationships and WOE. The
case studies should cross the different EPA regulatory programs in which WOE is used.
3
-------
In general, the SAB recommends that a comprehensive set of LOE be utilized in the WOE process. The
LOE should adequately characterize physical, chemical and biological conditions. This will ensure a
cumulative evaluation that considers commonly occurring stressors such as habitat, water flow and
nutrients. The weight given to particular LOE is likely to be very case-specific, and quality of the data
underlying a particular LOE should factor into the assigned weights. LOE that are clearly linked to
ecological population or community attributes should receive greater weights than those that are not.
Communication of Ecological Assessment Issues and Results to Decision-Makers and
Stakeholders
The Plan calls for development of methods for better communication of ecological assessment issues
and results to decision-makers and stakeholders. The SAB was asked to: (1) comment on whether the
project developed by an RAF Communication Technical Panel is an appropriate approach to address this
issue; and (2) provide observations on why ecological risk assessment has or has not been well
incorporated into decision-making.
The one-page RAF Communication Technical Panel project description indicates that the EPA intends
to: (1) gather data through surveys and interviews of EPA risk assessors and decision-makers to explore
how ecological risk assessments are used in agency decision-making; and (2) develop guidance for risk
assessors and decision-makers on how to better communicate and enable the use of ecological risk
assessment information. The EPA has chosen to limit the scope of this project to improving
communication between ecological risk assessment practitioners and decision-makers within the agency.
The SAB finds that, while the EPA's proposed project could lead to short-term improvements in the
agency's use of ecological risk assessments, it is a preliminary approach that will not provide all of the
information needed to develop better communication methods. As further discussed in section 3.4.1 of
this report, a much broader study is needed to achieve the RAF's stated goals of promoting full use of
ecological risk assessment across EPA programs and meeting managers' needs for useable ecological
risk assessment information to support decisions.
The SAB recommends that the RAF Communication Technical Panel project be completed and used as
the basis for a broader study that addresses how effective communication can be incorporated
throughout all phases of the assessment and management decision process. This broader study should
address better communication performance by all participants in the process including EPA risk
assessors and other scientists, managers and key stakeholders. Communication should be elevated to an
essential core activity of the ecological risk assessment process with its own performance standards and
success criteria. To provide a better basis for developing performance standards and guidance, the EPA
should conduct a systematic evaluation of the challenges and opportunities for better communication
across a range of different decision types. In undertaking a broader study, the EPA should also recognize
that communication strategies may vary with decision types and target audiences.
The RAF Communication Technical Panel project should focus on identifying guidance and support
tools that can be adapted to the needs of specific regulatory applications and a range of intended
audiences. The SAB strongly recommends that the EPA take advantage of a recent SAB study, Science
Integration for Decision Making at the U.S. Environmental Protection Agency, which provides
additional information about the interface between risk assessors and managers, and recommendations
to address the challenge of bringing science to bear on agency decisions.
4
-------
The SAB also recommends that, in developing guidance and tools for improved communication, the
EPA consider incorporating external technical input from social scientists, ethicists, marketing
professionals and media specialists who have a good understanding of risk communication and broader
environmental concerns. The SAB notes that ineffective communication is one reason why ecological
risk assessment output has not had an optimal impact on decisions. A robust communication process that
leads to a clear understanding of the context for decisions, and how various data or ecological risk
analysis improve or support decisions at hand, will lead to better alignment of assessments and
decisions.
Incorporation of Ecosystem Services into Ecological Risk Assessment Methods
The Plan calls for incorporation of ecosystem services endpoints into ecological risk assessment
methods. The SAB was asked to comment on whether a project developed by the RAF Ecosystem
Services Endpoints Technical Panel captures the full range of opportunities to incorporate ecosystem
services into the EPA's ecological risk assessment methods.
The Plan does not provide sufficient information to indicate how the proposed ecosystem services
project will be implemented. It states that an RAF Ecosystem Services Endpoint Technical Panel has
been created and expects to produce case studies and guidance on how to relate ecological endpoints to
ecosystem services. The Technical Panel's one-page project description indicates that the following
products will be developed: (1) a white paper interpreting conventional measurement and assessment
endpoints in the context of ecosystem services; (2) a case study(s) of applying the ecosystem services
concept in ecological risk assessment; and (3) an addendum to the RAF Generic Ecosystem Assessment
Endpoints (GEAE) guidance.
More information is needed to evaluate potential success of the Ecosystem Services Endpoint Technical
Panel project, although the material presented to the SAB by EPA staff indicates that considerable
progress has been made. The SAB encourages the EPA to complete the ecosystem services white paper
and recommends that the authors submit a shorter version to a peer-reviewed publication to make it
available to a wider audience. The decision to incorporate case studies into the white paper will increase
its value. The SAB also recommends that the white paper include an evaluation of the use of ecosystem
services endpoints throughout the entire risk assessment and risk management decision process. The
ecosystem services white paper should also include a description of how the concept of ecosystem
services is being used in other agencies and other countries. Many natural resources agencies (e.g., U.S.
Forest Service) routinely consider ecosystem services in their management strategies.
Neither the GEAE guidance nor the Colloquium Report captures the full range of concepts embodied by
the term ecosystem services. The EPA should incorporate more current ecosystem services concepts and
definitions into the GEAE document, replacing older terminology and meaning. The EPA should also
consider updating the original GEAE guidance rather than producing an addendum. If both an original
and an addendum are in circulation, a practitioner could mistakenly use only one, which could lead to
errors. The SAB also recommends that the RAF Ecosystem Services Endpoint Technical Panel consider
looking at the other policy focus areas in the Ecological Assessment Action Plan to determine where
ecosystem services information should be incorporated into those other topics.
5
-------
Use of Adaptive Management for Testing and Revising Risk Management Actions
The Plan calls for use of adaptive management as a tool to methodically improve risk management
decisions. The SAB was asked to comment on how adaptive management approaches can be developed
to provide optimal value for EPA programs.
The six elements of adaptive management identified by the NRC and described in section 6 of the
Colloquium Report are consistent with the EPA's existing ecological risk assessment framework. The
SAB recommends that the EPA take action to implement the goal of incorporating adaptive management
principles into the framework. However, implementing this goal may be difficult given the complexity
of consistent and continuous ecosystem monitoring and evaluation over appropriate time scales, and the
scope and magnitude of resource limitations currently facing the EPA. Incorporation of the adaptive
management approach into the risk assessment framework may be particularly useful for addressing
certain technical concerns (e.g., climate change) or management issues and decisions facing EPA
programs (e.g., Office of Pesticide Programs, Superfund, Office of Air). However, the adaptive
management approach may not be appropriate for all risk assessment applications in all EPA programs.
Monitoring and evaluation are key elements of adaptive management but they are not always fully
implemented and are often targeted for elimination when budgets are tight. However, monitoring and
evaluation provide an important means of assessing uncertainty and measuring the efficacy of
mitigation, and should be incorporated into the EPA's risk management activities. Further, monitoring
data form the underpinnings of an understanding of past trends and predictions of future conditions and
thus are essential for adaptive strategies. The SAB notes that the use of Bayesian approaches, causal
argumentation and probabilistic risk assessment would facilitate the development of hypotheses that
could be evaluated as part of the adaptive management process.
Strengthening the EPA's Ecological Protection Goals
The Plan indicates that there is little consensus in the EPA about goals for the protection of ecological
systems. The SAB was asked to comment on aspects of ecological risk assessment science that make
ecological risk information difficult to communicate and use in decision-making. The SAB was also
asked to provide recommendations to strengthen the EPA's ecological protection goals.
The EPA should develop stronger (i.e., clearly articulated, elucidated, and scientifically rigorous)
ecological protection goals. This need is particularly urgent because ecological protection goals are
likely to provide important context and guidance for the development of ecological assessment
approaches used by the agency. Clear articulation of ecological protection goals will lead to better
definition of agency decisions and an improved understanding of the ecological assessments needed to
support those decisions. Depending upon the nature of decisions to be made, different branches of
ecosystem, social, and behavioral sciences will have to be engaged. Improved communication will be
driven by the needs of the decision-making process. To strengthen EPA's ecological protection goals,
the agency's ecological scientists will need to develop information and perspectives that will enable
them to communicate more effectively with decision-makers and the public. Clearly, ecosystem function
and services are bound tightly to human health. In some cases, ecosystem function may conflict with
some aspects of human health (e.g., diseases transmitted by mosquitos; the presence of top predators
such as cougars), but there cannot be good human health without good ecosystem function. Ecosystem
function is therefore an input that should be considered as part of a human health risk assessment.
Incorporation of ecosystem services into the ecological assessment process will strengthen the EPA's
6
-------
ecological protection goals. Environmental justice is also a useful platform to highlight the relationship
between ecosystem condition and the health of vulnerable human populations.
Many of the risk assessment practices recommended in the 2007 SAB report on advancing the science
and application of ecological risk assessment would lead to the development of stronger ecological
protection goals at the EPA and bring about improved protection of ecosystem structure and function. In
particular, scale (both in time and space) should be explicitly considered in the problem formulation
stage of ecological risk assessments. Ecological risk assessments should link biomarkers of exposure to
biomarkers of effect, and post-remedial assessments and adaptive management programs should be
incorporated into the risk assessment and management process. Environmental cleanup success stories
should be developed and used to enhance the communication process between risk managers, assessors
and environmental scientists. Examples illustrating approaches that have been less successful may also
be useful. An overarching recommendation of the SAB is that ecological risk assessment teams should
use better communication techniques to educate managers, policy makers and the general public.
7
-------
2. INTRODUCTION
The Risk Assessment Forum (RAF) in the EPA Office of the Science Advisor (OSA) has developed a
draft Ecological Assessment Action Plan titled Integrating Ecological Assessment and Decision-Making
at EPA: 2011 RAF Ecological Assessment Action Plan (August 11, 2011). The draft Ecological
Assessment Action Plan (hereafter referred to as the "Plan") sets forth science policy and technical
practice initiatives to improve the quality, scope and application of the EPA's ecological assessments.
The Plan was developed in response to a 2007 EPA Science Advisory Board (SAB) Report, Advice to
EPA on Advancing the Science and Application of Ecological Risk Assessment in Environmental
Decision-Making (U.S. EPA SAB 2007) and a 2009 National Research Council (NRC) report, Science
and Decisions: Advancing Risk Assessment (NRC 2009). These reports put forward recommendations to
improve the application of ecological risk assessment in environmental decision-making. To address the
recommendations in these reports, the RAF convened an intra-agency colloquium that included
ecologists from across the EPA. Colloquium participants reviewed the EPA's ecological risk assessment
practices and guidance in light of the SAB and NRC advice and recommended actions to improve
ecological risk assessment in the agency. The initiatives in the EPA's Plan address recommendations in
the colloquium report, Integrating Ecological Assessment and Decision-making at EPA: A Path
Forward (U.S. EPA 2010b) (hereafter referred to as the Colloquium Report).
The Plan sets forth six science policy initiatives aimed at improving the agency's ecological assessments
and better informing decision-makers. These initiatives focus on: (1) developing a systems approach to
ecological assessment that integrates multiple media, endpoints and types of assessments; (2) developing
weight-of-evidence as an option for inference in ecological assessments; (3) improving communication
of ecological assessment issues and results; (4) incorporating ecosystem services into ecological risk
assessment methods; (5) using adaptive management for testing and revising risk management actions;
and (6) strengthening the EPA's ecological protection goals. The Plan also lists other specific initiatives
aimed at incrementally improving ecological risk assessment practice. These technical practice
initiatives focus on the need for improvements in: training; quality assurance; guidance to address
multiple and specific receptors; life cycle evaluations; uncertainty analysis; and access to information.
The EPA requested that the SAB review the Plan as well as descriptions of two projects undertaken by
RAF technical panels (the Communication Technical Panel and the Ecosystem Services Endpoint
Technical Panel) and provide advice on the technical merit and implementation of the proposed
initiatives. The background documents provided included the 2007 SAB report on ecological risk
assessment and the EPA Colloquium Report. In response to the OSA's request, the SAB Ecological
Processes and Effects Committee, augmented with experts who developed the previous SAB ecological
risk assessment report, held public teleconferences on February 22-23, 2012 to review the Plan. The
EPA's charge questions (provided in Appendix A) focus on the major science policy initiatives in the
Plan. The SAB was specifically asked to comment on: the technical merit of the initiatives in the Plan;
how the integrated assessment approach might contribute to better decision-making; the scientific merit
and limitations of the weight-of-evidence approach; the merit of proposed RAF communication and
ecosystem services projects; how adaptive management approaches can be developed to provide value
for the EPA; and how the EPA's ecological protection goals could be strengthened. This SAB report
provides the consensus advice and recommendations of the Committee.
8
-------
3. RESPONSES TO THE EPA'S CHARGE QUESTIONS
3.1. Overall Technical Merit of the Proposed Science Policy and Technical Practice Initiatives
Charge Question 1: The RAF Ecological Assessment Action Plan proposes six high priority
overarching science policy initiatives and seven specific technical practice initiatives to improve
the quality, scope and application of EPA 's ecological assessments. Please comment on whether
the initiatives proposed in the Plan are: (a) responsive to SAB and NRC recommendations; and (b)
reflect the most important set of activities needed to address the key scientific and technical
challenges for advancing the application of ecological risk assessment in environmental decision-
making. Please also consider whether there are other key science policy or technical practice
initiatives that should be consideredfor inclusion in the Plan.
The SAB strongly supports the investment of resources to strengthen the EPA's ecological assessments.
The six major science policy initiatives and the other technical practice initiatives recommended in the
Plan follow logically from the EPA Colloquium Report (U.S. EPA 2010b), are responsive to previous
SAB and NRC recommendations, and are very reasonable. The SAB has not identified any significant
omissions in the proposed set of initiatives but additional information should be included in the Plan to
indicate how it will be used and implemented by the agency. Many EPA decisions focus on human
health issues. However, ecological risk assessments are also used by the agency to make decisions
required under various statutes. These assessments support decisions involving product health and safety
evaluation (e.g., under the Federal Insecticide, Fungicide, and Rodenticide Act and Toxic Substances
Control Act); contaminated site management (e.g., under the Comprehensive Environmental Response,
Compensation, and Liability Act); and resource management (e.g., under the Clean Water Act).The
proposed initiatives in the Plan address subjects that have the potential to greatly improve environmental
assessment and decision-making at the EPA. The SAB commends the EPA for creating a Plan that is a
solid starting point for the agency's effort to integrate ecological risk assessment and decision-making.
However, after examining the relevant documents, observing the presentations made by EPA scientists
about the Plan, and deliberating on the response to the charge question, the SAB has raised a number of
issues to be addressed. The SAB urges the EPA to consider the following major recommendations for
refining and developing the Plan.
Recommendations
• The EPA is encouraged to carefully review related activities occurring in other countries, including
Canada, Australia, China and the European Union, U.S. States, and related work of
nongovernmental organizations. The Plan appears to be very myopic, that is, overly "EPA-centric."
On some important aspects of ecological risk assessment, it appears that the U.S., and the EPA in
particular, has lost its former leadership role, which is unfortunate. In any event, the agency can
benefit from attention to advancements in other countries. Although other jurisdictions look at
ecological risks in different ways, The EPA should take advantage of international advancements in
the science of ecological risk assessment.
• The EPA should explore relevant activities of other U.S. agencies that are involved in resource
management and have mandates related to the EPA's, including the National Oceanic and
Atmospheric Administration (NOAA),U.S. Forest Service (USFS) and U.S. Fish and Wildlife
Service (USFWS), and collaborate with those agencies on the integration of ecological risk
assessment and decision-making. In this regard, the USFS makes a number of "threat" assessments
9
-------
concerning fires, invasive species and climate change and federal agencies that are natural resource
trustees conduct natural resource damage assessments. Fishery managers directly manage specific
ecosystem services using probabilistic tools.
• The recommendations in the Plan are laudable. Due to imposed page limitations (four pages), the
RAF was limited in the level of detail the Plan could convey. More detailed information should be
developed to explain how the recommendations in the Plan will be achieved. The EPA has indicated
that this information will be provided as individual project plans concerning policy issues identified
in the Plan are developed. There appears to be heavy reliance on subsequent workshops to develop
implementation strategies. This generates concern that implementation discussions could lag behind
the evolution of science underlying ecological assessments and constantly emerging new ecological
problems. However, the SAB recognizes that the issues being addressed by the EPA and the RAF
are enormous in complexity and importance.
• The SAB recommends that the EPA incorporate additional input from social scientists as it
elaborates the current brief Plan into more detailed project plans. Social scientists are admittedly rare
in the EPA, but the Plan would benefit from their perspectives on issues such as sustainability,
adaptive management, communication and environmental justice. Failure to sufficiently incorporate
social sciences overlooks the reality that ecological risk assessments occur within a given social,
economic and political context and will be most useful when skillfully aligned with community
values and/or management objectives.
• Although the issue of environmental justice is not an explicit part of the Plan, highlighting the ways
that ecological risk assessments can support the Agency priorities related to environmental justice
would illustrate the cross-cutting impact of the Plan. For example, an evaluation of the loss of valued
ecosystem services to poor and minority communities would contribute to an assessment of the
vulnerability of these communities to impacts from environmental degradation. Such an evaluation
could be facilitated by using cumulative risk assessments that explicitly account for both background
and source-related exposures and their effects on ecosystem services.
• Incorporation of case studies and success stories is recommended as an approach to increasing the
utility of guidance documents. Examples illustrating approaches that have been less successful may
also be useful. The SAB also notes that, as recognized in the EPA Colloquium Report, development
of new guidance documents should include planning for training.
• In developing state of the art approaches for ecological risk assessments, the agency should
incorporate probabilistic quantitative approaches such as Bayesian methods. The SAB notes that
some EPA researchers are already doing work in this area (e.g., Carriger and Barron 2011).
Although Bayesian methods can be very useful, it is important to be mindful of their assumptions
and limitations. One limitation is that Bayesian methods require prior selection of appropriate
distributions for unknown parameters. This can be a concern when sample sizes are small.
• The SAB recommends that, as part of a broad communication strategy, the EPA consider the use of
community-based participatory research approaches that engage stakeholders throughout the entire
risk assessment process. This could serve as a way to incorporate traditional knowledge from
indigenous peoples.
10
-------
• The parameters of the decision-making process should be considered in determining the kind of
information needed from an ecological assessment (NRC 1996; Willis et al. 2004; Gregory et al.,
2012). The Plan should explicitly address the importance of the problem formulation stage of
ecological risk assessment in: (1) clearly articulating how information from ecological risk
assessments will inform agency decisions; (2) considering constraints; and (3) ensuring a systems-
level approach. Prior to and during problem formulation, an open dialogue among scientists, risk
assessors, risk managers, decision-makers and stakeholders is essential (likewise, broad engagement
also sets the stage for effective communication of results). One point highlighted by the 2007 SAB
workshop on ecological risk assessment was that ecological risk assessments have been most
effective when clear management goals were collaboratively developed and incorporated into
problem formulation. Review at the problem formulation stage would be an excellent strategy to
ensure that systems approaches are used. In particular, review by ecologists would make it more
likely that the ecological risk assessment sufficiently addressed ecological end points and protected
ecosystem function and services.
• In its presentation to the SAB, the EPA also requested advice concerning prioritization of the
initiatives in the Plan. The SAB finds that all six of the science policy initiatives proposed by the
EPA are critical to the long-term success of the Plan, but three of them could provide immediate
benefits in a shorter period of time than the others and should be addressed at the outset. These are
(in the order provided in the Plan):
- Use of weight-of-evidence (WOE) approaches in ecological risk assessments.
- Communication of ecological assessment issues and results to decision-makers.
- Incorporation of ecosystem services into ecological risk assessment methods.
The other science policy initiatives in the Plan are equally important and represent key beneficial long-
term changes in the culture and approach to ecological risk assessment.
3.2. Importance of Developing an Integrated Assessment Approach
Charge Question 2. The RAF Action Plan proposes that EPA develop a systems approach to
ecological assessments that includes multiple media and endpoints as well as integration of
different types of assessments as described by Cormier and Suter in A Framework for Fully
Integrating Environmental Assessment, Environmental Management 42:543-556, and in
chapter 3 of the EPA colloquium report, Integrating Ecological Assessment and Decision-
Making at EPA: A Path Forward. The framework focuses on resolving environmental
problems by integrating different types of assessments: (1) condition assessments to detect
chemical, physical and biological impairments; (2) causal pathway assessments to determine
causes and identify their sources; (3) predictive assessments to estimate environmental,
economic, and societal risks, and benefits associated with different possible management
actions; and (4) outcome assessments to evaluate the results of the decisions of an integrative
assessment. Please comment on how guidance for an approach to assessment that integrates
different media and endpoints and different types of assessments might contribute to better
decision-making (e.g., assessment of complex issues, cumulative risk assessment and
sustainability analysis).
11
-------
The SAB has considered two important aspects of this charge question: 1) how an integrated assessment
approach might contribute to improved decision-making; and 2) issues that should be addressed in
implementing an integrated framework. The SAB finds that Charge Question 2 (developing an
integrated approach) and Charge Question 3 (use of a weight-of-evidence approach) are closely related
because an integrated assessment approach involves weighing and integrating lines of evidence from
different kinds of assessment using different endpoints. As further discussed below, and in the response
to Charge Question 3, the primary benefit of an approach for weighing and integrating different lines of
evidence is that it would enable decision-makers to address complex environmental challenges by
combining the results of different studies into an assessment of assigned risk. In general, the SAB finds
that the EPA would clearly benefit from developing an ecological risk assessment approach that
combines multiple assessment types and integrates multiple and varied assessment activities across the
agency. The framework proposed by Cormier and Suter (2008) provides an approach for integrating
different components of the risk assessment process across the EPA. It is a good preliminary framework
but it lacks operational detail. The framework should be populated and further developed by providing
detailed operational information needed for implementation. Issues to be addressed in implementing the
framework are discussed in section 3.2.1 of this report. The key issues discussed below should be
addressed in order to develop a framework that will enable decision-makers to more effectively manage
complex environmental problems.
3.2.1. Weight-of-Evidence Determination and Integrity of Data and Models
Successful implementation of the integrated assessment framework will hinge on a WOE determination,
which is addressed in the response to Charge Question 3 in this report. Use of a WOE approach in
ecological risk assessment has clear scientific merit but additional guidance on its use and application is
required. Weight-of-evidence approaches integrate various types of lines of evidence (e.g., chemistry,
bioassay, and field studies) to provide information needed to draw conclusions and make decisions.
Tiered approaches have been proposed for sequential analysis of lines of evidence in ecological risk
assessments (e.g., Fairbrother 2003). The "bottom-line" is that integration of different lines of evidence
(LOE) is essential given that "today's environmental challenges are increasingly subtle and complex"
(Anastas 2012), particularly so given the reality of global climate change (cf. U.S. Fish and Wildlife
Service and National Oceanic and Atmospheric Administration 2012). Clarification of how WOE is
implemented in ecological risk assessment is likely to contribute to better decision-making across the
EPA. Bayesian approaches, which are especially useful in situations where data are sparse, are
recommended.
Full implementation of an integrated framework will hinge on the integrity of the data and models used
in each of the individual assessment types; it is not altogether clear whether there are sufficient data
(both quantity and quality) for different types of cases to fully implement this framework. It will be
important to ensure that the models and data used in each assessment type address the appropriate spatial
and temporal scales of the problem. Principles of landscape ecology and environmental heterogeneity
must be incorporated explicitly into this framework, but it is not clear that the proposed framework is
dynamic enough to do this. Further, the three types of stressors (chemical, physical, biological)
identified in the "Condition Assessment" compartment of the EPA's preliminary framework need to be
considered in terms of cumulative effects, both direct and indirect (e.g., trophic cascades). The
combined use of field data along with bench top data and models is recommended for that purpose.
Guidance and case studies need to consider both data-rich and data-poor situations to ensure that the
approach is protective of populations and communities of organisms and, where applicable, of
endangered species.
12
-------
Recommendations
• Models and data used for each type of assessment in the integrated framework should address the
appropriate spatial and temporal scales of problems.
• In the integrated framework, the EPA should consider the cumulative effects, both direct and indirect
(e.g., trophic cascades), of the three types of stressors (chemical, physical, biological) in "Condition
Assessments."
• The guidance and case studies developed for the integrated framework should consider both data-
rich and data-poor situations to ensure that the approach is protective of populations and
communities of organisms and, where applicable, of endangered species.
3.2.2. Consideration of Climate Change and Ecosystem Services
Each of the assessment processes in the proposed integrated framework must be considered in the
context of changing climate, particularly with respect to increased variability. Many of the assessment
endpoints used for condition assessments are subject to change as sensitive species are eliminated and
replaced by those less sensitive. Not only will populations and ecosystems change, but as temperatures
increase, so will sensitivity to other stressors, including chemicals. The combined effects of increased
temperatures and changing precipitation patterns is a special concern in the context of responses to a
multiple stressor environment (see the special issue of the Journal of the North American Benthological
Society 2010, 29(4) on the topic of bioassessment under a changing climate). The SAB is concerned
about exclusive reliance on the use of indices for such assessments because this may result in loss of
critical data (Chapman 2011; Green and Chapman 2011).
Consideration of ecosystem services endpoints is implicit but not explicit in the EPA's proposed
integrated assessment approach; explicit assessment end-points are needed. It is unclear why the EPA
Office of Research and Development (ORD) strategy for incorporation of ecosystem services into
assessments has not been mentioned in the Plan development process. Incorporation of ecosystem
services is an important aspect of the integrated assessment approach and previous SAB reports on the
EPA Ecosystem Services Research Program (U.S. EPA SAB 2008, 2009a) identified some of the
challenges that face the EPA with regard to integrating across agency programs. The SAB report,
Valuing the Protection of Ecological Systems and Services (U.S. EPA SAB 2009b) discusses the process
of identifying ecosystem services and developing ecosystem production functions that describe how
ecological responses affect the provision of services.
Recommendations
• Each of the assessment processes in the proposed integrated framework should be considered in
the context of changing climate.
• Ecosystem services endpoints should be explicitly incorporated into the integrated assessment
framework.
13
-------
3.2.3. Use of Adverse Outcome Pathway and Adaptive Management Approaches
The development and application of adverse outcome pathway (AOP) approaches is strongly
recommended as part of the integrative framework. An AOP is "a conceptual construct that portrays
existing knowledge concerning the linkage between a direct molecular initiating event and an adverse
outcome at a biological level of organization relevant to risk assessment (e.g., lethality; impaired
development; impaired reproduction; cancer; changes in population structure, recruitment, or extinction)
(Ankley et al. 2009). When developing AOPs, it is important to identify specific sublethal measures
associated with linkages between the molecular initiation events. Ankley et al. (2009) refer to the
initiating event as "anchor 1" and adverse outcomes at the individual and population levels of biological
organization as "anchor 2."
The integrated assessment framework could address the long-term perspective and "cultural" changes
that are needed within the EPA to implement the framework. This can be viewed in the context of
adaptive management. Adaptive management requires a long-term commitment to a specific goal, which
in this case is the maintenance of specific ecosystem services. Long-term management of ecological
systems will require a planning time frame that enables consideration of the dynamics of those systems,
often years and decades in scale. Management of major systems such as the Great Lakes and estuaries
such as the Puget Sound and Chesapeake are examples of systems that require such long-term
commitments. Mining and energy extraction can require a century or more of management from the
exploration of a site to the final remediation and closing.
For large-scale systems managed at time scales equal to or longer than a decade, climate change must be
considered in the risk assessment and adaptive management framework. Consideration of such persistent
multi-decadal changes to ecological systems has a number of implications: (1) managing to some ideal
reference or baseline state is unrealistic; (2) ecosystem services are the entity to be preserved, although
the species that provide those services are likely to change; (3) a long-term monitoring and response
system, innate to adaptive management systems, will have to become the norm across the EPA, and
National Pollution Elimination Discharge System (NPDES) permits, Total maximum daily load
(TMDL) guidance and restoration activities, and other long-term programs must be managed within the
context of climate change to meet the goals of legislation; and (4) adaptive management in the context
of climate changes means placing the most current science and techniques at the Regional and Program
level so that improved processes become operational. Delays in implementing management under
climate change simply increase the uncertainty that implemented regulations are effective in managing
vital ecosystem services.
Recommendations
• The development and application of AOP approaches is strongly encouraged as part of the
integrated assessment framework.
• Adaptive management approaches could be implemented in an integrated assessment framework
but it will be necessary for the EPA to adopt planning time frames for long-term management of
ecological systems. The time frames adopted should enable consideration of the dynamics of
those systems under changing climate.
14
-------
3.2.4. Consideration of Multiple Stressors
The SAB notes that the risks of multiple stressors to multiple endpoints have been calculated for
landscapes for over 15 years by groups around the world (e.g., Walker et al. 2001; Moraes and Molander
2004; Luxon and Landis 2005; Pollino et al. 2007; Gibbs 2007; Apitz 2012; Bartolo et al. 2012;
Glendining and Pollino 2012; Chen et al. 2012). This work has been poorly acknowledged throughout
the Plan. As part of a research effort in the U.S. Forest Service, Bayesian networks increasingly are
showing their worth as a tool for both assessing impacts and adaptive management (Marcot et al. 2001,
2006a,b; Nyberg et al. 2006). Drawing on this work and on the regional risk assessment model, Ayre
and Landis (in press) have demonstrated that risk assessment can be successfully combined with the
Bayesian approaches in a forest management context. It is critical that the EPA take advantage of this
ongoing research conducted outside of the agency to improve the risk assessment process.
Recommendation
• To improve the risk assessment process, the EPA should be mindful of multiple stressor research
conducted outside of the agency. In particular, the SAB notes that Bayesian networks are
increasingly showing their worth as tools for assessing impacts and adaptive management.
3.2.5. Importance of Social Science
The SAB notes that engagement of social scientists is essential for successfully implementing ecological
risk assessment. A leading risk assessment journal, Risk Analysis, publishes a large number of articles in
the social sciences addressing such topics as stakeholder communication, risk perception, decision-
making and expert elicitation. The EPA currently employs few experts in these areas. Having access to
this additional expertise would facilitate the EPA's communication of ecological problem(s) and risk
assessment results to informed professionals, stakeholders and the general public.
The application of social science tools can be best illustrated by developing a series of case studies
illustrating the utility of the social sciences in environmental assessment and adaptive management. A
similar approach was used to create and critique case studies for the EPA's Framework for Ecological
Risk Assessment (U.S. EPA 1992).
Recommendation
• The SAB recommends developing a series of case studies to illustrate the utility of the social
sciences in environmental assessment and adaptive management.
3.3. Use of a Weight-of-Evidence Approach in Ecological Risk Assessments
Charge Question 3. Although ecological risk assessments often involve multiple lines of
evidence, no guidance exists on how to weigh those lines of evidence to make inferences. The
RAF Action Plan proposes that EPA develop such guidance. Please comment on the
scientific merit and limitations of using a weight-of-evidence approach in decision-making
and offer any guidance on weighing ecological risk assessment lines of evidence.
WOE is an approach to evaluating and integrating multiple sources of evidence. A uniform definition of
WOE does not exist. The following WOE definition of Burton et al. (2002b) is perhaps the best one at
15
-------
present because it does not unduly limit the concept: "a process used in environmental assessment to
evaluate multiple LOE concerning ecological condition." The SAB has also previously described
ecological risk assessment as "a process, not just a technique" (U.S. EPA SAB 2007).
3.3.1. Scientific Merit of Using a Weight-of-Evidence Approach
The SAB strongly supports development of EPA guidance for weighing and integrating multiple lines of
evidence to make inferences in ecological risk assessments. The scientific merit of WOE is clear. There
is a large body of scientific literature supporting its use and as discussed in sections 3.3.3 and 3.3.4 of
this report, useful statistically-based approaches such as meta-analysis provide tools for practitioners to
systematically draw together results of numerous studies to address a problem. These approaches would
provide a solid basis for EPA guidance that could be structured to address the needs of agency programs.
However, significant work will be required to synthesize the science and develop this guidance. A series
of articles on WOE were published in the journal Human and Ecological Risk Assessment. Reviews of
WOE approaches (e.g., Burton et al. 2002a; Weed 2005; Linkov et al. 2009) uniformly recommend its
use, particularly in ecological risk assessment, but also note the need for transparency and guidance in
its use. As previously noted, WOE includes a number of quantitative approaches including meta-
analysis. Meta-analysis has been used for several decades to combine data from multiple studies. It has
been used in a variety of situations (Glass 1976, 1977; Hedges and Olkin 1985; Rosenthal 2001).
Quantitative meta-analysis can include tendencies and error terms describing uncertainty (Neill 2012).
The EPA has recognized that "today's environmental challenges are increasingly subtle and complex,"
and that research must not be just inter-disciplinary but in fact trans-disciplinary, "combining
perspectives to form entirely new concepts and reach new levels of scientific understanding" (Anastas
2012). It has also been recognized both inside and outside of the EPA that the WOE approach has merit;
this merit has been affirmed by the SAB in previous advice to the agency (U.S. EPA SAB 2007).
However, a scientifically rigorous WOE approach must rely less on best professional judgment and
more on statistically-based decision points. This will not be possible without EPA WOE guidance that is
program specific and that ideally provides structured decision-making frameworks. Specific
recommendations are required regarding the use of WOE.
As discussed below, there can be much argument over how much weight to give certain LOE or how to
integrate the various LOE into a decision-making framework. As a better understanding arises
concerning multiple and complex stressors, and as these are related to life history parameters (see the
response to Charge Question 7), there should be less argument over the strongest LOE and the optimal
integration process.
3.3.2. Limitations of Using the Weight-of-Evidence Approach
Challenges inherent in using WOE
The challenges inherent in using WOE for decision-making are well known (Batley et al. 2002; Burton
et al. 2002a,b; Wenning et al. 2005). WOE approaches vary widely in their scientific rigor and statistical
credibility (Burton et al. 2002a). Consequently, they may not reduce uncertainty as they are meant to,
and may in fact confound effective decision-making. WOE depends to a certain extent on best
professional judgment, which varies depending on the professionals making judgments (e.g., Bay et al.
2007; Thompson et al. 2012). The EPA has identified best professional judgment as a source of
uncertainty (U.S. EPA 2010b), and lack of agreement among experts about WOE extends beyond the
16
-------
environmental sciences (Large and Nielssen 2008). Bay et al. (2007) suggest that uncertainty related to
the use of best professional judgment must be recognized in ecological risk assessment and will be less
important at the extremes (e.g., sites that are clearly contaminated and toxic, and those that are clearly
not) than between the extremes. They recommend three steps to reduce uncertainty in the integration and
interpretation of multiple LOE:
1. Key elements of the assessment strategy should be determined during the problem formulation
phase of the ecological risk assessment. This could involve developing: the relative weight of
each LOE; the method of combining multiple LOE using techniques such as scores, ranks and
logic frameworks; and criteria for determining the ecological risk assessment conclusions.
2. Guidance should be developed on the specific methodology/methodologies for measuring and
assessing each LOE.
3. Training, including guidance documents, should be provided for individuals interpreting both
individual LOE and the overall WOE.
WOE interpretations in the context of risk assessment
The term weight-of-evidence appears to have a variety of interpretations in the context of risk
assessment. It begins with the general idea that more than a single line of inquiry is desirable when
assessing risk. At issue is how to integrate and synthesize evidence from different studies. The studies
might not all measure the same thing (e.g., chemical responses, individual organism responses,
community responses).
A classic paper on causes of occupational diseases (Hill 1965) has been used to identify causality criteria
formulated in the context of potential carcinogens and disease (Rothman and Greeland 2005). These
criteria can also be interpreted in terms of ecological risk assessment and they underscore the fact that
there are many factors involved in trying to quantify the process of linking exposure to something (e.g.,
a toxicant, a management practice) and a resulting effect. The EPA has incorporated these useful
concepts into its stressor identification evaluation guidance (U.S. EPA 2000) and CADDIS (Causal
Analysis / Diagnosis Decision Information System) approach (U.S. EPA 2010a).
The EPA has used a WOE approach to conduct human health risk assessments (e.g., to evaluate
toxicological data and assess carcinogens) (U.S. EPA 2005). However, it seems unlikely that ecological
risk assessments are as amenable to formalization as human cancer risk assessments, which have
discrete or common endpoints (i.e., mortality and morbidity) that can be compared between studies.
Many ecological risk assessments are inherently unique, and a high degree of flexibility will remain
desirable for the foreseeable future to address the nuances associated with particular assessments. As
noted in the response to Charge Question 2, adverse outcome pathways (AOPs) should be a key
component of ecological risk assessment. The overall goal should be to protect populations and
communities, not individuals (an exception is the protection of individuals in an endangered species).
However, this goal must include protection of the ecosystem services provided. Thus, measurement
endpoints must include responses that affect or could potentially affect those services (e.g., cancers in
edible crabs that reduce consumer interest in this food source are indicative of anthropogenic ecosystem
degradation if the cancers are related to human activities).
The 2007 SAB report on advancing the science and application of ecological risk assessment (U.S. EPA
SAB 2007) provided a number of recommendations concerning use of the WOE approach. The SAB
17
-------
recommended: development of a consistent approach to interpreting LOE and WOE in ecological risk
assessments (both to reduce uncertainty and to facilitate the use of this information in decision-making);
use of Bayesian analysis and causal argumentation to develop hypotheses or risk questions focused on
causal relationships and WOE; and development of "case studies and/or standards of practice for
interpreting lines of evidence and weight-of-evidence with an emphasis on application in decision-
making." The SAB continues to advise the EPA to apply these recommendations as the Plan is
developed and implemented. The SAB also notes that the EPA guidance document on generic ecological
assessment endpoints (GEAEs) for ecological risk assessment (U.S. EPA 2003) states that "as the
GEAEs are applied to ecological risk assessments, the experiences should be documented and published
as case studies." The SAB supports the development of such case studies. In particular, we recommend
that the case studies focus on whether some LOE carried more weight than others, or whether they were
ignored or too difficult to interpret or use. This information will assist in future weighting of ecological
risk assessment LOE. Case studies should cut across different EPA regulatory programs in which WOE
is used and should emphasize statistics, not just best professional judgment, for decision-making. As
discussed below, the SAB particularly recommends that the EPA develop case studies illustrating the
use of Bayesian approaches. Such approaches will provide more flexibility and more convincing
outcomes than reliance on best professional judgment.
3.3.3. Guidance on Weighting Ecological Risk Assessment Lines of Evidence
WOE is an approach to evaluating and integrating multiple sources of evidence, rather than a single
technique. As such, WOE should follow certain principles, but not a particular recipe nor algorithm.
Any effort that applies WOE to reach conclusions should be completely transparent with regard to the
different sources of evidence considered and any qualitative (e.g., expert opinion) or quantitative
weighting schemes used. This point is discussed in the context of evaluating epidemiology data for
cancer risk assessment (Swaen and Amelsvoort 2009). Data quality and the reliability of different
studies should also be considered in a WOE approach. Applying an arbitrary weighting scheme without
a solid theoretical foundation to integrate different LOE into a single risk score may not actually
improve decision-making (Weed 2005).
A well developed WOE framework would enable ecological risk assessors to assign quantitative weights
to results from different studies (with associated estimates of uncertainty) and to combine them into an
assessment of a defined risk. Thus far, this has been largely done in epidemiological contexts, but
having quantitative results from adaptive management experiments based on sound principles of
statistical design would make it easier to construct WOE arguments on ecological risk.
As alluded to above, meta-analysis is a WOE method. In its simplest form, meta-analysis is used when
different studies have provided estimates of the same effect. Estimated effects arising from different
studies are assigned weights according to associated variances and sample sizes. The effects are then
combined to produce an overall effect with a variance estimate. In this manner, the presence of many
studies with nearly statistically significant results can lead to an overall statistically significant result.
Thus, having more than a single study estimating the same effect can lead to a more powerful (i.e., able
to detect smaller changes) estimate of that effect. Variation among studies can also be investigated.
Other useful statistically-based WOE approaches have been reported (Bailer et al. 2002; Burton et al.
2002b; Grapentine et al. 2002; Reynoldson et al. 2002; Kapo and Burton 2006; Kapo et al. 2008). These
approaches address many weaknesses of qualitative approaches and provide a solid basis for EPA
guidance that could be structured to address unique program needs.
18
-------
The WOE process should be described in the problem formulation stage of a risk assessment and ensure
credible stakeholder input and a transparent understanding of what constitutes reference condition,
restoration goals, remedy objectives and/or ecological impairments in the context of site spatial and
temporal variations. This point was highlighted in the SAB report on improving the ecological risk
assessment process, which resulted in the current RAF process (Dale et al. 2008). The SAB provides the
following recommendations with regard to weighting ecological risk assessment LOE:
Recommendations
• In general, a comprehensive set of LOE should be utilized in the WOE process. The LOE should
adequately characterize physical, chemical and biological conditions. This will ensure a cumulative
evaluation that considers commonly occurring stressors such as habitat, water flow and nutrients.
The weight given to particular LOE is likely to be very case-specific, and quality of the data
underlying a particular LOE should factor into assigned weights. Beyond that, if other factors are
considered equal, LOE that are clearly linked to population or community attributes should receive
greater weights than those that are not.
• Arbitrary numerical weightings should not be assigned to LOE because site- and situation-specific
considerations will affect weightings. WOE assessments need to be "flexible, transparent and
defensible... [with] sufficient flexibility to accept all relevant evidence and generate creative
solutions to difficult problems" (Suter and Cormier 2011). The SAB agrees with the EPA assertion
that "weighing of evidence should be considered during each problem formulation, and a method for
weighing evidence should be included, as appropriate, in the analysis plan" (U.S. EPA 2010b).
• Multicriteria decision analyses should be further investigated as recommended by Linkov et al.
(2011): "Each WOE method is based on a unique rationale capable of considering a different scope
of LOEs. Thus, each method has specific benefits and drawbacks. The different nature of methods
means that one cannot a priori determine the superior method for a particular application. One must
consider the method employed in addition to the evidence."
3.3.4. Probabilistic Basis for the Weight-of-Evidence Approach
The evaluation of WOE has progressed over the years but as it is currently practiced, WOE is a
qualitative tool without a probabilistic basis. A WOE is essentially a Bayesian approach without a
realization of the calculation. Evidence should be taken that can differentiate between alternative
hypotheses and Bayesian networks can be used to improve analyses conducted in risk assessments
(Newman et al. 2007). The application of Bayesian networks to different environmental and resource
management problems is described in a series of articles in a special issue of the journal Integrated
Environmental Assessment and Management (volume 8, number 3, July 2012).
Bayesian networks can be tied directly to the cause-effect conceptual model that should be generated for
every risk assessment. Bayesian networks have long been used in this manner to create diagnostic tools
for medicine (Ben-Gal 2007). It has been suggested that Bayesian networks be used to optimize methods
specifically targeted to mammalian testing for carcinogenicity (Jaworska et al. 2010), but their broad
outline should be applicable to testing approaches for environmental toxicity and for protocols used to
determine causation when environmental degradation is observed. Bayesian approaches have also been
used to analyze gene expression data in the public domain to create an automated diagnostic data base
19
-------
for human health (Huang et al. 2010). Environmental toxicology research is also providing data on gene
expression that may be used for prediction of effects.
One question concerning the use of Bayesian tools and networks is how to effectively communicate the
results to managers and decision-makers. A case study using a Bayesian network in a comparison of the
risks of alternate medical diagnosis demonstrates how results can be effectively communicated (Fenton
and Niel 2010).Given the experience in medical diagnosis, a Bayesian approach would be an extension
of the basic WOE in deciding between alternative hypotheses and in the diagnosis of causality.
Recommendation
• Specific quantitative (i.e., statistical) guidance, with associated case studies, on interpreting LOE and
WOE in ecological risk assessments should be developed for use by EPA risk assessors and risk
managers. Case studies should cut across different EPA regulatory programs in which WOE is used.
In particular, the SAB recommends that more emphasis be placed on the use of Bayesian approaches
(e.g., to assess the probability that a certain state of ecosystem services is the desired state of
ecosystem services, or to develop the best possible information for decision-making in the face of
uncertainty - for instance for data-poor case studies). It is critically important to show success in case
studies under data-limited conditions to convince non-scientists (e.g., managers) of the utility and
value of WOE and of ecological risk assessment.
3.4. Communication of Ecological Assessment Issues and Results to Decision-Makers and
Stakeholders
Charge Question 4. The RAF Action Plan calls for the development of methods for better
communication of ecological assessment issues and results to decision-makers and
stakeholders. This applies to communicating ecological assessment issues during both
planning of assessments and presentation of results. In part, this may be a matter of the
inability of assessors to communicate the significance of the loss of species, changes in
community structure and other endpoints. The RAF has developed a communication
technical panel project description. Please comment on whether the RAF's planned project is
an appropriate way to proceed, and what obstacles might exist to either interpreting or
utilizing ecological information in risk assessment. Please include any observations on why
ecological risk assessment has or has not been well incorporated into decision-making in
general.
3.4.1. Comments on RAF Communication Technical Panel Project
The SAB commends the RAF for recognizing risk communication as an important aspect of science
policy. We understand that the RAF has limited resources and has therefore chosen to limit the focus of
its Communication Technical Panel project to improving communication between ecological risk
assessment practitioners and decision makers within the EPA. That said, the SAB encourages the RAF
to reach out to others in the agency to develop a broader long-term initiative to improve communication
at all steps of the risk assessment and management process and to make communication an essential and
effective core attribute of the risk assessment process. The SAB has provided advice on the narrowly
focused RAF Communication Technical Panel project as well as approaches that EPA should consider
in undertaking a broader initiative. The one-page RAF Communication Technical Panel project
description provided to the SAB indicates that the Panel intends to: (1) gather data through surveys and
20
-------
interviews of EPA risk assessors and decision-makers to explore how ecological risk assessments are
used in agency decision-making; and (2) develop guidance for risk assessors and decision-makers on
how to better communicate and enable the use of ecological risk assessment information.
The SAB has been given a minimal amount of detail upon which to base this review. However, the
available information indicates that the RAF Communication Technical Panel's approach to data
collection is quite general, and the scope of the proposed investigation is somewhat one-directional and
narrow (i.e., it is not designed to provide exchange of information about communication issues
important to participants involved in each step of the risk assessment and management process). Such a
narrow focus on better communication in risk assessment is not consistent with recommendations from a
variety of panels, committees, advisors (NRC 1994, 1996, 2009) that have emphasized that multi-
directional communication is a critical part of risk assessment. The SAB has previously advised the EPA
that early and continuing engagement of risk assessors with decision-makers and stakeholders is
important in order to determine what is valued, and what outcomes are desired (U.S. EPA SAB 2007).
The RAF Communication Technical Panel has intentionally targeted the interface between the
ecological risk assessor and the risk manager as the point at which communications can be improved in
the assessment process. Such a limited focus could lead to some short-term improvements in managers'
confidence in and use of ecological risk assessments. However, a much broader study will be required to
achieve the RAF Communication Technical Panel's stated goals of promoting full use of ecological risk
assessment across EPA programs, and meeting the managers' needs for useable ecological risk
assessment information to support decisions. That broader study should address how effective
communication can be incorporated throughout all phases of the assessment and management decision
process. The SAB recommends that the Technical Panel complete its initial task and then use the results
as a basis to advocate for a broader study to be undertaken either by ORD or through other appropriate
means. Communication should be required and documented as an essential core activity of the
ecological risk assessment process that runs from problem formulation, through analysis and remedy
selection to final communication of the decision and proposed management strategy to stakeholders.
There is no indication that the RAF Communication Technical Panel project will address the role and
responsibility of management to frame problems correctly, or that the project will focus on how a well-
communicated assessment can improve the communication of management decisions, particularly with
regard to the importance or value of impaired endpoints.
Barriers to effective risk communication can be associated with the risk assessment process (e.g., lack of
information or access to information and incomplete stakeholder participation) as well as social
processes of human interaction and decision-making (e.g., differences in societal perception of risks;
differences in societal concerns about risks; societal characteristics such as language barriers and
cultural differences; and problems deriving from institutional and political systems) (NRC 1989; FAO
1999). Overcoming these barriers will require the engagement of experts in the social science disciplines
such as psychology, economics, and sociology. The EPA should explore the questions of what
constitutes successful communication and what the criteria are for measuring that success. One
communication recommendation in the Colloquium Report was to enhance communication among risk
assessors, but that does not seem to be a part of the Plan. Establishing a formal exchange among EPA
ecological risk assessors would be a valuable way to transfer experience and practice among the agency
assessor community.
EPA should expand the scope of the RAF Communication Technical Panel project to address the need to
develop a variety of risk communication strategies to reach different audiences. In this regard,
21
-------
ecosystem services might be recognized as more or less valuable by different audiences and thus
become a critical element of targeted communication strategies. Therefore, concerted cross linkage
between the Communication Technical Panel and the separate RAF Technical Panel on Ecosystem
Services is advised.
The SAB finds that the methodology of the Communication Technical Panel survey to develop a basis
for designing better communications seems at best to be a preliminary approach that will not provide the
information needed by the EPA. The use of surveys/interviews of both risk assessors and decision-
makers (i.e., risk managers) asking high level questions such as "is ecological risk assessment
information being used for ?" will likely lead to many "yes, no and maybe" answers. The description
of the proposed survey contains insufficient detail to fully understand the extent to which the surveys are
meant to delve into the questions posed by the RAF with regard to how to communicate the significance
of the results, such as the loss of species, changes in community structure and other endpoints. Such
surveys are very labor intensive and may reinforce the need to act, but not get at the matter of what to
change and how to make a change. The original EPA colloquium was a survey of sorts and already has
indicated the need for change. The Communication Technical Panel would be better advised to analyze
the risk assessment process by deconstructing specific decisions with a cross disciplinary team.
The SAB strongly recommends that the EPA take advantage of a recent SAB survey of the agency's use
of science in decisions. A description of the survey and recommendations to strengthen science
integration for EPA decision-making are provided in the SAB report, Science Integration for Decision
Making at the U.S. Environmental Protection Agency (U.S. EPA SAB 2012). The results of the SAB
survey provide information about the interface of assessors and managers across EPA regions and
programs. Ecological risk assessment and associated science are a subset of the SAB study.
Recommendations
• Recognizing that the resources available for the RAF Communication Technical Panel's project are
limited, the SAB finds that the task outlined is only a good start and recommends that the EPA find
additional resources to broaden the scope of the project to include a focus on improved
communication across the entire risk assessment and management process. Communication should
be elevated to an essential core process activity with its own performance standards and success
criteria. The EPA should not focus on just communicating the current process and the outputs it
generates, but consider process redesign to make effective communications an integral component of
the ecological risk assessment process.
• The EPA should focus on identifying forms of guidance/support tools that can be adapted to the
needs of specific regulatory applications. The EPA's work should recognize that communication
strategies may vary with the type of regulatory application. The EPA makes a broad range of
decisions across its programs and regions, and different management and supporting assessment
processes are used within the EPA for given regulatory programs (e.g., wastes are managed in
accordance with requirements of the Resource Conservation and Recovery Act and the
Comprehensive Environmental Response, Compensation, and Liability Act). These decisions are
supported by a variety of assessment-decision processes. It should be no surprise that the risk or
hazard assessment methods and outputs that support these varied processes would also vary. This
suggests that, for communication between ecological risk assessors and risk managers to achieve
maximum impact for a given agency decision, the question of how to improve that outcome must be
addressed at the level of the specific decision process.
22
-------
• The SAB recommends that the RAF Communication Technical Panel actively reach out to other
RAF Technical Panels (e.g., Ecosystem Services) as sources of information to enhance better
communication of risks and impacts. There is an obvious and strong linkage between risk
communication and some of the other policy areas addressed in the Plan, such as ecosystem services
and the integrated assessment framework. Results of an ecological risk assessment consist of the
scientific facts, but generally lack the interpretation of how humans will be affected. Ecological and
human health risk assessments have the potential to provide complementary information that can be
used in an integrated "holistic" approach to risk assessment. The key to effective communication
may lie in understanding and communicating why the risk matters and how managing it will provide
a valued benefit.
• The EPA should actively engage experts in communication to inform its efforts. The EPA should
consider incorporating external technical input to its process from social scientists, ethicists,
marketing professionals and media specialists who have a good understanding of risk
communication and broader environmental concerns. Particularly good resources include
environmental non-governmental organizations, university extension professionals and academic
communities that have considerable expertise communicating the value of ecosystems and their
services to the public (e.g., State University of New York Stony Brook School of Journalism Center
for Communicating Science; http://www.centerforcommunicatingscience.org/; and the Ecosystem
Commons http://ecosystemcommons.org; http://ecosystemcommons.org/soapbox/Madsen).
• Ultimately, the EPA should conduct a more thorough analysis of the communication issue. A
systematic evaluation of how to address decisions requiring integrated and multi-criteria analysis and
the challenges and opportunities for better communication across a range of different decision types
would provide a better basis for developing performance standards and guidance. The analysis
should address better communication performance by all participants including risk assessors,
managers and key stakeholders. As noted previously, a multidimensional approach to effective
communication in the risk assessment process is needed to improve the impact of risk assessment in
the risk management process. In addition, a range of communication strategies or techniques needs
to be made available to both risk assessors and risk managers. Establishing technical platforms for
peer networks to exchange experience and practices may go a long way toward advancing
communication of risk assessments. A more thorough analysis must recognize that communication
strategies should be adapted to the intended stakeholder audience. This analysis would assist EPA
decision-makers in communicating how assessments drive actions that will be required. Knowing
why the risk matters to the receiving audience is as important as quantifying any given risk
The following is just one such approach illustrating steps the EPA might take in this kind of evaluation:
1. Create a matrix of decision types (programmatic, regional, etc.) and identify: (a) those decision
types where EPA staff or interviewees indicate that ecological risk assessment is having the
appropriate impact; and (b) one decision type where there is clear indication that ecological risk
assessment is being grossly under-utilized.
2. Create a work team with representation from all staff involved in the specific decision process.
This work team should include assessors and decision-makers and real or mock stakeholder
representatives.
23
-------
3. Provide the team with a detailed case study problem and have it review the decisions while fully
discussing: (a) how ecological risk assessment information, or more broadly "science," can
support better decisions; and (b) how and when ecological risk assessment can be better
communicated. Use this exercise to discuss the manager's information needs, useable
information format and the importance of context in communicating the ecological risk
assessment output (but more importantly in shaping the ecological risk assessment through the
problem formulation). The exercise should involve all aspects of the decision process including
the selection of the technology or remedy and communicating the decision. The exercise should
be facilitated by a non-participant in the assessment-decision exercise. The team should have a
clear charge, information reporting requirements and sets of questions that need to be addressed
at each stage of the decision process.
4. After running several of these exercises, evaluate the results and determine whether a
protocol/process template can be developed for independent use by others (e.g., EPA Program
Offices and Regions). If such a template cannot be developed, run several additional "exercise
sessions" with different decisions. At the end, the goal is to create a self-assessment tool that
can be used by a program or regulatory group for a specific regulatory application in order to
identify how ecological risk assessment can have more impact on a specific decision, and how
better communication of ecological risk assessment information and science can facilitate those
results.
5. Finally, if the EPA wants to build general guidance, the SAB suggests developing a document
built on understanding gained from the case study problem and also creating a "go-by" tool that
risk assessors can use to improve communication as they are participating in the ecological risk
assessment process. The tool could be simply a check list of considerations and questions that
are clustered according to the typical steps in most decision processes, including problem
formulation, assessment design, risk analysis, options selection and decision communication.
Development and use of such a self-guided process to improve performance would elevate both
consistency and alignment of decisions and science.
• The SAB strongly recommends that the EPA take advantage of a recent SAB survey of the agency's
use of science in decisions. A description of the survey and recommendations to strengthen science
integration for EPA decision-making are provided in the SAB report, Science Integration for
Decision Making at the U.S. Environmental Protection Agency (U.S. EPA SAB 2012).
The SAB is supportive of the EPA's efforts to improve the communication of ecological risk assessment
information and interested in the direction that the EPA is taking in the RAF Communication Technical
Panel project. We stand ready to provide future advice as the EPA completes work on this important
project. When the Ecological Assessment Action Plan has been refined to provide more detail, or after
the initial path of data collection, it may be useful to receive additional advice from the SAB.
3.4.2. General Observations on Incorporation of Ecological Risk Assessment into Decision
Making
The SAB was asked to comment on why ecological risk assessment has or has not been well
incorporated into decision-making. There are likely many reasons why ecological risk assessment output
has not, in the view of risk assessors, had an optimal impact on decisions. Not all of these reasons
involve effective communication. In some cases, failure to identify, prioritize and communicate
24
-------
uncertainties that may affect the quality of risk management decisions has prevented ecological risk
assessments from being well-incorporated into decision-making. The SAB previously recommended
that, in the problem formulation phase of a risk assessment, the EPA identify uncertainties that could
profoundly affect the results and outcomes, and also identify the information needed to reduce
uncertainties (U.S. EPA SAB 2007). Incorporating adaptive management principles into ecological risk
assessment and management would enable EPA decision-makers to determine whether changes in
approach may be needed and appropriate. Focusing principally on the aspect of improving success
through better communication, the SAB suggests that the EPA Risk Assessment Forum think more
broadly about communication throughout the entire assessment-management process.
As stated previously, the communication of the risk output is not the only point of communication that
could be improved. The risk assessor and the manager overseeing the decision process need to clearly
understand what data are required for a decision, or from which a decision could benefit, and how the
data inform the selection of alternatives. A robust communication process that leads to a clear
understanding of the context for the decisions, and how various data or ecological risk analysis improve
or support a decision at hand, will lead to better alignment of assessment and decision. As discussed in
section 7.1.2 of the EPA Colloquium Report (U.S. EPA 2010b), risk assessors need to understand all
aspects of the contexts of problems and decisions in order to help managers do the "balancing act"
required in specific regulatory applications.
Unfortunately, ecological risk assessment still finds limited use even in the EPA. For example, a recent
SAB report on methods for the treatment of vessel ballast water (U.S. EPA SAB 2011) recommended
including a specific risk assessment process, Hazard Analysis Critical Control Point (HACCP). HACCP
is currently used in food safety and many other applications. HACCP was recommended by the SAB as
a method for analytically determining an appropriate ballast water treatment methodology and managing
the treatment. Although the EPA's response to the SAB report indicated that specific parts of the
Agency's vessel general permit follow SAB recommendations, those requirements do not appear to
resemble a HACCP-like approach.
There are many barriers to the use of ecological risk assessment in decision-making. The barriers to be
overcome include making stakeholders or decision-makers comfortable with quantitative assessments
and providing realistic training so that EPA staff can apply risk assessment to decision-making
processes. Further information about the limitations or barriers to fuller use of ecological risk
assessment by the EPA is provided in Landis (2009).
3.5. Incorporation of Ecosystem Services into Ecological Risk Assessment Methods
Charge Question 5. Ecosystem services can be used to describe potential outcomes of
environmental management decisions in terms that can be more effectively communicated to
decision-makers and the public. The RAF expects to produce guidance on how to relate
ecological risk assessment endpoints to ecosystem services. This information will be used to
update the EPA guidance document, Generic Ecological Assessment Endpoints (GEAEs) for
Ecological Risk Assessment (U.S. EPA 2003). Please consider Appendix B (page 52) of the
generic ecological assessment endpoints guidance document and the project description of
the RAF Technical Panel on Ecological Services Assessment Endpoints and comment on
whether they capture the full range of opportunities to incorporate ecosystem services into
EPA's ecological risk assessment methods.
25
-------
The concept of ecosystem services has emerged as a means of conveying the direct value of the
environment to human well-being by including both market and nonmarket goods and services. The
ecosystem services paradigm has been operative for some time (Costanza et al. 1997) and its integration
with ecological assessments is both fundamental and overdue. Thus the SAB finds that its inclusion in
the Plan is appropriate. The use of ecosystem services endpoints affords the opportunity to move
ecological risk assessment from identifying what is at risk to why that risk matters to humans. This will
help EPA managers more clearly communicate outcomes in terms of the benefits/values derived from
the ecosystem attributes that are protected, and support an understanding of why decisions were made.
The concept of ecosystem services can be easily grasped by all stakeholders including scientists,
managers, policy makers and informed public. It can also help managers better understand the trade-offs
in their decisions, although assessing trade-offs among services is one of the most difficult aspects of the
use of services as an endpoint in risk assessment. Often, optimizing delivery of a given service may
reduce or impair another (Mooney 2010). Agriculture provides a primary example. The enhanced
provisioning of food can result in loss of clean water and stress to biodiversity that supports other
services. Incorporating ecosystem services into risk assessment is of high priority. It takes advantage of
the expertise being developed around ecosystem services in ORD, and therefore should benefit both
ecological risk assessment and the ORD program. Incorporating ecosystem services is also likely to
stimulate progress on the first science policy initiative in the Plan, namely incorporation of systems
analysis into ecological risk assessment.
3.5.1. Comments on RAF Ecosystem Services Technical Panel Project
The Plan is vague on how the ecosystem services activity will be implemented. It indicates that an RAF
Ecosystem Services Endpoint Technical Panel has been created whose findings will be incorporated into
the EPA guidance document on ecological assessment endpoints; the brief project description from the
Technical Panel does not provide much additional information. The Panel's project description indicates
that the products to be developed include: (1) a white paper interpreting conventional measurement and
assessment endpoints in the context of ecosystem services; (2) a case study(s) of applying the ecosystem
services concept in ecological risk assessment; and (3) an addendum to the RAF Generic Ecosystem
Assessment Endpoints guidance (U.S. EPA 2003). The addendum would expand the GEAE document to
include a broader range of ecosystem services and more explicitly address linkages between the original
GEAEs and services. It is not clear that the development of "case studies and guidance on how to relate
ecological endpoints to ecosystem services" will be successful in achieving the desired goals of
incorporating ecosystem service endpoints into the ecological risk assessment during problem
formulation, analysis and risk characterization. Greater detail is needed to evaluate the potential success
of this activity. The EPA's presentation to the SAB indicated that considerable progress has been made
in developing the white paper on the use of ecosystem services as an endpoint in ecological risk
assessment. The decision to incorporate several case studies into this document will increase its value.
The SAB has some concern that the document may stress concepts at the expense of application. It will
be most valuable to the community of practitioners if application is stressed, for example, how endpoints
could be created and applied in different situations. The SAB encourages the EPA to complete this white
paper and recommends that, in addition, the authors consider submitting a briefer version to a peer-
reviewed publication that would make it available to a wider audience. Such a publication could benefit
the EPA by describing how its actions fit into the research on ecosystem services that is being conducted
in ORD and elsewhere around the world.
It would have helped the SAB to have had a copy of the white paper in hand to better evaluate the
direction this effort will take the agency. It would be beneficial for the RAF Ecosystem Services
26
-------
Endpoint Technical Panel to engage the SAB to make suggestions on the guidance being developed.
One SAB concern that might be assuaged by having the white paper available is the limited view
implied by the statement (on page 1 of the Ecosystem Services Endpoint Technical Panel presentation)
that the value to the EPA of integrating ecosystem services into ecological risk assessment is "improved
means of communicating risk and informing risk management decisions." The SAB notes that
integration of ecosystem services can also help in designing an ecological risk assessment that is well
aligned with the decision context and those aspects of the decision that matter to the public. Integration
of ecosystem services can also be used to inform the selection of technologies and remedies by
expressing the reduction of risk in terms of benefits that can potentially be valued in monetary terms. In
addition, tying the ecological risk assessment to ecosystem services should help managers communicate
with stakeholders. This may have been implied in the statement in the EPA's presentation to the SAB,
but that is not clear without the details that are likely to be provided in a white paper. The SAB
recommends that the white paper include evaluation of the use of ecosystem services endpoints
information throughout the entire risk assessment and risk management decision process. By
considering the entire range of process steps, it seems likely that using ecosystem services as endpoints
will help achieve the goals defined in Charge Question 4, namely making ecological risk assessment
more frequently used and more useful to managers in making the their decisions.
The white paper could benefit from describing how the concept of ecosystem services is being used in
other agencies and other countries. Many natural resources agencies (e.g., U.S. Forest Service) routinely
include ecosystem services in their management strategies (e.g., Agee 2003; Ager et al. 2007; Barbour et
al. 2007). Other agencies are considering how to use ecosystem services endpoints in the context of
integrated ecosystem assessments (Levin et al. 2009). The EPA should investigate lessons that can be
learned from the experiences of other agencies. There is a risk and threat assessment literature that
routinely incorporates ecosystem services in its calculations (Marcot et al 2006a; Apitz 2012;
Glendining and Pollino 2012). The EPA should assess how the agency's proposed guidelines compare
with those efforts and results. The international community is also using ecosystem services in risk
assessment (e.g., SETAC 2012), but it is not clear that insights from this work are being incorporated
into the documents being developed by the EPA.
Recommendations
• More information is needed to evaluate the potential success of the RAF Ecosystem Services
Endpoint Technical Panel project. The SAB encourages the EPA to complete the ecosystem services
white paper and recommends that, in addition, the authors consider submitting a briefer version to a
peer-reviewed publication that would make it available to a wider audience. The decision to
incorporate several case studies into this document will increase its value.
• The SAB recommends that the ecosystem services white paper include an evaluation of the use of
information related to ecosystem services endpoints throughout the entire risk assessment and risk
management decision process. Considering the entire range of process steps will help achieve the
EPA's defined goals, namely making ecological risk assessment more frequently used and more
useful to managers in making the their decisions.
• The ecosystem services white paper should include a description of how the concept of ecosystem
services is being used in other agencies and other countries. Many natural resources agencies (e.g.,
U.S. Forest Service) routinely include ecosystem services in their management strategies
27
-------
3.5.2. Definition of Ecosystem Services Endpoints
The SAB was asked to comment on the range of ecosystem services described in Appendix B of the
EPA's generic ecosystem assessment endpoints guidance. Neither the GEAE guidance nor the
Colloquium Report, Integrating Ecological Assessment and Decision-Making at EPA: A Path Forward
(U.S. EPA 2010b) captures the full range of concepts embodied by the term ecosystem services. More
current ecosystem services concepts and definitions, provided in Table 1 (Millennium Ecosystem
Assessment, 2005; U.S. EPA SAB 2011), need to be incorporated into the EPA's guidance, replacing
older terminology and meaning (e.g., Appendix B Table B-l of the GEAE guidance). This would
provide the EPA with continuity of thought and concepts with the published literature on ecosystem
services, and would provide clarity as to what constituted ecosystem services. The term "ecosystem
services" is used frequently in the Colloquium Report, but there is no list of services or discussion of the
broad range of tangibles and intangibles included in the term.
It is difficult to judge how the proposed addendum to the table in Appendix B of the GEAE guidance
will build on the material that is currently in the document. Presumably, the text of the white paper will
provide further explanation and support. The SAB recommends considering updating the original
guidance rather than producing an addendum. If both an original and an addendum are in circulation, a
practitioner could mistakenly use only one, which could lead to errors.
The GEAE document lists ecosystem functions as a possible endpoint, but only for wetlands. Some
evaluation of the experience in using this endpoint would be a valuable aspect of the EPA's proposed
analysis because there is more experience in using structural rather than functional endpoints. Section 4
of the GEAE document proposes that there be a place (e.g., a website) where experiences with these and
other endpoints could be posted. This is a useful suggestion.
The incorporation of ecosystem services into ecological risk assessment at the EPA should be
straightforward. As previously discussed, there are a number of examples from work outside of the
agency that can be used as models. In addition, Suter et al. (2005) have presented an approach for
endpoint definition that can easily be applied to ecosystem services. They define an endpoint as an entity
plus its attribute(s). Although they did not single out ecosystem services in their paper, this foundation is
applicable. For example, a common ecosystem service is the persistence of commercial and sports
fisheries. With that as the ecosystem service, the attributes would be those entities that embody that
service. In the Willamette River in Oregon, fisheries are defined by the Oregon Department of Fish and
Wildlife as a number of each salmonid species for each of the segments of the river and their tributaries.
The specifications for the types of fish and the numbers are the attributes of the commercial and sport
fishing ecosystem service. For the South River of Virginia that same service is defined by the
Commonwealth of Virginia as a certain number of trout in one part of the river and by the number of
sunfish and bass in other segments. The ecosystem service of flood control can be defined by the
number and extent of floods. Water quality and quantity have attributes that are mandated by local,
regional and federal standards.
The SAB recommends that when the RAF Ecosystem Services Endpoint Technical Panel completes its
listed objectives at the end of fiscal year 2012, it consider looking at the other policy focus areas in the
Plan to determine where ecosystem services information should be incorporated into those other topics.
Although this is not currently part of the Technical Panel's charge, the linkage with communications is
28
-------
Table 1. Global Status of Provisioning, Regulating and Cultural Ecosystem Services
(Millennium Ecosystem Assessment 2005)
Service
Sub-category
Status
Notes
Provisioning Services
Food
Crops
+
Substantial production increase
Livestock
+
Substantial production increase
Capture fisheries
-
declining production due to overharvest
Aquaculture
+
substantial production increase
Wild foods
-
declining production
Fiber
Timber
+/-
forest loss in some regions, growth in others
Cotton, hemp, silk
+/-
declining production of some fibers, growth in others
Wood fuel
-
declining production
Genetic resources
-
lost through extinction and crop genetic resource loss
Biochemicals, natural medicines
-
lost through extinction, overharvest
Freshwater
-
unsustainable use for drinking, industry and irrigation
Regulating Services
Air quality regulation
-
decline in ability of atmosphere to cleanse itself
Climate regulation
Global
-
net source of carbon sequestration since mid-century
Regional and local
-
preponderance of negative impacts
Water regulation
+/-
varies depending on ecosystem change and location
Erosion regulation
-
increased soil degradation
Water purification and waste treatment
-
declining water quality
Disease regulation
+/-
varies depending on ecosystem change
Pest regulation
-
natural control degraded through pesticide use
Biological control, trophic structure
+/-
trophic-dynamic regulations of populations
Pollination
-
apparent global decline in abundance of pollinators
Natural hazard regulation
-
loss of natural buffers (wetlands, mangroves)
Supporting Services
Soil formation
+
Weathering of rock and erosion
Photosynthesis
+
Primary production
+
net primary production has increased
Biodiversity
-
loss of species
Nutrient cycling
Nitrogen
-
large-scale changes from general eutrophication
Phosphorus
-
Water cycling
-
major changes from structural changes in rivers, water withdrawal and
climate change
Habitat, refugia
-
habitat for resident and transient populations
Cultural Services
Spiritual and religious values
-
rapid decline in sacred groves and species
Aesthetic values
-
decline in quantity and quality of natural lands
Recreation and ecotourism
+/-
more areas accessible but many degraded
Status indicates whether the condition of the service globally has been enhanced (+) or degraded (-) in the recent past.
29
-------
obvious, and other important linkages with topics such as strengthening the EPA's protection goals and
applying systems and integrated approaches may be uncovered. The RAF Ecosystem Services Endpoint
Technical Panel could act as a cross review group with expertise in ecosystem services that could
contribute to the other tasks.
Recommendations:
• The EPA should incorporate more current ecosystem services concepts and definitions (e.g., Table
1) into the GEAE document replacing older terminology and meaning. In this regard, the SAB notes
that cultural services to indigenous communities should not be understated even though they are hard
to articulate, quantify and value.
• The EPA should consider updating the original GEAE guidance rather than producing an addendum.
If both an original and an addendum are in circulation, a practitioner could mistakenly use only one,
which could lead to errors.
• EPA guidance should include evaluation of experience in using the functional wetlands endpoint
because there is more experience in using structural rather than functional endpoints.
• When the RAF Ecosystem Services Endpoint Technical Panel completes its listed objectives at the
end of fiscal year 2012, it should consider looking at the other policy focus areas in the Ecological
Assessment Action Plan to determine where ecosystem services information should be incorporated
into those other topics.
3.6. Use of Adaptive Management for Testing and Revising Risk Management Actions
Charge Question 6. In its 2007 report, Advice to EPA on Advancing the Science and
Application of Ecological Risk Assessment in environmental Decision-Making, the SAB
recommended that EPA use adaptive management to address uncertainties in decision-
making. The application of adaptive management in risk assessment and risk management is
discussed in section 6.3 of the EPA colloquium report, Integrating Ecological Assessment
and Decision-making at EPA: A Path forward, and the RAF Action Plan proposes the
development of adaptive management as a tool to methodically improve risk management
decisions. Please comment on how adaptive management approaches can be developed to
provide optimal value for EPA programs.
3.6.1. General Comments on Developing Adaptive Management Approaches
Adaptive management is a process intended to reduce the uncertainty in the decision-making process
through the use of monitoring efforts and the iterative evaluation of the data from these monitoring
programs. Adaptive management has been defined in many different contexts. In one definition, the
NRC (2004) identified six elements as key principles of adaptive management:
(1) "resources of concern are clearly defined;
(2) conceptual models are developed during planning and assessment;
(3) management questions are formulated as testable hypotheses;
30
-------
(4) management actions are treated like experiments that test hypotheses to answer questions and
provide future management guidance;
(5) ongoing monitoring and evaluation are necessary to improve accuracy and completeness of
knowledge; and
(6) management actions are revised with new cycles of learning."
Adaptive management has not been adopted as a formal policy in EPA programs; however, its concepts
have been adopted by other regulatory agencies responsible for the management of environmental
concerns.
Both the SAB and the RAF have previously considered the use of the adaptive management approach, at
least in part. In its 2007 report, Advice to EPA on Advancing the Science and Application of Ecological
Risk Assessment in Environmental Decision-Making, (U.S. EPA SAB 2007) the SAB recommended that
the EPA use adaptive management to address uncertainties in decision-making. Subsequently, in its
2010 Colloquium Report, the EPA RAF concluded that "Adaptive management is potentially a highly
useful strategy, but its implementation would require changes in fundamental agency science policies
and practices." The SAB provides the following comments and recommendations concerning
development of adaptive management approaches to provide value for EPA programs.
• The six elements of adaptive management that were identified by the NRC (2004) and described in
section 6 of the RAF Colloquium Report are consistent with the general risk assessment framework
typically employed in conducting an ecological risk assessment in accordance with existing EPA
policy (U.S. EPA 1992; 1998). The SAB therefore concludes that the use of adaptive management
approaches is a logical recommendation and an appropriate application in the risk assessment
framework. Adaptive management offers an opportunity to improve practices in ecological risk
management and to document successes so that decision-makers have greater appreciation for the
practice of ecological risk management.
• The SAB notes that one of the six adaptive management elements, ongoing monitoring and
evaluation, is a key aspect of adaptive management that is not always fully implemented in the risk
management framework and the evaluation of mitigative actions. This is likely due to a lack of
regulatory authority or cost considerations, but the importance of these activities as a means of
"uncertainty" evaluation and a measure of "validation" of the risk assessment or the "efficacy" of
mitigation actions should not be overlooked. Monitoring and evaluation should be incorporated in
any risk management activities. The SAB notes that Bayesian approaches, causal argumentation and
probabilistic risk assessment would facilitate the development of hypotheses that could be evaluated
as part of the adaptive management process
• Incorporation of adaptive management principles into the EPA's risk assessment framework is an
appropriate goal for the agency. However, it is recognized that implementation of this goal may be
difficult given the complexity of consistent and continuous ecosystem monitoring and evaluation
over appropriate time scales and the scope and magnitude of resource limitations currently facing the
EPA. Nonetheless it is a goal to be aspired to and implemented with time.
• Incorporation of the adaptive management approach into the risk assessment framework may be
particularly useful for addressing certain technical concerns (e.g., climate change) or management
issues and decisions facing EPA programs (e.g., Office of Pesticide Programs, Superfund, Office of
Air). However, the adaptive management approach may not be appropriate for all of the complex
31
-------
"risk assessment applications" throughout the various EPA programs using the risk assessment
framework. The following factors can be considered in deciding whether to apply adaptive
management principles to complex risk assessment and management problems: spatial and temporal
scale of the problem; relevant dimensions of uncertainty; the suite of cost, benefits and risks; and the
degree to which there is stakeholder and institutional support (Gregory et al. 2006). These factors are
associated with various challenges to successful application of adaptive management.
• Comments from EPA staff suggest that the adoption of all the principles in the adaptive management
approach may involve the need to understand the mechanism(s) of "why we see what we see." This
may require more than just an understanding of the "validation" or "lack of validation" of the
management actions taken. EPA staff is concerned that this interpretation of adaptive management
may be more onerous than is actually needed. The SAB appreciates the EPA's concern in this regard
and suggests that recommendations for the use of adaptive management principles should indicate
that "aspects of the adaptive management approach" are appropriate for inclusion in the risk
assessment process.
• Risk assessment is, by its very nature, an iterative process and this is consistent with the adaptive
management approach. The approach can be described as "Plan, do, check, adapt." One must always
look back to check that actions had their intended consequences. It is difficult to predict at the
problem formulation phase of any risk assessment all of the issues or concerns that may arise;
therefore, changes in approach may be needed and appropriate. This appears to be a key principle of
the adaptive management approach and one that should garner greater focus in the implementation
of the risk assessment framework.
Recommendations
• The SAB recommends that the EPA take action to implement the goal of incorporating adaptive
management principles into the agency's risk assessment framework. However, the SAB recognizes
that implementation of this goal may be difficult given the complex problems and the scope and
magnitude of resource limitations currently facing the EPA.
• Monitoring and evaluation should be incorporated into the EPA's risk management activities.
Monitoring and evaluation are an important means of "uncertainty" evaluation and a measure of
"validation" of a risk assessment or the "efficacy" of mitigation actions. EPA Regional and
Program Offices have limited support for monitoring. Resources should be provided to these offices
to develop improved capacity for monitoring and sampling to support ecological risk assessment.
3.6.2. Importance of Applying Statistical Design Principles in Adaptive Management
Adaptive management effectively occurs when natural resource managers apply the principles of
rigorous statistical design of experiments to evaluation of management actions. In the best cases, this
can result in powerful "management experiments." Such management experiments may have to occur at
large temporal or spatial scales, and therefore require careful thought and planning. In order to compare
selected practices or policies, one can state management questions in terms of testable null hypotheses
about the system being managed. This usually requires a good deal of knowledge about the particular
ecological process(es) being studied.
32
-------
Clearly defined study objectives are needed in the design of any project. One has to decide what the
different "treatments" being compared are; in adaptive management these could be different risk
management practices or policies. There may be ancillary variables ("covariates") that also affect the
response; if so, they must be recorded and included in the data analysis. Type II" regression, which
examines the covariance of variables of concern in time and space, or in response to some other process
or variable, can be a useful modeling tool. Careful thought must be given to the experimental units to
which the different "treatments" are being applied. For example, in comparing different fishery
management practices, an experimental unit might be a large region to which a particular management
practice has been assigned. Inferences are the most solid when true random assignment of "treatments"
to experimental units can actually be done, although this is not always the case.
There may be a need to separate experimental material into "blocks" (e.g., in space or time) such that
there is more homogeneity within blocks, more heterogeneity between blocks, and each "treatment"
occurs once in each block. An example is a boat using different types of fishing gear meant to decrease
accidental seabird bycatch, in a set time period, in randomized order. Another is using different forest
management practices (the "treatments") in a relatively homogeneous area of land.
The concept of a "control" also requires some thought. In ecological studies where few areas are really
pristine, the concept of regional reference sites as "controls" has been used. Statistical replication is
necessary in order to get useful inference from the results. This means multiple experimental units must
receive the same "treatment." For example, in comparing different fishery management practices, at
least two regions would need to be subjected to each management practice. In the forest management
example mentioned above, several large blocks of land would be required; each block would have the
different forest management practices occur within it. One can obtain measurements on the same
experimental unit over time or space; that can add useful information, but would not be statistical
replication in the sense of adding more true experimental units. Accumulating lots of data points is not
necessarily the same as adding more statistical replicates. The usefulness of the subsequent data analysis
depends upon measuring meaningful responses at appropriate scales and using a good sampling design,
paying attention to the original objectives. Quality control at every step of the way is crucial.
It is also important to consider "what, when, where and how." "What" refers to the response variables
being monitored; these must be decided with care. Sometimes the response variables are obvious from
knowledge of the process being studied, sometimes they are not (see statement about pilot studies,
sample size and power analysis below). "When" and "where" refer to the temporal and spatial aspects of
monitoring, and "how" to the actual method of obtaining the data. In the context of adaptive
management, this could comprise carefully designing a long-term, large, or multi-stage monitoring
study, with chosen milestones when actual hypothesis testing or estimation of important parameters
occur.
Paying attention to Type I and Type II errors and their costs is also important. In classical statistics, a
Type I error occurs when data lead one to reject a specified null hypothesis (i.e., a hypothesis of "no
difference," or "no change") in favor of an alternative hypothesis, when the null hypothesis is actually
the true state of nature. A Type II error occurs when data result in the failure to choose the alternative
hypothesis when the alternative is actually the true state of nature. In adaptive management, attention
must be paid to the costs of making each type of error. Pilot studies can yield valuable information in
this regard before a large experiment or massive monitoring effort takes place. Pilot studies can also aid
in sample size determination, or even in the choice of response variable (e.g., one which has the most
statistical power to detect a certain level of change). Results from hypothesis tests or estimation of
33
-------
certain effects can then be incorporated into future decisions. While there are limitations to applying
statistical design of experimental principles when comparing management practices, one can still strive
to meet such principles.
Adaptive management requires scientists to look at every ecological monitoring effort as an experiment,
through the prism of statistical design, and to implement that monitoring using rigorous statistical
principles. This is not a trivial effort. Perhaps the biggest point in favor of an adaptive management
approach is that by applying statistical design principles to assess and compare ecological risk
management practices, any subsequent WOE arguments would then incorporate rigorous quantitative
results, with associated estimates of uncertainty.
Recommendation
• In order to implement effective adaptive management approaches, the EPA should apply principles
of rigorous statistical design.
3.7. Strengthening the EPA's Ecological Protection Goals
Charge Question 7. The RAF Action Plan indicates that there is little consensus in EPA
about goals for the protection of ecological systems, and that important and well-developed
ecological science principles (e.g., systems analysis, landscape ecology, ecosystem services
and adaptive management) have not been systematically integrated into the agency's science
policy framework. Are there aspects of ecological risk assessment science that make the
information difficult to communicate, use and process by decision-makers? What
recommendations does the Committee have to strengthen EPA 's ecological protection goals?
Please comment on how ecological assessment science can be used to strengthen EPA's
ecological protection goals.
EPA should develop stronger (i.e., clearly articulated, elucidated, and scientifically rigorous) ecological
protection goals. This need is particularly urgent because ecological protection goals are likely to
provide important context and guidance for the development of ecological assessment approaches used
by the agency. Clear articulation of ecological protection goals will lead to better definition of agency
decisions and an improved understanding of the ecological assessments needed to support those
decisions. Depending upon the nature of decisions to be made, different branches of ecosystem, social,
and behavioral sciences will have to be engaged. Improved communication will be driven by the needs
of the decision-making process. As mentioned in the Plan, increasing the representation and influence of
ecological scientists in the agency is likely to be crucial to strengthening and sustaining ecological
protection goals over the long term. As the EPA's ecological risk assessments evolve toward the concept
of ecosystem services, economists and other social scientists would also play a larger role in ecological
risk assessments in assessing public benefits from ecosystem protection. This is especially important as
EPA risk assessments move beyond such service endpoints as human health effects (e.g., number of
cancer cases reduced) toward other endpoints like commercial and recreational uses, and especially
aesthetic values and non-use values. For example, wetlands might have an adverse human health effect
associated with vector-borne disease, but provide beneficial services in the form of habitat for migratory
waterfowl and nurseries for species of fish. It is important to understand human values of the full range
of wetland services.
34
-------
Incorporation of ecosystem services into the ecological assessment process is an effective strategy to
strengthen the EPA's ecological protection goals because an ecosystem services framework explicitly
recognizes the interdependence of the ecosystem and human health. This framework could be further
expanded to include environmental justice, which recognizes that poor ecological conditions and/or
health can exacerbate exposure and the magnitude of negative impacts on vulnerable populations.
The concept and application of ecosystem services has been an important component of recent global
efforts by United Nations agencies to reduce excess nutrients delivered to coastal zones and the negative
environmental impacts associated with these excess nutrients, which are primarily harmful algal blooms
and hypoxia. Ecosystem services have been particularly important in conveying risks of continued
increases in hypoxia from excess nutrients. For example, very preliminary estimates indicate a possible
0.5% loss of global ecosystem services due to hypoxia. These services are valued at approximately 500
billion in 2012 dollars (R. Diaz and colleagues, unpublished data1). The factors that will lead to long-
term sustainability of ecosystem services are central to development of management strategies for
reducing nutrient loadings and consequently hypoxia (STAP 2011). The consequences of oil spill
impacts on estuarine and bay habitats, linking ecosystem structure to function, have also recently been
discussed (NRC 2012). Thus, the concept of ecosystem services could provide an overall framework for
assessment and lead to increased effectiveness and responsiveness of the EPA to the social, economic
and ecologic components of risk assessment. There are other clear examples linking ecosystem function
and services to human health. The assessment of the Milltown Reservoir on the Clark Fork River,
Montana and the assessment of the Coeur D'Alene River in Idaho (NRC 2005) are two examples.
Damaged ecosystems are a bellwether to damaged human health and economic and social well being. To
strengthen the EPA's ecological protection goals, agency risk assessors should make a more direct case
connecting human health to ecosystem function and services and follow this with development of
relevant ecosystem evaluation methods. In some cases, ecosystem function may conflict with some
aspects of human health (e.g., diseases transmitted by mosquitos; the presence of top predators such as
cougars), but there cannot be good human health without good ecosystem function. Ecosystem function
is therefore an input that should be considered as part of a human health risk assessment.
Many of the risk assessment practices recommended in the 2007 SAB report on advancing the science
and application of ecological risk assessment and in Dale et al. (2008) would lead to the development of
stronger ecological protection goals at the EPA and bring about improved protection of ecosystem
structure and function. Ultimately, the recommendations would not only lead to more robust
assessments, but also better understanding of ecosystem structure and function. This understanding, in
turn, would lead to better predictability of effects and wider applicability of ecological assessments in
similar situations.
Recommendations
• Scale, both in time and space, should be explicitly considered in the problem formulation stage of
ecological risk assessments. Aspects such as life history and scope-for-growth analyses are
important for predicting if a population, subject to stressors of a variety of types, will grow, stabilize
or shrink.
• Ecological risk assessments should link biomarkers of exposure to biomarkers of effect.
1 Robert Diaz, Virginia Institute of Marine Science, and colleagues estimated the value of global ecosystem services lost due to hypoxia. The
estimate is based upon the approach taken by Costanza et al. (1997). An inflation factor of 3% yr"1 was applied to report the value of ecosystem
services lost in 2012 dollars.
35
-------
• As previously discussed, post-remedial assessments and adaptive management programs should be
incorporated into the risk assessment and management process in order to adjust the remediation
approaches should this be necessary after the assessment.
• Success stories, in which environmental cleanup has led to cleaner air and water, should be
developed and used to enhance the communication process between risk managers, assessors and
environmental scientists. Examples illustrating approaches that have been less successful may also
be useful. An overarching recommendation of the SAB is that risk assessment teams, assessors and
managers use better communication techniques to educate managers, policy makers and the general
public.
36
-------
REFERENCES
Agee, J. 2003. Historical range of variability in eastern Cascade forests, Washington, USA. Landscape
Ecology 18:725-740.
Ager, A., A. McMahan, J. Hayes, and E.L. Smith. 2007. Modeling the effects of thinning on bark beetle
impacts and wildfire potential in the Blue Mountains of northeastern Oregon. Landscape and
Urban Planning 80:301-311.
Anastas, P.T. 2012. Fundamental changes to EPA's research enterprise: The path forward.
Environmental Science and Technology 46: 580-586.
Ankley, G., R.S. Bennett, R. J. Erickson, D. J. Hoff, M. W. Hornung, R. D. Johnson, D. R. Mount, J. W.
Nichols, C. L. Russom, P. K. Schmieder, J. A. Serrano, J.E. Tietge, and D.L. Villeneuve. 2009.
Adverse outcome pathways: a conceptual framework to support ecotoxicology research and risk
assessment. Environmental Toxicology and Chemistry 29:730-741.
Apitz, S.E. 2012. Conceptualizing the role of sediment in sustaining ecosystem services: Sediment-
ecosystem regional assessment (SEcoRA). Science of the Total Environment 415:9-30.
Ayre , K.K. and W.G. Landis. In press. A Bayesian approach to landscape ecological risk assessment
applied to the Upper Grande Ronde watershed, Oregon. Human and Ecological Risk Assessment.
Bailer, A. J., M.R. Hughes, K. See, R. Noble, and R. Schaefer R. 2002. A pooled response strategy for
combining multiple lines of evidence to quantitatively estimate impact. Human Ecological Risk
Assessment 8:1597-1611.
Barbour, R., M. Hemstrom, and J. Hayes. 2007. The Interior Northwest Landscape Analysis System: A
step toward understanding integrated landscape analysis. Landscape and Urban Planning
80:333-344.
Bartolo, R.E., R.A. van Dam, and P. Bayliss. 2012. Regional ecological risk assessment for Australia's
tropical rivers: Application of the relative risk model. Human Ecological Risk Assessment 18:16-
46.
Batley, G.E., G.A. Burton Jr., P.M. Chapman, and V.E. Forbes. 2002. Uncertainties in sediment quality
weight of evidence (WOE) assessments. Human Ecological Risk Assessment 8:1517-1547.
Bay, S., W. Berry, P.M. Chapman, R. Fairey, T. Gries, E. Long, D. MacDonald, and S.B. Weisberg.
2007. Evaluating consistency of best professional judgment in the application of a multiple lines
of evidence Sediment Quality Triad. Integrated Environmental Assessment and Management
3:491-497.
Ben-Gal, I. 2007. Bayesian networks. In Ruggeri, F., F. Faltin, and R. Kenett (Eds.), Encyclopedia of
Statistics in Quality and Reliability, Wiley & Sons, New York, NY.
Burton, G.A. Jr., P.M. Chapman, and E.S. Smith. 2002a. Weight-of-evidence approaches for assessing
ecosystem impairment. Human Ecological Risk Assessment 8:1657-1673.
37
-------
Burton, G.A. Jr., G.E. Batley, P.M. Chapman, V.E. Forbes, E.P. Smith, T. Reynoldson, C.E. Schlekat,
P.J. den Besten, A.J. Bailer, A.S. Green, and R.L. Dwyer. 2002b. A weight-of-evidence
framework for assessing sediment (or other) contamination: improving certainty in the decision-
making process. Human Ecological Risk Assessment 8:1675-1696.
Carriger, J. and M.G. Barron. 2011. Minimizing risks from spilled oil to ecosystem services using
influence diagrams: The Deepwater Horizon spill response. Environmental Science and
Technology 45:7631-7639.
Costanza, R., R. d'Arge, R. de Groot, S. Farberk, M. Grasso, B. Hannon, K. Limburg, I. Shahid Naeem,
R. V. O'Neill, J. Paruelo, R. G. Raskin, P. Sutton, and M. van den Belt. 1997. The value of the
world's ecosystem services and natural capital. Nature 387:253-260.
Chapman, P.M. 2011. Indices: attractive delusions. Integrated Environmental Assessment and
Management 7:313.
Chen, Q., J. Liu, K.C. Ho, and Z. Yang. 2012. Development of a relative risk model for evaluating
ecological risk of water environment in the Haihe River Basin estuary area. Science of the Total
Environment 420:79-89.
Cormier, S.M., and G.W. Suter II. 2008. A framework for fully integrating environmental assessment.
Environmental Management 42:543-556.
Dale, V.H., G.R. Biddinger, M.C. Newman, J.T. Oris, G.W. Suter II, T. Thompson, T.M. Armitage, J.L.
Meyer, R.M. Allen-King, E.F. Benfield, G.A. Burton, P.M. Chapman, Conquest LL, I.J.
Fernandez, W.G. Landis, L.L. Master, W.J. Mitsch, T.C. Mueller, C.F. Rabeni, A.D. Rodewald,
J.G. Sanders, and I.L. van Heerden. 2008. Enhancing the ecological risk assessment process.
Integrated Environmental Assessment and Management 4:306-313.
Fairbrother, A. 2003. Lines of evidence in wildlife risk assessments. Human and Ecological Risk
Assessment 9:1475-1491.
FAO (Food and Agriculture Organization of the United Nations). 1999. The Application of Risk
Communication to Food Standards and Safety Matters. Report of a Joint FAO/WHO Expert
Consultation, Rome, 2-6 February 1998.
[Available at: http://www.fao.Org/docrep/005/xl271e/X1271E00.htm#TOC]
Fenton, N., and M. Neil. 2010. Comparing risks of alternative medical diagnosis using Bayesian
arguments. Journal of Biomedical Informatics 43:485-495.
Gibbs, M. 2007. Assessing the risk of an aquaculture development on shorebirds using a Bayesian belief
model. Human and Ecological Risk Assessment 13:156-179.
Glass, G.V. 1976. Primary, secondary, and meta-analysis of research. Educational Researcher 5:3-8.
Glass, G.V. 1977. Integrating findings: The meta-analysis of research. Review of Research in Education
5:351-379.
38
-------
Glendining, N.S., and C.A. Pollino. 2012. Development of Bayesian network decision support tools to
support river rehabilitation works in the Lower Snowy River. Human and Ecological Risk
Assessment 18: 92-114
Grapentine, L, J. Anderson, G.A. Burton Jr., P.M. Chapman, C. DeBarros, C. Gaudet, G. Johnson, C.
Marvin, D. Milani, S. Painter, T. Pascoe, T. Reynoldson, L. Richman, K. Solomon, and R.
Turner. 2002. Development of a decision-making framework for sediment assessment. Human
and Ecological Risk Assessment 8:1641-1655.
Gregory, R., D. Ohlson, and J. Arvai. 2006. Deconstructing adaptive management: criteria for
applications to environmental management. Ecological Applications 16(6): 2411-2425.
Gregory, R., L. Failing, M. Harstone, G. Long, T. McDaniels, and D.Ohlson. 2012. Structured Decision
Making: A Practical Guide to Environmental Management Choices. Wiley-Blackwell, Hoboken,
NJ.
Green, R., and P.M. Chapman. 2011. The problem with indices. Marine Pollution Bulletin 62:1377-
1380.
Hedges, L.V., and I. Olkin. 1985. Statistical Methods for Meta-Analysis. Academic Press, Inc., Orlando,
FL.
Hill, A.B. 1965. The environment and disease: Association or causation? Journal of the Royal Society of
Medicine 58:295-300.
Huang, H., C-C. Liub, and X.J. Zhoub. 2010. Bayesian approach to transforming public gene expression
repositories into disease diagnosis databases. Proceedings of the National Academy of Sciences
107:6823-6828.
Jaworska, J., S. Gabbert, and T. Aldenberg T. 2010. Towards optimization of chemical testing under
REACH: a Bayesian network approach to integrated testing strategies. Regulatory Toxicology
and Pharmacology 57:157-67.
Kapo, K., and G.A. Burton Jr. 2006. A geographic information system based weights-of-evidence
approach for diagnosing aquatic ecosystem impairment. Environmental Toxicology and
Chemistry 25: 2237-2249.
Kapo, K.E., G.A. Burton Jr., D. De Zwart, L. Posthuma, and S.D. Dyer. 2008. Quantitative lines of
evidence for screening-level diagnostic assessment of regional fish community impacts: a
comparison of spatial database evaluation methods. Environmental Science and Technology
42:9412-9418.
Landis, W.G. 2009. Why has ecological risk assessment found such limited application? Human and
Ecological Risk Assessment 15:849-857.
39
-------
Large, M.M., and O. Nielssen. 2008. Factors associated with agreement between experts in evidence
about psychiatric injury. Journal of the American Academy of Psychiatry and the Law 36: 515-
521.
Levin, P.S., M.J. Fogarty, S.A. Murawski, and D. Fluharty. 2009. Integrated ecosystem assessments:
developing the scientific basis for ecosystem-based management of the ocean. PLoS Biol 7(1):
el000014. doi: 10.1371/journal.pbio. 1000014
Linkov I., D. Loney, S. Cormier, F.K. Satterstrom, and T. Bridges. 2009. Weight-of-evidence evaluation
in environmental assessment: Review of qualitative and quantitative approaches. Science of the
Total Environment 407:5199-5205.
Linkov I., P. Welle, D. Loney, A. Tkachuk, L. Canis, J.B. Kim, and T. Bridges. 2011. Use of
multicriteria decision analysis to support weight of evidence evaluation. Risk Analysis 31:1211-
1225.
Luxon, M., and W.G. Landis. 2005. Application of the relative risk model to the upper Willamette River
and lower McKenzie River, Oregon. In W.G. Landis (Ed.) Regional Scale Ecological Risk
Assessment Using the Relative Risk Model. CRC Press, Boca Raton, FL.
Marcot, B., R.S. Holthausen, M.G. Raphael, M.M. Rowland, and M.J. Wisdom. 2001. Using Bayesian
belief networks to evaluate fish and wildlife population variability under land management
alternatives from an environmental impact statement. Forest Ecology and Management 153:29-
42.
Marcot, B., P. Hohenlohe, S. Morey, R. Holmes, R. Molina, M.C. Turley, M.H. Huff, and J.A.
Laurence. 2006a. Characterizing species at risk II: Using Bayesian belief networks as decision
support tools to determine species conservation categories under the Northwest Forest Plan.
Ecology and Society. 1 l(2):article 12.
[Available at: http://www.ecologyandsociety.org/articles/1781.html]
Marcot, B., J. Steventon, G. Sutherland, and R. McCann. 2006b. Guidelines for developing and updating
Bayesian belief networks applied to ecological modeling and conservation. Canadian Journal of
Forest Research 36:3064-3074.
Millennium Ecosystem Assessment. 2005. Ecosystems and Human Well-being: Synthesis. Island Press,
Washington, DC.
Mooney, H.A. 2010, The ecosystem-service chain and the biological diversity crisis. Philosophical
Transactions of the Royal Society B 365: 31-39.
Moraes, R., and S. Molander. 2004. A procedure for ecological tiered assessment of risks (PETAR).
Human Ecological Risk Assessment 10:349-371.
Neill, J. 2012. Meta-analysis Research Methodology.
http://wilderdom.com/research/meta-analysis.html [Accessed April 29, 2012]
40
-------
Newman, M.C., Y. Zhao, and J.F. Carriger. 2007. Coastal and estuarine ecological risk assessment: the
need for a more formal approach to stressor identification. Hydrobiologia 577: 31-40.
NRC (National Research Council). 1989. Improving Risk Communication. National Academy Press,
Washington, DC.
NRC. 1994. Science and Judgment in Risk Assessment. National Academy Press, Washington, DC.
NRC. 1996. Understanding Risk: Informing Decisions in a Democratic Society. National Academy
Press, Washington, DC.
NRC. 2004. Adaptive Management for Water Resource Planning. Committee to Assess the U.S. Army
Corps of Engineers Methods of Analysis and Peer Review for Water Resources Project Planning.
National Academy Press, Washington, DC.
NRC. 2005. Superfund and Mining Megasites. National Academies Press, Washington, DC.
NRC. 2009. Science and Decisions: Advancing Risk Assessment. National Academies Press,
Washington, DC.
NRC. 2012. Approaches for Ecosystem Services Valuation for the Gulf of Mexico After the Deepwater
Horizon Oil Spill: Interim Report. National Academies Press, Washington, DC.
Nyberg, J., B. Marcot, and R. Sulyma. 2006. Using Bayesian belief networks in adaptive management.
Canadian Journal of Forest Research 36:3104-3116.
Pollino, C., A. White, and B. Hart. 2007. Examination of conflicts and improved strategies for the
management of an endangered Eucalypt species using Bayesian networks. Ecological Modeling
201:37-59.
Reynoldson, T.B., E.P. Smith, and A.J. Bailer. 2002. A comparison of three weight-of-evidence
approaches for integrating sediment contamination within and across lines of evidence. Human
and Ecological Risk Assessment 8:1613-1624.
Rosenthal, R. 2001. META-ANALYSIS: Recent Developments in Quantitative Methods for Literature
Reviews. Annual Review of Psychology. 52:59-82.
Rothman, K.J., and S. Greenland. 2005. Causation and causal inference in epidemiology. American
Journal of Public Health 95 (Suppl 1): S144-150.
SETAC (Society of Environmental Toxicology and Chemistry). 2012. Ecosystem Services: From Policy
to Practice.
http://sesss05. setae. eu/programme/presentations/?contentid=565&pr_id=470&last=487
[Accessed June 15, 2012]
STAP (Scientific and Technical Advisory Panel). 2011. Hypoxia and Nutrient Reduction in the Coastal
Zone. Advice for Prevention, Remediation and Research. A STAP Advisory Document. Global
Environment Facility, Washington, DC.
41
-------
Suter, G.W. II, and S.M. Cormier. 2011. Why and how to combine evidence in environmental
assessments: Weighing evidence and building cases. Science of the Total Environment 409:
1407-1417.
Suter, G.W. II, S.B. Norton, and A. Fairbrother. 2005. Individuals versus organisms versus populations
in the definition of ecological assessment endpoints. Integrated Environmental Assessment and
Management 1:397-400.
Swaen, G., and L. van Amelsvoort. 2009. A weight of evidence approach to causal inference. Journal of
Clinical Epidemiology 62: 270-277.
Thompson, B., S.B. Weisberg, A. Melwani, S. Lowe, J.A. Ranasinghe, D.B. Cadien, D.M. Dauer, R.J.
Diaz, W. Fields, M. Kellogg, D.E. Montagne, P.R. Ode, D.J. Reish, and P.N. Slattery 2012. Low
levels of agreement among experts using best professional judgment to assess benthic condition
in the San Francisco Estuary and Delta. Ecological Indicators 12:167-173.
U.S. EPA (U.S. Environmental Protection Agency). 1992. Framework for Ecological Risk Assessment.
EPA/630/R-92/001. U.S. Environmental Protection Agency Risk Assessment Forum,
Washington, DC.
[Available at: http://www.epa.gov/raf/publications/framework-eco-risk-assessment.htm]
U.S. EPA. 1998. Guidelines for Ecological Risk Assessment. EPA/630/R-95/002F, U.S. Environmental
Protection Agency Risk Assessment Forum, Washington, DC.
[Available at: http://www.epa.gov/raf/publications/guidelines-ecological-risk-assessment.htm]
U.S. EPA. 2000. Stressor Identification Guidance Document. EPA 822-B-00-025. U.S, Environmental
Protection Agency, Washington, DC
[Available at:
http://water.epa.gov/scitech/swguidance/standards/criteria/aqlife/biocriteria/upload/stressorid.pdf]
U.S. EPA. 2003. Generic Ecological Assessment Endpoints (GEAEs) for Ecological Risk Assessment.
EPA/630/P-02/004F, U. S. Environmental Protection Agency Risk Assessment Forum,
Washington, DC.
[Available online at: http://www.epa.gov/raf/publications/geae.htm]
U.S. EPA. 2005. Guidelines for Carcinogen Risk Assessment. EPA/630/P-03/001F, U.S. Environmental
Protection Agency, Washington, DC, USA.
[Available at: http://www.epa.gov/cancerguidelines/]
U.S. EPA. 2010a. Causal Analysis/Diagnosis Decision Information System (CADDIS). U.S.
Environmental Protection Agency Office of Research and Development, Washington, DC.
www.epa.gov/caddis [Accessed, May 8, 2012]
42
-------
U.S. EPA. 201 Ob. Integrating Ecological Assessment and Decision-Making at EPA: A Path Forward.
Results of a Colloquium in Response to Science Advisory Board and National Research Council
Recommendations. EPA/100/R-10/004, U.S. Environmental Protection Agency Risk Assessment
Forum, Washington, DC.
[Available at: http://www.epa.gov/raf/publications/pdfs/integrating-ecolog-assess-decision-
making.pdf]
U.S. EPA SAB (U.S. Environmental Protection Agency Science Advisory Board). 2007. Advice to EPA
on Advancing the Science and Application of Ecological Risk Assessment in Environmental
Decision Making. EPA-S AB-08-002.
[Available at:
http://yosemite.epa.gov/sab/sabproduct.nsf/7140DC0E56EB 148A8525737900043063/$File/sab-
08-002.pdf]
U.S. EPA SAB. 2008. SAB Advisory on the EPA Ecological Research Program Multi-Year Plan. EPA-
SAB-08-011.
[Available at:
http://yosemite.epa.gov/sab%5Csabproduct.nsf/3F81610BFF770E30852574D600718D4F/$File/
EPA-SAB-08-011-unsigned.pdf]
U.S. EPA SAB. 2009(a). Consultation on EPA 's Implementation of the Ecosystem Services Research
Program. EPA-SAB-09-019.
[Available at:
http://yosemite.epa.gov/sab%5Csabproduct.nsf/91190EEC56A44B3F85257641006BB7D7/$File
/EPA-S AB-09-019-unsigned.pdf]
U.S. EPA SAB. 2009(b) Valuing the Protection of Ecological Systems and Services. EPA-SAB-09-012.
[Available at:
http://yosemite. epa.gov/sab/sabproduct.nsf/F3DB lF5C6EF90EE1852575C500589157/$File/EP
A-SAB-09-012-unsigned.pdf]
U.S. EPA SAB. 2011. Efficacy of Ballast Water Treatment Systems: a Report by the EPA Science Advisory
Board. EPA-SAB-11-009.
[Available at:
http://yosemite.epa.gov/sab/sabproduct.nsf/6FFFlBFB6F4E09FD852578CB006E0149/$File/EPA-
SAB-1 l-009-unsigned.pdf]
U.S. EPA SAB. 2012. Science Integration for Decision Making at the U.S. Environmental Protection
Agency. EPA-SAB-12-008.
[Available at:
http://yosemite. epa.gov/sab/sabproduct.nsf/8AA27AA419BlD413 85257A330064A479/$File/EP
A-SAB-12-008-unsigned.pdf]
U.S. Fish and Wildlife Service and National Oceanic and Atmospheric Administration. 2012. National
Fish, Wildlife & Plants Climate Adaptation Strategy, Public Review Draft, January 2012,
http://www.wildlifeadaptationstrategy.gov [Accessed, March 20, 2012]
43
-------
Walker, R., W.G. Landis, and P. Brown. 2001. Developing a regional ecological risk assessment: A case
study of a Tasmanian agricultural catchment. Human and Ecological Risk Assessment 7:417-39.
Weed, D.L. 2005. Weight of evidence: A review of concept and methods. Risk Analysis 25:1545-1557.
Wenning, R., C. Ingersoll, G. Batley, and M. Moore (Eds.) 2005. Use of Sediment Quality Guidelines
(SQGs) and Related Tools for the Assessment of Contaminated Sediments. SET AC Press,
Pensacola, FL.
Willis, H.H. , M.L. DeKay, M.G. Morgan, H.K. Florig, and P.S. Fishbeck. 2004. Ecological risk
ranking: development and evaluation of a method for improving public participation in
environmental decision making. Risk Analysis 24(2) 363-378.
44
-------
APPENDIX A: THE CHARGE TO THE SAB
ACTION AGENCY
January 20, 2012
MEMORANDUM
SUBJECT: Transmittal of RAF Ecological Action Plan and Charge to the SAB EPEC
The RAF Ecological Action Plan, prepared following deliberation »n SAB
recommendations, is attached for transmittal to the SAB EPEC for their review and
comment. Also attached is a charge to the HPEC offering guidance for their review
scheduled for February 22 and 23. 2012. We are looking forward to the outcome of this
timely meeting regarding SAB/EPKC's feedback on the Action Plan.
Thank you for your assistance in this endeavor.
Attachment
FROM:
Edward Ohanian. Chair
EPA Risk Assessment forum
TO:
Thomas Armitage
EPA Science Advisory Board Staff Office
A-l
-------
Charge to the SAB Ecological Processes and Effects Committee for the Advisory on the EPA Risk
Assessment Forum's Ecological Assessment Action Plan
February 22 - 23,2012
The Risk Assessment Forum (RAF) in the EPA Office of the Science Advisor has developed an
Ecological Assessment Action Plan identifying six high priority overarching science policy initiatives
and seven specific technical practice initiatives to improve the quality, scope, and application of EPA's
ecological assessments. The initiatives in the Ecological Assessment Action Plan address high priority
recommendations in the report of an EPA colloquium, Integrating Ecological Assessment and Decision-
Making at EPA: A Path Forward. The EPA colloquium, which included ecologists from across the
Agency, was held in response to a 2007 SAB report titled, Advice to EPA on Advancing the Science and
Application of Ecological Risk Assessment in Environmental Decision-Making and the National
Research Council (NRC) report Science and Decisions: Advancing Risk Assessment (National Research
Council, 2009).
Summary of RAF Ecological Assessment Action Plan
The following science policy initiatives are proposed by the RAF in the Ecological Assessment
Action Plan to transform and improve the Agency's Ecological Risk Assessments:
• Develop Guidelines for Application of Systems Approaches to Ecological Assessments and
Integration of Different Types of Assessments to Solve Broad Environmental Problems
The design and conduct of complex large-scale assessments currently facing EPA (e.g., global
change, sustainability, estuarine and costal hypoxia, integrated nitrogen control, hydraulic
fracturing of deep geologic formations for methane extraction, mountain top mining, and deep
sea oil spills) requires a broad assessment framework. The RAF recommends that EPA develop a
systems approach to ecological assessments that includes multiple media and endpoints as well
as integration of different types of assessments described in Cormier and Suter (2008) and in
chapter 3 of the EPA colloquium report Integrating Ecological Assessment and Decision-Making
at EPA: A Path Forward. The framework focuses on resolving environmental problems by
integrating different types of assessments: (1) condition assessments to detect chemical, physical,
and biological impairments; (2) causal pathway assessments to determine causes and identify
their sources; (3) predictive assessments to estimate environmental, economic, and societal risks,
and benefits associated with different possible management actions; and (4) outcome
assessments to evaluate the results of the decisions of an integrative assessment.
• Improve Communication of Ecological Assessment Issues and Results
The RAF Action Plan calls for the development of methods for better communication of
ecological assessment issues and results to decision-makers and stakeholders. This applies to
communicating ecological assessment issues during both planning of assessments and
presentation of results. In part, this is a matter of the inability of assessors to communicate the
significance of the loss of species, changes in community structure, and other endpoints. In
addition, it involves the lack of standards for acceptability like those in human health assessment,
2 Cormier, S.M., and G. Suter. 2008. A Framework for Fully Integrating Environmental Assessment. Environmental
Management 42:543-556.
A-2
-------
the plethora of assessment methods employed, and difficulties in conveying variability and
uncertainty. Currently there is no EPA guidance for communicating ecological risks. A Risk
Assessment Forum panel is studying this issue.
• Incorporate Ecosystem Services and Benefits in Ecological Risk Assessments
The outcomes of research into ecosystem services and benefits are potentially transformational
for environmental sciences and decision-making. Ecosystem services can be used to describe
potential outcomes of environmental management decisions in terms that can be more effectively
communicated to decision-makers and the public. A Risk Assessment Forum panel is addressing
this issue and expects to produce case studies and guidance on how to relate ecological risk
assessment endpoints to ecosystem services. This information will be used to update the EPA
guidance document Generic Ecological Assessment Endpoints (GEAEs) for Ecological Risk
Assessment (U.S. EPA, 2003).
• Strengthen Science Policies that Promote Agency-wide Ecological Protection Goals
There is little consensus in the Agency about goals for protection of ecological systems or the
importance of ecological effects. In addition, important and well-developed ecological science
principles (e.g., systems analysis, landscape ecology, ecosystem services, and adaptive
management) are unfamiliar and have not been systematically integrated into the Agency's
science policy framework. If the Agency is to successfully incorporate ecology, it must consider
ways to elevate representation and influence of ecological scientists in its programs, regions, and
Intra-Agency science policy development and coordinating bodies.
• Incorporate Adaptive Management as a Formal Science Policy for EPA
Adaptive management is a process that determines the outcomes of actions and uses that
information to improve assessments that inform decisions, thereby improving the efficacy of
those decisions. Adaptive management has not been adopted as a policy at EPA. However, it is
conceptually well developed and has been widely adopted in numerous federal and state agencies
charged with ecological, fisheries, and wildlife management. The RAF recommends the
development of adaptive management for testing and revising risk management actions.
• Develop Weight-of-Evidence as an Option for Inference in Ecological Assessments
Although ecological assessments often involve multiple lines of evidence, there is no guidance
on how to weigh those lines of evidence to make inferences. The SAB identified a need for
guidance, case studies, and standards of practice for weighing multiple lines of evidence to
support decision-making. The weight-of-evidence should be used and fully documented during
problem formulation, data analysis and interpretation, and risk characterization. The RAF
recommends development of guidance on the use of weight-of-evidence.
The following specific technical practice initiatives are also proposed by the RAF in the
Ecological Assessment Action Plan:
A-3
-------
• Training and Improved Access to Information for Ecological Assessment - Risk assessor and
manager training and increased access to information will lead to improved quality of risk
assessments.
• Quality Assurance and Data Quality Objectives for Ecological Assessment - Quality assurance
and data quality objectives for ecological risk assessment will formalize ecological assessment
standards.
• Assessing the Risks of Multiple Stressors - Development of guidance is proposed for assessing
the risks of multiple stressors.
• Receptor-specific and Stressor-specific Guidance - Development of guidance is proposed for
common receptor and stressor-specific assessments.
• Life Cycle Analysis for Product Safety Evaluations - Development of guidance is proposed for
assessing new chemicals and other products using a life cycle approach. This will improve the
quality of assessments and decisions.
• Uncertainty Characterization and Communication - Guidance is proposed for characterizing
uncertainty and preparing risk communication information.
• State-of-the Science, Best Practices Reports, Exemplary Case Studies, and Success Stories -
This initiative will provide timely information on best practices to risk assessors.
Overarching Charge Question
Charge Question 1. Overall technical merit of the proposed science policy and technical practice
initiatives.
The RAF Ecological Assessment Action Plan proposes six high priority overarching science
policy initiatives and seven specific technical practice initiatives to improve the quality, scope,
and application of EPA's ecological assessments. Please comment on whether the initiatives
proposed in the Plan are a) responsive to SAB and NRC recommendations; and b) reflect the
most important set of activities needed to address the key scientific and technical challenges for
advancing the application of ecological risk assessment in environmental decision-making.
Please also consider whether there are other key science policy or technical practice initiatives
that should be considered for inclusion in the Plan.
Specific Charge Questions
Charge Question 2. Importance of developing an integrated assessment approach.
The RAF Action Plan proposes that EPA develop a systems approach to ecological assessments
that includes multiple media and endpoints as well as integration of different types of
assessments as described by Cormier and Suter in "A Framework for Fully Integrating
Environmental Assessment", Environmental Management 42:543-556, and in chapter 3 of the
EPA colloquium report Integrating Ecological Assessment and Decision-Making at EPA: A Path
Forward. The framework focuses on resolving environmental problems by integrating different
types of assessments: (1) condition assessments to detect chemical, physical, and biological
A-4
-------
impairments; (2) causal pathway assessments to determine causes and identify their sources; (3)
predictive assessments to estimate environmental, economic, and societal risks, and benefits
associated with different possible management actions; and (4) outcome assessments to evaluate
the results of the decisions of an integrative assessment. Please comment on how guidance for an
approach to assessment that integrates different media and endpoints and different types of
assessments might contribute to better decision making (e.g., assessment of complex issues,
cumulative risk assessment and sustainability analysis).
Charge Question 3. Use of the weight-of-evidence approach in ecological risk assessments.
Although ecological assessments often involve multiple lines of evidence, no guidance exists on
how to weigh those lines of evidence to make inferences. The RAF Action Plan proposes that
EPA develop such guidance. Please comment on the scientific merit and limitations of using a
weight of evidence approach in decision making and offer any guidance on weighing ecological
risk assessment (ERA) lines of evidence.
Charge Question 4. Communication of ecological assessment issues and results to decision-makers and
stakeholders.
The RAF Action Plan calls for the development of methods for better communication of
ecological assessment issues and results to decision-makers and stakeholders. This applies to
communicating ecological assessment issues during both planning of assessments and
presentation of results. In part, this may be a matter of the inability of assessors to communicate
the significance of the loss of species, changes in community structure, and other endpoints. The
RAF has developed a communication technical panel project description. Please comment on
whether the RAF's planned project is an appropriate way to proceed, and what obstacles might
exist to either interpreting or utilizing ecological information in risk assessment. Please include
any observations on why ERA has or has not been well incorporated into decision making in
general.
Charge Question 5. Incorporation of ecosystem services into ecological risk assessment methods.
Ecosystem services can be used to describe potential outcomes of environmental management
decisions in terms that can be more effectively communicated to decision-makers and the public.
RAF expects to produce guidance on how to relate ecological risk assessment endpoints to
ecosystem services. This information will be used to update the EPA guidance document
Generic Ecological Assessment Endpoints (GEAEs) for Ecological Risk Assessment (U.S. EPA,
2003). Please consider Appendix B (page 52) of the generic ecological assessment endpoints
guidance document and the work plan for the RAF Technical Panel on Ecological Services
Assessment Endpoints and comment on whether they capture the full range of opportunities to
incorporate ecosystem services into EPA's ecological risk assessment methods.
Charge Question 6. Use of adaptive management for testing and revising risk management actions.
In its 2007 report, Advice to EPA on Advancing the Science and Application of Ecological Risk
Assessment in Environmental Decision-Making, the SAB recommended that EPA use adaptive
management to address uncertainties in decision-making. The application of adaptive
management in risk assessment and risk management is discussed in section 6.3 of the EPA
A-5
-------
colloquium report Integrating Ecological Assessment and Decision-Making at EPA: A Path
Forward, and the RAF Action Plan proposes the development of adaptive management as a tool
to methodically improve risk management decisions. Please comment on how adaptive
management approaches can be developed to provide optimal value for EPA programs.
Charge Question 7. Strengthening EPA 's ecological protection goals.
The RAF Action Plan indicates that there is little consensus in EPA about goals for the
protection of ecological systems, and that important and well-developed ecological science
principles (e.g., systems analysis, landscape ecology, ecosystem services, and adaptive
management) have not been systematically integrated into the Agency's science policy
framework. Are there aspects of ERA science that make the information difficult to
communicate, use and process by decision makers? What recommendations does the committee
have to strengthen EPA's ecological protection goals? Please comment on how ecological
assessment science can be used to strengthen EPA's ecological protection goals?
A-6
------- |