United States
          Environmental Protection
            Office of Research and
            Washington DC 20460
February 2000
Workshop Report on
Characterizing Ecological
Risk at the Watershed Scale
July 7-8, 1999
Arlington, VA

                                          February 2000
 Workshop Report on Characterizing
Ecological Risk at the Watershed Scale
                July 7-8, 1999
                Arlington, VA
  National Center for Environmental Assessment-W
        Office of Research and Development
       U.S. Environmental Protection Agency
               Washington, DC


This document has been reviewed in accordance with U.S. Environmental Protection Agency
policy and approved for publication. Mention of trade names or commercial products does not
constitute endorsement or recommendation for use.

As ecological risk assessment evolves, it is moving beyond a focus on single species toward
addressing multiple species and their interactions, and from assessing effects of simple chemical
toxicity to the cumulative impacts of multiple interacting chemical, physical, and biological
stressors. While EPA and others have developed guidance and have considerable experience in
applying the ecological risk assessment paradigm in source-based approaches (such as those
focused on particular chemical contaminants), specific guidance for using it in "place-based"
approaches (such as those conducted on a watershed-wide scale) is  still limited.

One of the principal challenges in applying ecological risk assessment to watershed management
and decision making is the need for a framework for characterizing  risks that involve numerous
stressors, interconnected pathways, and multiple endpoints. To develop the needed guidance  in
this area a workshop was held that gathered 35 participants with extensive experience in relevant
disciplines such as watershed ecological risk assessment, ecological risk assessment, watershed
management, or regional-scale assessment.  To focus workshop discussions, several charge
questions addressing the issues of greatest concern for characterizing risk at the watershed scale
were developed prior to the workshop.  This report provides the proceedings from the workshop
and includes an introduction, a synthesis of discussion and recommendations, and summaries  of
presentations and breakout sessions.

LIST OF TABLES AND FIGURES                                                 v
FOREWORD                                                                    vii
AUTHORS AND REVIEWERS                                                  viii

SECTION ONE - INTRODUCTION                                              1-1
       1.1   Background and Purpose 	 1-1
       1.2   Workshop Organization	 1-1
       1.3   Charge Questions	 1-2

       2.1   Synthesis of Discussion Points and General Recommendations	2-1

       3.1   Welcome and Introductory Comments on Place-Based Ecological
            Assessments	3-1
       3.2   Workshop Goals and Approach  	3-2
       3.3   Risk Characterization Summary  	3-3
       3.4   Traditions of Environmental Management and Their Implications	3-7

       4.1   Big Darby Creek	4-1
       4.2   Middle Platte 	4-10
       4.3   Waquoit Bay  	4-11
       4.4   Middle Snake	4-12
       4.5   Clinch Valley 	4-14

SECTION FIVE - FACILITATOR PRESENTATIONS                              5-1
       5.1   Jack Gentile, Center for Marine and Environmental Analysis, University
            of Miami	5-1
       5.2   William Smith, Yale School of Environment and Forestry Studies, Yale Universi3y2
       5.3   Patrick Bourgeron, INSTAAR University of Colorado, Colorado State
            University	5-3

SECTION SIX - BREAKOUT SESSION SUMMARIES                             6-1
       6.1   Breakout Team 1  	6-1
       6.2   Breakout Team 2  	6-4
       6.3   Breakout Team 3  	6-9

                            CONTENTS (continued)


APPENDIX A: List Of Workshop Participants                                   A-l

APPENDIX B: Workshop Agenda                                             B-l

APPENDIX C: Ecological Risk Assessment Guidelines, Section on Risk
Characterization  	C-l

                                  LIST OF TABLES
2-1    Middle Snake River Case Study: Sample Table Integrating Stressors, Responses, and
       Recovery Potential  	2-2
                                 LIST OF FIGURES

2-1    Clinch Valley Case Study:  Conceptual Model  for Clinch Eco-risk and Assessment
       Approaches Used	2-5

2-2    Middle Snake River Case Study:  Map of Snake River Showing Probability of Life
       Stage Impairment for Rainbow Trout	2-7

2-3    Waquoit Bay Case Study:  Management Decision Process for Waquoit Bay
       Assessment 	2-10

3-1    Ecological Risk Assessment Framework	3-4

3-2    Proximity of Management to Action on the Ground	3-9

4-1    Big Darby Creek Case Study: Sample Box Plots of IBI Metrics Showing Source
       Signatures 	4-2

4-2    Big Darby Creek Case Study: Map Showing IBI Scores by Location	4-4

4-3    Big Darby Creek Case Study: Sample Plots of ICI Versus River Mile	4-5

4-4    Big Darby Creek Case Study: Sample Radar Plots of Trend Analysis of Response
       Variables	4-6

4-5    Big Darby Creek Case Study: Map Showing Erosion Potential Based on Model
       Predictions	4-7

4-6    Big Darby Creek Case Study: Trends in Stressors Using Radar Plots  	4-8

4-7    Big Darby Creek Case Study: Longitudinal Plot of Stressor Versus River Mile	4-9

                            LIST OF FIGURES (continued)
4-8    Middle Snake River Case Study: Snake River Illustration Linking Stressors to
       Recovery Potential for Rainbow Trout	4-13

6-1    Clinch Valley Case Study: Map Showing Land Use Versus Threatened and
       Endangered Species  	6-5

Risk assessment is playing an increasingly important role in determining environmental policies
and decisions at the U.S. Environmental Protection Agency (EPA). EPA's first Agency-wide
guidelines for ecological risk assessment, published in May 1998, provided a broad framework
applicable to a range of environmental problems associated with chemical, physical, and biological
stressors (U.S. EPA, 1998). However, while EPA has considerable experience in applying the
ecological risk assessment paradigm in source-based approaches (such as those focused on
particular chemical contaminants), specific guidance  on "place-based" approaches (such as those
conducted on a watershed-wide scale) is still limited.

As ecological risk assessment evolves, it is moving beyond a focus on single species toward
addressing multiple species and their interactions,  and from assessing effects of simple chemical
toxicity to the cumulative impacts of multiple interacting chemical, physical, and biological
stressors. Ecological risk assessment will play a major role in bringing science into place-based
decision making, but its application faces several challenges. Among  those challenges is the need
for a framework for characterizing risks that involve numerous stressors, interconnected
pathways, and multiple endpoints.

The risk characterization phase of ecological risk assessment is the culmination of planning,
problem formulation, and analysis.  The objective is to weave together the various lines of
evidence to formulate a clear and compelling summary of the exposure and effects data. Because
it is intended to support scientifically sound decisions by risk managers, the risk characterization
must also address the uncertainties, assumptions, and qualifiers embedded in the risk assessment.
The recommendations developed from this workshop represent a significant step forward in
meeting the challenges of risk characterization on a watershed scale.
                                  Michael Slimak
                                  Associate Director of Ecology
                                  EPA, National  Center for Environmental Assessment

                           AUTHORS AND REVIEWERS
The National Center for Environmental Assessment-Washington Office (NCEA-W) of EPA's
Office of Research and Development was responsible for the preparation of this document. The
first draft was prepared by Tennessee Valley Authority under Interagency Agreement No.
DW64866901 for which Victor B. Serveiss of NCEA-W served as the EPA Project Officer.
Victor B. Serveiss
National Center for Environmental Assessment
U.S. Environmental Protection Agency
Washington DC 20460

Janice P. Cox
Jennifer Moses
Bruce L. Yeager
Tennessee Valley Authority
Knoxville, TN 37901

James Andreasen, U.S. EPA, ORD
Ed Bender, U.S. EPA, ORD
Pat Cirone, U.S. EPA, Region 10
Bill Ewald, U.S. EPA, ORD
Jerry Diamond, Tetra Tech
Susan Ferenc, ILSI Risk Science Institute
Bob Fenemore, U.S. EPA, Region 7
Jack Gentile, University of Miami
Ken Jones, Green Mountain Institute for Environmental Democracy
Robert Murphy, Alliance for the Chesapeake Bay
Sue Norton, U.S.  EPA, ORD
Don Rodier, U.S. EPA, OPPTS
Anne Sergeant, U.S. EPA, ORD
Michael Slimak, U.S. EPA, ORD
Parti Tyler, U.S. EPA, Region 1
William Smith, Yale University
Barry Tonning, Tetra Tech (at time of workshop-Council of State Governments)
William van der Schalie, U.S. EPA, ORD


This workshop was funded through an interagency agreement between EPA and the Tennessee
Valley Authority (TV A).  I would like to thank TV A, EPA, and other staff for their support and
hard work in making this workshop a success. First, thanks to Bill van der Schalie and Sue
Norton for their insight provided in the many discussions we held to plan the workshop. From the
TVA group, Angela Stiles coordinated the vast majority of the logistical activity to get the right
people in one good place at the same time. Jennifer Moses, Janice Cox, and Bruce Yeager took
the notes that formed the basis for the workshop report. Forrest Rich conceived the idea that
EPA and TVA should work together in putting on this workshop because TVA was already
involved in addressing watershed risk communication issues and had experience putting on
workshops.  Janice Dockery helped with formatting, typing and distribution of the report.

Although many panel members provided input into developing the charge questions, the extensive
assistance of Sue Norton and Bill van der Schalie (EPA National  Center for Environmental
Assessment [NCEA]), Maggie Geist (Waquoit Bay National Estuarine Research Reserve), Pat
Cirone (EPA Region I) and Jerry Diamond (Tetra Tech) warrants specific acknowledgment.  Jerry
Diamond and Pat Cirone also made a major contribution to the workshop by developing draft
watershed ecological risk characterizations that provided examples for participants to consider in
responding to the charge questions.  Jerry Diamond and Pat Cirone's presentations on Clinch
Valley and Middle Snake assessments, respectively, provided the  basis for many of the
recommendations contained in this report. The success of the Clinch Valley assessment would
not have been possible without the dedicated efforts of Don Gowan (The Nature Conservancy)
and Roberta Hylton (U.S. Fish and Wildlife Service).

The initial plenary sessions successfully  set the stage for the discussions that followed, and I wish
to thank Mike Slimak, Sue Norton and Bill van der Schalie, all from the EPA NCEA, for their
various perspectives on ecological risk assessment and interactions with risk management.
Several others presented their work on watershed or regional-scale ecological risk assessments,
including Susan Cormier (EPA National Exposure Research Laboratory, Cincinnati); Bob
Fenemore and Maria Downing (EPA Region 7); Maggie Geist and Parti Tyler (EPA Region  1);
Jack Gentile (University of Miami); William Smith (Yale University); and Patrick Bourgeron
(University of Colorado). Their presentations contributed to the workshop discussion and
recommendations. Drs. Gentile, Smith,  and Bourgeron also served as facilitators to guide and
maintain the focus of the break-out discussions in developing responses to the charge questions.

Finally, while there are too many to list individually here (see Appendix A), I acknowledge with
gratitude everyone who participated in the workshop and contributed to this report. Addressing
the charge questions was not an easy task, and the subject required open and frank
communication and sharing and documenting this collective wisdom.
                                  Victor B. Serveiss, Workshop Organizer
                                  U.S. EPA, National Center for Environmental Assessment

                                    SECTION ONE

The workshop described in this report was conducted to further develop and document the
process for the risk characterization phase of watershed-scale ecological risk assessment. The
recommendations reflect the responses of workshop participants to charge questions addressing
those aspects of watershed-scale ecological risk characterization deemed most in need of a
procedural framework.

Participants were invited to the workshop because of their experience in ecological risk
assessment, watershed management, or regional-scale assessment. Many of the attendees have
been active in five EPA-sponsored watershed assessments that were initiated in 1993 to
demonstrate application of the ecological risk assessment process to large scale, place-based
studies. The idea to hold this workshop originated when a few of the chairs of these assessments
noted a lack of specific guidance for developing such watershed risk characterizations. The
workshop was also designed to provide feedback to help complete and improve these five
watershed-scale assessments.

The recommendations developed through the workshop are intended to supplement the
Guidelines for Ecological Risk Assessment (U.S.  EPA, 1998) and the Risk Characterization
Handbook, presently being developed by EPA (U.S. EPA, 1999).  While the proceedings from the
workshop include comments that replicate material in the 1998 EPA Guidelines and the
Handbook, this report assumes the reader is already familiar with the principles of risk
characterization as described in the Guidelines (included as Appendix C).

The workshop was held on July 7 and 8, 1999, at the Crystal City Marriott in Arlington, Virginia.
Participants at the workshop represented federal agencies, academia, consulting firms, and
environmental organizations. The workshop was led by three facilitators with extensive
experience in large scale ecological assessment.  The other attendees included 13  participants in
one or more of the prototype watershed assessments, five authors of the Guidelines and 22 others
with backgrounds in ecological risk assessment or ecological resource management at various
spatial scales. The program began with individual presentations, then continued as breakout
group discussions that generated recommendations for conducting watershed-scale ecological risk
characterization.  A list of workshop participants is included as Appendix A.

The workshop began with opening remarks from four EPA National Center for Environmental
Assessment (NCEA) staff.  First, Mike Slimak, Associate Director of Ecology, welcomed
everyone to the workshop and provided a historical background. Next, Vic Serveiss, the
workshop organizer, provided an overview of the workshop plans.  Bill van der Schalie then
summarized the major principles of risk characterization as described in the Guidelines and


Handbook, and Sue Norton discussed different traditions in environmental management and their
implications for watershed ecological risk assessment.  The remainder of the day was dedicated to
presentations on the five prototype watershed assessments.

The second day of the workshop opened with presentations from each of the three facilitators
based on their experience in ecological assessment as related to the charge questions.  The group
then divided into three facilitator-led breakout sessions, each with the goal of developing
recommendations for conducting watershed ecological risk characterization.  The results from
each breakout session were reported back to the entire group.  The workshop agenda is included
as Appendix B.

The following charge questions were developed to focus workshop discussions:

•   How can the watershed assessment drafts (or plans) for Clinch Valley, Middle Snake, and
    Waquoit Bay be improved, especially in regard to the other charge questions listed below?

•   How should exposure and effects data be integrated in a watershed context to generate a risk

•   How should uncertainty be addressed and presented (e.g., incomplete data or analyses,
    qualitative estimates, data at different spatial and organizational levels)?

•   What are the best ways to address and present the integration of qualitative and quantitative
    lines of evidence for a) an individual assessment endpoint and b) drawing overall conclusions?

•   How should the degree of adversity of predicted or observed effects in watersheds be
    described (e.g., considering nature and intensity of effects, spatial and temporal  scale, and the
    potential for recovery)?

•   How are alternative management options selected to have their risks characterized and how is
    management informed of the consequences?

•   How should ecological risk be communicated to the manager and the public (journal articles,
    computer programs, presentations, town meetings, Internet, fact-sheets)?

                                    SECTION TWO

This section summarizes recommendations made in response to the charge questions and is based
on the individual presentations, the three breakout team sessions and the report-back
presentations that followed the breakout sessions. More complete documentation of the
individual break-out group discussions is included in Section 6 of this report.  Attempts to
characterize the degree of agreement on discussion points or recommendations below are based
on notes taken and observations made during the discussions among the participants who actively
expressed opinions. The group was not polled on any of the points presented.
CHARGE QUESTION: How should exposure and effects data be integrated in a watershed
context to generate a risk estimate?

Use the conceptual model to review the purpose
•  As an introduction to presenting the data, revisit the purpose of the assessment, the
   assessment endpoints, and the pathways described by the conceptual model.  This helps
   reiterate the value of the findings to the risk manager.

Use models, associations, and multivariate analysis
•  Dose-response curves are very effective in a single stressor - single endpoint scenario.
   However, in watersheds the more common scenario will require analyzing the impacts of
   multiple stressors on multiple endpoints.  In such situations, models and multivariate analysis
   can be used to analyze and present associations between stressors and impacts.

Work with multiple lines of evidence
•  Lines of evidence may be brought together to increase confidence in associations between
   stressors and responses.  This effective approach was taken in the Middle Snake River
   assessment and is presented in summary form in Tables 17-20 of the draft project report (see
   Table 2-1).  References to the literature are supportive.  Using multiple lines of evidence from
   observations in the field may actually be more useful than dose-response information derived
   from laboratory data.

Look for associations by category
•  Risk assessors often strive to present the relationships between stressors and effects in the
   form of dose-response curves. However, categorical information can sometimes provide a
   clearer picture of a relationship than is evident from continuous data. In examining
   relationships between stream habitat quality and biological condition in the Clinch Valleyjisk
   assessment, for example, continuous "dose-response" relationships were not apparent.
   However, by categorizing habitat quality as either poor, fair, good,  or excellent  (based on
   predetermined decision criteria) and relating these habitat categories to either continuous or

     Table 2-1. Middle Snake River Case Study: Sample Table Integrating Stressors, Responses, and Recovery Potential

     Factors limiting reproduction, growth, and survival of the rainbow trout population in the Middle Snake River.
Number of
Loss of adult habitat
(e.g., stream side
overhanging banks,
and woody debris)
Sedimentation, high
water temperature,
and land use on
Unstable stream flow
during the spring and
high water
Loss of habitat (e.g.,
deep holes and large
woody debris)
Sedimentation and
increased water
Hybridization with
stocked fish
An increase in the
population size is not
possible with low or no
Population of native fish
can not recover without
successful reproduction
Population can not
recover without
successful recruitment
Poor growth as
invertebrate fauna do
not support cold water
Poor survival with
mixed genotype
Low, field studies show low
numbers of adult fish are
present in the Middle Snake
Moderate, historic spawning
areas in the main channel have
not been documented
Rearing areas in the main stem
of the Middle Snake River
identified using habitat
suitability indices
High, no information available
on the amount of
overwintering habitat in the
main stem
Moderate, adequate analysis of
sampling information has not
been completed
Low, effects of hybridization
on native fish are known
Lack of habitat is a
main factor limiting
the size of the adult
Poor spawning
success attributed to
poor water quality
Rearing habitat
important for
maintaining and
increasing adult
Overwintering habitat
can limit the size of
the adult population
Food supply can limit
the size of a rainbow
trout population
An adequate
population of native
fish remains
Recovery Potential
Good, if habitat improvements
can be improved, but low if a
stable annual flow regime is
not maintained
Low, without improving water
quality, reducing
sedimentation, and controlling
land/water use on tributaries
Low, the carrying capacity for
native fish was likely
permanently reduced by dam
construction in the Middle
Snake River
Low, without improving lotic
conditions, lowering water
temperature, and controlling
Good, provided that existing
native fish are protected from
stocked hatchery fish
     Note: 1.  LOE - lines of evidence, HSI - habitat suitability indices, WQS - water quality standards, Field - field  surveys in the Middle Snake River, LIT - literature,
         BPJ - best professional judgement.

categorical expressions of the biological condition, the relationships between habitat and biology
were much easier to visualize.

 Use the right scale and data
    Consider partitioning the spatial scale into smaller units with regard to availability of data,
    occurrence of problems, subwatersheds or sub-ecosystem types, and management authorities.
    In some cases, it may be possible to assess relationships within subwatersheds that have
    comprehensive data bases, and then extrapolate to the larger area. On the other hand, some
    problems may impact only a small subset of the area being assessed. More discussion on the
    scale issue is available in a workshop report intended to develop a problem formulation
    process for large spatial scales (TN & Associates, 1999).

•   Partitioning a watershed into smaller units, especially those with similar attributes such as
    hydrology, structure, function, and habitats, makes the effort more manageable. An example
    of this is described in more detail on Page 5-1.

    The data should be appropriate for the scale being addressed: for example,  satellite remote
    sensing data might be appropriate for larger scales, but smaller scales may require field
CHARGE QUESTION:  How should uncertainty be addressed and presented (e.g.,
incomplete data or analyses, qualitative estimates, data at different spatial and
organizational levels)?

Focus on confidence
•  It is preferable to discuss the "degree of confidence" in the results rather than the "level of
   uncertainty," especially when communicating with a relatively non-technical audience.

•  Recognize that most decision makers don't require the degree of certainty that scientists seek
   to attain.

•  In cases where the information is inadequate to define a stressor-response curve or even
   document a causal relationship, the discussion  should address the strength of association using
   standard models or methods for causal inference, such as  those presented in the EPA
   Guidelines (U.S. EPA, 1998).

Consider the risk manager's perspective
   The "acceptable" degree of uncertainty (or level of confidence) in a risk assessment may be
   influenced by the perspective of the audience (e.g., scientist, risk manager, stakeholder), the
   magnitude of the risk,  and the cost associated with decreasing the degree of uncertainty.
   Tables can be used to compare accuracy and precision versus the degree of risk or the cost to
   reduce uncertainty, although it may be preferable to use such tables for generating estimates
   without incorporating them into the actual risk characterization report.

•  Information may be characterized simply as having a low, medium, or high degree of
   confidence. This approach can be applied to the integrated effects and exposure data or to
   each set of data individually.

•  Risk characterization should consider the context in which risk management decisions are
   being made and the implications of risk management decisions. For example, reducing
   uncertainties in data may have a low priority if there are no feasible management options.

   The simplified form of the Clinch Valley conceptual model is a good example of how to
   illustrate and communicate the pathways, linkages, associations and locations of uncertainties
   (see Figure 2-1).

Focus on the key uncertainties but acknowledge all of them
•  Participants were divided on the issue of the scope for the discussion of uncertainties.  Some
   participants preferred a narrow focus to address only "key" uncertainties (those with the
   potential to influence the outcome),  thus avoiding an exhaustive "laundry list" of all potential
   uncertainties.  Others preferred presentation of a comprehensive list of uncertainties, feeling
   that omissions were more costly errors than inaccuracies.

•  An intermediate approach mentioned was to assemble a comprehensive inventory of
   uncertainties, but then to partition the list based on importance (e.g., ranking as high, medium,
   or low). However, one danger identified with using ranking is that it adds another layer of
   judgment and may lead the reader to disregard important information.

•  Although it can be useful to identify the factors most likely to affect the outcome, the risk
   characterization should strive to make the uncertainties transparent so the reader can judge
   which ones are more important.

•  Distinguish between qualitative uncertainties (e.g., is there a cause-and-effect relationship?)
   and quantitative uncertainties (e.g.,  what does the  stressor-response curve  look like?).
   Address uncertainties quantitatively when possible, but avoid conducting more quantitative
   uncertainly analysis than is genuinely necessary. The quantitative uncertainty evaluation
   (including references to standard tests, metrics, etc.) should be explicitly presented, but may
   be included in an appendix.
CHARGE QUESTION: What are the best ways to address and present the integration of
qualitative and quantitative lines of evidence for a) an individual assessment endpoint and
b) drawing overall conclusions?

Use visuals
•  Many participants favored the format used in the Snake River assessment (see Table 2-1). An
   advantage of this format is that it can summarize both qualitative and quantitative lines of
   evidence. Such a table can be supplemented with a narrative text that refers the reader to

   Conceptual Model for Clinch EcoRisk and Assessment Approaches Used
(Not meant to be an exhaustive model, rather for purposes of the Analysis Plan)
Sources: — i
- GIS/statistical analyses
of CPRATS data;
- Riparian buffer analyses
Copper Creek
Stressors: — '
Habitat Q

^^ embeddedness ^) f
Other Factors: ^"Elev



Land Use

^* 	 ^\

S^T . , ~^x /- — S x-— ~~~~ ^~~~-< f Transportation \
CAgriciilture^ (Mining) (^UAan/IndustiM^ ( Corridors )
4 v
' ^
V \' \'
uality (CPRATS) Water/Sediment C
^ 	 ^ ( ^-— — ^j.

v ^/^ "
^( Spills

(historical records, Lone
Mln, APCO)
T r

)uality Hydrology

Cover ^ /^ Riparian \ (Toxic^) (^trients C?T> /^rought^N
^-_-^ 	 1 (STORE

ography^) (e.g., slope*)

an CorridorJ^) (looked at in Copper Creek*)

topographyj^) (looked at in Copper Creek*)
Mussel Recruitment/Diversity

T^ v^ood^y
(USGS gages)

Native Fish
Multi-variate and
other analyses on
CMCP data and
T&E species and
land use
Measurement Endpoints:    T&E species
                                       EPT *      CMCP            T&E species

                        Denotes that aspect has been examined in the risk assessment thus far.
Figure 2-1. Clinch Valley Case Study: Conceptual Model for Clinch Eco-risk and Assessment Approaches Used.

    other sections of the report for supporting information, and the very detailed data can be
    moved to appendices. It was also suggested that such a table include citations to technical
    references, or in cases where conclusions are based on best professional judgment, identify the
    scientist(s) drawing the conclusion.

•   Maps and map overlays may be helpful in illustrating stressors and impacts.

Use rankings
    Qualitative lines of evidence should be considered and could be ranked as having a high,
    medium, or low level of importance or confidence. Risk assessor's need to use professional
    judgement to sort qualitative aspects of issues such as degree of adversity, strength of
    association, level of confidence, importance to endpoint, etc. into some coarse categories.
    Addressing  the ranked issues in order of importance avoids being overwhelmed because there
    is never enough data to address all concerns and avioids getting into a never-ending analysis
    of all the potential uncertainties.

    Several participants recommended avoiding techniques that numerically rank or weight
    various lines of evidence; others recommended that such techniques only be used with great
    caution; and still others considered such techniques to be extremely useful.  The concern with
    weighting qualitative data, lines of evidence, or measures of strength of association was that it
    introduces an additional source of uncertainty.

Presentation style
•   As each assessment endpoint is discussed, keep the presentation format relatively constant to
    enhance continuity of the text.

•   Any contradictions between the various lines of evidence should be explained.
CHARGE QUESTION: How should the degree of adversity of predicted or observed
effects in watersheds be described (e.g., considering nature and intensity of effects, spatial
and temporal scale, and the potential for recovery)?

Use visuals
•   The table in the Snake River assessment (reproduced as Table 2-1 for this report) and the map
    showing degree of impairment for various life stages along the stream reach (see Figure 2-2)
    are a useful format for illustrating observed effects and predictions for recovery. Such visuals
    should be accompanied by narrative descriptions.

Consider the historical context
•   Trend assessment and comparison with historical data (including historical maps or photos)
    can be useful. However, caution is warranted because historical data may be of poor or
    unknown quality, and because historical trends are not always predictive of future trends.

                                                     Straight Lin* Representation of Impairment and River Miles

                                         f« , ff, , #, , P, f fft , |» , ffi , ffn ff, . ftf , f»M ff, , ff, , ("I", , fTU Pfl  fl'i , f'l I ff, I ff, I ff. I ff M ffl I f"
          it r» Ow Irte flage » * lundton of Oat m B
           [IIMWI tapfcn*)
           VdKl grtilu HI i and >m tan « BJJ>| B J
FIGURE    Snake River Probability of Life Stage Impairment for Rainbow Trout
       Life Stage (bottom to upper line) Spawn, Incubation, Fry. Adult
                                                            0  **•
                                                            FT]  hwitaiBilMM
                                                            ' |Tj  TlWMK
Figure 2-2.    Middle Snake River Case Study:  Map of Snake River Showing
                  Probability of Life Stage Impairment for Rainbow Trout

Additional Considerations
•  Discussions of recovery should include a clear statement of the recovery goal (including
   whether recovery is to be natural or "assisted") and the potential impact of natural and
   anthropogenic factors, including ecosystem interactions.

•  If an adverse effect has a substantial probability, the risk assessor should explore the potential
   for secondary adverse effects (e.g., loss of mussel species following loss offish host species).
CHARGE QUESTION: How are alternative management options selected to have their
risks characterized and how is management informed of the consequences?

Interactions with risk managers
•  Most participants who expressed an opinion felt that regular consultation with risk managers
   and stakeholders is necessary throughout the process, especially when alternative management
   options will be evaluated. A few mentioned that this suggestion should be added to the
   Guidelines. It was recognized that such ongoing involvement has the potential to change the
   direction of a study.

•  One participant noted that only rarely will an individual or distinct committee have complete
   responsibility for making a risk management decision.  It is more likely (as in the five case
   studies in this pilot project) that the "risk managers" will include landowners, local officials,
   state officials, environmental groups, news media, and many others.  In order to be useful for
   decision making in these situations, the risk characterization should reflect close adherence to
   a collaboratively designed problem formulation and analysis plan.

•  A few participants and presenters felt it was not appropriate to consider management
   alternatives in the risk assessment itself, and view risk assessment and evaluation of
   management alternatives as separate activities.

•  Ideally, if management alternatives are to be considered in the risk assessment, they should be
   introduced by risk managers and discussed in the problem formulation phase. However, it
   should be recognized that if management alternatives are crystallized too early in the process,
   the risk assessment may eventually show that some of the favored alternatives are  not feasible
   or that some good alternatives were overlooked.  In addition, it was pointed out that some
   management options may be "discovered" during the risk assessment process itself.
•  Many believed the risk characterization section should cover both the current situation and
   reasonable "what if scenarios supplied by risk managers. "What to include" and "where to
   stop" are affected by the type of management information needed (e.g., to determine if there is
   a problem or if action is needed, to identify priority areas for action, or to determine the best
   action to take).  The risk characterization itself, however, was  seen as a summary  of the
   science that explains the probability of an adverse effect on endpoints.

•  Recognize that many individuals and organizations may need to be involved when a variety of
   managers and agencies have authority over subsets of the watershed, stressors, and ecological

•  The approach being used in Waquoit Bay is to have the managers decide "how much is too
   much" (i.e., "How much eelgrass loss is acceptable?").  Researchers are using models to
   estimate the stressor load (in this case nitrogen)  necessary to meet the goal and to evaluate
   the effectiveness of specific management alternatives (see Figure 2-3).

•  A "click-on" web site can be developed to allow planners to conduct "what if scenarios to
   evaluate various habitat and landscape feature management options for themselves.
CHARGE QUESTION: How should ecological risk be communicated to the manager and
the public (journal articles, computer programs, presentations, town meetings, Internet,

Packaging the product
•   There was considerable discussion during the workshop concerning who constitutes the
    audience for the risk characterization.  One primary audience is the risk manager(s) who needs
    to be informed about the scientific basis for the conclusion in the risk assessment. The
    stakeholders also represent an important audience. Stakeholders may be a diverse audience
    depending on whether they are directly affected by management decisions or are more
    interested in the technical, regulatory, or political aspects of the assessment. Length, detail
    and style of the report should be tailored to meet the needs of the intended audience.

•   Many participants agreed that the risk characterization should be a technical, stand-alone
    document. Yet, many others considered it acceptable to have separate versions of the risk
    characterization report targeted for various audiences. One option many participants favored
    was to have a single scientific report with  "executive summary" and "relevant findings"
    sections in "plain language"  geared toward managers. An alternative suggestion was for the
    risk characterization to be written in straight-forward language understandable by lay persons,
    with technical appendices prepared for scientists to explain the studies, lines of evidence, etc.,
    mentioned in the risk characterization report. A third alternative that several seemed to favor
    was to have three reports: an executive summary for risk managers, a scientific report for staff
    and scientists, and a colorful brochure-type product for the public.  One participant
    recommended that EPA consider field-testing targeted communication products to determine
    which report format options  were most effective.

•   One participant expressed an opinion that there should only be one risk characterization report
    and that multiple versions were undesirable for decision-making on policy issues.


                  1) Stakeholders choose end-points (black arrows)

                  2) Researchers define response of end-point measures to agents of
                     change (black curves) and use the curves to find values of agent of
                     change that correspond to selected end-points (grey arrows)


                              Ave. annual cone, of DIN in the estuary
                                               N load (kg N ha"1 y1)

                 3) Models used to identify management options that might allow
                    critical value of agent of change to be reached

                 4) Stakeholders evaluate acceptability of effective management

                 5) Develop plan to implement effective, acceptable options
Figure 2-3.  Waquoit Bay Case Study: Management Decision Process for Waquoit Bay

•  A few individuals suggested that non-technical stakeholders should be the primary target
   audience. One participant pointed out that if the problem formulation incorporates significant
   stakeholder involvement and reflects the values of the affected public, then the
   characterization should respond to stakeholder information needs.

•  Progress and sample reports are very helpful.  For instance, during the Clinch case study,
   drafts of the report with preliminary findings were developed as progress reports to determine
   how to best present and how technical the writing the final risk characterization should be for
   the resource managers.

   Some though not all, thought that the first time a technical term is used in the report, it may be
   helpful to highlight the word using a bold font, and then either define the term on that page or
   put the term in a glossary. If a term is a central concept and will be used repeatedly, an
   expanded definition (i.e., a paragraph or more) may be appropriate.

•  Consider other options to paper products (e.g., the Internet, an interactive what-if scenario).

Use visuals
   Tables such as the one in the Snake River report (reproduced as Table 2-1  for this report) are
   a good way to present the lines of evidence in one place.

•  Color can be effective if used well, but the writer needs to take into consideration that some
   readers will be color-blind. A good example of maps created to accommodate color blindness
   can be found in Pickle, et al., 1996.

•  Help decision makers see the connection between human activities in a watershed and
   achievement of the goals set for the assessment endpoints. This can be accomplished by
   referring back to the conceptual model diagram developed during the problem formulation.
   The context in which decisions are being made should also be considered since the linkages
   between natural and social systems are so strong that risk assessors cannot ignore them. We
   must deal with all the compartments of the natural, land use, and socio-economic systems and
   drivers affecting resource endpoints.

Communicating with the public
•  Besides the standard written report information can be disseminated via the Internet, videos,
   print media, public meetings, school programs, fair exhibits, and public radio.

•  In the Clinch case study, for the public, a more lay-version executive  summary of the report
   was prepared with input from the entire working group prior to distribution. Feedback from
   public meetings was used to evaluate the effectiveness of the Executive Summary and to
   determine whether other presentation materials were advisable.

General communication principles
•  Clearly state why the risk assessment was done (e.g., to support a particular management
   decision or to explain an observed effect).

Communicate early, often, and in a straightforward manner.  The vast majority strongly felt
that regular interactions with risk managers were key to the success of the assessment. Some
felt that this should also apply to stakeholders, though others did not. One of the lessons
learned in the Middle Platte study was that it is critical to have the risk assessors and risk
managers in constant touch and that risk assessors need to be aware of local socioeconomic
and political concerns.  Recognize that individuals and organizations are more apt to
participate or view the assessment differently when they may be directly affected by
management options under consideration.

Consider presenting the assessment as an opportunity to help stakeholders develop tools to
manage their watershed, as opposed to having EPA doing it for them.

Follow the seven cardinal rules of communication (plan carefully; coordinate and collaborate
with other sources; involve the public; listen to the public; be honest, frank, and open; speak
clearly and with compassion; and meet the needs of the media) and "TCCR" (transparency,
clarity, consistency, and reasonableness) as outlined in Text Box 6-2 and Text Box 5-9,
respectively,  in the 1998 EPA Guidelines.

Recognize that multiple stressors and large-scale, place-based efforts may make it necessary
to involve multiple resource management authorities.

                                   SECTION THREE
This section presents a summary of the opening remarks and background information presented
during the first session of the workshop.
     Mike Slimak, Associate Director for Ecology, NCEA

In the early days of the risk assessment process, risk assessment was separate from risk
management. Emphasis then was on human health assessments, and cancer was the main
endpoint of concern. While health assessors never debated the "so what" question for cancer, it
has become a frequently-asked question in the realm of ecological risk assessment. It is therefore
necessary to integrate risk assessment and risk management from the very beginning of an
ecological risk assessment. Dialog during problem formulation is very important to ensure that
risk assessors and risk managers are thinking along the same lines. Selecting assessment
endpoints that the risk manager understands is crucial  for successful risk characterization, which
in turn is the key to an effective risk management decision.

The purpose of this workshop is to learn from the experience the participants have in conducting
watershed-scale and place-based ecological risk assessments and to incorporate that knowledge
into improving EPA's ecological risk assessment guideline products. Ecological risk  assessments
are evolving from single species to multiple species, single stressors to multiple stressors,
chemicals to all stressors, and now, place-based assessments.  The place-based approach takes
risk assessment to the next level, moving beyond the command-and-control approach of the past.

EPA has a major commitment to place-based assessments. Places represent the real world where
exposure to multiple stressors occurs with effects on multiple species in the context of real
ecosystems. Stakeholder involvement is integral to this approach and adds a sense of ownership
not seen with chemical-based risk assessments. Building and maintaining effective relationships
with stakeholders is crucial when working in these complex situations, and methods for creating
positive relationships with stakeholders is an area worthy of research. The scale issue is another
area worthy of research as the place-based approach is developed. Watersheds range in size from
small (e.g., Waquoit Bay) to mid-sized (e.g., Big Darby Creek) to very large (e.g., Mid-Atlantic
Integrated Assessment, or Environmental Monitoring  and Assessment Program western pilot).

The ecological risk assessment process consists of problem formulation, characterizing exposure
and effects in an analysis phase, and integrating those  data in the risk characterization phase. Risk
characterization, which is the focus of this workshop, is the final phase of the risk assessment and
is the culmination of the planning, problem formulation, and analysis. However, risk
characterization actually begins during the characterization of ecological effects in the analysis
phase of the assessment.  The characterization of ecological effects consists of three primary
elements that lead to derivation of the stressor-response curve:


•  Plausibility that effects may occur as a result of exposure to stressors

•  Relationship between stressor levels and ecological effects

•  Linkages between measurable ecological effects and assessment endpoints when the latter
   cannot be directly measured

Derivation of a dose-response curve is not an easy task for complex systems with differing
stakeholder values. Derivation of the curve can be a "nested" process, as in the Waquoit Bay case

•  risks to estuarine  fauna as a function of submerged aquatic plants;
   submerged aquatic plants as a function of water clarity;
•  water clarity as a  function of algal blooms;
   algal blooms as a function of nutrient levels;
•  nutrient levels as  a function of nitrogen input;  and finally,
•  nitrogen input as  a function of the density of septic systems.

Interpretation and presentation of the exposure and effects must contain a clear explanation of the
lines of evidence leading to the conclusions, including a description of the uncertainties,
assumptions, and qualifiers in the risk assessment. A compelling, well-communicated risk
characterization will allow risk managers to  make a decision instead of asking, "So what?"
     Vic Serveiss, NCEA Environmental Scientist and Workshop Manager

Participants were invited to the workshop because of their experience in ecological risk
assessment, watershed management, or regional-scale assessment.  The goal is to use the
collective wisdom of attendees to further develop and document a process for conducting
watershed ecological risk characterization.

The Framework for Ecological Risk Assessment (EPA/630/R-92/001) was developed in 1992. In
1993, EPA's Office of Research and Development (ORD) and Office of Water (OW) selected five
demonstration sites to further develop, demonstrate and test the ecological risk assessment
paradigm by applying it to real world situations with multiple stressors. The five  sites were
selected based on several factors including availability of data, willingness of organizations to
participate, and the existence of multiple stressors. In addition, the study  sites came from a
variety of geographic regions, ecoregions, and EPA regions.  The watershed assessment sites
chosen were:

•   Big Darby Creek, Ohio;
   Clinch Valley, Virginia;
•   Waquoit Bay, Massachusetts;
•   the middle segment of the Platte River, Nebraska; and,
•   the middle segment of the Snake River, Idaho.


The plan is to complete individual draft assessment reports for each of these five sites between fall
1999 and December 2000, send the assessment reports out for peer review, and then publish
them as individual EPA reports.

Many of the attendees have been involved in these assessments. As the five assessments
approached the risk characterization phase, several participants, also at this workshop, noted that
there was no specific guidance available on how to develop and present a watershed-scale
ecological risk characterization. The current guidance that is available, though predominantly
focused on single stressors and endpoints, includes:

•   Guidelines for Ecological Risk Assessment (U. S. EPA, 1998)
•   EPA's draft Risk Characterization Handbook (U.S. EPA, 1999)
•   Improving Risk Communication (National Research Council, 1989)

Such guidance is helpful, and participants had some ideas of their own on how to proceed, but
were also enthusiastic to get additional suggestions. Hence, a workshop seemed appropriate to
develop and document additional guidance for watershed and place-based ecological risk
characterization.  To focus workshop discussions, several charge questions were developed that
address the issues of greatest concern for characterizing risk at the watershed scale.  It is expected
that the workshop proceedings will develop answers to  seven selected charge questions. These
"answers" will be used  to format recommendations to supplement the existing body of literature
on risk characterization and will provide guidance to place-based risk assessors and managers.

Draft watershed risk characterizations for the Clinch Valley and Mid-Snake assessments were
provided as background materials to participants.  These documents should serve as helpful
references as responses to the charge questions are developed. A second anticipated benefit of
this workshop is to provide information and general guidance to help complete and improve the
five demonstration water shed-scale assessments. A draft report on the workshop will be prepared
with the help of the TVA and will be sent out for review by workshop attendees.  Following a
peer review the final  workshop products will be an EPA report and a journal article.
     Bill van der Schalie, Ecologist and Guidelines Author, NCEA

Ecological risk assessment is a process in which all the elements build on each other.  The risk
characterization phase of the process consists of risk estimation and risk description, followed by
presentation of the risk characterization findings, as shown in Figure 3-1.

Risk Manager/
                Ecological Risk Assessment
                  PROBLEM FORMULATION

                                                                c  S
                                                                p* *"
                                                                w  s
                           Communicating Results
                            to the Risk Manager
                           Risk Management and
                         Communicating Results to
                              Interested Parties
           Figure 3-1.  Framework for Ecological Risk Assessment.

Risk estimation integrates the exposure and effects information.  The Ecological Risk Assessment
Guidelines (U.S. EPA, 1998) reviewed a number of approaches that can be used,  depending on
the situation and available data:

•   Categories and rankings - Categories and rankings are a qualitative means of estimating risk.
    This method is useful in complex situations with little available data. Expert judgment is
    required and should be documented. Categories and rankings are useful for stressor
    prioritization, as in the Waquoit Bay problem formulation.

•   Single point comparisons - Single point comparisons use the quotient method.  This is  a
    good screening tool that is simple to use; however, it is not quantitative, does not cover
    indirect effects, and does not consider uncertainty.

•   Methods incorporating variability in exposure and effects - The incorporation of variability
    can be accomplished using stressor-response curves or toxicity distributions.  Advantages of
    these methods are that quantitative measures show the magnitude of change in response to a
    stressor and allow uncertainty to be addressed. However, this method does not consider
    indirect effects and can  address only some of the uncertainties.

•   Process models - Mechanistic process models can be used to integrate exposure and effects.
    These models incorporate indirect effects, and are useful for scenario analyses. Disadvantages
    are that validation is not always possible, and underlying assumptions may be hidden (i.e., not

•   Field observational studies - Field observational studies are advantageous because they
    include direct measurement of exposure and effects, and they can take into account multiple
    stressors and relationships.  Disadvantages can include poor statistical power,  lack of
    replication, and difficulty in establishing causality.

It is clear that  each of these methods of risk estimation has both strengths and weaknesses.
Because no tool is perfect, the best option may be to use a combination of these approaches to
provide different lines of evidence.

Development of the risk description follows risk estimation in the risk characterization  process. A
risk description interprets the significance of ecological risk by evaluating the available lines of
evidence and by determining the ecological adversity (the "so what" question). Interpretation and
discussion of lines of evidence should include data quality considerations, uncertainly
considerations, and relationships to assessment endpoints and risk hypotheses. Ecological
adversity considerations include the nature and intensity of effects, spatial scales, temporal scales,
and the potential for recovery.

The risk description should:

•   Describe risks to assessment endpoints;
•   Revisit critical problem  formulation choices;
•   Address the risk manager's needs;


•  Use varying lines of evidence; and
•  Highlight critical assumptions and uncertainties, while avoiding a laundry list of all potential

Effective presentation of the risk characterization is the key to an effective management decision.
The principles of 'TCCR' described in the EPA draft Risk Characterization Handbook (U.S.
EPA, 1999) are important to a successful risk characterization report and can be summarized as

•  Transparency - Making the process open and frank helps achieve full disclosure of
   assumptions and uncertainties and permits ready identification of default or policy-driven
   assumptions. Key elements are to highlight the critical assumptions and uncertainties and their
   effects on the results; separate science from policy (example - protection of 95 percent of the
   organisms is policy); acknowledge any conflicting scientific interpretation of major issues; and
   describe the level of confidence in major risk conclusions.

   Clarity - Making the product clear makes the assessment complete and understandable.  Key
   elements are that the level of writing should be appropriate for the audience; major risk
   conclusions should be clearly stated;  and graphs and diagrams should be simple and

   Consistency - Consistency provides a context for the reader by ensuring the assessment uses
   an approach similar to other assessments (or offers an explanation if the approach or findings
   differ).  Key elements are to follow Agency-wide ecological risk assessment guidelines, if
   appropriate; define and explain the risk assessment purpose and level of effort; and consider
   results of other similar risk assessments.

•  Reasonableness - The findings should be reasonable within the context of the best-available
   science, the default assumptions, and the policy choices made. Key elements include ensuring
   the risk characterization is based on the best scientific information available; using generally
   accepted scientific knowledge in the assessment; making sure policy judgments use common
   sense in light of statutory requirements and  agency guidance; and presenting plausible
   alternative risk estimates for the various management options under consideration.

In conclusion, risk characterizations should be prepared using principles from the Guidelines
(U.S. EPA, 1998) and the EPA  draft Risk Characterization Handbook, which is scheduled for
release in 2000. The major risk characterization challenges in place-based assessments include
complexity and multiple stressors, lack of data  and large uncertainties, and providing clear
information in a usable format to risk managers.

     Sue Norton, Ecologist and Guidelines Author, NCEA

Risk characterization is the phase of risk assessment during which the results and conclusions of
the risk assessment are reached and summarized in a way that is useful for managers. Risk
managers are defined as individuals and organizations who have the responsibility or authority to
take action or to require action to be taken (U.S.  EPA, 1998). This definition encompasses a
broad spectrum from senior managers in regulatory agencies to the farmer who decides to build a
fence to limit livestock access to a stream.

The linkage between risk characterization and the decision to be made (i.e., the link between
science and management) varies from one guidance document to another:

•  Understanding Risk (NRC, 1996) - National Research Council (NRC) principles for risk
   characterization iteratively link the characterization with the decision, beginning in the
   problem formulation phase.  It requires a broad understanding of harms and consequences and
   is an outcome of on-going interactions between the scientific analysis and the societal

•  Risk Assessment and Risk Management in Regulatory Decision-Making (The
   Presidential/Congressional Commission on Risk Assessment and Risk Management, 1997) -
   The framework diagram  in this document shows no management interaction until near the end
   of the process at  decision-making time.

•  Ecological Risk Assessment Guidelines (U.S. EPA, 1998) - Management options are
   mentioned in the planning phase, but not in risk characterization.

•  draft Risk Characterization Handbook (U.S.  EPA, 1999) - Risk characterizations are written
   in a generic way to support a variety of decisions. The document makes no mention of the
   specific decisions the risk characterization is  intended to support.

There is no consistent guidance  on the interface between risk assessment and risk management.
As a consequence, some characterizations appear to be decision-driven, while others appear to be
a more generic presentation of information.

Insights  into effective ways to link science with decisions can be found by thoughtfully examining
the variety of environmental  management processes for place-based ecological assessments,

•  Biological assessment
•  Watershed approach
•  Community-based environmental protection
•  Watershed management
•  Adaptive environmental assessment
•  Adaptive management


•  Regional vulnerability analysis
•  Ecosystem management
•  Decision analysis
•  Natural resource damage assessment
•  Risk Assessment for Superfund (includes the no action alternative and comparison of remedial

These approaches all seek to bring science into decision making, and have the common goal of
making a difference that will result in environmental improvement. Most are also geared toward a
specific decision.  There are also ways in which the approaches differ. They may have different
starting points (e.g., effects, scenario, source, stressor); they may emphasize prospective or
retrospective points of view; they may build on different academic fields, traditions, and tools; and
they involve varying degrees of interaction with stakeholders.

However, the two most important ways in which the approaches differ are:  1) the degree of
dispersion of risk management authority, and 2) the proximity of management authority to on-the-
ground action. Decision making or management authority for different types of assessments may
be concentrated in one individual or management chain (e.g., Superfund assessments,
environmental impact assessments, biological assessments, decision analysis). Alternatively,
management authority may be more dispersed, as is the case with the watershed protection
approach, ecosystem management, futures analysis, and community-based environmental

The full spectrum in the proximity of management to on-the-ground action can be represented by
the list of considerations and associated activities shown in Figure 3-2.

The implications of this discussion for watershed risk characterizations are that the five ongoing
watershed assessments support all these types of decisions, and may have a dispersed management
audience. As a result, watershed risk characterizations may have different formats. Although
there is  currently little written guidance that is directly applicable to watershed risk
characterization, the variety of environmental management approaches discussed above provides a
wide range of styles and formats to build on.

As we develop recommendations for conducting watershed ecological risk characterization and
for improving the five assessments underway, the following questions should be considered:

•  Where are the case studies on the decision continuum toward taking action on the ground?
   (The five case studies appear to be at the point of prioritizing issues/risks and deciding if
   action is required.  Waquoit Bay is further along and now involves considering different
   management options.)

      Proximity to action on the
            Is there a problem that requires
            further work or action?
             - Futures analysis
             - Ecological/Biological assessment
             - Status and trends analysis
            Where are the priority areas or
            subjects for action
             - Regional vulnerability analysis
             - Risk-based scoring schemes
            Is any action required at all?
             - Superfund baseline risk assessment
             - Most agency chemical assessments
            What is the best action?
             - Environmental impact assessment
             - Superfund remedial alternatives analysis
             - Decision analysis
            Take an action already!
             - Adaptive management
Figure 3-2.  Proximity of Management to Action on
the Ground

•  How dispersed are the people who will be making decisions based on our analysis?

•  What type of decisions are our risk characterizations expected to support?

•  How can we present our risk characterizations in a way that facilitates decision making?

One workshop participant suggested that the degree to which risk assessment information is
feeding into and impacting management decisions provides an indirect measure of success in
addressing the last of these four questions.

                                    SECTION FOUR
This section provides summaries of the presentations made by chairs of the five EPA-sponsored
watershed ecological risk assessments.  Each presentation included an overview of the case study
with emphasis on the risk characterization phase. Any discussion related to answering the charge
question, "How can the watershed assessment draft or plan be improved?" is included in this
     Susan Cormier, EPA NERL, Big Darby Creek Assessment Co-chair

The Big Darby Creek watershed is located west of Columbus, Ohio, in the Eastern Corn Belt
Plains ecoregion.  The drainage area of 1440 km2 (557 mi2) is predominately row crop agriculture
with forests along the stream.  There are several small towns, but the area is  relatively unimpaired
by urban and industrial use.  The stream is both a Federal and State Scenic River and still has high
species diversity; however, there has been a 20 to 25 percent decline in aquatic diversity since the
1970s, and the area is in danger of losing  additional species.

Based largely on stakeholder input solicited during the first year of the project, the following
management goal was developed for Big Darby Creek: "Protect and maintain native stream
communities of the Big Darby Creek ecosystem." The approach to achieving this goal has been
to let the stakeholders know the assessment team was there to help them take care of the
watershed themselves, not to do it for them.  A primary task of the assessment team has been to
develop new tools for conducting the assessment and identifying interrelationships among
stressors and endpoints.

One  of the lessons learned in this case study is that the risk assessor should be able to show
exposure pathways and effects and explain them clearly to stakeholders.  This helps the risk
assessor build personal and professional credibility in the eyes of the stakeholders.  The
conceptual  model has been helpful for this type of communication.

Risk characterization to date has been mostly through presentations to stakeholders and at
scientific meetings.  These presentations identified and explained key risks and protective factors
using some of the following specific methods and techniques:

•   Box plots of Index of Biotic Integrity (IBI) metrics showing source signatures (see Figure

                  Source Signature
    Complex Toxic
    Agricultural Non-Point Source

Figure 4-1.  Big Darby Creek Case Study: Sample Box Plots of IBI Metrics
         Showing Source Signatures

•  IBI scores plotted on a map using different colors for categories of scores (although a
   problem was noted with showing temporal variability and measurement uncertainty) (see
   Figure 4-2).

•  Plot of Invertebrate Community Index (ICI) versus river mile (see Figure 4-3).

•  Trend analysis of response variables (IBI and ICI scores from 1970 to the present) using radar
   plots (see Figure 4-4).

•  Map showing erosion potential based on model predictions (see Figure 4-5).

   Trends in stressors using radar plots (see Figure 4-6).

•  Longitudinal plots of stressors versus river mile showing variability and trend of medians (see
   Figure 4-7).

The study team identified protective factors (e.g., steep gradient, presence of refugia, forested
streambanks, etc.) and key risks (e.g., livestock, housing developments, etc.). The results
suggested that useful management approaches would focus on the good stream segments by
protecting springs and groundwater, and preventing problem spots from impacting the good
stream segments.

The presentations have been successful in prompting stakeholders to take action to improve water
quality.  Actions taken to date include: removal of lowhead dams (with subsequent improvement
in fish communities); landowner action in response to erosion potential models; a proposal by the
U.S. Fish and Wildlife Service to designate a Darby Prairie National Wildlife Refuge (which
would protect approximately one-tenth of the study area, including the areas with the best aquatic
biological conditions and the recharge area); and use of the risk assessment results by the Natural
Resources Conservation Service to re-target their erosion control efforts.

Additional technical risk characterization and management options will be considered later as new
tools are developed and better data become available. These actions may include development of
a "click-on" web site (Ohio Watersheds Modeling Project) with models that allow planners to
conduct "what if?" scenarios to evaluate various habitat and landscape feature
management options (e.g., if a development of a given size goes in at a location, what impact will
that have on endpoint measures?). The models would be based on empirical relationships that
link IBI scores to "what if questions associated with land use, wastewater discharges, etc.

                         Ecological Conditions
                     1992 Index oF Biotic Integrity
                                  Buck Run
                                           Sugar Run
                                               Plain CHy

                                          Big Darby Creek
            West Jefferson
                                              Hellbranch Run
                 Little Darby Creek

                    IBI Scores
                   01 ISflo 19
                 5 • 20 to 29
                 9   30 to 39
                 9 H 39 to 45
                     46 to 49
                     I 50 to 60
Figure 4-2. Big Darby Creek Case Study: Map Showing IBI Scores By

           Longitudinal Profile
              I ! ! . . I , . . . I . . . . I . . I . I . . . , I . . . , I . . , .
                                            EWH Criterion
                                            WWH Criterion
        80   70   60  50   40   30   20  10   0
                       River Mile
Figure 4-3. Big Darby Creek Case Study: Sample Plots of ICI Versus River Mile

                  Trend  Analysis
                       Big Darby fish indicators, by year

                               Darter spp.
                                          Sunfish spp.
          %-Slmple lithophils
             %-Round suckers

                    %-Top carnivores'
              1979-1961 (n=4€)

              1966-19t9 (n=37)

            —1990-1993 (n=M)

        Sucker spp.

           Intolerant spp.
                                                 Total species
      Rel. total *flsh

                     Big Darby benthic indicators, by year

                               Total * taxa
           Qual. EPT taxa
                                            * Mayfly
             •1979-1981, n»14

             •1986-1989, n=S

             '1990-1993, n-18
Figure 4-4.   Big Darby Creek Case Study: Sample Radar Plot of Trend
             Analysis of Response Variables

                 EROSION INDEX (El) VALUES
                   Site Specific Stressor Trends
         Relative Median Concentrations for Big Darby Creek
                        12:12, p=0.002*
       12:12, p=0.002
               5:14, n.s.
      2:7, n.s.
9:10, p=0.017
n improved, n total, p*



   * one-tailed sign tests
        Figure 4-6.  Big Darby Creek Case Study Trends in Stressors
                    Using Radar Plots
Figure 4-6.  Big Darby Creek Case Study: Trends in Stressors Using Radar Plots

                      NO2 + NO3 Concentrations

                           Big Darby Creek 90-93


               Median trend
     JS >
     jg c
     W >,   — ^  „.
     LU .a   -f,>.  s
     ^ ro   B5  ro
     •60   2!  J
c    » o   £3  *
S    ° «   Ij^i  JB
       _i   x°-  o
        90    80    70    60
SO   40    30

               20    10
Figure 4-7. Big Darby Creek Case Study: Longitudinal Plot of Stressor Versus

      River Mile

     Bob Fenemore, EPA Region 7, Middle Platte Assessment Chair, and
     Maria Downing, EPA Region 7, Former Middle Platte Assessment Chair

The Middle Platte study area encompasses 5,000 square miles, making it the largest of the five
case studies.  The stream reach in the study area is 165 miles long. The watershed is 99 percent
agriculture and in private ownership. There are approximately 200,000 people living in the area,
and there are seven major communities, ranging in population from 1,500 to 42,000.

The Middle Platte is a highly appropriated river system with intense competition for water.
Stakeholders in the area have a history of competition rather than cooperation, with little
communication.  Key issues causing divisiveness included a dam and its operation, water flow
rights, and use impairments identified in a Clean Water Act Section 303(d) report.

When the 1996 draft of the planning and problem formulation phase of the risk assessment was
issued, the stakeholders were observed to be less concerned with what the risk assessment said
than with who said it. One of the panel members pointed out that this demonstrates how the
personal credibility of the risk assessor is critical to how the message is received by stakeholders.

A key issue for this case study was how best to communicate a risk characterization to such a
diffuse, diverse, divisive, and untrusting group.  Among the lessons learned were that it is critical
to have the risk assessors and risk managers in constant touch, and that risk assessors need to be
aware of local socio-political "hot buttons." It is also important to know stakeholders on a
personal basis and to understand local communication styles. The chair  of this assessment team
visited the affected Natural Resource Districts, municipalities, developers,  environmental groups,
etc.,  several times a month to develop a trusting relationship with these stakeholders.

EPA was particularly disliked by some stakeholders in the region, so it was important to separate
the risk assessment from any regulatory role.  Local stakeholders were concerned that information
from the risk assessment could later be used in enforcement actions.  However, even if that should
happen at some point in the future, stakeholders will have been involved throughout the process
so that findings will be representative of their concerns and should not come as a shock.

One panelist noted that it might help to clearly separate the management decisions from the
science. The speaker commented that close coordination between risk assessment and risk
management is necessary to be able to present the risk characterization in a palatable manner,
although as a scientist she agrees that they should be separate. It was also suggested that because
EPA is held in such low regard by some stakeholders in the region, it might be helpful to present
the assessment as an opportunity for the stakeholders to help develop the tools to manage their
own watershed, rather than having EPA develop tools to "regulate" them.

     Patti Tyler, EPA Region 1, Waquoit Bay Assessment Co-chair, and
     Maggie Geist, Waquoit Bay NERR, Waquoit Bay Assessment Co-chair

The Waquoit Bay watershed covers 21 square miles on the southeast coast of Cape Cod,
Massachusetts.  Initially valued for the hunting, farming, and fishing opportunities it provided,
20 percent of the land is now used for residential purposes.  There are seven subwatersheds with
different population densities. With sandy soils and low permeability, the watershed hydrology is
entirely groundwater driven, and groundwater quality is being impacted by the profusion of on-
site septic systems and by contaminated groundwater plumes from the Massachusetts Military
Reservation (MMR). Nonetheless, biological resources are many and varied. There are a number
of critical  habitats including eelgrass beds, coastal-plain, pond-shore communities, anadromous
fish runs, salt marshes, shellfish habitats, and barrier beaches. There is a great diversity of
shellfish and finfish species.  The area is along the Atlantic coast flyway, and there are numerous
shorebird  species. Several federal and state listed birds and plants occur in the area. As a result,
Waquoit Bay has a variety of special considerations and designations, including being a National
Estuarine Research Reserve, a U.S. Fish and Wildlife Refuge, a Land Margin Ecosystem
Research Site, and a state-designated "Area of Critical Environmental Concern."

Early in the assessment, the watershed stakeholder community was  asked what they wanted to
protect and what they thought the stressors were in Waquoit Bay. This stakeholder input was
used to help develop a management goal for the risk assessment. A comparative risk analysis was
conducted to define which stressors and endpoints needed to be examined further. "Fuzzy Set"
decision analysis was used to rank stressors according to their relative contribution to risk to  the
endpoints. Results of the decision matrix showed excess nutrient loading was the principal
stressor that was preventing attainment of the management goal for the watershed and that the
effect of nutrient loading on eelgrass beds was the primary stressor/assessment endpoint
relationship that needed to be investigated.

The team is now developing a model to relate the response of eelgrass coverage to nutrient
loading within Waquoit Bay and eventually to predict future loadings under different development
scenarios (e.g., density of septic systems, which are the primary  source of the nutrients).  The plan
is to provide resource managers with the information needed to develop guidelines that would
protect estuarine resources.

Exposure  (i.e., nitrogen loading) was estimated using two models. The Nitrogen Loading Model
(NLM) was used to predict the nitrogen load from various sources based on assumptions about
landscape features, atmospheric deposition, groundwater movement, etc. This model tracked
nitrogen delivery to the water's edge. The Estuarine Loading Model (ELM) was then used to
estimate the load of nitrogen in the estuary that was available to producers (i.e., the effect of
nitrogen load on algal growth).

Risk managers must first decide "how much is too much" (e.g., How much eelgrass loss is
acceptable?).  Researchers can then use the models to estimate the N load necessary to meet the
goal selected by the managers and to evaluate the effectiveness of various management
alternatives.  A  description of how this will be accomplished was shown above in Figure 2-3.


     Patricia Cirone, EPA Region 10, Middle Snake Assessment Chair and Guidelines

The Snake River study area is only a portion (a 160 km stretch of the total 1,000 km river) of the
watershed. A travelogue slide presentation along the Middle Snake River pointed out that 85
percent of the Idaho population lives on this  stretch of the river, and 90 percent of the trout
served in U.S. restaurants are from aquaculture operations along this stretch of the river.

This assessment began in 1987, and analysis began in 1994.  The Snake River Team is made up of
federal and state agencies, county and local governments, private organizations, and academia.
Major stressors identified by the team include: flow diversion for irrigation and water supply,
flow alteration by impoundments, sediment and nutrient inputs from trout hatcheries, municipal
wastewater discharge, cattle grazing, feedlots, and returns from irrigated land.

Much of the native fisheries resource has been lost.  Five species of snails are endangered and
several others already extinct. Management goals were developed to address these and other
water quality related problems, as well as stakeholder interests related to recreational  use of the
river. The goals included attaining  state water quality goals, establishing Total Maximum Daily
Loads (TMDLs) for water quality-limited stretches, recovering threatened and endangered
species, sustaining economic activity, and increasing tourism and river use.

The assessment endpoints drove the decisions for "what are we going to analyze?" In this case
the endpoints selected were growth and survival of cold water biota (Mountain Whitefish, White
Sturgeon,  Trout);  growth, survival, and diversity of molluscs; and growth of aquatic macrophytes
and algae  (which drove most of the public interest).  Measures of effect selected for analysis were
water quality criteria (dissolved oxygen, temperature, ammonia, total phosphorus and nitrogen);
Habitat Suitability Indices; and presence, absence, and abundance  of flora and fauna.

Risk estimation was based on four lines of evidence. These were:

•  Field observations and studies (empirical  models)
•  Literature (field and best professional judgment)
   Stochastic or probabilistic estimates for habitat suitability and water quality criteria
•  Process models for eutrophication

Estimates  of the likelihood of recovery were  also developed and presented using both figures and
tables in the case study report (see Figure 4-8 and Table 2-1 of this report).  Recovery is an
endpoint that is important to stakeholders and is easily understood. While there was much praise
for these figures and table, one comment to improve the table was to include citations to technical
references or, in the cases where best professional judgment was used, identify the scientist(s)
drawing the conclusion.

    Figure 21.  Factors limiting Rainbow Trout growth & survival
                         Rainbow Trout
                           Dams are
                                                  and stocking
  water, low,
 steady flow;
in spring side
channles; small
Figure 4-8. Middle Snake River Case Study:  Illustration Linking
         Stressors to Recovery Potential for Rainbow Trout

It is anticipated that the information in the ecological risk assessment can be used to help establish
total maximum daily loads (TMDLs) for river sections suffering water quality impairment; used in
the review of permits for stream-affecting activities; and used to help evaluate management
options for identification and control of nonpoint source pollution.
     Jerry Diamond, Tetra Tech, Inc., Clinch Valley Assessment Principal Investigator

The study area for the Clinch Valley assessment case study is in the unimpounded portion of the
basin in Virginia.  There has been an enormous decline in the fish and mussel species diversity.
Stressor sources include dams, logging, some crops, diminished riparian corridor, mining, spills,
and livestock pastures.

The management goal developed for the Clinch Valley ecological risk assessment was to maintain
or restore the biological  diversity of the Clinch-Powell watershed surface and subsurface aquatic
ecosystem. Assessment endpoints selected for evaluation were native mussels, native fish and
cave-dwelling species.   (Ultimately, cave-dwelling species were not considered  as part of this
assessment due to lack of data.)

The Clinch Valley assessment is analyzing:

   The upland and riparian land use and its relationship to in-stream habitat quality and mussel
   distribution and abundance;

•  Riparian vegetation, connectivity, and width in relation to measures of in-stream resource

   Temporal trends of water quality and mussels; and,

•  Water quality, land use, and riparian corridor characteristics in relation to distributions of
   mussels and native fish.

Scale issues were paramount. The primary question for land use was, "What scale of information
is related to the biological endpoint measurement?"

Preliminary conclusions to date include:

   Cumulative impacts  appear to be responsible for the decline in species richness and
   distributions. A stressor index was developed to indicate cumulative impacts based  on
   proximity (> or <2 km) to pasture/cropland, urban areas, mining, and major  transportation

•  As the number of stressors identified in an area increased (0, 1,2, or 3), the maximum number
   of mussel species present declined.

•  Percent forest cover is not necessarily an accurate indicator of in-stream conditions due to
   effects of other factors, especially close proximity to mining.

•  A relatively large riparian area upstream of a site can have substantial effects on in-stream
   habitat quality and distribution of mussels and fish. The best fit (predictor) between biological
   endpoint measures and riparian zone size was with an area 100 m wide on each side of the
   river that also extended 1,000-1,500 m upstream of the siHn narrow floodplains or areas of
   higher gradient, a larger upstream reach needs to be considered.

•  Relationships which were found to be significant included cropland (over 3 percent) versus
   sedimentation, pasture versus riparian vegetation integrity, and urban/barren versus quality of
   in-stream cover.

•  Fish IBI was a better predictor of mussel populations than was EPT (sum of Ephemeroptera,
   Plecoptera, and Trichoptera taxa in  a sample of benthic macroinvertebrates).

The major stressors for mussels were found to be proximity to mining (10 %), and percentages of
cropland (>19 %).


•  Limitations of the Data - Habitat data were qualitative and not always available from the same
   sites as biological data; taxonomy for EPT was based on family-level taxonomy (genus may
   have yielded more conclusive results);  water quality data were sparse; flow data were sparse
   and difficult to relate to the risk assessment.

   Sources of Uncertainty - Differences in sampling sites for mussels, fish and EPT; scales used
   for calculating land use percentages may not be transferable from stream to stream; and given
   that not all land uses co-occur everywhere, apparent correlations with a source may actually
   be due to presence or absence of other sources of stress.

                                    SECTION FIVE
                          FACILITATOR PRESENTATIONS
5.1  Jack Gentile, Center for Marine and Environmental Analysis, University of
     Miami"Large-scale Assessment and Risk Characterization - The South Florida
     Regional Assessment as an Example"

The South Florida Regional Assessment and Restoration Strategy for the area of Florida south of
Lake Okeechobee (including The Everglades),  is a risk assessment that combines both
retrospective and prospective elements in understanding the future risks associated with specific
restoration actions.

Restoration goals were developed by the Governor's Commission for a Sustainable South Florida.
This commission met quarterly and was tasked with developing the societal and environmental
goals for the restoration. The scale and complexity of this activity have tremendous associated
uncertainty; consequently, a decision was made to construct conceptual models for the whole
system. The first technical problem facing investigators was how to partition an area as large as
this (i.e., approximately 3,000 mi2). Ultimately, the  study area was characterized as a mosaic of
13 areas with distinct hydrology, habitat assemblages, and habitats. Alternatively, the assessment
could have labeled several habitats, each occurring in one or more areas.

Conceptual models developed for each of these landscape units allowed the development of
hypotheses that described the causal basis for the current state  of the ecosystem as well as for
future recovery. These 13 landscape units became the focus of the South Florida Restoration
Strategy to recover south Florida and the Everglades.

The assessment was a tool for managers who needed a method for seeing how "what if scenarios
affected the various endpoints of concern to them. The stress-response relationships developed
were the engines driving the analysis.  The risk characterization involved building dynamic
visualization tools.

In the analysis phase, a 2- by 2-km hydrodynamic model was used to develop exposure
information based on a 30-year record of precipitation data.  Model outputs of stage height,
duration, and flow were generated for various climate scenarios.   Stressor-response curves were
developed for relationships between percent cover types and hydrologic parameters.

The effects that optimizing hydrology would have on other endpoints (e.g., wading birds,
panthers, threatened and endangered species) became a major issue. The model indicated that it
was desirable to flood certain areas, but flooding would have had significant social effects (Native
Americans now own the area) and ecological effects (habitat of the endangered Cape sparrow
would have been eliminated, although the species did not occur in this area until the land was
drained by the historical diversion of water).  The difficult management decisions have not yet
been made.

Information on the assessment has been relayed via area-wide public meetings of the governor's
commission.  The group has also prepared technical reports, had pieces in publications, and
developed brochures and fact sheets several times during the process.

The South Florida case study illustrates the full application of the ecological risk assessment
paradigm. Problem formulation was used in a retrospective or deductive manner to identify the
sources of the problem as well as to characterize the current state of the system and understand
how it got to be where it is today. Hypotheses were developed for actions that, based upon
several lines of evidence (e.g., historic, baseline monitoring), would lead to recovery.  The
Analysis Phase or inductive/prospective component of the guidelines included the examination of
a variety of scenarios with sensitivity analyses to identify the most important variables in the
system. In the Risk Characterization Phase, the outcomes were examined again, using multiple
lines of evidence and often weight of evidence to help optimize the recovery in different landscape
units.  While numerical models played an important part in forecasting recovery, expert judgment
was equally important. And, in fact, the latter often had a much higher degree of confidence than
the former.
5.2  William Smith, Yale School of Environment and Forestry Studies, Yale University

The watershed concept has utility in that it is universal, allows for examining effects on different
scales, allows for hypothesis testing, and is useful as an environmental management unit. Three
goals of regional or watershed-scale risk assessments are to explain observed effects, evaluate
actions with regional implications, and evaluate the status of key endpoints.

Successfully meeting these challenges involves developing good conceptual models with source-
stressor-effects linkages, and comprehensively including what we know to be important.  Clarity
on the  purpose of the risk assessment (e.g., whether the assessment is being done to answer a
scientific question or to support a management decision) is also critical.

Ecological risk assessors should move away from the term "uncertainty" and think in terms of
"degree of confidence." But while risk assessors should strive to be honest and clear about
uncertainties in the assessment, they should also  avoid being hamstrung or silenced by those
uncertainties. Most major risk management decisions have a political context, and decision
makers do not usually need the 95 percent confidence level that scientists strive for. In the real
world,  risk management decisions will continue to be made even in the face of great uncertainty.

Several panelists pointed out that the perceived need for certainty increases when risk
management decisions involve the expenditure of large sums of money or an attempt to  change
ingrained  land management practices.  Another panelist commented that the perceived need for
certainty may decrease when it is very expensive to collect and analyze the data to increase the
level of confidence, or when the level of uncertainly ceases to be significant relative to the specific
endpoint of interest.

Focusing specifically on the charge questions for the risk assessment case studies, Dr. Smith
offered the following suggestions:


    Clearly state why the risk assessment was done (e.g., to support a particular management
    decision or to explain an observed effect). This should be repeated in the risk characterization
    even though it was stated initially in the problem formulation.

    Consider the potential benefits of a ranking or weighting system based on relative importance
    when dealing with qualitative lines of evidence and focus first on those judged most important.
    A ranking  approach would also be useful in discussing the degree of adversity of predicted or
    observed effects in a watershed.

    When appropriate, include and develop qualitative lines of evidence. Sometimes both
    qualitative and quantitative lines of evidence are required to "tell the story" effectively.

    Keep risk assessment and evaluation of management options as two separate activities.

    Risk assessment reports should be polished, highly readable, professional-looking
    communication products. They should include a high quality executive  summary targeted to
    decision makers.
5.3  Patrick Bourgeron, INSTAAR University of Colorado, Colorado State University

The linkages between natural and social systems are so strong that risk assessors cannot ignore
them. We must deal with all the compartments of the natural, land use and socio-economic
systems and drivers affecting resource endpoints.  Too often, there is a disconnect between the
system components considered by landscape ecologists and those considered by land or regional

Selecting appropriate data for the scale examined is a crucial decision in assessing risk. In
general, moving toward more conceptual viewpoints of the landscape (i.e., going from the real
landscape, to spatial representation in mapping, to including temporal and dynamic features, to the
strategic level of considering scenarios) necessitates a movement from high levels of precision to
more general probabilities.  In addition, as the domain under consideration enlarges (i.e., from
local, to sub-region, to watershed, to regional), the "grain" of data changes from fine, to medium,
to coarse.  Use of finer or coarser data than is  appropriate for a scale will cause serious problems
in the ability to discern exposure-response relationships, interpret results, and define uncertainty.

There are typically several spatial areas to be considered in developing large-scale assessments.
Ideally, the areas are one and the same, but this situation rarely  occurs. The areas to consider are:

•  Assessment area (the area over which data are  available or will be collected)

•  Analysis area (the area over which analyses are conducted)

•  Planning area (the area over which community planning will occur)

•  Response units (the lowest level on which you can expect a response)

   Cumulative impact areas (the area that management or land use activities will affect)

•  Reporting Units (the area on which the risk assessor is reporting)

Tables presenting accuracy/precision versus cost are one method to communicate key information
about the level of uncertainty. The information may be expressed as simply as low, medium, or
high ranges.  This information can be used in discussion of the appropriate level of effort in
minimizing uncertainly.

One  panelist commented that it might be useful to include an accuracy/precision versus cost table
in the risk assessment to help communicate the justification for accepting a particular level of
uncertainty. He also noted that there should be consideration of whether the increased cost of
achieving a higher level of confidence produced a comparable value-add in decision quality.

                                     SECTION SIX
                         BREAKOUT SESSION SUMMARIES
This section presents a summary of each team discussion during the breakout session.
     Jack Gentile, Facilitator

CHARGE QUESTION:  How should exposure and effects data be integrated in a
watershed context to generate a risk estimate?

•  There is value in using a "process diagram" similar to that used in the Clinch Valley
   presentation (see Figure 2-1) to summarize what could or could not be addressed with the
   available data.  To be effective, such a diagram need not be a complete reproduction of the
   detailed conceptual model, but rather should  provide an "in a nutshell" review of what was
   addressed and what the key issues are. The approach would be especially useful in situations
   where the risk characterization focuses on a single assessment endpoint within a complex
   ecosystem. A similar diagram could also be developed to highlight where the major
   uncertainties lie.

   Stakeholders may only be interested in certain endpoints  or certain sub-areas of a complete
   watershed and the characterization should place greater emphasis on those parameters.

•  Although the risk assessor may  recognize complex interrelationships between  stressors, or
   between a single stressor and multiple effect endpoints, models tend to approach such
   relationships linearly and one at a time. A potential approach to writing a risk characterization
   within this framework of complexity begins with a reiteration of the goals of the analysis and
   the linkage between those goals and the assessment endpoints. The second step would be
   presentation of a simplified version of the conceptual model and a summary of why the
   assessment focuses on selected endpoints within a fabric of cumulative effects. From this
   point, the risk characterization would proceed into a summary of the cause-and-effect
   relationships or benchmarks that will be used to evaluate exposure  data.
CHARGE QUESTION:  How should uncertainty be addressed and presented (e.g.,
incomplete data or analyses, qualitative estimates, data at different spatial and
organizational levels)?

•  Discussions of uncertainty should focus on the key uncertainties (i.e., those that could change
   the decision) rather than simply reciting a "laundry list" of all possible uncertainties.

•  When uncertainties are due to a gap in the available data, the risk characterization should
   make recommendations for filling in the data gaps.  The text might preface the

   recommendation with a statement such as: "If 'X' data were available, the probability of this
   response could be predicted with 'Y' degree of confidence."

   When the data are not adequate to develop a stressor-response curve, the risk characterization
   should assess the strength of the association between putative stressor and effect.  The
   references on biological inference cited in the Guidelines for Ecological Risk Assessment
   (e.g., Fox [1991], Hill [1965], Susser [1986], and Suter [1993]) were identified as providing a
   useful conceptual framework  for this analysis.

   When conclusions are based on comparison of data with benchmarks (such as water quality
   criteria), the uncertainties inherent in the development of the benchmarks should be

   The group discussed the differences in how risk assessors and risk managers might view
   uncertainty in the analysis. For example, a risk manager might choose to act based on a
   potential for an adverse effect, even if there is no conclusive evidence of causality.
   Discussing the degree of confidence in the results is a critical step in developing a credible risk
   characterization even if the uncertainties are only a minor consideration in the risk
   management decision making process.

   As scientists, we tend to demand a high degree of confidence before we present results.
   However, much of the uncertainty in ecological risk assessment cannot be captured in terms
   of standard statistics, even in  those cases where quantitative information on the variability of
   exposure or endpoint data is available. Nonetheless, a qualitative, semi-intuitive estimate of
   confidence can usually be made based on best professional judgment, and this estimate of
   confidence may be adequate for the risk manager.
CHARGE QUESTION:  What are the best ways to address and present the integration of
qualitative and quantitative lines of evidence for a) an individual assessment endpoint and
b) drawing overall conclusions?

   The group liked Table 17 on page 99 (see Table 2-1) of the Middle Snake River report as a
   way of displaying and summarizing multiple stressors and lines of evidence. The addition of a
   "references cited" column on the right-hand side of the table was identified as a potential
   improvement in the format.  The group suggested that a table of this type, if used,  should be
   supplemented by a narrative treatment of the various lines of evidence.

•  Any contradictions between the various lines of evidence should be explored and, if possible,
   explained. The text should address whether the contradictory lines of evidence nullify the risk
   assessment process or whether it is reasonable to proceed with the risk characterization.

   The agreement or concordance of various lines of evidence, which individually may have a
   high degree of uncertainty, can strengthen confidence in the conclusion.

   One participant noted that EPA's Office of Research and Development and Office of Water
   are presently developing a document that addresses stressor identification and evaluation
   within the context of defining causal relationships.

   The group cautioned against numerically weighting qualitative data.
CHARGE QUESTION: How should the degree of adversity of predicted or observed
effects in watersheds be described (e.g., considering nature and intensity of effects, spatial
and temporal scale, and the potential for recovery)?

•  A variety of qualifying and modifying factors should be addressed in a description of
   adversity, including:
              spatial scale
              temporal scale
              biological scale (i.e., are adverse effects apparent on an individual organism scale
              or on a population scale?)
              functional redundancy in the ecosystem that may obscure or limit apparent adverse
              ecological effects (e.g., species composition may change, but primary production
              rates may not).

•  If the potential for recovery is an important part of determining the degree of adversity,
   address the following:
              probability of recovery
              spatial and temporal scale of recovery expected (including alerting the risk
              manager to any potential "quick successes")
              whether recovery will be natural or "assisted" (e.g., species reintroductions,
              population supplementation by stocking)
              what other factors (natural and anthropogenic) might limit recovery when the
              primary stressor is reduced or eliminated.
CHARGE QUESTION: How are alternative management options selected to have their
risks characterized and how is management informed of the consequences?

•  A question was raised about whether it was "heresy" to address management alternatives in a
   risk assessment (based on the separation of risk assessment from risk management in the NRC
   analytical deliberative paradigm).

•  Depending on the purpose of the risk assessment (e.g., characterizing ecosystem dynamics
   versus supporting informed decisions by risk managers), evaluation of management
   alternatives may or may not be an integral part of the effort.  If it is the goal of the risk
   assessment to evaluate management alternatives, this should be clearly reflected in the
   problem formulation.  However, the group noted that even if the ultimate purpose is to
   support evaluation of management alternatives, the suite of alternatives may be developed
   through an iterative process that involves bringing stakeholders into the process.


CHARGE QUESTION: How should ecological risk be communicated to the manager and
the public (journal articles, computer programs, presentations, town meetings, Internet,

    Set the stage for clear communication in the risk characterization by briefly reiterating the
    goal(s) of the risk assessment (e.g., characterizing a complex system; providing the
    information to support a specific management decision).

•   Ecological  risk communication should follow the "7 cardinal rules of communication" as listed
    in Text Box 6-2  of the 1998 Guidelines for Ecological Risk Assessment.

•   The risk characterization should strive for TCCR as discussed in Text Box 5-9 of the 1998
    Guidelines for Ecological Risk Assessment.

    The risk characterization presentation style should be tailored for the intended audience.  The
    potential audiences are diverse and range from other environmental professionals to
    congressional representatives to land owners. At a minimum, there should be a scientifically
    credible, reasonably detailed and technical version that summarizes and documents the results
    of the assessment for the technical reader. Depending on the goal(s) of the assessment, it may
    be desirable to prepare a relatively nontechnical executive summary suitable for a broad
    audience. The executive summary should be tightly focused, and possibly arranged in a
    "bulleted" format.
     William Smith, Facilitator

CHARGE QUESTION: How should exposure and effects data be integrated in a
watershed context to generate a risk estimate?

   Output from mathematical models comprised a large portion of the case study presentations,
   and the breakout group was surprised at the limited use of mapping overlays to summarize
   results. Although several presentations used mapping overlays, there was a feeling that there
   were many additional opportunities to use the technique. When used effectively, mapping
   overlays can make the data more user friendly and aid the scientist in understanding it. For
   example, in the Clinch Valley case study, maps of endpoint measurement data and stressor
   data were effectively used to portray relationships and linkages (see Figure 6-1).

                       Copper Creek Watershed
                         Land Use/Land Cover
       |	|  Pasture
       |  Agriculture
       |	|  Deciduous Forest
       I  J  Coniferous Forest
       |  |  Forested Wetland
       |  I  Non^Forested Wetland
       Q  Scrub/Shrub
       ! J  Open Water
       |  Urban
       |	|  Barren
       ^|  Mines
       |  |  Qoud/Nodata
      Number of T & E species
        1   1
        •   2-3
        •    3
      * Source for land cover data is Lanrisat Thematic Mapper data
                                                              SCALE 1:198,326
                                                              1 inch represent J. 13 miles
                                                          J  0  J  2   J 4 i t 7 SEikc
                                                          i  i  i  i—ii—ii—ii
                                                            0   J   :   i   4   ifcEi.
Figure 6-1.  Clinch Valley Case Study: Map Showing Land Use Versus Threatened & Endangered Species

   The group felt that examining small subwatersheds was an effective way to decrease
   complexity of analysis and presentation, and to focus on the effects of individual stressors.
   Subsequently (as in the Clinch Valley case study), the analysis can be re-expanded (with
   assumptions clearly identified) to evaluate the cumulative impacts of stressors. One
   participant noted that the Big Darby investigators had used an analogous approach by sorting
   out the stressors using Principal Components Analysis (PCA) and then developing a
   multivariate model that "reassembled" the effects.

   Ranking techniques and development of stressor indices (but not weighting) for the watershed
   were considered reasonable methods of assembling the data into usable information.  (There
   was no detailed discussion on either of these approaches.)

   The picture charts and tables relating stressors (see Figure 4-8 and Table 2-1, respectively),
   lines of evidence, description of risk, uncertainty, assumptions and recovery potential from the
   Snake River case materials in the workshop handout binder were considered to be the best
   examples of integrating and presenting  information.
CHARGE QUESTION:  How should uncertainty be addressed and presented (e.g.,
incomplete data or analyses, qualitative estimates, data at different spatial and
organizational levels)?

•  Discussion began with the level of comfort of risk assessors in addressing uncertainty, and the
   participants warned against being paralyzed by "uncertainty block."

•  Risk assessments are likely to be questioned at some point, and one group member remarked
   on the fear that a risk assessment might be challenged in court. Another participant noted that
   in the NEPA arena courts tend to find for the initial investigator rather than the challenger as
   long as the investigator has conducted and documented a "reasonable" scientific approach to
   assessing the magnitude and likelihood of impacts.  It is generally sufficient to show that the
   issue has been considered, the investigator has used a reasonable scientific process to reach
   conclusions (conducting studies, developing conceptual linkages and consolidating lines of
   evidence), and the degree of uncertainty has been identified. "Sins of omission" (e.g., leaving
   out an important stressor or endpoint) are worse than "sins of inaccuracy."

•  Paying due respect to gaps in conceptual models is critical.

•  The levels of confidence needed for scientist and managers differs. Risk managers make many
   decisions with levels of uncertainty that are considered high for the risk assessor.  The
   confidence level needed may be affected by the management decision to be made (e.g., it may
   be correlated with capital involved and political context). In public policy decisions, the
   uncertainty about effects from economics or cultural issues that influence decisions may be far
   greater than the uncertainty associated with factors in the risk assessment.  This does not,
   however, let the investigator off the hook: uncertainty should be clearly addressed with good
   science and judgment.

•  Uncertainty can be captured either quantitatively or qualitatively, depending on what is most
   appropriate for the data or information being considered. Quantitative data should be used

   whenever possible, but in many cases, uncertainty may simply be characterized as "high,"
   "medium" or "low."  The appropriate level of uncertainly analysis may be non-quantitative for
   some data or lines of evidence.

   The objectives should be to lay out the degree of uncertainty with clarity, honesty and
   openness; and to be as comprehensive as possible in your "uncertainty" inventory.

   It is often helpful to reference standards, tests, and accepted metrics (e.g., IB I) in support of
   level of uncertainty.
CHARGE QUESTION: What are the best ways to address and present the integration of
qualitative and quantitative lines of evidence for a) an individual assessment endpoint and
b) drawing overall conclusions?

•  For the technical audience, use an approach similar to that of the Snake River presentation
   was good (see Table 2-1).  The advantage of a tabular presentation was that the information
   regarding the lines of evidence (e.g., model, field observations, similar sites comparison, etc.)
   could be seen in one place. Use of qualitative categories (high, medium or low) was both
   appropriate and useful in some instances. The addition of numeric references at the side of the
   table was suggested as a useful change in format.
CHARGE QUESTION: How should the degree of adversity of predicted or observed
effects in watersheds be described (e.g., considering nature and intensity of effects, spatial
and temporal scale, and the potential for recovery)?

•  The Snake River table (see Table 2-1 above) in conjunction with the maps showing degree of
   impairment for various life stages along the stream reach (see Figure 2-2 above) are good
   formats to follow.

•  Temporal scale analyses and presentations should indicate trends for stressors and endpoints.
   Participants noted the need for caution about making subjective judgments.
CHARGE QUESTION: How are alternative management options selected to have their
risks characterized, and how is management informed of the consequences?

•  Management decision making goes on throughout the risk assessment and continues after it is
   completed, yet risk assessments need closure.

•  Management options and issues should be identified to the extent possible in the
   planning/problem formulation stage, but risk managers (whether urban planners or farmers)
   should be involved along with the risk assessor in the iterative development of risk scenarios
   during the risk assessment process.

   The risk characterization section should cover both the current situation and reasonable "what
   if scenarios supplied by risk managers. "What to include" and "where to stop" are affected
   by the type of management information needed (e.g., to determine if there is a problem or if
   action is needed, to identify priority areas for action, or to determine the best action to take).
   The risk characterization itself, however, is seen as a summary of the science that explains the
   probability of an adverse effect on an endpoint.

   The risk assessor should not feel obligated to go further than covering the management
   options that were identified in the planning/problem formulation stage.

   When considering management options, recognize that many individuals may need to be
   involved because different managers and agencies may have authority over subsets of the
   spatial scale,  stressors, and ecological endpoints.
CHARGE QUESTION: How should ecological risk be communicated to the manager and
the public (journal articles, computer programs, presentations, town meetings, Internet,

•   There is a strong need for continued communications to "bring the public along all the way."
    This is especially true when considering management options.

•   Although somewhat simpler in presentation, executive summaries should have enough "meat"
    to give the reader the full understanding of major risks, stressors, and impacts to important
    endpoints identified in early stakeholder discussions.

•   Other routes for sharing risk assessment information that were either done in the course of the
    demonstration risk assessments or were thought to be good ideas include:  exhibits at a county
    fair, school audiences, use of print media and development of "slicks" for the public as well as
    a technical report for the scientific community. Full text versions of risk assessments should
    be made available on web sites.

•   Recognition of your audience was a major point of discussion. A key question is, "Do you
    know who your audience is?"  There were three basic audiences identified:  risk managers, the
    public, and other scientists.  To be maximally effective, correct identification of the
    management audience was thought to be critical.  Reports should be  appropriate for 1)
    scientific review; 2) risk management utility; and 3) public access and readability.

    Tables such as the  one in the Snake River report (see Table 2-1 above) are a good way to
    present the lines of evidence in one place. Using qualitative categories (high, medium, or low)
    can be useful sometimes.  The addition of numeric references at the side  of the table was
    suggested as a useful change in format.
     Pat Bourgeron, Facilitator, and


     Patti Tyler, Report-Back Volunteer

CHARGE QUESTION:  How should exposure and effects data be integrated in a
watershed context to generate a risk estimate?

•   The dose-response curve format is useful for a single stressor dose-response (e.g., using
    eelgrass as an assessment endpoint for nutrients in the Waquoit Bay assessment).

•   Multivariate analysis can be used in a system with multiple stressors to help determine which
    stressor is having the greatest impact.

•   The group liked Table 17 on page 99 of the Snake River report (reproduced as Table 2-1 for
    this report) as a way to display and summarize the impact of multiple stressors and lines of
    evidence.  A table of this type can capture both quantitative and qualitative evaluations.

•   Evidence of exposure, evidence of effects, magnitude of effects, confidence, and risk potential
    can all be ranked as high, medium, or low. This qualitative evaluation can be quite useful.
    For example, if there is medium confidence of low risk, it is unlikely that additional data
    would change the conclusion that the risk is low.

    Sometimes it may be best to use best professional judgment rather than to try to quantify. For
    instance, better information may be derived from the best professional judgment of a local
    biologist than from an in-depth study of limited data.

    The group discussed the scale issue and how to extrapolate from subwatersheds to entire
    watersheds. Applying specific subwatershed relationships to entire watersheds works better if
    the entire watershed is similar in nature. It is possible to develop templates for different
    habitats which may then be extrapolated to similar locations rather than the whole watershed.
    In the Clinch Valley Assessment, data from study stations were used to define trends based  on
    water quality data, land use, habitat, etc., which were then extrapolated to specific  sites with
    mussels to help explain  existing conditions or help predict future recovery.

•   It may not always be appropriate to manage a stressor for the whole watershed if only a small
    or specific area needs to be protected.

•   It is also good to be able to present risk estimates at different scales. Data should be used for
    an appropriate area and reported for an appropriate area.

    The relationship between exposure and effects must be communicated so that the manager can
    understand all the links between the stressor and the endpoint. Plugging data into the
    conceptual model is one way to demonstrate effects, and it is best if it can be shown at
    different spatial scales.

CHARGE QUESTION:  How should uncertainty be addressed and presented (e.g.,
incomplete data or analyses, qualitative estimates, data at different spatial and
organizational levels)?

•  Be aware that users of the report aren't as interested in uncertainty as are scientists.

•  Nutrient modeling is an example of where a model can demonstrate variability.  Confidence in
   predictive numbers, limitations of model output, standard deviations,  standard errors, etc., can
   be determined.

•  When it is not possible to describe uncertainty quantitatively, references to the literature may
   help support a conclusion.

•  One participant commented that variability cannot be reduced, but that the associated
   uncertainty can be defined/reduced through use of probabilistic analysis to reach a quantitative
   estimate of uncertainty.  Uncertainty factors may  also be useful in some situations to account
   for uncertainties associated with  extrapolation.

•  Describe how uncertainty is based on available data and analysis. For example, in the Clinch
   Valley Assessment a less than optimal taxonomic level was used for the benthic
   macroinvertebrate analysis. Investigators questioned whether a benthic macroinvertebrate
   index could serve as a surrogate for mussels in their study. The fish IBI had a much stronger
   correlation with the mussels than did the EPT index in that study, possibly because taxonomic
   data below the family level was unavailable.

•  At the end of the process the risk assessor should address which uncertainties have the most
   impact on the conclusion. Uncertainties should be ranked for the risk manager.

   The degree of uncertainty should be addressed as the degree  of confidence in discussions with
   the public.  While it is better to present what we are confident about, uncertainties in some
   exposures will still have to be explained.  Example: A certain degree of change in an exposure
   may not predict with certainly that recovery will occur because of uncertainties in the

•  Risk characterization should consider the risk management context. For example, reducing all
   uncertainties in data may be meaningless if there are no feasible management alternatives.
CHARGE QUESTION: What are the best ways to address and present the integration of
qualitative and quantitative lines of evidence for a) an individual assessment endpoint and
b) drawing overall conclusions?

   The team recommended that conclusions be presented in a summary table with references to
   the location of in-depth data.  Table 17 on page 99 of the Snake River report (reproduced for
   this report as Table 2-1) is a good example.  One participant noted that a similar approach is
   described in a report from the Society for Environmental Toxicology and Chemistry


   (SETAC) Pellston Workshop on Multiple Stressors in Ecological Risk and Impact Assessment
   (Pellston Workshop Report is now available. See Foran andFerenc, 1999).  The Pellston
   approach uses a series of matrices to 1) show stressor/risk rankings which provide managers
   with relative rankings of Stressors within each scenario, including uncertainty information; 2)
   show information about the magnitude, duration, and spatial extent of a stressor and the
   probability the effect will be expressed, including uncertainty information; and 3) allow
   development of a set of management scenarios with associated sets of Stressors and
   management options.

   The team agreed that maps showing Stressors and assessment endpoints are excellent tools for
   presentation; however, it was pointed out that people often don't realize there are
   uncertainties involved (accuracy and confidence associated with points on the map) when
   viewing map presentations. Web sites with maps were also suggested.

   Multiple Stressors and multiple endpoints often make it difficult to concisely state a
   conclusion. Use of a matrix, summary table, or the conceptual model may help in presentation
   of the results and conclusions.
CHARGE QUESTION: How should the degree of adversity of predicted or observed
effects in watersheds be described (e.g., considering nature and intensity of effects, spatial
and temporal scale, and the potential for recovery)?

•  The Snake River case study is an example where it is easy to illustrate present versus historical
   conditions using photographs and maps.

•  Maps showing individual Stressors help explain the overall situation, especially if there is some
   ranking or scoring system.

•  Recovery potential is included in Tables 17 and 19 on pages 99 and 101 of the Snake River
   report (see Table 2-1 above).

•  Refer to the literature on similar situations to give best professional judgment about what
   might happen.  Sometimes the predicted recovery does not occur.  In the Clinch Valley some
   improved areas which now provide a suitable habitat still do not have mussels. Another
   example is areas where stocking of salmon and habitat improvement have not resulted in
   restored populations.

   -   Temporal scales for  recovery vary  for different species (e.g., earthworms versus bear),
       thus making recovery difficult to estimate.  Spatial considerations should be appropriate
       for the assessment.

   -   Units of adversity, such as percent  mortality, may be more useful than rankings if they are
       understood by the public and directly relate to something the public cares about.

CHARGE QUESTION: How are alternative management options selected to have their
risks characterized and how is management informed of the consequences?

•  Ideally, management options should be discussed during planning or problem formulation;
   however, after some findings are available later in the process, additional discussions with
   management may be needed because some options may no longer be realistic and new
   alternatives must be considered.  At the beginning of an assessment it may not be known
   whether recovery is possible, so it may not be clear what alternatives should be considered.  If
   management options are not discussed throughout the process, the risk assessment may not
   produce meaningful findings.

•  Later in a study, the assessor may be able to determine subsections of a study area that need
   different management options to achieve management goals.

•  A meeting with the stakeholders following the analysis phase was advised to plug in different
   scenarios based on stakeholder input.  In addition, periodic meetings with stakeholders should
   be held to discuss management options and to receive other viewpoints that might help explain

•  When stakeholders are included throughout the process, they understand the process and have
   buy-in from the beginning.  When conclusions are reached, the stakeholders  are more likely to
   understand them,  and the process does not have to be explained from the beginning. The
   stakeholders were a part of the entire assessment.

•  A recommendation for more extensive involvement and consultation with risk managers and
   stakeholders during the analysis phase should be added to the guidelines. Such involvement
   could change the direction of a study.

   -   Public advisory committees are a forum to exchange information and meet regularly.
       Public meetings with stakeholders should always be scheduled in advance, and scientists
       and investigators should be available for questions and discussions.
CHARGE QUESTION: How should ecological risk be communicated to the manager and
the public (journal articles, computer programs, presentations, town meetings, Internet,

•   In risk communication, a few participants felt writers should apply the "teenager rule" (i.e.,
    the report should be explainable to a teenager) or use writing at a seventh-grade level. For
    example, say "no dissolved oxygen," rather than "anoxia."

    Several members of the team felt the risk characterization should be written at the level of the
    technical literacy of the risk manager.  Any version of the risk characterization should be
    written as appropriate for the target audience.

•  Many felt the risk characterization should be a scientific report, but it should have a "relevant
   findings" section for the manager.

•  Alternatively a "plain language" watershed plan/risk characterization could be written with
   technical appendices. This is in agreement with the approach provided in the Guidelines and

•  Terms not in the dictionary should be briefly defined in the text or in a glossary. If a term is a
   central concept that will be used repeatedly, it may need even more definition (possibly
   expanded to a paragraph or two). Words that are contained in the glossary should be in bold
   type the first time they appear in the text.

•  Technical terms should not be used in the executive summary or the relative findings  section
   of the risk characterization.

•  Encourage stakeholders to participate in presentations. It is advantageous if the
   communicator is part of the audience group or is otherwise considered to be credible by the

•  Have open houses for school children.

•  Meet the needs of the media.

•  Reporting units used for the results should be compatible with the display of other results.
   Consistency in presentation is helpful for the readers.

Foran, J., and S. Ferenc (Eds.) (1999) Multiple stressors in ecological risk and impact assessment.
Proceedings from the Pellston Workshop on Multiple Stressors in Ecological Risk and Impact Assessment:
13-18 September 1997, Pellston, Michigan. Society for Environmental Toxicology and Chemistry,

Fox, G.A. (1991) Practical causal inference for ecoepidemiologists. J. Toxicol. Environ. Health 33:359-

Hill, A.B. (1965) The environment and disease: Association or causation?  Proc. R.  Soc. Med. 58:295-

National Research Council. (1989) Improving risk communication.  Washington, DC: National Academy

National Research Council. (1996) Understanding risk: informing decisions in a democratic society.
Washington, DC: National Academy Press.

Pickle, L. W., M. Mingle, G. K. Jones, and A. A. White.  (1996) Atlas of United States mortality.
Hyattsville, Maryland: National Center for Health Statistics.

TN & Associates, Inc. (1999) Workshop Report on Developing a Problem Formulation Process
Large Spatial Scales. Oak Ridge, TN.

The Presidential/Congressional Commission Risk Assessment and Risk Management. (1997) Risk
assessment and risk management in regulatory decision-making, Volume 2.  Washington, DC.

Susser, M.  (1986) Rules of inference in epidemiology. Regul. Toxicol. Pharmacol. 6:116-128.

Suter, G. W. (1993) A critique of ecosystem health concepts and indexes. Environ. Toxicol. Chem.

U.S. EPA (U. S. Environmental Protection Agency). (1992) Framework for ecological risk assessment.
Washington, DC: Risk Assessment Forum, U.S. Environmental Protection Agency.  EPA/630/R-92/001.

U.S. EPA.  (1998) Guidelines for ecological risk assessment. Washington, DC: Risk Assessment Forum,
U.S. Environmental  Protection Agency. EPA/630/R-95/002F.

U. S. EPA. (1999) Risk characterization materials for peer review -March 1999.  Washington, DC:
Office of Research and Development, U.S. Environmental Protection Agency.  EPA/600/R-99/025.


Attendee Name
Michael Slimak
ECay Austin

Jack Gentile
William Smith
Patrick Bourgeron

Panel Members:
Vic Serveiss
Bill van der Schalie
Sue Norton
Jerry Diamond
Roberta Hylton
Don Gowan
Maaaie Geist


University of Miami
Yale University
Colorado State University

Tetra Tech
Southwest Virginia Field Office
The Nature Conservancy
Waciuoit Bav NERR

401 M Street, SW, 8601D
Washington, DC 20460
401 M Street, SW, 8 60 ID
Washington, DC 20460

Center for Marine and Environmental Analysis
University of Miami
4600 Rickenbacker Causeway
Miami, FL 33149-4152
Yale School of Environment & Forestry Studies
285 Prospect
New Haven, CT 065 11
INSTAAR University of Colorado
Campus Box 450, 1560 30th Street
Boulder, CO 80309

Mail Code 8623D
40 1M Street, SW
Washington, DC 20460
National Center for Environmental Assessment
Washington Office
Mail Code 8623D
40 1M Street, SW,
Washington, DC 20460
Mail Code 8623D
40 1M Street, SW
Washington, DC 20460
10045 Red Run Blvd., Suite 110
O wings Mills, MD 21 117
988 West Main Street
AbmgdonVA, 24210
151 W. Main Street
149Waquoit Highway
Waciuoit MA 02536





william smith@y ale. edu
Patrick. Bourgeron@Colorado.EDU

roberta hylton@fws.gov
m2eist@,c ap ec od. net
Interest in Meeting

Associate Director for Ecology, National Center
for Environmental Assessment
Senior Science Advisor for Ecology, National
Center for Environmental Assessment

Watershed Ecological Risk Assessor
Ecological Risk Assessor
Regional Ecological Assessor

Watershed Prototype Assessments & Workshop
Guidelines Author
Guidelines Author & Big Darby Creek Researcher
Watershed Ecological Risk Assessor
Fish & Wildlife Service, Southwest Virginia
Field Office
Clinch Valley Assessment Co-chair
Waciuoit Bav Assessment Co-chair

Attendee Name
Panel Members continued:
Patti Tyler
Bob Fenemore
Maria Downing
Patricia Cirone
Susan Cormier

Additional Participants:
John Miller
James Andreasen
Catriona Rogers
Anne Sergeant
Bill Ewald
Susan Ferenc
Ed Bender
Lawrence Martin
Laura Gabanski
Ken Jones

USEPA Region 1

ILSI Risk Science Institute
Green Mountain Institute

EPA New England Region 1
Office of Ecosystem Assessment
60 Westview Street
Lexington, MA 02421
Region 7
901 North Fifth Street
Kansas City, KS 66101
Region 7
901 North Fifth Street
Kansas City, KS 66101
Region 10
1200 Sixth Avenue
Seattle WA 98101
26 West Martin Luther King Drive
Cincinnati OH, 45268

40 1M Street, SW
Washington, DC 20460
401 M Street, SW
Mail Code 8623D
Washington, DC 20460
401 M Street
Mail Code 8601D
Washington, DC 20460
401 M Street, SW
Mail Code 8623D
Washington, DC 20460
MD52, USEPA Mailroom
Research Triangle Park, NC 2771 1
11 26 16th Street NW
Washington, DC 20036
Office of Science Policy (ORD-8 1 03R)
Washington, DC 20460
USEPA Headquarters, 8103R
40 1M Street, SW
Washington, DC 20460
4503F, 401 M Street, SW
Washington, DC 20460
104 East State Street
Montoelier. VT 05602



tyler.patti^epamail. epa.gov
Do wning.Marla(SJepamail. epa.gov
Cormier. Susan(S5epa.gov

miller. johne(55epa.gov
Rogers. Catriona(5);epa.gov
sergeant. anne(tf5epamail. epa.gov
Bender .Ed(S),epamail. epa.gov
Martin.Lawrence(5);epamail. epa.gov
Gabanski.Laura(tf5epamail. epa.gov
ki onesfoJgmied. org
Interest in Meeting

Waquoit Bay Assessment Co-chair
Middle Platte Assessment Chair
Former Middle Platte Assessment Chair
Middle Snake Assessment Chair & Guidelines
Big Darby Creek Assessment Co-chair

Former Watershed Prototype Assessment Project
Proj ect Manager of Columbia River Eco Risk
Regional Assessment of Ecological Impacts of
Multiple Stressors
Developer of Eco Risk Assessment Training
Course and Guidelines Author
Neuse River Risk Assessment
Ecological Risk Assessor
Cumulative (Eco and Health Risk) Coordinator
ORD State and Local Relations Coordinator
Former Participant Waquoit Bay Assessment,
Developer of Eco Risk & Decision Making
Green Mountain Institute

Attendee Name [Organization
Additional Participants continued:
Barry Terming
Robert Murphy
Margaret Stewart
Jennifer Moses
Bruce Yeager
Angie Stiles
Ray Alexander
Dennis Yankee
Michelle Witten
Janice Cox
Donald Rodier
The Council of State
Alliance for the Chesapeake Bay
Watershed Environment Research
Tennessee Valley Authority
Tennessee Valley Authority
Tennessee Valley Authority
Tennessee Valley Authority
Tennessee Valley Authority
Green Mountain Institute
Tennessee Valley Authority

2760 Research Park Drive
PO Box 11910
Lexington, KY, 40578-1910
6600 York Rd, #100
Baltimore, MD 21212
601 Wythe Street
Alexandria, VA 22314-1994
Reservation Rd, CEB 3A
Muscle Shoals, AL 35662-1010
17 Ridgeway Road (Box 920)
Noms, TN 37828
400 W. Summit Hill Drive, WT 8C
ECnoxville, TN 37902
1101 Market Street, HB 2A
Chattanooga, TN 37402
129 Pine Road
Noms, TN 37828
104 East State Street
Montpeher, VT 05602
1101 Market Street, CST 17D
Chattanooga, TN 37402
Science Support Branch, Risk Assessment Division (7403)
Office of Pollution Prevention and Toxics
401 M Street, SW
Washinaton DC 20460


btonning(5);csg. org
bmurphy (Sjacb-online. org
mstewartt^wef . org
rkalexander@tva. gov
mwitten@gmied. org
Rodier DonaldfSepa 2ov
Interest in Meeting

Watershed Monitoring, Assessment,
Management, and Watershed Management
Watershed Restoration
Watershed Management
TVA Technical Writer
TVA Technical Writer
TVA Logistics Coordinator
Manager, TVA Remote Sensing
TVA GIS & Landscape Ecologist
Green Mountain Institute
TVA Risk Assessment Specialist
Ecolo2ical Risk Assessor and Guidelines Author



                 Watershed Ecological Risk Characterization Workshop
                                      July 7-8, 1999
I.    Purpose of Workshop: To further develop and document the process for conducting
     watershed ecological risk characterization.

II.   Desired Output: Develop recommendations for how to conduct a watershed ecological
     risk characterization. Discuss the desirable items to include in such a characterization and
     how to address common situations.

III.  Workshop Approach: We are developing additional guidance for how one should develop
     the risk characterization phase of a watershed scale ecological risk assessment. We are
     advancing what is documented in the ecological risk assessment guidelines.  While we will
     attempt to comply with the guidelines, we will also deviate from them if justifiable. Where
     applicable we may also develop supplemental and deviating guidelines for the analysis phase
     of risk assessment.

     The Agency is also developing a risk characterization handbook. These will be revised
     based on peer review comments. The peer review comments noted that  the document needs
     to more fully incorporate ecological risk assessment principles.  The latest draft does
     conclude that the risk characterization should summarize the earlier phases of the risk
     assessment including planning and problem formulation.  The current draft also requires the
     risk characterization to be  written in "plain language" and to be concise.

     We need an approach that attempts to comply with the Agency ecological risk assessment
     guidelines and the Agency risk characterization guidelines.  To achieve scientific credibility
     while also using plain language and concise explanations we will develop a scientific risk
     characterization report that will also include a relevant findings section in plain language. In
     addition, the executive  summary for the entire assessment will also be written to be
     understandable to an educated non-scientist.

IV.  Charge questions to consider when reviewing the provided risk characterizations and other
     background materials.

     A.  How can the three submitted watershed assessment drafts (or plans) be improved,
         especially in regards to the other charge questions listed below?
     B.  How should exposure and effects data be integrated in a watershed context to generate
         a risk estimate?

     C.  How should uncertainty be addressed and presented (e.g., incomplete data or analyses,
         qualitative estimates, data at different spatial and organizational levels)?

     D.  What are the best ways to address and present the integration of qualitative and
         quantitative lines of evidence for (a) an individual assessment endpoint and (b) drawing
         overall conclusions?

     E.  How should the degree of adversity of predicted or observed effects in watersheds be
         described (e.g., considering nature and intensity of effects, spatial and temporal scale,
         and the potential for recovery)?

     F.  How are alternative management options selected to have their risks characterized and
         how is management informed of the consequences?

     G.  How should ecological risk be communicated to the manager and the public (journal
         articles, computer programs, presentations, town meetings, Internet, fact-sheets)?

V.   Background Documents:

     A.  Ecological Risk Assessment Guidelines,  Section on Risk Characterization

     B.  Draft Risk Characterization Handbook, Sections 1, 2, & 3

     C.  Waquoit Bay Risk Characterization Plan

     D.  Review Comments on Waquoit Bay Risk Characterization Document

     E.  Draft Middle Snake Watershed Risk Characterization

     F.  Draft Clinch Valley Watershed Risk Characterization

                       Watershed Risk Characterization Workshop
                                     July 7 & 8 1999
July 7:
8:00           Registration and conversation

8:45           Opening remarks, value of this workshop
               Mike Slimak, Associate Director for Ecology
               National Center for Environmental Assessment (NCEA)

9:00           Workshop goals and approach, progress on five case studies, review agenda
               Vic Serveiss, Environmental Scientist, NCEA

9:15           Introductions

9:25           Risk characterization summary
               Bill van der Schalie, Environmental  Scientist, NCEA

9:50           Different traditions of environmental management and their implications for
               watershed ecological risk assessment
               Sue Norton, Ecologist,  NCEA

               10:15  Break

10:40          Presentation and discussion of five watershed assessments
               Present the risk characterization  draft or plan.
               Discuss charge question #1:  How can each watershed risk characterization be

10:45          Big Darby Creek, Susan Cormier

11:15          Middle Platte, Dennis Jelinski

11:45          Lunch

12:45          Waquoit Bay, Maggie Geist; Jack Gentile, facilitator

2:00           Middle Snake, Patricia Cirone;  Patrick Bourgeron, facilitator

3:15           Break

3:30           Clinch Valley, Jerry Diamond; William Smith, facilitator

4:45           Plan for tomorrow, Vic Serveiss

5:00-6:00      Optional conversation time (Crystal  Club Room)


July 8:
8:00           Optional conversation time

               Facilitators present responses to charge questions # 2-7
8:30           Facilitator #1   Jack Gentile

9:00           Facilitator #2   William Smith

9:30           Facilitator #3   Patrick Bourgeron

10:00          Break

10:15          Break-out groups led by facilitators developed recommendations for conducting
               watershed eco-risk characterization by developing replies to charge questions.

Break-out Session Team 1  - Crystal Club Room
Break-out Session Team 2 - Board Room
Break-out Session Team 3  - Salon F (Main Room)

12:00-1:00     Lunch

2:30           Report back by a volunteer from each group

3:30           Break

4:00           Group  synthesis/consensus, led by invited experts

4:45           Closing comments-Serveiss/Norton/van der Schalie

5:00           Adjourn

Break out teams:
   Crystal Club Room
   Team 1
   James Andreasen
   Kay Austin
   Patricia Cirone
   Janice Cox
   Maria Downing
   Bill Ewald
   Laura Gabanski
   Jack Gentile
   Don Gowan
   Ken Jones
   John Miller
   Margaret Stewart
   Bill van der Schalie
Board Room
Team 2
Ed Bender
Bob Fenemore
Jack Fowle
Maggie Geist
Roberta Hylton
Lawrence Martin
Robert Murphy
Sue Norton
Anne Sergeant
William Smith
Molly Whitworth
Dennis Yankee
Bruce Yeager
Salon F
Richard Batiuk
Joyce Brooks
Patrick Bourgeron
Susan Cormier
Jerry Diamond
Susan Ferenc
Jennifer Moses
Donald Rodier
Catriona Rogers
Vic Serveiss
Michael Slimak
Barry Tonning
Patti Tyler
Michelle Witten

              APPENDIX C

                            5. RISK CHARACTERIZATION

       Risk characterization (figure 5-1) is the final phase of ecological risk assessment and is the
culmination of the planning, problem formulation, and analysis of predicted or observed adverse
ecological effects related to the assessment endpoints. Completing risk characterization allows
risk assessors to clarify the relationships between stressors, effects, and ecological entities and to
reach conclusions regarding the occurrence of exposure and the adversity of existing or
anticipated effects. Here, risk assessors first use the results of the analysis phase to develop an
estimate of the risk posed to the ecological entities included in the assessment endpoints identified
in problem formulation (section 5.1). After estimating the risk, the assessor describes the risk
estimate in the context of the significance of any adverse effects and lines of evidence supporting
their likelihood (section 5.2). Finally, the assessor identifies and summarizes the uncertainties,
assumptions, and qualifiers in the risk assessment and reports the conclusions to risk managers
(section 5.3).
       Conclusions presented in the risk characterization should provide clear information to risk
managers in order to be useful for environmental decision making (NRC, 1994;  see section 6).  If
the risks are not sufficiently defined to support a management decision, risk managers may elect
to proceed with another iteration of one or more phases of the risk assessment process.
Reevaluating the conceptual model (and associated risk hypotheses) or conducting additional
studies may improve the risk estimate. Alternatively, a monitoring program may help managers
evaluate the consequences of a risk management decision.

       Risk estimation is the process of integrating exposure and effects data and evaluating any
associated uncertainties. The process uses exposure and stressor-response profiles developed
according to the analysis plan (section 3.5).  Risk estimates can be developed using one or more
of the following techniques: (1) field observational studies, (2) categorical rankings, (3)
comparisons of single-point exposure and effects estimates, (4) comparisons incorporating the
entire stressor-response relationship, (5) incorporation of variability in exposure and/or effects
estimates, and (6) process models that rely partially or entirely on theoretical approximations of
exposure and effects.  These techniques are described in the following sections.

                                           PROBLEM FORMULATION


                                           RISK CHARACTERIZATION
                                   Risk  \
                                   Risk   \
                   Communicating Results to the Risk Manager
                     Risk Management and Communicating
                         Results to Interested Parties
Figure 5-1. Risk characterization.

5.1.1.  Results of Field Observational Studies
       Field observational studies (surveys) can serve as risk estimation techniques because they
provide empirical evidence linking exposure to effects. Field surveys measure biological changes
in natural settings through collection of exposure and effects data for ecological entities identified
in problem formulation.
       A major advantage of field surveys is that they can be used to evaluate multiple stressors
and complex ecosystem relationships that cannot be replicated in the laboratory.  Field surveys are
designed to delineate both exposures and effects (including secondary effects) found
in natural systems, whereas estimates generated
from laboratory studies generally delineate either
exposures or effects under controlled or
prescribed conditions (see text box 5-1).
       While field studies may best represent
reality, as with other kinds of studies they can be
limited by (1) a lack of replication, (2) bias in
obtaining representative samples, or (3) failure to
measure critical components of the system or
random variations.  Further, a lack of observed
effects in a field survey may occur because the
measurements lack the sensitivity to detect
ecological effects.  See section 4.1.1 for additional
discussion of the strengths and limitations of
different types of data.
       Several assumptions or qualifications need
to be clearly articulated when describing the results of field surveys.  A primary qualification is
whether a causal relationship between stressors and effects (section is supported. Unless
causal relationships are carefully examined, conclusions about effects that are observed may be
inaccurate because the effects are caused by factors unrelated to the stressor(s) of concern.  In
addition, field surveys taken at one point in time are usually not predictive; they describe effects
associated only with exposure scenarios associated with past and existing conditions.
Text Box 5-1. An Example of Field Methods
Used for Risk Estimation

Along with quotients comparing field measures
of exposure with laboratory acute toxicity data
(see Text Box 5-3), EPA evaluated the risks of
granular carbofuran to birds based on incidents
of bird kills following carbofuran applications.
More than 40 incidents involving nearly 30
species of birds were documented. Although
reviewers identified problems with individual
field studies (e.g., lack of appropriate control
sites, lack of data on carcass-search efficiencies,
no examination of potential synergistic effects of
other pesticides, and lack of consideration of
other potential receptors such as small
mammals), there was so much evidence of
mortality associated with carbofuran application
that the study deficiencies did not alter the
conclusions of high risk found by the assessment
(Houseknecht, 1993).

  Text Box 5-2.  Using Qualitative Categories to
  Estimate Risks of an Introduced Species

  The importation of logs from Chile required an
  assessment of the risks posed by the potential
  introduction of the bark beetle, Hylurgus ligniperda
  (USDA, 1993). Experts judged the potential for
  colonization and spread of the species, and their
  opinions were expressed as high, medium, or low as
  to the likelihood of establishment (exposure) or
  consequential effects of the beetle.  Uncertainties were
  similarly expressed. A ranking scheme was then used
  to sum the individual elements into an overall
  estimate of risk (high, medium, or low). Narrative
  explanations of risk accompanied the overall
5.1.2.  Categories and Rankings
       In some cases, professional judgment
or other qualitative evaluation techniques
may be used to rank risks using categories,
such as low, medium, and high, or yes and
no. This approach is most frequently used
when exposure and effects data are limited or
are not easily expressed in quantitative terms.
The U.S. Forest Service risk assessment of
pest introduction from importation of logs
from Chile used qualitative categories owing
to limitations in both the exposure and effects
data for the introduced species of concern as
well as the resources available for the assessment (see text box 5-2).
       Ranking techniques can be used to translate qualitative judgment into a mathematical
comparison.  These methods are frequently used in comparative risk exercises. For example,
Harris et al. (1994) evaluated risk reduction opportunities in Green Bay (Lake Michigan),
Wisconsin, employing an expert panel to compare the relative risk of several stressors against
their potential effects.  Mathematical analysis based on fuzzy set theory was used to rank the risk
from each stressor from a number of perspectives, including degree of immediate risk, duration of
impacts,  and prevention and remediation management.  The results served to rank potential
environmental risks from stressors based on best professional judgment.

 5.1.3. Single-Point Exposure and Effects Comparisons
       When sufficient data are available to quantify  exposure and effects estimates, the simplest
approach for comparing the estimates is a ratio (figure 5-2a).  Typically, the ratio (or quotient) is
expressed as an exposure concentration divided by an effects concentration.  Quotients  are
commonly used for chemical stressors, where reference or benchmark toxicity values are widely
available (see text box 5-3).
       The principal advantages of the quotient method are that it is simple and quick to use and
risk assessors and managers are familiar with its application. It provides an efficient, inexpensive
means of identifying high- or low-risk situations that can allow risk management decisions to be
made without the need for further information.
       Quotients have also been used to integrate the risks of multiple chemical stressors:
quotients for the individual constituents in a mixture are generated by dividing each exposure

       a:  Comparison of point estimates
                             (e.g., mean
                                 (e.g., LC10)
    b:  Comparison of a point estimate of a stressor-response
    relationship with uncertainty associated with an exposure
    point estimate
e.g., uncertainty around
mean concentration
                     Intensity of Stressor (e.g., concentration)

         Figure 5-2. Risk estimation techniques, a. Comparison of exposure and stressor-response
         point estimates, b. Comparison of a point estimate from the stressor-response relationship
         with uncertainty associated with an exposure point estimate.
level by a corresponding toxicity endpoint (e.g., LC50, EC50, NOAEL).  Although the toxicity of a

chemical mixture may be greater than or less than predicted from the toxicities of individual

constituents of the mixture, a quotient addition approach assumes that toxicities are additive or

approximately additive.  This assumption may be most applicable when the modes of action of

chemicals in a mixture are similar, but there is evidence that even with chemicals having dissimilar

modes of action, additive or near-additive interactions are common (Konemann, 1981; Broderius,

1991; Broderius et al., 1995; Hermens et al., 1984a, b; McCarty and Mackay, 1993; Sawyer and

Safe, 1985).  However, caution should be used when assuming that chemicals in a mixture act
independently of one another, since many of the supporting studies were conducted with aquatic
organisms, and so may not be relevant for other endpoints, exposure scenarios,  or
species.  When the modes of action for
constituent chemicals are unknown, the
assumptions  and rationale concerning
chemical interactions should be clearly stated.
       A number of limitations restrict
application of the quotient method (see Smith
and Cairns, 1993; Suter, 1993a).  While a
quotient can  be useful in answering whether
risks are high or low, it may not be helpful to
a risk manager who needs to make a decision
requiring an incremental quantification of
risks. For example, it is seldom useful to say
that a risk mitigation approach will reduce a
quotient value from 25 to 12, since this
reduction cannot by itself be clearly
interpreted in terms of effects on an
assessment endpoint.
       Other limitations of quotients may be
caused by deficiencies in the problem formulation and analysis phases. For example, an LC50
derived from a 96-hour laboratory test using constant exposure levels may not be appropriate for
an assessment of effects on reproduction resulting from short-term, pulsed exposures.
       In addition, the quotient method may not be the most appropriate method for predicting
secondary effects (although such effects may be inferred).  Interactions and effects beyond what
are predicted from the simple quotient may be critical to characterizing the full extent of impacts
from exposure to the stressors  (e.g., bioaccumulation, eutrophication, loss of prey species,
opportunities for invasive species).
       Finally, in most cases, the quotient method does not explicitly consider uncertainty (e.g.,
extrapolation from tested species to the species or community of concern).  Some uncertainties,
however, can be incorporated into single-point estimates to provide a statement of likelihood that
the effects point estimate exceeds the exposure point estimate (figures 5-2b and 5-3).  If exposure
variability is  quantified, then the point estimate of effects can be compared with a cumulative
Text Box 5-3.  Applying the Quotient Method

When applying the quotient method to chemical
stressors, the effects concentration or dose (e.g., an
LC50, LD50, EC50, ED50, NOAEL, or LOAEL) is
frequently adjusted by uncertainty factors before
division into the exposure number (U.S. EPA, 1984;
Nabholz, 1991; Urban and Cook, 1986; see section, although EPA used a slightly different
approach in estimating the risks to the survival of
birds that forage in agricultural areas where the
pesticide granular carbofuran is applied
(Houseknecht,  1993).  In this case, EPA calculated
the quotient by dividing the estimated exposure levels
of carbofuran granules in surface soils (number/ft2) by
the granules/LD50 derived from single-dose avian
toxicity tests. The calculation yields values with units
of LD50/ft2. It was assumed that a higher quotient
value corresponded to an increased likelihood that a
bird would be exposed to lethal levels of granular
carbofuran at the soil surface. Minimum and
maximum values for LD50/ft2 were estimated for
songbirds, upland game birds, and waterfowl that
may forage within or near 10 different agricultural

e.g., uncertainty around
mean concentration
e.g., uncertainty around LC
         Intensity of Stressor (e.g., concentration)
                                    e.g., probability that LC10 > mean concentration
         Figure 5-3. Risk estimation techniques: comparison of point estimates with associated
exposure distribution as described in text box 5-4. Further discussion of comparisons between
point estimates of effects and distributions of exposure may be found in Suter et al., 1983.
       In view of the advantages and limitations of the quotient method, it is important for risk
assessors to consider the points listed below when evaluating quotient method estimates.

       •               How does the effect concentration relate to the assessment endpoint?

       •               What extrapolations are involved?

       •               How does the point estimate of exposure relate to potential spatial and
                      temporal variability in exposure?

                      Are data sufficient to provide confidence intervals on the endpoints?

5.1.4.  Comparisons Incorporating the
Entire Stressor-Response Relationship
       If a curve relating the stressor level to
the magnitude of response is available, then
risk estimation can examine risks associated
with many different levels of exposure (figure
5-4). These estimates are particularly useful
when the risk assessment outcome is not
based on exceedance of a predetermined
decision rule, such as a toxicity benchmark
       There are advantages and limitations
to comparing a stressor-response curve with
an exposure distribution.  The slope of the
effects curve shows the magnitude of change
in effects associated with incremental changes
in exposure, and the capability to predict changes in the magnitude and likelihood of effects for
different exposure scenarios can be used to compare different risk management options.  Also,
uncertainty can be incorporated by calculating uncertainty bounds on the stressor-response or
exposure estimates.  Comparing exposure and stressor-response curves provides a predictive
ability lacking in the quotient method. Like the quotient method, however, limitations from the
problem formulation and analysis phases may limit the utility of the results. These limitations may
include not fully considering secondary effects, assuming the exposure pattern used to derive the
stressor-response curve is comparable to the environmental exposure pattern, and failure to
consider uncertainties, such as extrapolations from tested species to the species or community of
                                                Text Box 5-4.  Comparing an Exposure
                                                Distribution With a Point Estimate of Effects
                                                The EPA Office of Pollution Prevention and Toxics
                                                uses a Probabilistic Dilution Model (PDM3) to
                                                generate a distribution of daily average chemical
                                                concentrations based on estimated variations in
                                                stream flow in a model system. The PDM3 model
                                                compares this exposure distribution with an aquatic
                                                toxicity test endpoint to estimate how many days in a
                                                1-year period the endpoint concentration is exceeded
                                                (Nabholz et al, 1993; U.S. EPA,  1988b).  The
                                                frequency of exceedance is based on the duration of
                                                the toxicity test used to derive the effects endpoint.
                                                Thus, if the endpoint was an acute toxicity level of
                                                concern, an exceedance would be identified if the
                                                level of concern was exceeded for 4 days or more (not
                                                necessarily consecutive). The exposure estimates are
                                                conservative in that they assume instantaneous
                                                mixing of the chemical in the water column and no
                                                losses due to physical, chemical, or biodegradation
5.1.5.  Comparisons Incorporating Variability in Exposure and/or Effects
       If the exposure or stressor-response profiles describe the variability in exposure or effects,
then many different risk estimates can be calculated.  Variability in exposure can be used to
estimate risks to moderately or highly exposed members of a population being investigated, while
variability in effects can be used to estimate risks to average or sensitive population

   0 0.50
                     distribution of
      comparison of
      90th percentile exposure
      with EC10
      comparison of
      50th percentile exposure
      with EC
                        Intensity of Stressor (e.g., concentration)
  Figure 5-4.  Risk estimation techniques:  stressor-response curve versus a cumulative distribution of
members. A major advantage of this approach is its ability to predict changes in the magnitude

and likelihood of effects for different exposure scenarios and thus provide a means for comparing

different risk management options. As noted above, comparing distributions also allows one to

identify and quantify risks to different segments of the population.  Limitations include the

increased data requirements compared with previously described techniques and the implicit

assumption that the full range of variability in the exposure and effects data is adequately

represented.  As with the quotient method,
secondary effects are not readily evaluated
with this technique.  Thus, it is desirable to
corroborate risks estimated by distributional
comparisons with
field studies or other lines of evidence. Text
box 5-5 and figure 5-5 illustrate the use of
cumulative exposure and effects distributions
for estimating risk.

5.1.6.  Application  of Process Models
       Process models are mathematical
expressions that represent our understanding
of the mechanistic operation of a system
under evaluation.  They can be useful tools in
both analysis (see section 4.1.2) and risk
characterization.  For illustrative purposes, it
is useful to distinguish between analysis
process models, which focus individually on
either exposure or effects evaluations, and
risk estimation process models, which
integrate exposure and effects information
(see text box 5-6). The assessment of risks
associated with long-term changes  in
hydrologic conditions in bottomland forest
wetlands in Louisiana using the FORFLO
model (Appendix D) linked the attributes and
placement of levees  and corresponding water
level measurements  (exposure) with changes
in forest community structure and wildlife
habitat suitability (effects).
       A major advantage of using process
models for risk estimation is the ability to
consider "what if scenarios and to forecast beyond the limits of observed data that constrain
techniques based solely on empirical data. The process model can also consider secondary
Text Box 5-5.  Comparing Cumulative Exposure
and Effects Distributions for Chemical Stressors
Exposure distributions for chemical stressors can
be compared with effects distributions derived
from point estimates of acute or chronic toxicity
values for different species (e.g., HCN, 1993;
Cardwell et al., 1993; Baker et al, 1994;
Solomon et al., 1996). Figure 5-5 shows a
distribution of exposure concentrations of an
herbicide compared with single-species toxicity
data for algae (and one vascular plant species) for
the same chemical.  The degree of overlap of the
curves indicates the likelihood that a certain
percentage of species may be adversely affected.
For example, figure 5-5 indicates that the 10th
centile of algal species' EC5 values is exceeded
less than 10% of the time.

The predictive value of this approach is evident.
The degree of risk reduction that could be
achieved by changes in exposure associated with
proposed risk mitigation options can be readily
determined by comparing modified exposure
distributions with the effects distribution curve.

When using effects distributions derived from
single-species toxicity data, risk assessors should
consider the following questions:

• Does the subset of species for which toxicity
  test data are available represent the range of
  species present in the environment?

• Are particularly sensitive (or insensitive) groups
  of organisms represented in the distribution?

• If a criterion level is selected—e.g., protect 95%
  of species—does the 5% of potentially affected
  species include organisms of ecological,
  commercial, or recreational significance?


| 90
CO 7°
0 50
>- 30
§ 10


Distribution of ^
water T
concentrations F

^^/ ^^^^^
/ /
/ /
V \
/ \
I i

Comparison of 90th centile |
concentration with 10th centile i
nf the I n,s .1
1 0~2 1 0"^ 1 0^ 1 0'' '
Figure 5-5. Risk estimation techniques:

Distribution of LC5s
ML Dunaliella tertiolocta
JL Porphyridium cruentum
^ Chlorella pyrenoidosa
^ Lemna gibba
Microcystis aeruginosa
^ Selenastrum capricornutum
Skelotonema costatum

Isochrysis galbana


1 —
90 K
70 =>
50 ES
30 a
1 1
10 °


I02 103
IN |jg/L
comparison of exposure distribution of an herbicide in
surface waters with freshwater single-species toxicity data. See text box 5-4 for further
discussion. Redrawn from Baker et al.,
1994. (Centile ranks for species LCS data were
obtained using the formula (100 x «/[N+l]), where n is the rank number of the LC5 and TV is the
total number of data points in the set; adapted from Parkhurst et al., 1995).
effects, unlike other risk estimation techniques such as the quotient method or comparisons of
exposure and effect distributions.  In addition, some process models can forecast the combined
effects of multiple stressors, such as the effects of multiple chemicals on fish population
sustainability (Barnthouse et al., 1990).
       Process model outputs may be point estimates, distributions, or correlations; in all cases,
risk assessors should interpret them with care. They may imply a higher level of certainty than is

appropriate and are all too often viewed
without sufficient attention to underlying
assumptions. The lack of knowledge on basic
life histories for many species and incomplete
knowledge on the
structure and function  of a particular
ecosystem is often lost in the model output.
Since process models are only as good as the
assumptions on which  they are based, they
should be treated as hypothetical
representations of reality until appropriately
tested with empirical data.  Comparing model
results to field data provides a check on
whether our understanding of the system was
correct (Johnson, 1995), particularly with
respect to the risk hypotheses presented in
problem formulation.
Text Box 5-6. Estimating Risk With Process

Models that integrate both exposure and effects
information can be used to estimate risk. During risk
estimation, it is important that both the strengths and
limitations of a process model approach be
highlighted. Brody et al. (1993; see Appendix D)
linked two process models to integrate exposure and
effects information and forecast spatial and temporal
changes in forest communities and their wildlife
habitat value. While the models were useful for
projecting long-term effects based on an
understanding of the underlying mechanisms of
change in forest communities and wildlife habitat,
they could not evaluate all possible stressors of
concern and were limited in the plant and wildlife
species they could consider.  Understanding both the
strengths and limitations of models is essential for
accurately representing the overall confidence in the
       Following preparation of the risk estimate, risk assessors need to interpret and discuss the
available information about risks to the assessment endpoints. Risk description includes an
evaluation of the lines of evidence supporting or refuting the risk estimate(s) and an interpretation
of the significance of the adverse effects on the assessment endpoints.  During the analysis phase,
the risk assessor may have established the relationship between the assessment endpoints and
measures of effect and associated lines of evidence in quantifiable, easily described terms (section If not, the risk assessor can relate the available lines of evidence to the assessment
endpoints using qualitative links. Regardless of the risk estimation technique, the technical
narrative supporting the risk estimate is as important as the risk estimate itself.

5.2.1.  Lines of Evidence
       The development of lines of evidence provides both a process and a framework for
reaching a conclusion regarding confidence in the risk estimate.  It is not the kind of proof
demanded by experimentalists (Fox, 1991), nor is it a rigorous examination of weights of
evidence.   (Note that the term "weight of evidence" is sometimes used in legal discussions or in
other documents, e.g., Urban and Cook, 1986; Menzie et al.,  1996.) The phrase lines of evidence
is used to  de-emphasize the balancing of opposing factors based on assignment of quantitative

values to reach a conclusion about a "weight" in favor of a more inclusive approach, which
evaluates all available information, even evidence that may be qualitative in nature.  It is important
that risk assessors provide a thorough representation of all lines of evidence developed in the risk
assessment rather than simply reduce their interpretation and description of the ecological effects
that may result from exposure to stressors to a system of numeric calculations and results.
       Confidence in the conclusions  of a risk assessment may be increased by using several lines
of evidence to interpret and compare risk estimates. These lines of evidence may be derived from
different sources or by different techniques relevant to adverse effects on the assessment
endpoints, such as quotient estimates,  modeling results,  or field observational studies.
       There are three principal categories of factors for risk assessors to consider when
evaluating lines of evidence:  (1) adequacy and quality of data, (2) degree and type of uncertainty
associated with the evidence, and (3) relationship of the evidence to the risk assessment questions
(see also sections 3  and 4).
       Data quality directly influences how confident risk assessors can be in the results of a
study and conclusions they may draw  from it.  Specific concerns to consider for individual lines of
evidence include whether the experimental design was appropriate for the questions posed in a
particular study and whether data quality objectives were clear and adhered to.  An evaluation of
the scientific understanding of natural variability in the attributes of the ecological entities under
consideration is important in determining whether there  were sufficient data to satisfy the analyses
chosen and to determine if the analyses were sufficiently sensitive and robust to identify stressor-
caused perturbations.
       Directly related to data quality issues is the evaluation of the relative uncertainties of each
line of evidence. One major source of uncertainty comes from extrapolations. The greater the
number of extrapolations, the more uncertainty introduced into a study. For example, were
extrapolations used to infer effects in one species from another, or from one temporal or spatial
scale to another? Were conclusions drawn from extrapolations from laboratory to field effects, or
were field effects inferred from limited information, such as chemical structure-activity
relationships? Were no-effect or low-effect levels used to address likelihood of effects?  Risk
assessors should consider these and any other sources of uncertainty when evaluating the relative
importance of particular lines of evidence.
       Finally, how directly lines of evidence relate to the questions asked in the risk assessment
may determine their relative importance in terms of the ecological entity and the attributes of the
assessment endpoint. Lines of evidence directly related to the risk hypotheses, and those that
establish a cause-and-effect relationship based on a definitive mechanism rather than associations
alone, are likely to be of greatest importance.

       The evaluation process, however, involves more than just listing the evidence that
supports or refutes the risk estimate.  The risk assessor should carefully examine each factor and
evaluate its contribution in the context of the risk assessment.  The importance of lines of
evidence is that each and every factor is described and interpreted.  Data or study results are often
not reported or carried forward in the risk assessment because they are of insufficient quality. If
such data or results are eliminated from the evaluation process, however, valuable information
may be lost with respect to needed improvements in methodologies or recommendations for
further studies.
       As a case in point, consider the two lines of evidence described for the carbofuran example
(see text boxes 5-1 and 5-3), field studies and quotients.  Both approaches are relevant to the
assessment endpoint (survival of birds that forage in agricultural areas where carbofuran is
applied), and both are relevant to the exposure scenarios described in the conceptual model (see
figure D-l).  The quotients, however, are limited in their ability to express incremental risks (e.g.,
how much greater risk is expressed by a quotient of "2" versus a quotient of "4"), while the field
studies had some design flaws (see text box 5-1). Nevertheless, because of the strong evidence of
causal relationships from the field studies and consistency with the laboratory-derived quotient,
confidence in a conclusion of high risk to the assessment endpoint is supported.
       Sometimes lines of evidence do not point toward the same conclusion. It is important to
investigate possible reasons for any disagreement rather than ignore inconvenient evidence. A
starting point is to distinguish between true inconsistencies and those related to differences in
statistical powers of detection. For example, a model may predict adverse effects that were not
observed in a field survey. The risk assessor should ask whether the experimental design of the
field study had sufficient power to detect the predicted difference or whether the endpoints
measured were comparable with those used in the model.  Conversely, the model may have been
unrealistic in its predictions.  While iteration of the risk assessment process and collection of
additional data may help resolve uncertainties, this option is not always available.
       Lines of evidence that are to be evaluated during risk characterization should be defined
early in the risk assessment (during problem formulation) through the development of the
conceptual model and selection of assessment endpoints. Further, the analysis plan  should
incorporate measures that will contribute to the interpretation of the lines of evidence, including
methods of reviewing, analyzing, and summarizing the uncertainty in the risk assessment.
       Also, risk assessments often rely solely on laboratory or in situ bioassays to assess adverse
effects that may occur as a result of exposure to stressors.  Although they may not be manifested
in the field, ecological effects demonstrated in the laboratory should not be discounted as a line of

5.2.2. Determining Ecological Adversity
       At this point in risk characterization, the changes expected in the assessment endpoints
have been estimated and the supporting lines of evidence evaluated.  The next step is to interpret
whether these changes are considered adverse. Adverse ecological effects, in this context,
represent changes that are undesirable because they alter valued structural or functional attributes
of the ecological entities under consideration. The risk assessor evaluates the degree of adversity,
which is often a difficult task and is frequently based on the risk assessor's professional judgment.
       When the results of the risk assessment are discussed with the risk manager (section 6),
other factors, such as the economic, legal, or social consequences of ecological damage, should be
considered.  The risk manager will use all of this information to determine whether a particular
adverse effect is acceptable and may also find it useful when communicating the risk to interested
       The following are criteria for evaluating adverse changes in assessment endpoints:

       •   Nature  of effects and intensity of effects
       •   Spatial  and temporal scale
       •   Potential for recovery.

       The extent  to which the criteria are evaluated depends on the scope and complexity of the
risk assessment. Understanding the underlying assumptions and science policy judgments,
however, is important even in simple  cases. For example, when exceedance of a previously
established decision rule, such as a benchmark stressor level, is used as evidence of adversity (e.g.,
see Urban and Cook, 1986, or Nabholz,  1991), the reasons why this is considered adverse should
be clearly understood. In addition, any evaluation of adversity should examine all relevant
criteria, since none are considered singularly determinative.
       To distinguish adverse ecological changes  from those within the normal pattern of
ecosystem variability or those resulting in little or no significant alteration of biota, it is important
to consider the nature and intensity of effects. For example, for an assessment endpoint involving
survival, growth, and reproduction of a species,  do predicted effects involve survival and
reproduction or only growth? If survival of offspring will be affected, by what percentage will it
       It is important for risk assessors to consider both the ecological and statistical contexts of
an effect when evaluating intensity. For  example, a statistically significant 1% decrease in fish
growth (see text box 5-7) may not be relevant to an assessment endpoint offish population
viability, and a 10% decline in reproduction may be worse for a population of slowly reproducing
trees than for rapidly reproducing planktonic algae.

       Natural ecosystem variation can make it very difficult to observe (detect) stressor-related
perturbations. For example, natural fluctuations in marine fish populations are often large, with
intra- and interannual variability in population levels covering several orders of magnitude.
Furthermore, cyclic events of various periods (e.g., bird migration, tides) are very important in
natural systems and may mask or delay stressor-related effects.  Predicting the effects of
anthropogenic stressors against this background of variation can be very difficult.  Thus, a lack of
statistically significant effects in a field study does not automatically mean that adverse ecological
effects are absent. Rather, risk assessors should then consider other lines of evidence in reaching
their conclusions.
       It is also important to consider the location of the effect within the biological hierarchy
and the mechanisms that may result in ecological changes.  The risk assessor may rely on
mechanistic explanations to describe complex ecological interactions and the resulting effects that
otherwise may be masked by variability in the ecological components.
       The boundaries (global, landscape, ecosystem, organism) of the risk assessment are
initially identified in the analysis  plan prepared during problem formulation.  These spatial and
temporal scales are further defined in the analysis phase, where specific exposure and effects
scenarios are evaluated.  The spatial dimension encompasses both the extent and pattern of effect
as well as the context of the effect within the landscape.  Factors to consider include the absolute
area affected, the extent of critical habitats affected compared with a larger area of interest, and
the role or use of the affected area within the landscape.
       Adverse effects to assessment endpoints vary with the absolute area of the effect.  A larger
affected area may be (1) subject to a greater number of other stressors, increasing the
complications from stressor interactions, (2) more likely to contain sensitive species or habitats, or
(3) more susceptible to landscape-level changes because many ecosystems may be altered by the
       Nevertheless, a smaller area of effect
is not always associated with lower risk. The
function of an area within the landscape may
be more important than the absolute area.
Destruction of small but unique areas, such as
critical wetlands, may  have important effects
on local and regional wildlife populations.
Also, in river systems, both riffle and pool
areas provide important microhabitats that
maintain the  structure  and function of the
Text Box 5-7.  What Are Statistically Significant
Statistical testing is the "statistical procedure or
decision rule that leads to establishing the truth or
falsity of a hypothesis . .  ." (Alder and Roessler,
1972).  Statistical significance is based on the number
of data points, the nature of their distribution,
whether intertreatment variance exceeds
intratreatment variance in the data, and the a priori
significance level (a). The types of statistical tests
and the appropriate protocols (e.g., power of test) for
these tests should be established as part of the analysis
plan during problem formulation.

total river ecosystem. Stressors acting on these microhabitats may result in adverse effects to the
entire system.
       Spatial factors are important for many species because of the linkages between ecological
landscapes and population dynamics.  Linkages between landscapes can provide refuge for
affected populations, and organisms may require corridors between habitat patches for successful
       The temporal scale for ecosystems can vary from seconds (photosynthesis, prokaryotic
reproduction) to centuries (global climate change). Changes within a forest ecosystem can occur
gradually over decades or centuries and may be affected by slowly changing external factors such
as climate. When interpreting adversity, risk assessors should recognize that the time scale of
stressor-induced changes operates within the context of multiple natural time scales.  In addition,
temporal responses for ecosystems may involve intrinsic time lags, so responses to a stressor may
be delayed. Thus, it is important to distinguish a stressor's long-term impacts from its
immediately visible effects. For example, visible changes resulting from eutrophication of aquatic
systems (turbidity, excessive macrophyte growth, population decline) may not become evident for
many years after initial increases in nutrient levels.
       Considering the temporal scale of adverse effects leads logically to a consideration of
recovery.  Recovery is the rate and extent of return of a population or community to some aspect
of its condition prior to a stressor's introduction. (While this discussion deals with recovery as a
result of natural processes, risk mitigation options may include restoration activities to facilitate or
speed up the recovery process.) Because ecosystems are dynamic and, even under natural
conditions, constantly changing in response to changes in the physical environment (e.g., weather,
natural disturbances) or other factors,  it is unrealistic to expect that a system will remain static at
some level or return to exactly the same state that it was before it was disturbed (Landis et al.,
1993). Thus, the attributes of a "recovered" system  should be carefully defined. Examples might
include productivity declines in a eutrophic system, reestablishment of a species at a particular
density, species recolonization of a damaged habitat, or the restoration of health of diseased
organisms. The Agency considered the recovery rate of biological communities in streams and
rivers from disturbances in setting exceedance frequencies for chemical stressors in waste
effluents (U.S. EPA, 1991).
       Recovery can be evaluated in spite  of the difficulty in predicting events in ecological
systems (e.g., Niemi et al., 1990). For example, it is possible to distinguish changes that are
usually reversible (e.g., stream recovery from sewage effluent discharge), frequently irreversible
(e.g., establishment of introduced species), and always irreversible (e.g., extinction).  Risk
assessors should consider the potential irreversibility of significant structural or functional changes
in ecosystems or ecosystem components when evaluating adversity.  Physical alterations such as

deforestation in the coastal hills of Venezuela in recent history and in Britain during the Neolithic
period, for example,  changed soil structure and seed sources such that forests cannot easily grow
again (Fisher and Woodmansee, 1994).
       The relative rate of recovery can also be estimated. For instance, fish populations in a
stream are likely to recover much faster from exposure to a degradable chemical than from habitat
alterations resulting from stream channelization.  Risk assessors can use knowledge of factors,
such as the temporal  scales of organisms' life histories, the availability of adequate stock for
recruitment, and the interspecific and trophic dynamics of the populations, in evaluating the
relative rates of recovery.  A fisheries stock or forest might recover in  decades, a benthic
invertebrate community in years, and a planktonic community in weeks to months.
       Risk assessors should note natural disturbance patterns when evaluating the likelihood of
recovery from anthropogenic stressors. Alternatively, if an ecosystem has become adapted to a
disturbance pattern, it may be affected when the disturbance is removed (e.g., fire-maintained
grasslands).  The lack of natural analogs makes it difficult to predict recovery from uniquely
anthropogenic stressors (e.g., synthetic chemicals).
       Appendix E illustrates how the criteria for ecological adversity (nature and intensity of
effects, spatial and temporal scales, and recovery) might be used in evaluating two cleanup
options for a marine oil spill. This example also  shows that recovery of a system depends not only
on how quickly a stressor  is removed, but also on how the cleanup efforts themselves affect the

       When risk characterization is complete, risk assessors should be able to estimate
ecological risks,  indicate the overall degree of confidence in the risk estimates, cite lines of
evidence supporting the risk estimates, and interpret the adversity of ecological effects.  Usually
this information  is included in a risk assessment report (sometimes referred to as a risk
characterization  report because of the integrative nature of risk characterization). While the
breadth of ecological risk assessment precludes providing a detailed outline of reporting elements,
the risk assessor should consider the elements listed in text box 5-8 when preparing a risk
assessment report.

       Like the risk assessment itself, a risk
assessment report may be brief or extensive,
depending on the nature of and the resources
available for the assessment. While it is
important to address the elements described
in text box 5-8, risk assessors should judge
the level of detail required. The report need
not be overly complex or lengthy; it is most
important that the information required to
support a risk management decision be
presented clearly and concisely.
       To facilitate mutual understanding, it
is critical that the risk assessment results are
properly presented. Agency policy requires
that risk characterizations be prepared "in a
manner that is clear, transparent, reasonable,
and consistent with other risk
characterizations of similar scope prepared
across programs in the Agency" (U.S. EPA,
1995b).  Ways to achieve such characteristics
are described in text box 5-9.
       After the risk assessment report is
prepared, the results are discussed with risk
managers.  Section 6 provides information on
communication between risk assessors and
risk managers, describes the use of the risk
assessment in a risk management context, and briefly discusses communication of risk assessment
results from risk managers to interested parties and the general public.
Text Box 5-8.  Possible Risk Assessment Report
•  Describe risk assessor/risk manager planning

•  Review the conceptual model and the assessment

•  Discuss the major data sources and analytical
   procedures used.

•  Review the stressor-response and exposure

•  Describe risks to the assessment endpoints,
   including risk estimates and adversity evaluations.

•  Review and summarize major areas of uncertainty
   (as well as their direction) and the approaches
   used to address them.

   *  Discuss the degree of scientific consensus in
      key areas of uncertainty.

   *  Identify major data gaps and, where
      appropriate, indicate whether gathering
      additional data would add significantly to the
      overall confidence in the assessment results.

   *  Discuss science policy judgments or default
      assumptions used to bridge information gaps
      and the basis for these assumptions.

   *  Discuss how the elements of quantitative
      uncertainty analysis are embedded in the
      estimate of risk.

Text Box 5-9. Clear, Transparent, Reasonable, and Consistent Risk Characterizations

For clarity:

    Be brief; avoid jargon.

    Make language and organization understandable to risk managers and the informed lay person.

    Fully discuss and explain unusual issues specific to a particular risk assessment.

For transparency:

    Identify the scientific conclusions separately from policy judgments.

    Clearly articulate major differing viewpoints of scientific judgments.

    Define and explain the risk assessment purpose (e.g., regulatory purpose, policy analysis, priority setting).

    Fully explain assumptions and biases (scientific and policy).

For reasonableness:

    Integrate all  components into an overall conclusion of risk that is complete, informative, and useful in
    decision making.

    Acknowledge uncertainties and assumptions in a forthright manner.

    Describe key data as experimental, state-of-the-art, or generally accepted scientific knowledge.

    Identify reasonable alternatives and conclusions that can be derived from the data.

    Define the level of effort (e.g., quick screen, extensive characterization) along with the reason(s) for
    selecting this level of effort.

    Explain the status of peer review.

For consistency  with other risk characterizations:

    Describe how the risks posed by one set of stressors compare with the risks posed by a similar stressor(s) or
    similar environmental conditions.

    Indicate how the strengths and limitations of the assessment compare with past assessments.

                            RISK MANAGEMENT DECISIONS
       After characterizing risks and preparing a risk assessment report (section 5), risk assessors
discuss the results with risk managers (figure 5-1).  Risk managers use risk assessment results,
along with other factors (e.g., economic or legal concerns), in making risk management decisions
and as a basis for communicating risks to interested parties and the general public.
       Mutual understanding between risk assessors  and risk managers regarding risk assessment
results can be facilitated if the questions listed in text box 6-1 are addressed. Risk managers need
to know the major risks to assessment
endpoints and have an idea of whether the
conclusions are supported by a large body of
data or if there are significant data gaps.
Insufficient resources, lack of consensus,  or
other factors may  preclude preparation of a
detailed and well-documented risk
characterization. If this is the case, the risk
assessor should clearly articulate any issues,
obstacles, and correctable deficiencies for the
risk manager's consideration.
       In making  decisions regarding
ecological risks, risk managers consider other
information,  such  as social,  economic,
political, or legal issues in combination with
risk assessment results. For example, the risk
assessment results may be used as part of an
ecological cost-benefit analysis, which may
require translating resources (identified
through the assessment endpoints) into
monetary values.  Traditional economic
considerations may only partially address
changes in ecological resources that are not
considered commodities, intergenerational
resource values, or issues of long-term or
irreversible effects (U.S. EPA, 1995a;
Costanza et al., 1997); however, they may
Text Box 6-1.  Questions Regarding Risk
Assessment Results (Adapted From U.S. EPA,

Questions principally for risk assessors to ask risk

•   Are the risks sufficiently well defined (and data
   gaps small enough) to support a risk management

•   Was the right problem analyzed?

•   Was the problem adequately characterized?

Questions principally for risk managers to ask risk

•   What effects might occur?

•   How adverse are the effects?

•   How likely is it that effects will occur?

•   When and where do the effects occur?

•   How confident are you in the conclusions of the
   risk assessment?
   What are the critical data gaps, and will
   information be available in the near future to fill
   these gaps?

   Are more ecological risk assessment iterations

   How could monitoring help evaluate the results of
   the risk management decision?

provide a means of comparing the results of the risk assessment in commensurate units such as
costs. Risk managers may also consider alternative strategies for reducing risks, such as risk
mitigation options or substitutions based on relative risk comparisons. For example, risk
mitigation techniques, such as buffer strips or lower field application rates, can be used to reduce
the exposure (and risk) of a pesticide. Further, by comparing the risk of a new pesticide to other
pesticides currently in use during the registration process, lower overall risk may result. Finally,
risk managers consider and incorporate public  opinion and political demands into their decisions.
Collectively, these other factors may render very high risks acceptable or very low risks
       Risk characterization provides the basis for communicating ecological risks to interested
parties and the general public.  This task is usually the responsibility of risk managers, but it may
be shared with risk assessors.  Although the final risk assessment document (including its risk
characterization sections) can be made available to the public, the risk communication process is
best served by tailoring information to a particular audience. Irrespective of the specific format, it
is important to clearly describe the ecological resources at risk, their value, and the monetary and
other costs of protecting (and failing to
protect) the resources (U.S. EPA, 1995a).
       Managers should clearly describe the
sources and causes of risks and the potential
adversity of the risks (e.g., nature and
intensity, spatial and temporal  scale, and
recovery potential).  The degree of
confidence in the risk assessment, the
rationale for the risk management decision,
and the options for reducing risk are also
important (U.S. EPA, 1995a).  Other risk
communication considerations are provided
in text box 6-2.
       Along with discussions of risk and
communications with the public, it is
important for risk managers to consider whether additional follow-on activities are required.
Depending on the importance of the assessment, confidence in its results, and available resources,
it may be advisable to conduct another iteration of the risk assessment (starting with problem
formulation or analysis) in order to  support a final management decision. Another option is to
proceed with the decision, implement the selected management alternative, and develop a
monitoring plan to evaluate the results (see section 1).  If the decision is to mitigate risks through
Text Box 6-2. Risk Communication Considerations
for Risk Managers (U.S. EPA, 1995b)

•  Plan carefully and evaluate the success of your
   communication efforts.

•  Coordinate and collaborate with other credible

•  Accept and involve the public as a legitimate

•  Listen to the public's specific concerns.

•  Be honest, frank, and open.

•  Speak clearly and with compassion.

•  Meet the needs of the media.

exposure reduction, for example, monitoring could help determine whether the desired reduction
in exposure (and effects) is achieved.