EPA/620/R-94/016
                                            February 1994
Environmental  Monitoring and
      Assessment Program
     Assessment Framework
                      by
                  Kent W. Thornton
                 FTN Associates, Ltd.
                Little Rock, AR 72211

                   Gary E. Saul
                 FTN Associates, Ltd.
                  Austin, TX  78735

                   D. Eric Hyatt
                   EMAP Center
            Research Triangle Park, NC 27711
                Contract #68-DO-0093
               Work Assignment Manager
                  Laura E. Jackson
                 EMAP Center (MD-75)
            U.S. Environmental Protection Agency
             Research Triangle Park, NC 27711
                   EMAP Center
        Environmental Monitoring and Assessment Program
            Office Of Research And Development
            U.S. Environmental Protection Agency
             Research Triangle Park, NC 27711
                                        Printed on Recycled Paper

-------
Assessment Framework
                                                  Abstract
                 The assessment framework proposed in this report provides a common approach for planning and
         conducting a wide variety of ecological assessments within EMAP. The framework also demonstrates how
         EMAP  complements EPA's assessment approach  proposed in the Risk Assessment Forum's  (RAF)
         Framework for Ecological Risk Assessment (RAF 1992). EMAP assessments can contribute directly to the
         problem formulation phase of the EPA-RAF Framework by identifying and quantifying factors that might
         contribute to the condition of ecological resources. EMAP assessments also will provide information needed
         to conduct ecological risk assessments that verify model predictions and the cumulative effectiveness of
         environmental protection and management decisions.

                 EMAP uses a retrospective or effects-oriented approach to assessment There are three phases in
         EMAP assessments: problem formulation, analysis, and interpretation and communication.  These three
         phases emphasize (1) formulating and refining assessment questions and issues with EMAP users, (2)
         identifying indicators of condition, (3) developing conceptual models, (4) analyzing data on ecological
         resources using weight of evidence and process of elimination approaches to infer factors contributing to
         observed trends in ecological effects, and (5) interpreting and effectively communicating assessment results
         in a policy-relevant context for users. There are five basic assessment products: quality-assured data,  annual
         statistical summaries, ecological resource assessments, assessment tools, and guidance.

                 Because it will take a number of years before all resources in all regions of the country will be
         routinely monitored, the ability of the program to conduct ecological resource assessments will depend on
         implementing its research, monitoring, and assessment activities in planned phases. Initial assessments will
         focus on data to determine extent, geographic coverage, and condition for individual ecological resources.
         Single region, single resource assessments will be conducted before assessments encompass multiple regions
         or national levels. Assessments of multiple ecological resources in a single region will be conducted as other
         resources start monitoring in that region.
         Key words:
                 USEPA-EMAP, ecology-decision making, ecology-risk assessment,  ecology-measurement,
         environmental monitoring—risk assessment, environmental policy, environmental risk assessment, indicators
         (biology), landscape assessment, risk assessment, risk communication, risk management, statistics.

         Preferred citation:
         EMAP (Environmental Monitoring and Assessment Program).  1994. Environmental  Monitoring and
           Assessment Program Assessment Framework. EPA/620/R-94/XXX. Research Triangle Park, NC: U.S.
           Environmental Protection Agency, Office of Research and Development, EMAP Center.

         Notice:
                 The information in this document has been funded wholly or in part by the U.S. Environmental
         Protection Agency under Contract #68-DO-0093 to Versar Inc. (FTN Associates, Ltd.) and Contract #68-C8-
         0006 to ManTech  Environmental  Technology,  Inc.  It has been subject to the Agency's peer and
         administrative review. It has been approved for publication as an EPA document. Mention of trade names
         or commercial products does not constitute endorsement or recommendation for use.

-------
                                                    The Environmental Monitoring and Assessment Program
                                          Preface
        Over the past several years, there has been an increased emphasis on comparative ecological risk
assessment within the U.S. Environmental Protection Agency (EPA), other agencies, and the scientific
community.  Also, there have been questions raised on the role of the Environmental Monitoring and
Assessment Program (EMAP) in comparative ecological risk assessment and its relationship with EPA's
Risk Assessment Forum.  This  report describes a framework, and its basic  elements, for conducting
assessments within EMAP as well as the relation of EMAP assessments to EPA's Risk Assessment Forum,
This document about EMAP's assessment framework is intended primarily for scientific administrators and
managers who require assessment information for making decisions related to environmental protection and
management; further, it provides a scientific explanation - i.e., a "definition" — of ecological assessment
in the context of the EMAP program, and should not be interpreted as a strategic planning document, or
any other form of planning or policy document.

        Requests  for  additional information on EMAP should be directed to: EMAP Director, Office of
Modeling, Monitoring Systems and Quality Assurance, Mail Code 8205, U.S.  Environmental Protection
Agency, 401 M Street, S.W., Washington, DC 20460.
                                              in

-------
Assessment Framework
                                          Acknowledgements
                 This Environmental Monitoring and Assessment Program (EMAP) report was prepared under the
         direction of the EMAP Technical Coordinator for Assessment and Reporting. Preparation of this document
         was initiated by Daniel A. Vallero (Atmospheric Research and Exposure Assessment Laboratory [AREAL],
         Research Triangle Park, NC) and is currently under the direction of D. Eric Hyatt (EMAP Center, Research
         Triangle Park, NC).

                 The principal authors were Kent W. Thornton (FTN Associates, Ltd., Little Rock, AR), Gary E.
         Saul (FTN Associates, Ltd., Austin, TX), and D. Eric Hyatt (EMAP Center, Research Triangle Park, NC)
         with contributions from Daniel A. Vallero (AREAL, Research Triangle Park, NC) and Lisa Gandy (FTN
         Associates, Ltd., Little Rock, AR). The final draft Assessment Framework was peer reviewed by Richard
         C. Bishop (University of Wisconsin-Madison, Madison, WI), Virginia H. Dale (Oak Ridge National
         Laboratories, Oak Ridge, TN), and David J. Rapport (University of Ottawa, Ottawa, ONT, Canada); it was
         edited by  Cynthia B. Chapman (Senior Technical Editor,  ManTech Environmental Technology, Inc.,
         Corvallis, OR).  Additional comments were provided by Daniel H. McKenzie (Acting Deputy Director,
         EMAP, Environmental Research Laboratory, Corvallis, OR), Sidney Draggan (Associate Director, EMAP-
         HQ, Washington, DC), and Marjorie M. Holland (Chief, EMAP-Integration and Assessment Team, EMAP
         Center, Research Triangle Park, NC).

                 A previous version of EMAP's Assessment Framework was reviewed by Joan Baker (Watershed
         Response Program  Leader, EPA Environmental  Research Laboratory,  Corvallis, OR), Laura Jackson
         (Scientist, EMAP Center, Research  Triangle Park, NC),  Linda Kirkland (Acting Technical Coordinator,
         EMAP-QA, Washington,  DC), Richard Latimer (Acting Assistant Technical Director, EMAP-Estuaries,
         Narragansett, RI), Anthony R. Olsen (Technical Coordinator, EMAP-Design and Statistics, Corvallis, OR),
         and Jugrid Schultz (EPA Office of Policy, Planning and Evaluation, Washington, DC). The contributions
         of these individuals are gratefully acknowledged.
                                                      IV

-------
                                                   The Environmental Monitoring and Assessment Program
                                        Contents
Abstract	•  ii

Preface	;	• iii

Acknowledgements	  iv

1 — Introduction	   1
   Purpose 	   1
   Assessment Defined	   1
   Ecological Risk Assessment Framework	   2
   Effects-Oriented Risk Assessment	   4
   Assessment Summary	   5
   Document Organization	   5

2 — Environmental Monitoring and Assessment Program	   7
   Rationale  	   7
   Goal and Objectives	   7
   Program Structure 	   8
   Assessment Products  	   9
   Evolving Process	• •  10

3 — Assessment Framework	  11
   Assessment Questions  	  12
       EMAP Users and Their Assessment Perspectives	  12
       Policy-relevant Questions	  12
       Indicators	  15
       Conceptual Models 	  16
   Data Analysis 	  18
        Quality Assurance/Quality Control	  19
        Geographic  coverage and extent	  19
        Diagnosis	  20
               Weight of Evidence	20
               Process of Elimination	  23
   Interpretation and Communication	  24
        Interpretation	  24
               Classification of Condition	  24
               Assumptions and Uncertainties	  25
        Communication	  25
        Assessment Products	  27
               Quality-Assured Data	  27
               Annual Statistical Summary Reports  	  27
               Ecological Resource Assessment Reports	  28
               Assessment Tools and Guidelines  	  28

4 — Evolving Program and Process	  30

Glossary	  31

References  	  39

Index	  43

-------
Assessment Framework
                                                   Figures

         Figure 1.  Framework for ecological risk assessment  	.	    3
         Figure 2.  EMAP's effects-oriented strategy compared to a stressor-oriented approach
                   for ecological assessments	    4
         Figure 3.  Relations of EMAP assessment to RAF's risk assessment framework.
                   EMAP assessments contribute primarily to problem formulation,
                   with more limited contributions to other phases	    6
         Figure 4.  EMAP structure	    9
         Figure 5.  Framework for EMAP assessments	   11
         Figure 6.  Major elements in the assessment questions and issues phase of
                   EMAP's assessment framework	12
         Figure 7.  Conceptual model for estuarine resources	   16
         Figure 8.  EMAP-Arid Ecosystems conceptual model	   17
         Figure 9.  Major elements in the data analysis phase of EMAP's assessment framework	   18
         Figure 10. Spatial display of major forest types in the conterminous United States	   19
         Figure 11. Association of median wet sulfate deposition and median surface water
                   sulfate concentrations in National Surface Waters Survey subregions	   21
         Figure 12. Process of elimination of possible sources of acidity to acidic lakes and streams	   23
         Figure 13. Percent of population influenced by dominant source of acidity	24
         Figure 14. Major elements in the interpretation and communication phase of
                   EMAP's assessment framework	   25
         Figure 15. Ranges of subnominal, marginal, and nominal conditions that
                   might be delineated along a condition continuum	   26
         Figure 16. One method of aggregating measures of resource condition among
                   different subregions (standard Federal region; EPA, USDA, or
                   Forest Service region; or biogeographical province)	   28
                                                   Tables
         Table 1.
         Table 2.
         Table 3.

         Table 4.
         Table 5.
         Table 6.

         Table 7.

         Table 8.

         Table 9.
Assessment definitions  	   1
Comparison of stress-oriented and effects-oriented risk assessment approaches	   5
Examples of different perspectives and different underlying data needs
to answer an assessment question	  13
Examples of policy-relevant questions to be addressed in EMAP	  14
Examples of policy-relevant questions that are not appropriate for EMAP	  15
Hill's (1965) criteria adapted for evaluating the likelihood, of probable cause from
associations in ecological assessments		  22
Example of weight of evidence approach for relating acidic deposition to surface
water acidification	  22
Suggestions  for how EMAP could establish  preliminary nominal-subnominal
categories 	27
Actions suggested by lessons learned in communicating between decision makers
and scientists through the global climate program	  29
                                                      vi

-------
                                                                  The Environmental Monitoring and Assessment Program
                                                       Introduction
Purpose

   This  document presents  a framework for  conducting
assessments in the Environmental Monitoring and Assessment
Program (EMAP). The framework describes basic elements of
the assessment process and provides a common foundation for
conducting assessments within EMAP. Because of its general
nature, the framework should be adaptable to a diverse set of
assessment questions and needs. Consequently, this document
is  written  to assist  science administrators  and  resource
managers in understanding the EMAP assessment process.
                    Assessment Defined

                       Assessment connotes different definitions and processes,
                    depending on the discipline, agency, and audience (Table 1).
                    Many Federal and State environmental assessments are based
                    on legislative or regulatory requirements that dictate explicit
                    purposes and approaches.  In general, these assessments are
                    site specific and range from addressing specific problems (e.g.,
                    the   Comprehensive   Environmental    Responsibility,
                    Compensation,     and  Liability \ Act   [CERCLA]   Natural
                    Resources Damage  Assessments) to broadly identifying or
                    disclosing  all  potential environmental  impacts  (e.g., the
      Table 1.  Assessment definitions.
                   Source
                      Definition of Assessment
                     EMAP
                  NEPA (1969)




             Deuel and D'Aloia (1989)


                  Streets (1989)


                  NAPAP (1991)



             Webster (Ninth ed. 1991)

                 Cowling (1992)


                  Suter (1993)
Assessment is the interpretation and evaluation of EMAP results for the purpose
of answering policy-relevant questions about ecological resources including:
(1) determination of the fraction of the population that meets a socially-defined
value and/or  (2) associations  among indicators of ecological condition and
stressors.

Assessment is the evaluation of the consequences of an action including short-term,
long-term, direct,  indirect, cumulative, and  irreversible, irretrievable  effects for the
purposes of avoiding to the fullest extent practicable undesirable consequences for the
environment.

Assessment  is a comprehensive  multifaceted  investigation  that  includes  data
acquisition, evaluation, conclusions, and recommendations.

Assessment is the translation of scientific  results into answers for policy-relevant
questions and issues within a decision framework.

Assessment is an  interdisciplinary activity wherein findings from diverse disciplines are
coordinated to produce a better understanding of the cumulative impacts of a stressor
(i.e., acidic deposition).

Assessment is the act of determining the importance, size and value of something.

Assessment is a process by which scientific and technological evidence is marshalled
for the purposes of predicting the outcomes of alternative courses of action.

Assessment is the  combination  of analysis with policy-related activities such as
identification of issues and comparison of risks and benefits.

-------
 Assessment Framework
National Environmental  Policy Act [NEPA] Environmental
Assessments/Environmental Impact Statements). Just as users
must understand the specific framework and elements of an
environmental assessment, assessors must understand from the
outset what the user needs from the assessment.

   Regardless of the definition, requirements, or approaches of
assessment, several features are common to almost every
environmental assessment First, there is a link with policy or
regulatory questions and issues. Second, mere is a value-added
perspective to assessments, ranging from a formal, quantitative
cost/benefit analysis  of  all  alternatives  to a  qualitative
improvement in our understanding of potential impacts or
effects. Finally, assessments synthesize and interpret scientific
information and present it in an understandable format for the
intended audience.  Over the past decade, environmental
assessments have evolved from analyzing and comparing solely
ecological effects from stressors to a wider consideration of the
risks to human and ecological health associated  with  these
stressors. A stressor is any physical, chemical, or biological
entity or process that can induce adverse effects on individuals,
populations, communities, or ecosystems (RAF  1992, xiv).

   Risk assessment is defined as the process of assigning
magnitudes and probabilities to the adverse effects of human
activities or natural catastrophes (Suter 1993). Guidelines for
conducting risk assessments on human health have been issued
by the U.S. Environmental Protection Agency (EPA 1976,
1986) and are being revised continually.  Ecological risk
assessment, however,  is  just emerging  as a process for
comparing  and evaluating the effects of multiple stressors on
ecological resources.

   EPA has embarked on a process to focus its efforts on the
environmental problems that pose the greatest risks rather than
those that receive the greatest public attention (Roberts 1990).
This process involves conducting comparative ecological risk
assessments so that the highest priority risks can be identified
and addressed.  The concept of comparative risk was initially
proposed in Unfinished Business: A  Comparative Assessment
of Environmental Problems (EPA 1987), which indicated the
greatest risks to the environment were not posed  by site-
specific problems  such as toxic  waste  dump sites, but by
regional and global scale  problems (e.g., nonpoint source
pollution,  habitat alteration, loss of biodiversity, or global
climate change). EPA's Science Advisory Board endorsed and
expanded the call for comparative ecological risk assessment,
recommending that EPA:  1) plan, implement, and sustain  a
long-term monitoring and research program; 2) report on the
status  and trends   in  environmental  quality; 3)  target its
environmental protection efforts on the basis of opportunities
for the greatest risk  reduction;  4)  improve  the data  and
analytical methods that support the  assessment, comparison,
and reduction of different environmental risks; and 5)  increase
its  efforts  to integrate  environmental  considerations into
broader aspects of public policy as fundamentally as economic
considerations are included in policy analysis  (SAB  1988,
1990). EPA has established a Risk Assessment Forum (EPA-
RAF) that is charting a strategic direction and developing
specific guidance for conducting ecological risk assessments.
The Framework for Ecological Risk Assessment (RAF  1992)
presents a basic structure and starting principles for conducting
EPA's ecological risk assessments.  The Framework initiates
a process in which long-term guidelines  for ecological risk
assessment can be organized (RAF 1992).
Ecological Risk Assessment Framework

   EPA defines ecological risk assessment as "the process that
evaluates the likelihood that adverse ecological effects may
occur or are occurring as a result of exposure to one or more
stressors" (RAF 1992, 37).  A risk is not considered to exist
unless (1) the identified stressor(s) has (have) the inherent
ability to cause adverse ecological effect(s) and (2) the stressor
co-occurs  with or contacts an  ecological  component  for a
sufficient time and at sufficient intensity to elicit the identified
effect(s).  Ecological  risk assessment may evaluate one or
several stressors or ecological components.

   In its Framework, EPA's Risk Assessment Forum describes
a flexible structure for its ecological risk assessment with three
sequential  phases,  namely,  1)  problem  formulation,  2)
analysis, and 3) risk characterization (Figure 1).

        Problem formulation is a planning and scoping
   phase that  links the regulatory or management goals to
   the risk assessment. It results in a conceptual model that
   identifies the environmental values to be protected (the
   assessment endpoints), the data needed, and the analyses
   to be used.

        The Analysis phase develops and links profiles of
   environmental exposure and profiles of ecological effects
   to stressors.   "The exposure profile characterizes the
   ecosystems in which the stressor may occur as well as
   the biota that may  be exposed.   It also describes the
   magnitude and spatial/temporal pattern of exposure. The
   ecological effects profile summarizes data on the effects
   of  the  stressor and  relates  them  to the assessment
   endpoints"  (RAF 1992, xiv).

        "Risk characterization integrates  the exposure and
   effects profiles" (RAF 1992, xiv). By comparing individual
   exposure and effects values, comparing the distributions of
   exposure and effects, or using simulation models, risks can
   be expressed either as qualitative or quantitative estimates.
   Results of  risk characterization describe  relations between
   the risks and social values or assessment endpoints; discuss
   ecological  significance of the effects; estimate the overall
   confidence or uncertainty in the assessment; and suggest

-------
                                                               The Environmental Monitoring and Assessment Program
                                      Ecological Risk Assessment
                                                Discussion Between the
                                                Risk Assessor and Risk
                                                     Manager
                                                     (Results)
                                                  Risk Management
                Figure 1.  Framework for ecological risk assessment (RAF 1992, 4).
effective approaches for communicating these risks to the user
and the risk manager.

   EPA-RAF's Framework "also recognizes several additional
activities that are integral  to, but separate from, the risk
assessment process" (RAF 1992, xv). First, early discussions
between the risk assessor and the risk manager ensure that the
assessment will provide information relevant to the decision
making process, that  the assessment addresses  all relevant
ecological  concerns, and that the manager  has a full and
complete understanding  of  the  conclusions,  assumptions,
limitations, and uncertainties associated with the assessment.
Next, data acquisition, verification, and monitoring studies
provide the information required for analysis, for validation of
the results of a specific assessment and the overall Framework
approach, and for improving the assessment process.

   The general risk assessment paradigm (NRC  1983), the
ecological risk assessment Framework (RAF 1992), and most
of the procedures and tools developed for risk assessment are
applicable for both retrospective  and predictive assessments,
but have been  used  primarily  for predictive assessments
(Suter 1993).   Predictive  assessments  usually are stress-
oriented, focusing on a particular  stressor and then estimating
future risks to the assessment endpoints (formal expressions of
EMAP's condition indicators) from this stressor.

-------
 Assessment Framework
   Other assessment approaches, such as epidemiological or
effects-oriented assessments, begin with an observed effect and
subsequently identify stressors that might have contributed to
this effect; EMAP's assessment strategy follows this effects-
oriented approach. EMAP's strategy complements the EPA-
RAF Framework by contributing to problem formulation and
providing corroborative information to the analysis and risk
characterization phases.
Effects-Oriented Risk Assessment

   Ths Framework for Ecological Risk Assessment (RAF 1992)
discusses a broad approach for conducting ecological risk
assessments, and it starts with a characterization of the stressor,
then describes exposure pathways from the sources of the
stressor to the associated ecological effects (Figure 2). While
this approach is equally applicable for both predictive and
retrospective analyses,  it typically emphasizes prospective
analyses using  simulation models to predict exposure and
stressor-effects  profiles.   Such predictive  approaches  are
dependent upon cause-effect relationships between stressors
and ecological effects.
   A complementary approach  to conducting  ecological
assessments is  the  strategy  being  developed in  EMAP, a
retrospective  approach  like  that  used  in  environmental
epidemiology (NRC1991) and the emerging area of ecosystem
health (Costanza et al.  1992, Rapport  1992).  Effects are
observed rather than the stressors (Figure 2). Effects-oriented
approaches  emphasize association, weight of evidence, and
process of elimination analyses to  identify possible factors
contributing to  the  observed ecological  effects.  Although
epidemiologic methods can include predictive analyses, its
initial emphasis—as well as EMAP's strategy—are based on
retrospective analyses. Both these approaches—retrospective
and predictive—were used in assessing the effects of acidic
deposition  on aquatic ecosystems  in  the  National  Acid
Precipitation Program (NAPAP 1991), illustrating how these
two approaches complement each other (Thornton 1993). Both
approaches represent scientifically valid approaches for
assessing ecological effects. In general, the predictive stress-
oriented approach is used—and better understood—than the
retrospective  effects-oriented   approach  in  conducting
environmental assessments  (Suter  1993).  Effects-oriented
strategies, however, will become increasingly important as
            Bfects-
            Orfented
            Approach
            (EMAP)
           Strew-
           Oriented
           Approach
                       r
                       \.
       Figure 2.  EMAP's effects-oriented strategy compared to a stressor-oriented approach for
       ecological assessments.

-------
                                                               The Environmental Monitoring and Assessment Program
assessments of larger scale problems are conducted because it
will become increasingly more difficult to establish specific
cause-effect relationships between a stressor and an effect.
Effects-oriented  approaches can help  eliminate  possible
stressors and pathways, and assist in identifying probable
sources of stress and pathways for predictive ecological risk
assessments.   Comparing the  characteristics  of these two
assessment approaches,  a user can better understand how
information from each  approach contributes  to ecological
assessment (Table 2).
      This document, EMAP's Assessment Framework, provides
   a  broad  outline of how EMAP  contributes  to  ecological
   assessment and how it  builds on the interrelationships  of
   assessment, monitoring, and research studies being conducted
   in EMAP. The results of EMAP's assessment framework will
   complement  studies  being  conducted  in  EPA's   Risk
   Assessment Forum and elsewhere.
           Table 2.   Comparison of stress-oriented and effects-oriented risk assessment
                       approaches.
                    Predictive, Stress-Oriented
      Retrospective, Effects-Oriented
                Critical Questions
                Stressor/Problem Oriented
                Individual Sites/Systems
                Link Stressor to Possible Responses
                Exposure Characterization
                Stressor-Effect Characterization
                ProspecfrVe/Retrospective
                Simulation Models/Causal Relationships
                Cause-Effect
Critical Questions
Effects Oriented
Target Populations
Link Condition to Possible Stressors
Effect/Exposure Associations
Effect/Stressor Associations
/fefrospecf/Vs/Prospective
Weight of Evidence/Process of Elimination
Association
Assessment Summary

  Ecological risk  assessment,  clearly,  is  in its  infancy.
Currently, we do not have effective methods and programs, at
regional and national scales, to monitor ecological conditions,
measure  and detect ecological trends, perform comparative
ecological risk assessments, and effectively communicate the
results to decision makers. EMAP is designed to contribute to
the  research  and  assessment  activities  of EPA's  Risk
Assessment  Forum  and  provide  essential  monitoring
information for comparative  ecological risk assessments
(Figure 3).
  For example, EMAP assessments will contribute directly to
the Problem Formulation phase of ecological risk assessment
through activities focused on  question formulation,  resource
characterization, and  conceptual  model development.   In
addition, EMAP can  contribute to the  Analysis and Risk
Characterization phases by  providing  information  which
characterizes resource condition; analyses  which  examine
associations among indicators of condition and stressors; data
sets for model development, data verification or confirmation,
and estimates of uncertainty.  Because data acquisition and
monitoring of the Nation's ecological resources is an integral
part of EMAP, the Program serves a separate but extremely
important role  for  EPA-RAF's  ecological  risk assessment
program  by providing quality assured data for performing
large-scale risk assessments.
   Document Organization

     EMAP's Assessment Framework, describes the structure and
   strategy EMAP will use in ecological assessments.

     The information in section 2—Environmental Monitoring
   and Assessment Program explains the rationale for EMAP,
   its  goal and objectives, program  structure, and assessment
   products.  Because EMAP's assessment framework is part of
   the process for achieving EMAP's goal and objectives, it is
   important for users of program information to understand what
   the program aims to accomplish and why.

     Section 3 — the Assessment Framework explains the three
   phases  for conducting  assessments in  EMAP:   problem
   formulation,   data   analysis,   and   interpretation  and
   communication. These phases emphasize (1) formulating and
   refining assessment questions and issues with EMAP users, (2)
   identifying indicators of condition, (3) developing conceptual
   models, (4) analyzing ecological resources data using effects-
   oriented strategies to answer the questions, and (5) interpreting
   and effectively communicating assessment results in a policy-
   relevant context for clients and other users.

     The  concluding  section 4  —  Evolving Program and
   Process discusses  the  implementation of EMAP and the
   evolving assessment process.
                                                             A list of references and glossary of terms complete EMAP's
                                                           Assessment Framework.

-------
Assessment Framework
              Environmental
              Monitoring
              Assessment
ECOLOGICAL  RISK
   ASSESSMENT

      Problem
    Formulation
                              	*-     Analysis
                                             Risk
                                       Characterization
                                              1
                                             Risk
                                         Management
           Figure 3.  Relations of EMAP assessment to RAF's risk assessment
           framework. EMAP assessments contribute primarily to problem
           formulation, with more limited contributions to other phases.

-------
                                                               The Environmental Monitoring and Assessment Program
                   2 — Environmental Monitoring and Assessment Program
  This section presents the rationale for EMAP, the Program's
goal and objectives, a brief description of EMAP's  structure,
and approaches for keeping EMAP assessments relevant
Additional information on EMAP can be obtained by writing
to the address listed in the Preface.
Rationale

  EMAP evolved from discussions about the basic elements
needed in a Federal monitoring and assessment program to
contribute to decision making on environmental protection and
management Seven elements summarized these discussions:
1)  A focus on social values and policy-relevant questions;
2)  Approaches that assess and translate scientific results into
    information useful for decision makers and the public;
3)  Ecological  indicators of  condition for  monitoring  key
    ecological resources rather than individual pollutants or
    stressors;
4)  Periodic estimates, with known confidence, of the status
    and trends in indicators of ecological condition;
5)  An integrated approach to monitoring that includes all
    ecological resources;
6)  National implementation with regional scales of resolution,
    rather than an individual site or local area orientation; and
7)  An interagency,  interdisciplinary program  in which all
    participating  agencies are cooperative  partners in the
    research, monitoring, and  assessment efforts.
Decisions on environmental protection and management require
that the important societal values associated with our ecological
resources and the related policy questions be identified and
clearly stated.

  The possible  uses of the data to support decisions must be
considered throughout the assessment process, not only after
the data are collected  but also initially to  determine what
information  is  needed.  Then, through the  design  of  a
scientifically  rigorous  monitoring  network,  appropriate
indicators are selected and monitored to provide the types of
information  required to address these questions. Measuring
these indicators within a network of probability samples, rather
than from sites selected using subjective  criteria, permits
estimates of the status  and trends  in ecological indicators of
condition on  a regional  and  national basis  with known
confidence. Existing  State and Federal monitoring networks
typically focus on a specific resource or medium, which often
results in "question-specific" designs. Aggregating data from
these designs  to address regional, multi-resource issues is
difficult if not impossible; therefore, a critical need exists for
a  complementary, integrated  program  mat monitors  all
ecological  resources.  Emerging  regional  and  national
environmental problems require monitoring networks designed
to provide information at these scales. Finally, cooperative and
collaborative   Interagency,  interdisciplinary  programs  are
required to address these complex issues.

  The technology and methods required to  design a cost-
effective,  nationwide monitoring program  of the scope of
EMAP are available, but they have never been fully tested.
Existing programs provide valuable information, but many
were designed  for  other  purposes  such as  compliance
monitoring, single-resource management, or problem-specific
monitoring. Although many monitoring programs  measure
specific elements of  environmental quality, reviews have
repeatedly found these programs to be inadequate (GAO 1988,
NRC 1990). By designing and implementing EMAP, EPA has
set  in  motion  an  ecological  research,  monitoring,  and
assessment program with a regional and national scope. These
basic programmatic elements are reflected in EMAP's goal and
objectives.
Goal and Objectives

  EMAP's goal is to monitor and assess the condition of the
Nation's ecological resources to contribute to decisions on
environmental protection and management To accomplish this
goal, EMAP works to attain four objectives:

1)  Estimate the current status, trends,  and changes  in
    selected  indicators  of the condition  of the Nation's
    ecological resources on a regional basis  with known
    confidence.
    EMAP will use selected indicators to monitor and assess
    the  condition  of  the Nation's  ecological  resources.
    Indicators are characteristics of the environment  both
    biotic  (biological) and abiotic  (non-living), that can
    provide  quantitative  information  on  the condition  of
    ecological  resources.  EMAP  emphasizes  biological
    indicators in contrast to  the  traditional approach  of

-------
 Assessment Framework
     monitoring chemical and physical indicators. Currently,
     the Nation's ecological resources are grouped by EMAP
     into  the  following  categories:    agroecosystems,  arid
     ecosystems, estuaries,  forests, the Great Lakes, surface
     waters (both lakes and streams), and wetlands. EMAP will
     monitor and assess these resources at the scale of the
     landscape, so landscape indicators also are important
     Status of each resource will be described through the
     distribution  of  scores  for  condition  indicators  for  a
     specified  time with relation to  the reference  condition
     associated with specific social values or desired  uses.
     Trends will be described by the changes in the distribution
     of scores  for indicators of condition over multiple time
     periods. Change is the difference in the distribution of
     measurements of condition indicators between two time
     periods. Because the design  has an underlying statistical
     basis,  the   proportion   of  resources   in   a  given
     condition—for instance, the proportion of lakes that are
     eutrophic—can be estimated with known confidence.

2)   Estimate  the geographic coverage and extent of the
     Nation's ecological resources with known confidence.
     Determining the national geographic coverage of multiple
     ecological resources has been a high priority among
     agencies and within scientific communities for several
     years. In  conjunction with other agencies, EMAP will
     provide information on geographic coverage for the
     Nation's ecological resources as spatial displays at specific
     scales of resolution, for example, at the scale of satellite
     Thematic  Mapper images with 30 m resolution.  EMAP
     will estimate the  extent or amount  of a resource, for
     example, the acres of forest, miles of streams, or numbers
     of lakes. Each of these estimates will be presented with
     known confidence. Also, EMAP  will monitor and assess
     changes and trends in geographic  coverage and extent

3)   Seek associations among selected indicators of natural
     and anthropogenic stresses and indicators of condition of
     ecological resources.
     EMAP  will  seek associations or relationships among
     selected indicators of natural and anthropogenic (human-
     induced) stresses and  ecological  condition to identify
     factors that might be contributing to the condition which
     the ecological indicators express. Any stressors proposed
     for EMAP are selected to aid in interpreting the indicators
     of ecological condition. When  monitoring a  stressor,
    EMAP requires that an explicit relationship exist between
     the selected  indicator of stress  and the  indicator  of
     condition or that there be a testable hypothesis regarding
     this relationship.

4)  Provide annual  statistical   summaries  and  periodic
    assessments of the Nation's ecological resources.
    EMAP information will be made readily available to all
    individuals, organizations, and agencies mat are interested
    in the condition  of our ecological  resources. Annual
    statistical summaries will be prepared for each ecological
     resource and  distributed in a timely fashion.   EMAP
     results will be interpreted or translated into answers for
     specific questions from users and decision makers.  These
     assessments will be conducted on both a periodic and ad
     hoc basis, as described in the remainder of this document

   These objectives support EMAP's goal and seek to provide
 scientific information useful to decision makers.
 Program Structure

  To  achieve its  programmatic objectives,  EMAP  has
 developed an  organizational  structure  in which  research,
 monitoring, and  assessment  activities are coordinated  and
 integrated throughout the Program (Figure 4).

  EMAP will monitor  all  of the  aforementioned  major
 categories of ecological resources and determine the patterns
 of these ecological resources on the landscape. An interagency
 resource group is responsible for each ecological resource as
 well as one for landscapes. For these eight resource categories
 (including landscapes), EMAP will monitor selected indicators
 of ecological condition and will collect and compile data on
 selected stressor indicators (including atmospheric deposition).
 The program will integrate its monitoring of indicators within
 and across resources, such as forests,  surface waters,  and
 wetlands, so that changes in indicators of ecological condition
 and landscape patterns can be detected over time. This large-
 scale integration represents  one of the  greatest  technical
 challenges to the program.

  The scope  and complexity of EMAP require extensive
 coordination  if the Program  is  to be successful and fully
 integrate data into  its  assessments.   Consequently, EMAP
 places a high priority on coordination at both the technical and
 administrative levels.   Cross-cutting  groups (Figure 4) are
 responsible for ensuring  there is consistent compatible  and
 comparable approaches among resource groups for design and
 statistical  analyses,  indicator  development  information
 management,   assessment   and   reporting,   landscape
 characterization, development of methods, logistics, and quality
 assurance. Also, EMAP  coordinates its research, monitoring,
 and assessment activities among a variety of other agencies
 and programs.

  The concept of assessing monitoring information in a policy-
relevant  context represents a central theme underlying all
EMAP activities.  Assessment in EMAP is the process of
interpreting and evaluating EMAP results for the purpose of
answering policy-relevant questions about ecological resources
including determination  of  the  fraction of the  resource
population that meets socially-defined values and associations
among indicators of ecological condition and selected stressors.
In turn, assessment results can describe where refinements in
monitoring are required.  One of the primary lessons learned
from the 10-year National Acid Precipitation Assessment

-------
                                                               The Environmental Monitoring and Assessment Program
                       Environmental Monitoring and Assessment Program (EMAP)
            Resource Monitoring
                 Landscapes
              Agroecosystems
              Arid Ecosystems
                 Estuaries
                  Forests
                Great Lakes
               Surface Waters
                 Wetlands
Integration and Assessment
  Figure 4.  EMAP structure.
  Assessment and Reporting
     Design and Statistics
    Indicator Development
   Information Management
   Landscape Characterization
    Logistics and Methods
      Quality Assurance
Program Coordination
    Atmospheric and
    Deposition Data
                                                                                         International Activities
    Regional EMAP
       (R-EMAP)
  EPA Risk Assessment
   Forum (EPA-RAF)
Program (NAPAP) was that monitoring and research require a
continuous emphasis  on  assessment (ORB 1991).   This
emphasis on assessment is reflected in the development of this
EMAP assessment framework.
Assessment Products

  EMAP activities will produce verified, aggregated resource
data; annual statistical summaries; assessments; and assessment
tools.  EMAP will produce two  basic types of ecological
assessment  reports:  pre-planned,  periodic  Status  of the
Ecological Resource reports and  special topic assessments.
The principal differences between these two assessments is that
assessments for a Status of the Ecological Resource report
have been planned as part of the programmatic design, based
primarily  on information being  collected in  EMAP,  and
scheduled for periodic production.  This assessment focus has
ensured that monitoring information will be available a priori
to address identified  assessment  questions.   Special topic
assessments will likely address a specific environmental issue
or set of specific policy-relevant questions that are not planned
into the program design and that make greater use of auxiliary
data.  Either of these assessments can be done at any of four
          levels—single resource, single region; single resource, multiple
          regions;  multiple  resources, single  region;  and  multiple
          resources,  multiple regions. Assessment reports  will  be
          produced as collaborative efforts with partner agencies,  by
          other agencies using EMAP data, by EMAP resource groups
          and by EMAP's Assessment and Reporting cross-cutting group.
          These  reports will evaluate the status  and  trends in the
          ecological condition of resources and will suggest possible
          factors contributing to this condition.

            EMAP can contribute to formulating hypotheses about the
          causes of ecological resource condition for investigation  by
          other programs conducting cause-effect  studies (Figure  3).
          Also, EMAP's assessment activities will contribute information
          to support the efforts of other offices and agencies: assisting
          development  of biocriteria  by  EPA's   Office  of Water,
          developing ecological indicators to assist resource management
          agencies,  evaluating  environmental  and  spatial  statistical
          methods  in the scientific community, and providing other
          assessment tools, particularly regional assessment tools, as the
          process evolves. Because the diversity of potential assessments
          that may be  conducted  within EMAP  is large, EMAP's
          Assessment Framework  emphasizes  the entire assessment
          process.

-------
 Assessment Framework
Evolving Process

  The EMAP assessment process is evolving; it will provide
policy-relevant information on important environmental issues,
will stimulate new ideas and raise new questions about the
status of and trends in conditions of the Nation's ecological
resources, and will identify by association factors that might be
contributing  to  these conditions.   The  lessons learned
conducting assessments, not only in EMAP but also in other
programs, will provide new ideas and new approaches.  Based
on a review of other national programs, e.g., National Acid
Precipitation  Assessment Program (Cowling 1992, NAPAP
1991, ORB 1991, NRC 1986) and the Global Climate Change
Program (SPA 1992), several elements were identified that can
improve the EMAP assessment process and its probability of
success:

  • Open process. An open and inclusive process must be
    followed in conducting large-scale assessments  for
    subsequent decisions to be accepted by primary users,
    interested parties  and the public. The  concerns  and
    issues of all parties must be heard and addressed, even
    if their  suggestions and  recommendations  are  not
    included  in the assessment

  •  Review  and oversight   Both scientific and policy
    reviews are critical to ensure each assessment is based
    not only on sound, scientifically defensible approaches
    and information but also addresses important policy-
    relevant questions and issues.

  • Social, economic and policy perspectives and values.
    Decisions typically  are  made on socioeconomic  or
    policy values, rather than scientific values or ecological
    indicators.  Ecological indicators must be linked or
    related to the values or variables on which decisions
    will be made.

  • Interagency cooperation and collaboration. To address
    large-scale  environmental  issues in  an effective  and
    economical manner,  collaborative  interagency studies
    should be conducted.  Each agency can contribute critical
    information to addressing questions and issues related to
    different social perspectives.

  • Interdisciplinary teams. Interdisciplinary teams  for
    assessments  must include  contributions from  social
    scientists,   economists,   policy   analysts,  and
   administrators in  addition  to  natural  scientists and
   engineers.   Scientists, professional communicators,
   policy analysts, and decision makers each  play a key
   role in  the assessment process, particularly for large-
   scale environmental problems and issues (Cowling
    1992).
   • Multiple methods. Weight of evidence methods and
    explicit uncertainty analyses are an integral part of the
    assessment process.  Multiple methods, models, and
    technologies must be used in order to corroborate findings.
    Research should reduce those uncertainties that contribute
    to answering policy-relevant questions.

   • Conflict resolution procedures. Different perspectives
    among natural, social and behavioral scientists, agencies,
    and societal groups will result in conflicts over the
    interpretation, certainty and translation of scientific and
    technological information for decision makers.  Conflict
    resolution procedures  need to be established before
    conflicts arise.   These procedures  might  include
    minority reports, arbitration, or similar strategies.

  Each of these elements is being evaluated to determine how
to appropriately include them in the EMAP assessment process.
Many  of these  elements  were incorporated in  the  initial
Program design, for example, interagency collaboration, and
interdisciplinary  teams.  These evaluations are part of the
iterative and evolving assessment process.
                                                        10

-------
                                                              The Environmental Monitoring and Assessment Program
                                      3 — Assessment Framework
  There are three phases and two direct
outputs  from the EMAP assessment
process   (Figure  5).     Assessment
questions,   data   analysis,   and
interpretation  and  communication
constitute the  phases  of  EMAP's
assessment framework.   The outputs
include  policy  relevant  information
and research and development tools
and results.

  In the assessment questions phase,
the users'  assessment  questions and
issues  are identified  and validated,
appropriate indicators of condition (i.e.,
assessment endpoints) are selected, links
among  indicators  of condition and
potential stressors are conceptualized,
and data needs are determined. In data
analysis, the relevant data needed for
the   assessment  are   acquired,
summarized,  integrated,  and analyzed
using association analyses, weight  of
evidence, and process  of elimination
methods.  During interpretation and
communication, results from  the data
analysis  phase  are interpreted in the
context   of   the  users'  values and
perspectives,   then   policy   relevant
information  is   communicated  in   a
format that is understandable  and that
addresses the assessment questions.  In
addition,  lessons   learned  from
conducting  the assessment  can  be
incorporated  in the Program  to help
EMAP maintain its policy relevance and
responsiveness.  These lessons or new
tools might  include revised analytical
approaches   or   procedures,   new
indicators, new linkages in conceptual
models, or possible modifications to the
monitoring  design   (Research  and
Development in Figure S).
                  EMAP Assessment Framework
                          ASSESSMENT QUESTIONS
                                   AND
                                  ISSUES
                               DATA ANALYSIS
                              INTERPRETATION
                                   AND
                              COMMUNICATION
                                                             Ecological   |
                                                               Risk    |
                                                          _  Assessment  I
                                                          L	I
Figure 5. Framework for EMAP assessments.
                                                        11

-------
 Assessment Framework
   One of EMAP's  unique attributes  is  that  the
 assessment process  is  inexorably  coupled with
 monitoring information. This assessment focus ensures
 that monitoring information will be available a priori
 to address identified assessment questions.
 Assessment Questions

   The most important phase of the assessment process
 is identifying the client's  questions and determining
 what  information  is  needed  to  make informed
 decisions  about   environmental   protection  and
 management (Figure 6).

 EMAP Users and Their Assessment Perspectives

   Addressing environmental issues involves more than
 finding purely  technical solutions (Bardwell  1991).
 Environmental  questions and issues reflect political,
 societal, regulatory, and  management  values and
 expectations as much as, if not more than, scientific
 and technical information (Ehrlich 1980, Sampson and
 Hair  1990, Schnaiberg 1980, Waide et al.  1992).
 Contributing to decision-making processes requires that
 assessment information addresses questions based on
 these underlying expectations and perspectives.

  EMAP users  bring a variety of perspectives to
 ecological assessments ranging from:

 • Social perspectives, which incorporate the broadest
  spectrum of environmental goals and values desired
  by contemporary society and expressed through the
  legislative process.

 •  Administrative perspectives,  which include
  manage-ment  and regulatory agencies  and their
  legislative mandates  to  protect and manage both
  specific  ecological   resources  and   the  total
  environment
                      VALUES
                        •Social
                        * Administrative
                        • Scientific
                     POLICY-RELEVANT
                       QUESTIONS
Figure 6.  Major elements in the assessment questions
and issues phase  of EMAP's assessment framework.
 *  Scientific  perspectives,  which include basic scientific
  principles and knowledge of  ecological  structure  and
  function  as  well as causal understanding  of ecological
  responses to human disturbances and  natural variability
  (Figure 6).

  Identifying social values is a critical first step in the EMAP
assessment process because it provides the link to the user.
Each of EMAP's resource groups has identified an initial set
of values for their respective resources. This is an on-going
and iterative process which reflects the changing values of
society, changes in the administrative procedures to ensure
these values are being attained, and advances in scientific
understanding of how ecosystems function and how selected
condition indicators relate to environmental values.
        Policy-relevant Questions

          The next step in the assessment process identifies questions
        based on the underlying perspective and background of the
        user. Social and administrative policy perspectives must be
        incorporated in the assessment questions and process (Cowling
        1992).

          Posner (1973)  states  that the initial  representation of a
        question is  the single  most  crucial  factor governing the
        likelihood of a satisfactory answer to the question; whether the
        question is answered, and how long it takes to provide the
        answer, depends a great deal on the initial statement of the
        question.
                                                        12

-------
                                                                The Environmental Monitoring and Assessment Program
  The implications  and frustrations  of inadequate  initial
question formulation can be significant  Bardwell (1991) cites
a study that indicates about 90% of real world problem solving
is spent

1)  solving the wrong problem,
2)  stating the question so it can not be answered,
3)  solving a solution,
4)  stating questions too genetically, or
5)  trying to  get agreement on the answer before there is
    agreement on the question.

An example of how these different perspectives  might be
expressed in formulating assessment questions is illustrated in
Table 3.
                                    A question of direct interest to the fishing sector of society
                                  might be, "Is it safe to eat the fish we catch?" Because EMAP
                                  addresses questions related to resource populations at multiple
                                  sites,  this initial question  must  be  rephrased  as,  "What
                                  proportion of the lakes in the region have catchable fish that
                                  are not safe to eat."  The rephrased question also reflects how
                                  it is  generally easier  to define the range of unacceptable
                                  conditions than acceptable conditions.

                                    From a regulatory perspective, however, there is not a
                                  straightforward answer to this general social question; the
                                  answer depends on the specific contaminant, the particular fish
                                  species, the criteria established (if  any)  for  that particular
                                  species, and other factors.  This question might have to be
                                  rephrased: "What proportion of the lakes in the region have
  Table 3.  Examples of different perspectives and different underlying data needs to answer an
  assessment question.
       Perspective
                                Assessment Question
  Social
1.    Is it safe to eat the fish I catch?
1 a.  EMAP Question: "What proportion of lakes have catchable fish that are safe to eat in the
     Region? [Subnominal criteria typically are easier to establish than nominal criteria, so the
     question becomes".. .are riot safe to eat".]
  Scientific
  Administrative             2.    What is the risk to humans from consuming various quantities of fish flesh with different
                                contaminant concentration?
                           3.    What is the type and condition of the consumer (i.e., pregnant woman, adult male, child
                                under 12 yr)?
                           4.    What is the risk to human health from fish with different gross external pathologies?
                           5.    What different risk factors exist for different fish species?
                           6.    What are the cumulative effects of consuming contaminated fish?
                           7.    What proportion of lakes have catchable fish with edible tissue contaminant concentrations
                                that exceed  FDA Action Levels?
                           8.    What proportion of lakes have catchable fish with edible tissue with detectable contaminant
                                concentrations?
                           9.    What proportion of lakes have gamefish populations with contaminated flesh?
                           10.   What additional contaminant/pathological  criteria or standards exist in addition to FDA
                                Action Levels?
                           11.   What are possible sources of contamination?
12.  What are the associations between tissue contamination concentrations and sediment
     concentrations? aqueous concentrations?
13.  What are the associations between concentrations in food sources (i.e., prey species) and
     tissue concentrations, indicating possible pathways for exposure?
14.  What are the appropriate criteria for assessing acute,  chronic, and cumulative human
     physiological responses?
15.  What proportion of lakes have the catchable fish with  gross external pathologies or
     contaminated fish flesh?
16.  What are the characteristics of lakes that contain a high proportion of catchable fish with
     gross external pathobgies or contaminated fish flesh?
                                                         13

-------
 Assessment Framework
legally  harvestable largemouth bass  with methyl mercury
concentrations  exceeding  1  mg/kg in  the edible  tissue?"
Scientifically,  additional  information  is  required  on  the
bioaccumulation and biomagnification of this contaminant, by
fish species, over time, the depuration rate of the chemical in
the fish species and humans, the acute and chronic toxicity of
the chemical, the status of the human species (e.g., pregnant
female versus adult male), the quantity of fish flesh consumed,
and similar information before an informative answer can be
provided to the relatively simple, initial question, "Is it safe to
eat the fish we catch?"

  Formulating  the policy-relevant assessment questions is
particularly important in EMAP because the national scope and
regional scale of resolution represents a different perspective
from that underlying most research studies as well as local and
State monitoring programs.  Examples of  broad assessment
questions that EMAP might address are presented in Table 4.
   It is equally important to identify the types of questions and
 issues that EMAP will not address. EMAP is neither designed
 to provide site-specific, compliance-oriented monitoring nor to
 provide information on specific, local-scale issues. It is not
 intended  to  provide  substantial  information  about  any
 individual sampling site, such as a specific lake, wetland, forest
 stand, or agroecosystem.  Questions and issues at the local
 scale can be addressed more effectively by existing or site-
 specific monitoring networks.   EMAP is  not designed to
 determine if any particular ecological effect is caused by  a
 specific pollutant; EMAP will not be able to describe the
 dynamics of any particular ecological process, such as nutrient
 cycling.  EMAP will not, and is not  intended  to, replace
 existing  compliance-oriented   or   resource-management
 monitoring programs, but it will supplement and add value to
 the  information  being  obtained from  these programs by
 assessing  large-scale  patterns  and  trends  in   ecological
 condition. Some of the questions EMAP will not address are
 listed in Table 5.
  Table 4.  Examples of policy-relevant questions to be addressed in EMAP.
                        Question
         Reason Appropriate for EMAP
  What proportion of estuarine area in large estuaries, tidal
  rivers, and small estuaries has fish with gross pathologies?

  What proportion of the Nation's lakes are eutrophic,
  mesotrophic, and olkjotrophic?

  What proportion of wetlands have less than the expected
  number and composition of native plant species?

  How is the area and geographic coverage of forest cover
  types In the  U.S. changing?

  What proportion of forests have vegetative structure and
  functions to sustain forest biodiversity?

  What proportion of the surficial sediments in the Great Lakes'
  harbors and embayments are toxic to aquatic organisms?

  What proportion of the southeastern U.S. has fragmented or
  simplified landscapes?

  What proportion of ark) ecosystems are experiencing
  desertification?  What anthropogenic stressors are associated
 with desertification  of arid ecosystems?
 EMAP focuses on biological indicators with societal value.
EMAP produces regional and national population estimates
for lake trophic state estimates, not for individual lakes.

EMAP targets ecosystem properties like community
structure.

EMAP estimates the extent and geographic coverage of the
Nation's ecological resources.

EMAP focuses on national environmental issues such as
biodiversity and sustainability.

EMAP's condition  indicators include both biotic and selected
abiotic measures.

EMAP also addresses interactions among ecological
resources on the landscape.

EMAP assesses the cumulative effects of multiple stressors
and identifies possible factors that might contribute to
condition.
                                                        14

-------
                                                                The Environmental Monitoring and Assessment Program
  Table 5. Examples of policy-relevant questions that are not appropriate for EMAP.
                        Question
        Reason Not Appropriate for EMAP
  What proportion of lakes in New Jersey are hypereutrophic?


  What proportion of degraded wetlands are caused
  specifically by inappropriate agricultural management
  practices?
EMAP is not a State level program, but the design is flexible
and can be enhanced for State level resolution.

EMAP is not a cause-effect program. Associations might
provide strong inference but do not establish causality..
  What is the trophic state of Lake Tahoe?

  What proportion of improved grassland condition can be
  associated with the implementation of the Conservation
  Reserve Agricultural Program?
EMAP reports on populations of resources, not on individual
systems or entities.

EMAP addresses the cumulative influence of national and
regional policies, not the effectiveness of individual
regulations or policies.
  What proportion of degraded estuaries in the Virginian
  Province are associated with storm event loadings?
EMAP uses an index sampling concept to describe ecological
condition. It is not designed for short-frequency, episodic
events but rather for detecting longer term trends in
ecological condition.
  EMAP's  strategy  for formulating  assessment questions
reflects different user perspectives: the first step in this strategy
is to identify a tentative list of assessment questions that are
considered to have policy-relevance based on the selected
values  for  a  resource.    Then,  EMAP  scientists  and
administrators  meet with  decision  makers  and  resource
managers in  Federal  agencies,  Congressional  staff,  and
representatives of environmental organizations to solicit their
review and  comment   The revised questions are used to
identify additional indicators for testing and evaluation.

  Formulating  these questions is an  ongoing and iterative
process between  EMAP scientists  and  users of  EMAP's
information, continually evolving as additional issues and new
users are identified.

Indicators

  Li the literature, indicators are defined as characteristics of
the environment that provide quantitative information  on the
condition of ecological resources, the magnitude of stress, or
the exposure of a biological component to stress (Hunsaker and
Carpenter 1990, Olsen  1992).  Condition of an ecological
resource is  determined by the interaction of all physical,
chemical, and biological components in the system.  Because
it is impossible to measure all components, EMAP's strategy
is to emphasize indicators that  represent the condition  of
ecological resources relative to social values.  EMAP selects,
develops, and evaluates indicators mat (1) describe the overall
condition of ecological resources, (2) permit the detection of
changes and trends in condition, and (3) provide preliminary
diagnosis  of factors  (e.g.,  human-induced  versus natural
stressors) that might contribute to observed conditions.
   EMAP defines two general types of ecological indicators:
 condition  and  stressor.    A  condition indicator  is  any
 characteristic of the  environment that provides quantitative
 information  on  the  state of  ecological resources  and  is
 conceptually tied to  a value.  Condition indicators may be
 classified as biotic or abiotic measures of condition, and  in
 EMAP  they  are conceptually  equivalent to  measurement
 endpoints (Suter 1989, 1990, 1993; Kelly and Harwell 1990;
 Hunsaker  and Carpenter  1990;  Olsen  1992). EMAP  will
 estimate the regional distribution of scores or  measures for
 each of these indicators within and among ecological resource
 categories.

   The program emphasizes the development and evaluation  of
 biological condition indicators because they incorporate and
 express the  cumulative effects of  the complex interactions
 among  physical, chemical and  biological components  of
 ecosystems.

   Stressor indicators are characteristics of the environment that
 are suspected to elicit  a change  in  the condition  of an
 ecological resource,   and they include  both  natural  and
 anthropogenic stressors.  The stressor indicators proposed for
 EMAP are selected to aid in interpreting the  indicators  of
 condition. EMAP requires an explicit relationship between the
 selected stressor indicator and the indicator of condition—or a
 testable hypothesis regarding this relationship—to monitor a
 stressor indicator.    EMAP  will  seek associations  among
 indicators  of ecological condition and stressor indicators  to
 help identify the factors  that might be  contributing to the
 observed condition. These associations can provide insight and
 direction for other  regulatory,  management,  or research
 programs in establishing causal relationships.
                                                         15

-------
 Assessment Framework
Conceptual Models

  Conceptual models help the assessment process because
they:
1)   identify links among values, indicators of condition or
     stress, and potentially important ecological pathways and
     processes;
2)   guide association analyses among indicators; and
3)   indicate factors that might be contributing to the status
     and trends in ecological condition (Olsen 1992).

  One model for the estuarine resources (Figure 7) illustrates
how values, indicators and stressors  can be schematically
linked; it also indicates potential pathways for transport, uptake
and exposure  of chemical  stressors,  as  well as potential
pathways for physical and biological stressors to affect desired
uses or values of estuarine resources.
  EMAP's Arid Ecosystems resource group has developed a
conceptual model to illustrate relationships among external
forces, resource class indicators, measurements, and values
(Figure 8).   To move through this model, start with the
external components (layer 1) which drive the arid ecosystem.
The broad indicator classes (layer 2) respond to these drivers
and interact with each other via major indicator components or
process (layer 3). Layer 4 contains measurement parameters
that reflect indicator components; these are used to assess
resource status.  The current resource status then can be
evaluated from a trend perspective (layer S) by coupling it to
retrospective  indicators  and  long-term,  historical  data.
Ultimately, resource status and trend data  are integrated to
address societal values and  issues  (layer 6). These two
examples demonstrate the usefulness of different types  of
conceptual models.
STRESSORS INDICATORS VALUES

1 CHrnnlA 1 	 >

Habitat
• Salinity
• Temperature
1 ~ 	 J • Depth
: /
i /
1 / /
i / /
i ' /
1 !/ s
Anlhropogmlc f*
* f.pok>K.n IXniJty
• Un* UK 	 "
• MaiMojmMtt PracBcM
• FwiB Fu.1 U»
V VX
\ 'N,
\
X
(Harvesting
___ . 	 • Sediment
**" "^ '
Pools S" / 	 X /
\ r^
Pathogens I. \S 1 Hi h TrepMe lewis 	 m
Tissue Levels 1 Subenjonhm Orgonltm Population 0.1,^,. |nt~,rf(y
• Wafer — . n,h ' • Btoot
• Sediment -i "n 	 j- * Sh.i


• Water - . . nu. -, // !
• Sediment , Oth«r 1 // 1
1 /j/
Nutrients 1' /0
• Water fi / / '
• Sediment ' j^^^ r7 ~T 1
wmlool • PoHnlogy • Sl» 	
• 6m* • Dhnrtlr - . ,.„„,„„„
le • B.prorfurtlon • DIomoM ^ ^
• Olh»r • Olhir • Abundanei
f V L J
/\
/ \

SolW» |
-------
                                                                  The Environmental Monitoring and Assessment Program
                                            Chemistry
                                                                   Socio-Economie
                                                     Soil Chemistry, Uaf 4 Utter
                                                    Chemistry, Nutrient Transport /£\
                                                     C, N, P Son Interactions.
                                                      Methane Production,
                                                        Loaf & Utter NSf.
                                                         So3 Salinity
                                                        uesenmcauan   / &*
                                                       Water Resources / P
                      Index of layer categories

                      1 - External Driver Components
                      2 - Broad Resource Indicator Classes
                      3 - Major Indicator Components and Processes
                      4 - Assessment Endpoints
                      5 - Resource Trend Analyses
                      6 - Ecological Risk Assessment
                    Figure 8.  EMAP-Arid Ecosystems conceptual model.
  Conceptual models are particularly  helpful in describing
relationships at large  spatial scales.   Different  processes
operate at different temporal and spatial scales (O'Neill et al.
1986), so the indicators and postulated associations are likely
to be different at larger scales than at the local scale.

  For example, Sala et al. (1988) evaluated the association
among soil  indicators  and  net primary productivity  for
grasslands throughout the Midwest and Great Plains. At any
individual site, net primary productivity was a function of
several variables,  including soil texture, moisture holding
capacity, soil nitrogen concentrations, precipitation, and solar
insolation.  The relative contribution of individual variables
influencing net primary productivity also varied among sites.
However, at the regional scale of the Midwest or the Great
Plains, annual precipitation alone accounted for 90% of the
variability (i.e.,  r2  =  0.90) in  net  primary  productivity.
Conceptual models permit these links to be hypothesized and
tested as EMAP  data  become available.    In addition,
conceptual  models  document links with other ecological
resources and assist in  identifying  common or comparable
indicators that might assist in interpreting regional patterns in
                                                           17

-------
 Assessment Framework
ecological condition among  resources. The data analyses
conducted within  EMAP assessments should evaluate the
strength of the relationships and hypothesized links identified
in the conceptual models for the ecological resource.

Data Analysis

  The data analysis phase in EMAP's assessment framework
can be partitioned into  three distinct, interactive segments.
These segments directly relate to  three of EMAP objectives,
namely, estimating geographic coverage and extent, estimating
condition, and  diagnosing  those  factors that might  be
influencing the condition of ecological resources and resource
classes (Figure 9).

  These analyses depend on and support EMAP's resource
monitoring and research activities. Moreover, EMAP's data
analyses  contribute  directly  to  the problem  formulation
activities  of EPA's Risk Assessment Forum.  For example,
EMAP's  analysis of a resource's geographic coverage and
extent will assist RAF in characterizing ecological resources.
                          GEOGRAPHIC
                           COVERAGE
                              AND
                            EXTENT
                                 INTERPRETATION AND COMMUNICATION
              Figure 9.  Major elements in the data analysis phase of EMAP's
              assessment framework.
                                                     18

-------
                                                               The Environmental Monitoring and Assessment Program
Similarly, EMAP's analyses of condition indicators  can be
used by RAF  to determine ecological  effects.   Finally,
association analyses can contribute to exposure and ecological
effects characterization in EPA-RAF.

Quality Assurance/Quality Control

  All data used in EMAP assessments will undergo rigorous
quality assurance and  quality  control (QA/QC)  analyses,
approvals, and documentation during resource monitoring and
research activities. Protocols for QA/QC have been established
for the Program and  implemented within certain  EMAP
components.  Quality assured data are primary assessment
products derived from routine monitoring that will be available
directly to EMAP users and clients.

Geographic coverage and extent

  As EMAP periodically  determines  national geographic
coverage and estimates  extent of ecological resources, it will
provide a basis  for examining  current condition, future
changes,  and trends  in ecological resources.   EMAP  will
provide information, such as spatial displays of forest cover
types (Figure 10) at specific scales (e.g., satellite Thematic
Mapper images  at 30 m resolution), and estimate with known
confidence the extent of such ecological resources as hectares
                                           of large estuaries, kilometers of streams, or number of lakes.
                                           As evidenced by U.S. Census Bureau data, the knowledge of
                                           geographic coverage, the extent of resources, and the change
                                           in coverage and  extent carries  significant policy-relevant
                                           information for decision makers and resource managers.  In
                                           addition,  these data contribute to the diagnosis of factors
                                           influencing resource condition.

                                           Condition

                                             The probability-based sampling design employed throughout
                                           EMAP provides a basis for estimating the current condition of
                                           ecological resources.  Descriptive  statistics  (e.g., means,
                                           medians,  standard deviations, quantiles,  and  cumulative
                                           distribution  functions)  will  be  used  to characterize  the
                                           distribution and central tendencies of indicators of condition.
                                           Status will be portrayed through cumulative distributions and
                                           visual displays of spatial patterns. Measurements of indicators
                                           of condition over time provides data to  examine changes in
                                           status and detect trends in resource condition. One essential
                                           feature  of  this approach  is the ability to  estimate  the
                                           cumulative proportion of a resource class with a condition
                                           indicator score less than or equal to some specified  threshold
                                           score that can be related to achieving some societal value (e.g.,
                                           nominal-marginal-subnominal   condition—see   section  on
                                           Interpretation and Communication).
    Figure 10.
    1965).
Spatial display of forest vegetation in the conterminous United States (Powells
                                                        19

-------
 Assessment Framework
   The  complexity  of ecological resources requires that
 indicators be considered in concert rather than individually.
 Although EMAP has selected a few individual indicators, the
 program has based its indicator selection on combinations, or
 suites, of indicators of ecological condition.  One approach to
 using suites of indicators is the development of indices. For
 example, the Index of Biotic Integrity (Karr 1987,1991; Karr
 et al. 1986) incorporates an array of biological measurements
 from the study of entire fish communities (e.g., total number
 of species, number of individuals, and proportion of  top
 carnivores)  to  produce  an  index  of condition  of fish
 communities at a sampling site.

   Properly developed indices of ecosystem condition should be
 compared more easily across regions than the measurements
 from which they are derived (Hughes 1989).  The underlying
 model or aggregation process, however, can be controversial
 and mathematically complex,  and  the  results  tend to be
 extremely dependent on the model,  the indicators and  the
 aggregation procedures used (Westman 1985).   Also,  the
 formation  of  indices  reduces multivariate measures  of
 ecosystem  condition to  a  more limited  set  of metrics.
 Although reduction in the number of indicators typically loses
 specific information, indices are one option for integrating or
 summarizing measures of ecosystem condition.

 Diagnosis

   EMAP uses  retrospective  approaches  to  systematically
 examine data in its effects-oriented analyses to identify factors
 which might be contributing to the current status or trends in
 ecological resource condition. This strategy is analogous to
 diagnosis in the branch of medicine called environmental
 epidemiology (NRC1991) and the emerging area of ecosystem
 health  (Costanza  et  al.  1992,  Rapport  1992).   EMAP's
 conceptual models propose  links among condition indicators
 and stressors, and initial association analyses will focus on
 these links.  Association  analyses, in the  broadest context,
 describe a range of exploratory and multivariate statistical
 procedures that will be applied to EMAP's data, ancillary data,
 and auxiliary data.

   Ancillary information includes data collected from studies
 within EMAP but not used directly in the computation of an
 indicator. Ancillary data can help characterize parameters and
 assist in the interpretation of data sets; time, stage of tide, and
 watershed characteristics  are examples of  ancillary  data.
 Auxiliary data are derived from a  source other than EMAP,
 that is,  from field studies  or other monitoring or sampling
programs. The  sampling  methods  and  quality  assurance
protocols of auxiliary data must be evaluated before the data
are used, and it  is always important to establish the population
represented by  auxiliary data. Auxiliary data might  include
climatic data, flow or discharge data, and census information
on population demographics.
   Association analyses will range from simple bivariate plots
 (Figure  11)  among indicators  to  more  computationally
 extensive multivariate techniques.  Land cover  and land use
 information as  well as landscape  indicators  (e.g.,  fractal
 indices, connectivity) will be used in conjunction with specific
 indicators of condition and stressors to help identify factors
 contributing to the condition of ecological resources at regional
 and national scales.

   These association analyses will be evaluated both over space
 and over time. Spatial associations among atmospheric sulfate
 deposition and lake  sulfate concentrations at the subregional
 scale (Figure 11),  for example,  were  used to infer  that
 atmospheric deposition of sulfate was influencing the acidity
 of lakes in various regions of-the United States (Baker et al.
 1990).  Association analyses conducted at the  Great  Plains
 regional scale found a strong spatial relationship between total
 annual precipitation and net primary productivity even though
 the association was not that strong at any individual site (Sala
 et al. 1988).

   The temporal associations among indicators  also will be
 investigated because the associations among  stressor  and
 condition indicators might be displaced in time. For example,
 there is a strong relationship between land use changes and the
 loss of productivity in arid  ecosystems through time (i.e.,
 desertification) (Reining 1978, UNEP 1992).

   Association analyses, however, are not causal.  EMAP is not
 designed to establish cause  and effect relationships.  The
 design, however, does not diminish the power for explaining
 and interpreting factors  that might  be contributing to the
 condition of ecological resources.   Association analyses are
 part of the weight-of-evidence strategy  that will be used to
 conduct EMAP  assessments  and contribute  hypotheses for
 testing in other programs.

 Weight of Evidence

   A weight of evidence approach uses multiple statistical and
 other analytical techniques, ancillary and auxiliary information,
 scientific literature, and both  probability and non-probability
based  field  observations to  reach  assessment  conclusions.
While  each  of these  approaches  might  provide   only
circumstantial evidence supporting a conclusion, multiple lines
of evidence strengthen the conclusion (NR(f 1991).

  Establishing causal relations  for  large-scale  phenomena
requires a weight of evidence approach because it is difficult,
if not  impossible, to conduct  cause-effect experiments  at
regional  or subregional scales  of  resolution.    However,
Mosteller and  Tukey (1977) state  that if twd  of  three
criteria—consistency, responsiveness, and  mechanism—are
satisfied, then causation can be implied.  Consistency implies
that the  relationship between variables is consistent across
                                                         20

-------
                                                              The Environmental Monitoring and Assessment Program
         300-
                                     NLS Subregions
                                             Wet SO42" deposition (kg/ha/yr)
 Figure 11. Association of median wet sulfate deposition and median surface water sulfate
 concentrations in National Surface Waters Survey subregions (after Baker et al. 1990).
populations, in both  direction  and amount (NRC  1986).
Responsiveness involves experimentally manipulating a system
by changing one variable and observing the expected change
in the response variable.  Mechanism implies a step-by-step
path from  the cause  to the  effect with links established
between each step (NRC 1986).  Based on the Mosteller and
Tukey (1977) criteria, the NRC (1986) inferred that in eastern
North  America,  a  causal  relationship  existed between
anthropogenic  sources of SO2 and the presence of sulfate
aerosols, reduced visibility and wet  deposition of  sulfate.
Similarly,  Hill  (1965)  suggested,  and  Rothman  (1986)
corroborated, a set of nine criteria for inferring causality (Table
6).

  A weight of evidence approach was used to demonstrate the
effects of acidic deposition on aquatic  ecosystems in the
United State (Table  7).   In  the early  1980's there was
anecdotal information  that acidic deposition contributed to
surface  water  acidification, but groups  were polarized at
opposite ends of a continuum, considering the acidic deposition
problem  either a trivial issue or  a major environmental
catastrophe. As aquatic research and surveys were conducted
during  the  1980s, additional information accumulated mat
provided additional weight to support the hypothesis that acidic
deposition resulted in surface water acidification and negatively
impacted aquatic ecosystems. In 1984, a NRC Panel agreed on
the processes affecting surface water acidification (NRC 1984).
In 1986, the NRC stated national  surface water surveys
estimated the population of acidic lakes and lakes sensitive to
acidic deposition in selected eastern subregions (Linthurst et al.
1986).  In  1987, subregions in the eastern United States in
which acidic lakes were located were determined to be in
steady state with sulfate deposition, which was consistent with
the watershed processes assumed  to  control surface water
acidification (Rochelle and Church 1987). In 1988, a linear
relationship between wet sulfate deposition and surface water
sulfate concentrations was demonstrated for lakes in the eastern
United States on a subregional basis  (Sullivan et al. 1988)
(Figure 11). The relationship was consistent with predictions
and measurements of prevailing wind directions and subregions
with the highest  sulfate deposition from the Ohio Valley to the
Atlantic Ocean  (NADP 1988).   In 1989, paleolimnological
studies in acidic lakes  within these same  eastern  subregions
determined a temporal change in diatom species that reflected
                                                       21

-------
Assessment Framework
 Table 6.  Hill's (1965) criteria adapted for evaluating the likelihood of probable cause from
 associations in ecological assessments  (after RAF 1992, Suter 1993).        	
      Hill's Criteria*
                                       Associations
         Strength

       Consistency

        Specificity

       Temporality
    Biological Gradient
        Plausibility

        Coherence

   Experimental Evidence

  	Analogy	
A high magnitude of effect is associated with exposure to the stressor, e.g., a large proportion of the
population responding in the exposed area relative to the reference area.
The association of observed effect and stressor is repeatedly observed under different stressor
circumstances.
The more specific the effect, the more likely it is to be diagnostic of the stressor. Also, the more specific
the stressor, the easier it is to associate with an effect.
The stressor always precedes the effect in time.
The effect should increase or decrease with corresponding changes in the stressor.
The association is consistent with our current understanding of physical, chemical and biological
principles and the characteristics  of the stressor and effect.
The hypothesis or postulated relationship between stressor and effect is consistent with available
evidence.
Changes in effects following changes in the stressor, observed through experimental manipulation or
through recovery of the population following abatement of the stressor.
Similar stressors are associated with similar effects.     .	___^
     'Not aH these criteria need to be satisfied, but each additional criteria that is satisfied adds to the strength of the inference that there
  is probable cause.  Negative  evidence  does not rule out a causal relationship  but likely indicates  incomplete knowledge of the
  association between the stressor and the effect (Rothman 1986).
  Table 7.  Example of weight of evidence  approach for relating acidic deposition to surface water
  acidification.
  Year
                               Evidence
  1984       Watershed processes controlling surface water acidification elucidated.
  1986       Anthropogenic sources of SO2 are causally related to wet deposition of sulfate.
  1986       Surface water surveys identified acidic and sensitive lakes in selected subregions of the eastern United States.
  1988       Wet sulfate deposition is linearly related to surface water sulfate concentrations in eastern lakes.
  1988       Sulfate deposition gradients correspond with prevailing wind directions based on predicted and observed measures.
  1988       No western lakes are acidic, corresponding with low sulfate deposition.
  1989       PateoKmnological evidence correlates changes in diatom assemblages and pH with changes in surface water acidification.
  1989       Historical changes in surface water acidification correspond with historical changes in emissions and coal combustion.
  1989       Watershed models predict similar distributions of acidic and sensitive lakes based on historical deposition patterns.
  1990       Laboratory, arjd field experiments indicate .surface water acidification (pH changes, associated aluminum and metal changes) i
             associated with toss of 'sensitive fish species arid changes in community structure.
                                                               22

-------
                                                                 The Environmental Monitoring and Assessment Program
 declining surface water pH in lakes and corresponded with
 temporal increases in regional emissions over the same time
 period (Charles et al. 1989).  Watershed models predicted
 similar distributions  of  acidic and sensitive lakes  in the
 surveyed regions based on historical deposition patterns and
 indicated that if acidic deposition continued, additional lakes
 and streams in selected subregions might become acidic in the
 future (Church et al. 1989). Laboratory and field experiments
 indicated  that  the  changes   in   aquatic   community
 structure—loss  of fish species sensitive  to pH,  changes in
 aluminum and  other  metal concentrations—were consistent
 with observations made on lakes receiving acidic  depositions
 (Schindler 1988, Baker et al. 1990).  Similar changes were not
 occurring in the western lakes where sulfate deposition was
 significantly  lower;  no acidic lakes were found during a
 western lakes survey (Landers et al. 1987).

     By 1990, there  was sufficient evidence  to support the
 statement that acidic deposition had contributed to surface
 water acidification in  some eastern  aquatic systems, sensitive
 fish species had been lost, and this condition would continue
 unless emissions decreased (NAPAP1991). While each of the
 individual pieces of information was not sufficient to support
 these conclusions, the weight of evidence clearly supported the
 conclusions and satisfied the Mosteller-Tukey criteria and
 many of Hill's criteria.   Weight-of-evidence methods are
 complemented by process-of-elimination procedures.

 Process of Elimination

     This method reduces the number of factors that might be
 contributing to an observed change in a condition indicator by
 associating stressors common to many indicators.  Process of
 elimination provides  insight and possible direction for other
 programs, offices, or agencies in determining  the underlying
 causal factors.  Establishing criteria for distinguishing among
 possible stressors is an initial step.

     Continuing  the acidic deposition example, a process of
 elimination approach was used to identify  the dominant
 source(s) of acidity contributing to the target population of
 acidic lakes and streams in the eastern United States. Acidic
 deposition was one source of acidity, but there also were other
 acidic sources that might have contributed to acidic ecosystems
 such as acid mine drainage, watershed  sulfur sources,  or
 organic acids. Criteria were established to distinguish each of
 the possible sources  and determine what proportion of the
 target population of acidic lakes and streams was likely acidic
 because of this source (Baker et al. 1990).  For example,
 streams that had sulfate concentrations > 2000 ueq/1 or specific
 conductivity  values  >  500  uS/cm were  assumed  to be
' significantly influenced by acid mine drainage (Figure 12).  If
 sulfate concentrations exceeded two times the expected value
 based on atmospheric deposition and evapoconcentration, then
 watershed sources of sulfate  were assumed to significantly
 influence surface water acidity. If organic  acid concentrations
         National
         Surface
          Water
          Survey
Figure 12. Process of elimination of possible
sources of acidity to acidic lakes and streams.

exceeded the sum of sea-salt corrected sulfate plus nitrate, then
organic acidity was assumed to influence surface water acidity.
Chloride acidity also was  considered (Baker et al.  1990).
Initially, there were 1,132 lakes and 4,768 stream reaches that
were estimated to be acidic (Baker et al.  1990). Through the
process of elimination, about 3% of the acidic lakes were
assumed to be influenced by watershed sources, 22%  of the
acidic lakes were influenced by organic acids, with about 75%
of the acidic lakes being  influenced by  acidic deposition
(Figure 13).  For the acidic stream reaches, the process of
elimination combined with weight of evidence, indicated about
26% and 27% of the acidic stream reaches were assumed to be
influenced  by  acid  mine  drainage  and  organic  acids,
respectively, with about 47% of the stream reaches influenced
by acidic  deposition (Figure 13, Baker et al.  1990).   The
process of elimination, combined with the weight of evidence,
indicated that acidic deposition was a likely factor  in the
number of acidic lakes and streams in the eastern United States.
                                                         23

-------
 Assessment Framework
    The assumptions and uncertainties  inherent  in  the
association analyses also are documented in the Data Analysis
phase.  This information contributes to the interpretation of
assessment results. Furthermore, users of the information need
to understand how these assumptions and uncertainties can
influence or support conclusions across resources or regions.
The next phase in EMAP's assessment framework addresses
how these assessment results are communicated to the users of
the information.
Interpretation and Communication

    EMAP's success will be measured by the contributions of
Program  information  to   decisions   on  environmental
management and protection at regional and national scales.
For assessment results and conclusions to be used, they must
be  communicated and  understood by the client or  user,
including full disclosure of the assumptions and uncertainties
and  their   implications   for  the  assessment   conclusions
(Figure 14). Assessments must provide information that relates
to the  status, trends, changes, and possible factors affecting
those resource attributes valued by users of the information.
The first step in this process is to interpret the analysis results.

Interpretation

    Scientifically defensible interpretation involves experience
and judgement  Judgement is used to  answer assessment
questions in relation to social values, desired resource uses,
and other criteria.    EMAP is developing an approach to
delineate nominal, marginal, and subnominal categories for the
condition of ecological resources. These categories and criteria
provide a basis for incorporating different societal perspectives
and values in the assessment process.

Classification  of Condition

    Nominal, marginal, and subnominal conditions are defined
and illustrated in Figure 15 as:

S Nominal condition means the social  value (e.g.,  desired
    use) is being achieved when  compared  with specific
    criteria.

S Subnominal condition means the social value is NOT being
    achieved when compared with specific criteria.

S Marginal  condition  exists  when  the nominal  and
    subnominal criteria are NOT coincident.

    Typically, identification of the  extremes or ends of the
condition continuum is relatively straightforward, and general
consensus can  be achieved among  individual  scientists or
agencies. There is general agreement, for example, that rivers
with no fish and no dissolved oxygen, matted with sewage-
                                   Acidic
                                   Streams
                        3%
         Deposition Dominated

         Organic Dominated
Watershed Sulfate
Dominated
Acid Mine Drainage
    Note: Acidic lakes are 4% of the NSWS lake population, and acidic
    streams are 8% of the NSWS stream population.
Figure 13. Percent of population influenced
by dominant source of acidity.
related fungus, and that catch fire regularly are subnominal
systems; conversely,  relatively pristine  areas,  minimally
impacted by humans, with  rich species diversity, that are
aesthetically pleasing, with abundant wildlife are acknowledged
as nominal systems. It is in the intermediate portions of this
continuum, not the extremes, that the assessment of resource
condition becomes more difficult

    A number of approaches, strategies, and methods for
developing nominal, marginal, subnominal ranges to use in
EMAP's assessments have been initiated (Table 8), but clearly
this is an on-going and evolving process.  These approaches
must integrate the natural variability in ecological systems as
well as  the variability in societal values into these nominal-
subnominal ranges.

    In addition to providing ways for  incorporating social
values in the interpretation of assessment results, establishing
a nominal-subnominal continuum also provides a strategy to
normalize information across resource groups or across regions
within a resource group. For example, there might be different
nominal/subnominal categories and classification criteria for
different subregions within a region (Figure 16). The nominal-
subnominal strategy permits the proportion of the resource in
each of the three subregions that is classified as subnominal
(marginal  or nominal) to be aggregated into an estimate of
subnominal  condition  for the  larger  administrative region.
Similarly, estimates of subnominal resource condition in these
regions  can be aggregated for national estimates.  EMAP
scientists also  are testing the nominal/subnominal strategy to
see if it provides a method for aggregating data across resource
groups.  For example, the area  (hectares) of wetlands, surface
waters, forests, arid ecosystems, or other ecological resources
in subnominal condition in a subregion,  region,  or at the
national scale can be estimated based on this strategy.  While
procedures for combining or associating these estimates are
still  being developed, the  general approach  does permit
                                                         24

-------
                                                              The Environmental Monitoring and Assessment Program
comparisons  based on  social values to  be included in
interpretations of assessment results together with the attendant
assumptions and uncertainties.

Assumptions and Uncertainties

    A complete discussion of assumptions and uncertainties is
beyond the scope of EMAP's Assessment Framework, these
topics are developed in Finkel (1990), Rolling (1978), Suter
(1990), and Bartell et al. (1992). Uncertainty analysis identifies
and quantifies (if possible) the uncertainties encountered in
conducting an ecological assessment, provides an evaluation of
the overall impact of those uncertainties on the interpretations
and conclusions derived  from the  assessment, and (when
feasible) provides information to reduce uncertainty.

    In its Framework for Ecological Risk Assessment, EPA's
Risk  Assessment  Forum lists four  primary sources  of
uncertainty in ecological  risk assessments:   incomplete or
flawed conceptual model formulation, incomplete information
and data, natural variability  (stochasticity),  and introduced
errors (RAF 1992).

 ,   Incomplete knowledge is the source of uncertainty that can
not be quantified; it is one of the greatest concerns in decision-
making. This source of uncertainty includes lack of knowledge,
which can only be reduced through research and investigation.
It also includes assumptions,  inferences about non-measured
species or systems, incomplete conceptual model structures and
similar sources of uncertainty. Weight of evidence approaches
can minimize but not eliminate this uncertainty.

    Stochastic uncertainty can be described and quantified but
can not be reduced because it is inherent in the system being
assessed (Suter 1993).

    Analytical error includes measurement, sampling, analysis,
model input, parameter and similar sources of error. These
sources can be quantified through QA/QC  programs and
stochastic modeling methods.  This is the source of error that
can be reduced through proper analytical procedures.

    EMAP  has  integrated a  research component in  the
program to continually work toward reducing uncertainty due
to incomplete information and data.   In addition, data from
EMAP's monitoring activities undergo  strict QA/QC  to
quantify natural variability, reduce analytical error, and check
assumptions.'
Communication

    The purpose of environmental risk communication is to
provide people with the uncertainties, assumptions, facts, and
interpretations they need to make informed judgements about
risks to the environment (Morgan et al. 1992). Only then can
                                                                          INTERPRETATION
                                                                              AND
                                                                           CONCLUSIONS
                                                                           COMMUNICATION
                                                                      •Data
                                                                      •Statistical Summaries
                                                        Figure 14.  Major elements in the interpretation
                                                        and communication phase of EMAP's assessment
                                                        framework.
assessment results contribute to environmental management
decisions.  Moreover, communication between decision makers
and researchers is  the most important element in keeping
research and assessment relevant to policy needs (SPA 1992).

    Rubin et  al. (1992)  suggest that NAPAP's failure to
influence recent environmental legislation resulted from lack of
communication: findings were not reported in a timely fashion
and results and conclusions were not understandable to policy
makers. In a study conducted as part of EPA's Global Climate
Change Program, workshops were held separately with
                                                       25

-------
Assessment Framework
                        Extinct.
                       Depleted
                            Pristine,
                           Flourishing
                                            Range of Possible Resource Conditions
                                                Marginal
                                               Conditions
                Nominal
               Conditions
                                           Range at Upper Limit
                                          of Subnominal Conditions
            Range at Lower Limit
            of Nominal Conditions
                                                                 \    \
                                               Ecological Indicator Score
                     .o

                     1
                      Q. QJ
                      o
                      E
                                 I    I
                                               Ecological Indicator Score
                   Figure 15.  Ranges of subnominal, marginal, and nominal conditions
                   that might be delineated along a condition continuum.
decision  makers and scientists  to  evaluate the needs  and
expectations each group had of the other (SPA 1992). Both
groups discovered many useful communication lessons by
conducting this exercise (Table 9). These lessons will be used
as a primer for improving communication between EMAP and
its information users.

    The  results  of EMAP assessments will be tailored to
client's needs, contain important information relevant to the
assessment issues,  and be understandable and useable.   The
forms  and media  by  which these results are to be most
effectively communicated will differ according to the target
audience.   Complex scientific reports providing 'Significant
detail on the analytical methods and copious presentations of
data and results may  meet  the needs  of the scientific
community, but they may not be satisfactory for presenting
information to other users. Spatial displays, photographs, line
art, conceptual models, graphics, and other visuals also can
effectively present relationships or interpretations of data. The
use of decision support systems such as computerized data
summaries  and interactive graphic presentations can help
decision  makers quickly examine management alternatives.
The use of focus groups, user networks,  and extensive client
interactions are other ways to fine-tune assessment results to
meet clients' needs.
                                                         26

-------
                                                                  The Environmental Monitoring and Assessment Program,
        Table 8. Suggestions for how EMAP could establish preliminary nominal-subnominal
        categories.
              Type
  Suggestion
            Approaches       1.   Adopt documented desired uses, societal perspectives, and social values for different resource
                                classes.

                            2.   Use public meetings, hearings, focus groups, special interest groups and other outreaching
                                activities to determine what the public considers to be nominal and subnominal condition.

            Strategies       3.   Use retrospective analyses and paleoecotogical analyses to define a nominal background or
                                baseline condition that can be compared with current and future conditions.

                            4.   Conduct specific experimental research directed at determining nominal/subnominal scores and
                                ranges.

                            5.   Set priorities for indicators based on their relative position in the indicator development scheme;
                                criteria might establish priorities by how indicators respond to change. Scientists currently are
                                close to achieving consensus on the nominal/subnominal continuum for some indicators, but others
                                are still in the research stage.

             Methods        6.   Use Delphi procedures to refine the range and scores for subnominal and nominal classes.

                            7.   Test hypotheses where a priori expected values are compared with actual measured indicator
                                scores in "good" and 'bad* systems.

                            8.   Evaluate temporal (e.g., succession) and spatial (geographic location and landscape position)
                                processes that might influence the nominal/subnominal scores  and ranges.

                            9.   Assess the literature for consensus in the professions.

                            10.  Collate scores and ranges in peer review papers.

                            11.  Conduct additional workshops, to review state-of-the-art research in environmental indicators
Assessment Products

    EMAP will produce four basic assessment products
(Figure 14):

    •      Quality-assured data.
    •      Annual statistical summaries.
    •      Ecological resource assessments.
    •      Assessment tools and guidance.
Quality-Assured Data

    Many clients and users want access to the data being
collected by EMAP. Hie demand for data in EPA's STORET
and USGS  WATSTORE information management  systems
attests to the interest users have in performing  their own
assessments. EMAP's policy is to ensure the integrity, utility,
and accountability of its data so that decision makers can have
confidence in the summaries and assessments based on those
data.  EMAP data will be verified and validated prior to its
release  to the user community.  The data will not be the
individual site data but rather aggregated data to maintain data
confidentiality requirements. One of the early products from
EMAP will be this aggregated data.
Annual Statistical Summary Reports

    EMAP's annual statistical summaries will contain such
descriptive statistics as means, medians, distributions, ranges,
and standard deviations for indicators monitored within the
sampling frame. EMAP expects these reports to be similar to
annual summaries prepared by the Bureau of Labor Statistics,
National Agriculture Statistics Surveys, and USGS Water Data
Summaries;  these  data  summaries  have  proved  to be
exceptionally useful even without substantial interpretation of
the results. The status of ecological resource condition can be
assessed by including  the pertinent nominal/subnominal
criteria.

    Statistical summaries will be prepared for standard Federal
regions. Additional summaries might be provided for regions
based on biogeographic or political designations appropriate for
a specific  resource category  (e.g.,  Great Basin biographic
region for arid ecosystems or USDA Forest Service Northeast
Region for forests). The format will be refined as monitoring
results for other resource groups become available  and are
presented to the users.  Annual statistical summaries will be
produced within one year  following the date of the last field
sampling to ensure that EMAP results are provided to the users
in a timely manner.
                                                          27

-------
 Assessment Framework
                          25
    S
   Index Of
Biotic Integrity
                                                                 Federal  Region
                      . .so
                     I
                                s
                              Index Of
                           Biotic Integrity
              90
20
                                                             S
                                                     Index Of
                                                  Biotic Integrity
               70
                        X% = [0.4[#Lakes(A)]+0.8[#Lakes(B)]+0.3[#Lakes(C)]]
                                              Regional  No.  of Lakes
              Figure 16.  One method of aggregating measures of resource condition among
              different subregions (standard Federal region; EPA, USDA, or Forest Service
              region; or biogeographic province).
Ecological Resource Assessment Reports

    EMAP will produce two general types  of assessment
reports: preplanned assessments, called Status of the Ecological
Resource reports, and special request assessments that address
new issues and concerns of EMAP's information users.  The
principle difference between  these  assessments is  that
assessments for status of an ecological resource are planned to
be based primarily on EMAP data, produced on a periodic
basis (e.g., every 4-5 years), and focused on the regional and
national condition of ecological resources. EMAP expects
special assessments to address specific issues or problems that
arise on an ad hoc basis. Both typos  of assessments  will
assist in the continual  improvement and evolution of the
                              monitoring and research activities in EMAP.  Status of the
                              ecological resource reports  will be prepared for individual
                              resources and for multiple resources both within and across
                              administrative or biogeographic reporting regions.  These
                              reports will assess ecological resource condition and suggest
                              possible  factors contributing to this condition; they will also
                              evaluate the cumulative effectiveness of regulations and
                              policies in managing and protecting the environment

                              Assessment Tools and Guidelines

                                  As  EMAP  develops  its assessment capabilities,  the
                              techniques and guidelines developed in the process will be of
                              value  to environmental managers  and other research  and
                                                      28

-------
                                                                    The Environmental Monitoring and Assessment Program
             Table 9. Actions suggested by lessons learned in communicating between decision
             makers and scientists through the global climate program (SPA 1992).
             Action
                          Lesson Learned
             Supply interim information.
             Tie assessment to important
             policy time frames.
             Provide useful information
             before reliable predictions.
             Know decisions are not
             either/or.
             Place comparative risk in
             relative context

             Address urgent need for
             education.
             Manage uncertainty.
             Know research does not
             always provide the answer.
             Develop an ongoing
             assessment process for
             research.
 Decision makers need interim information on relevant findings, in addition to
 periodic assessments.  Decisions dp not conform to the dates for assessment
• reports. Interim information is required for informed decisions driven by short-time
 frames.

 Assessments should be keyed to important dates such as international policy
 discussions, reauthorization of environmental legislation (e.g., Clean Water Act
 Amendments, Wetland  Protection Act Amendments), policy formulation and other
 decision making activities.

 Assessments can provide useful information even if the "penultimate" model
 predictions of the effects of stressors are not yet available. Existing information
 can be synthesized and integrated with results from simple empirical models,
 statistical analyses and similar analytical approaches to provide useful, interim
 information for decision makers.

 Decision makers' choices are not simply either to pursue research or to implement
 management alternatives. The challenge is to define the appropriate levels of
 each over time.  Researchers need to provide a broad array of information to
 address the complex and interacting decisions.  Decision makers, for their part,
 need to recognize the long time scales involved in research and, thus, the
 importance of continuity of funding and program goals.

 Assessing change in a relative risk context is difficult, but extremely important


 A concerted effort is needed to educate decision makers on the facts and
 uncertainties of any environmental issue. Considering that public concern is often
 the impetus for formulating policy, scientists need to communicate technical
 information to the public more effectively, as well as more frequently.  In addition,
 scientists need to learn  more about the decision making process and the types of
 information most useful for policy.  Frequent two-way communication between
 decision makers and researchers is fundamental if research is to play an effective
 role in the decision making process.

 There are more ways to manage uncertainties than simply trying to reduce them.
 For example, building resilient institutions and methodologies would provide a
 flexible response to any future change albeit at potentially significant costs.
 Contingency plans could allow decision makers to prepare for possible climate
 outcomes through R&D technologies, without needing to employ them.

 Decision makers need to realize that additional research actually could increase
 the amount of uncertainty in some areas.  Researchers should inquire about how
 much certainty decision makers are requiring  to take a specific action.  To this
 end, uncertainties that do not matter for decision making should be so identified.

To improve communication and better inform  decision makers, research efforts
 should include an iterative assessment process.  These assessments would not
only help identify the  relevant questions, but also serve to structure the research
results and, thus, facilitate clearer communication between the two communities.
 Furthermore, the assessment process would provide valuable input to the
planning of policy-relevant research.
monitoring   programs.     EMAP's  strategy  emphasizes
monitoring   ecological   indicators   with   state-of-the-art
technology  to  provide  a  new  generation  of ecological
monitoring methods that may be applied in other programs.
The monitoring design provides agencies an opportunity to
evaluate problems at finer or smaller scales man the regional
resolution  of EMAP by  enhancing  the monitoring grid.
Regional and other large-scale  tools developed to integrate
                       monitoring  data across landscapes  and  resources will be
                       invaluable   to  other   large-scale  monitoring   programs.
                       Guidelines developed to guide the aggregation, analysis and
                       interpretation  of ecological  resource assessments  will help
                       EMAP  evolve  and maintain  its  policy  relevance.   Peer-
                       reviewed scientific articles, technical presentations, and reports
                       written by EMAP scientists, partners, or interested individuals
                       also will help  advance assessment technology.
                                                            29

-------
Assessment Framework
                                 4 — Evolving Program and Process
    Currently, EMAP is demonstrating the capability  of a
network to monitor the Nation's ecological resources; however,
it will take a number of years before routine monitoring of all
resources  in all  regions of the country  will be  fully
implemented.  As a result,  the  ability of the Program to
conduct ecological resource assessments will correspond to the
phased  implementation   of  its  research  and  monitoring
activities. Initial assessments will focus on determining extent,
geographic coverage, and condition for individual ecological
resources. Single region, single resource assessments will be
conducted before assessments encompassing multiple regions
or national levels. Assessments of multiple ecological resources
in a single region will be conducted as other resources start
monitoring in that region.
    The  assessment framework proposed  in this  report
provides  a common strategy for planning and conducting a
wide variety of ecological assessments within EMAP.  The
framework also demonstrates how EMAP complements the
assessment  approach proposed  in  EPA's  Ecological  Risk
Assessment Framework (RAF 1992).  EMAP assessments can
contribute directly to the problem formulation phase of EPA-
RAF's proposed ecological risk assessment  framework by
identifying and quantifying factors which might be contributing
to condition of ecological resources. Other EMAP products
(e.g.,  ecological  resource condition  and stressor data) will
provide significant information needed to conduct ecological
risk assessments that  verify  model predictions  and the
cumulative   effectiveness  of  environmental  protection
regulations and environmental management decisions.
                                                        30

-------
                                                               The Environmental Monitoring and Assessment Program
                                                   Glossary
abiotic:  Nonliving  characteristic of the environment; the
    physical and chemical components that relate to the state
    of ecological resources. (Term added 1993. See related:
    biotic, condition indicator, indicator.)

acid deposition:   "A complex  chemical and atmospheric
    phenomenon that occurs when emissions of sulfur and
    nitrogen compounds and other substances are transformed
    by chemical processes in the atmosphere, often far from
    the original sources, and then deposited on earth in either
    a wet or dry form. The wet forms, popularly called "acid
    rain," can fall as rain, snow, or fog.  The dry forms are
    acidic gases or participates" (EPA 1992, 1).

agroecosystem:  A dynamic association of crops, pastures,
    livestock, other flora and fauna, atmosphere,  soils and
    water.    Agroecosystems are contained  within  larger
    landscapes that  include  uncultivated  land,  drainage
    networks, rural communities, and wildlife.

ancillary data: Data collected from studies within EMAP but
    not used directly in the computation of an  indicator.
    Ancillary data can help characterize parameters and assist
    in the interpretation of data sets; time, stage of tide, and
    weather conditions are examples of ancillary data. (Term
    added 1993. See related auxiliary data.)

annual statistical summary:   A document that presents a
    brief and comprehensive report of EMAP data collected
    on a single EMAP resource for a specific year. Annual
    statistical  summaries   may   include  cumulative
    distributions,  estimates  of the  extent  of nominal or
    subnominal condition, comparisons  among regions, or
    comparisons of data over time.

arid ecosystems:   Terrestrial  systems  characterized  by a
    climate regime where the potential evapotranspiration
    exceeds precipitation, annual precipitation ranges from less
    than 5 cm to not more than 60 cm, and daily and seasonal
    temperatures range from -40*C to 50*C. The vegetation
    is dominated by woody perennials, succulents, and drought
    resistant trees.

assessment: Interpretation and evaluation of EMAP results for
    the purpose of answering policy-relevant questions about
    ecological resources, including (1) determination of the
    fraction of the population that meets a socially defined
    value and (2) association among indicators of ecological
    condition and stressors.

assessment  endpoint:  Formal  expressions of the actual
    environmental value that is to be protected (Suter 1990).
    Risk  Assessment Forum defines  this as  an  "explicit
    expression of the  environmental  value  that is to be
    protected" (RAF 1992, 37).  Operationally in EMAP, an
    assessment  endpoint  is  the  range,  proportion, or
    percentage of a resource that is known with confidence to
    be in a  specified  condition (See related: condition
    indicator,   nominal,   measurement   endpoint,
    subnominal.)

attribute:  Any  property,  quality,  or characteristic of a
    sampling unit. The indicators and other measures used
    to characterize  a sampling  site or resource  unit are
    representations of the attributes  of that unit or site. A
    characteristic  of a map feature (point, line, or polygon)
    described by numbers or text; for example attributes of
    a tree, represented by a point, might include height and
    species. (See related: continuous, discrete resource.)

auxiliary data: Data derived from a source other than EMAP,
    that is, from an experiment or from another monitoring or
    sampling program, either Federal or State. The sampling
    methods and quality assurance protocols of auxiliary data
    must be evaluated before the  data are used.  It is always
    important  to  establish the population  represented by
    auxiliary data.  (Preferred term 1993; replaces "non-
    EMAP data"  and "found data," deleted in 1993; see
    related: ancillary data.)
                                                       31

-------
Assessment Framework
bias:   In a sampling  context, the difference between  the
    conceptual weighted average value of an estimator over all
    possible samples and the true value of the quantity being
    estimated.  An estimator is  said to be unbiased if that
    difference is zero. The "systematic or persistent distortion
    of a measurement process which deprives the  result of
    representativeness (i.e., the expected sample measurement
    is different than the sample's true value.  A data quality
    indicator" (QAMS 1993, 3).

biodiversity:   The  variety  and variability  among living
    organisms and the ecosystems in which they occur.
    Biodiversity includes the numbers of different items and
    their relative frequencies; these  items are organized at
    many levels,  ranging from complete ecosystems to  the
    biochemical structures  that are the molecular basis of
    heredity. Thus, biodiversity encompasses expressions of
    the relative abundances of different ecosystems, species,
    and genes (OTA 1987).

biotic:  Of or pertaining to living organisms.  (See related:
    abiotic,  indicator,   condition   indicator,  stressor
    indicator. In 1993, biotic condition indicator replaced
    "response indicator.")
cdf.  Cumulative  distribution function. (See:
    distribution).
cumulative
change:  As used in EMAP, the difference in the distribution
    of measurements of condition  indicators between two
    time periods. (See related: status, trends.)

characterization: Determination of the attributes of resource
    units, populations, or sampling units. A prominent use
    in EMAP is characterization of 40-hexes.

classification: The process of assigning a resource unit to one
    of a set of classes defined  by  values  of specified
    attributes. For example, forest sites will be classified into
    the designated  forest types,  depending on the  species
    composition of the forest  Systematic arrangement of
    objects into groups or categories according to established
    criteria

community:  "All of the populations occupying a given area
    (Odum 1959, 6); Odum's definition was adapted by the
    Risk Assessment Forum  to  read:   "an assemblage of
    populations of different species within a specified location
    in space and time" (RAF 1992, 37). "In ecology, a group
    of interacting populations in time and space. Sometimes
    a particular subgrouping may be specified, such as the fish
    community in a lake or the soil arthropod community in
    a forest" (EPA 1993, 6).

comparability: The degree to which different methods, data
    sets  and/or  decisions agree  or can  be  represented as
    similar; a data quality indicator" (QAMS 1993, 6).

completeness:  The amount of valid data obtained compared
    to the planned amount, and [it is] usually expressed as a
    percentage; a data quality indicator" (QAMS 1993, 6).

conceptual model:  A "conceptual model describes a series
    of working hypotheses of how the stressor might affect
    ecological components.   The conceptual model also
    describes the ecosystem potentially at risk, the relationship
    between  measurement [endpoints]  and  assessment
    endpoints, and exposure scenarios" (RAF 1992, 37).

condition:   The distribution of  scores describing  resource
    attributes without respect to any societal value or desired
    use, that is, a state of being. (New term 1993.)

condition indicator: A characteristic of the environment that
    provides quantitative estimates of the state of ecological
    resources and is conceptually tied to a value. (New Term
    1993;  replaces environmental  indicator.  See related:
    indicator, abiotic, biotic, stressor indicator.)

continuous:   A  characteristic  of  an  attribute that is
    conceptualized as a surface over some region. Examples
    are certain attributes of a resource, such as chemical
    stressor indicators measured in estuaries.

cross-cutting group:  A group of scientific and administrative
    personnel  headed by a technical coordinator and charged
    with addressing specific cross-program, integrative issues
    in EMAP. These groups  are named Assessment  and
    Reporting, Design and Statistics, Indicator Development,
    Information  Management, Landscape Characterization,
    Logistics  and Methods, and Quality Assurance. (See
    related resource group.)

cumulative distribution:   A means of representing  the
    variation of some attribute by giving running totals of the
    resource with attribute values less than or equal to a
    specified series of values.  For example, a cumulative
    areal distribution of lakes would give, for any value a of
    area, the total area covered by lakes with individual area
    less  than  or  equal to a.   A  cumulative frequency
    distribution for lake area would give the  total number of
    lakes with area less  than or equal to a.  The cumulative
    distribution function (cdf) of some specified attribute of
    a population is the function F(x) that gives the proportion
    of the population with value of the attribute less than or
                                                         32

-------
                                                                The Environmental Monitoring and Assessment Program
     equal  to x, for any choice of x.   For example, if the
     attribute was lake area in hectares, F(a) would give the
     proportion of lakes with area less than or equal to a ha.
     (In some cases, the word "cumulative" may be omitted in
     discussions of  the edf,  and the  cdf is called the
     distribution function.)
      uo
      our
 F(x)
      04-
      0.2-
      
    2.5  cm), greater than 260 km2, and with an aspect ratio of
    greater than 20. Small estuaries and small tidal rivers are
                                                         33

-------
Assessment Framework
    those systems whose surface areas fell between 2.6 km2
    and 260 km2. (See related: wetlands.)
  H
forest: Land with at least 10% of its surface area stocked by
    trees of any size  or  formerly having had such trees as
    cover  and not  currently  built-up or  developed for
    agricultural use (USDAFS  1989).

fragmented:  Being divided or convoluted into distinct parts,
    rather than entire. In EMAP, the spatial fragmentation of
    resources  and the  spatiaVtemporal fragmentation of
    resource  attributes affect the  precision  of certain
    population statistics, so that attention must be given to
    this state. (See related:  entire.)

frame:  A representation of a population, used to implement
    a sampling strategy  as, for example,  (1) a list  frame,
    containing  the  identity  of  all   the  units  in  the
    population—for instance,  a list of all  the lakes  in the
    United States between 10  and 2000 ha—or (2) an  area
    frame that consists of explicit descriptions of a partition of
    the areal.extent of an areal  universe—like the  NASS
    frame.  (See related: sampling unit.)
geographic  information system (GIS):   A collection of
    computer  hardware,  software,  and  geographic  data
    designed to  capture, store, update, manipulate, analyze,
    and display geographically referenced data.

Great Lakes:  Li EMAP, the resource that encompasses
    the five Great Lakes—Superior, Michigan, Huron,
    Erie, and Ontario, including river mouths up to the
    maximum  extent  of  lake  influence;  wetlands
    contiguous to the lakes; and the connecting channels,
    Lake  St.  Clair and the  upper portion of the St
    Lawrence Seaway.

grid:  A  data structure commonly used to represent map
    features.  A cellular-based data structure composed of
    cells or pixels arranged in rows and columns (also called
    a "raster").

grid, triangular (EMAP):   A lattice of points  in exact
    equilateral triangular structure on a plane.  The  EMAP
    grid points are 27.1  km apart.
                                                            habitat: "The place where a population (e.g., human, animal,
                                                               plant,  microorganism) lives and its  surroundings, both
                                                               living  and non-living" (EPA 1992, 14).
index: Mathematical aggregation of indicators or metrics.

indicator:    Any  expression  of  the environment  that
    quantitatively  estimates  the  condition  of  ecological
    resources, the magnitude of stress, the  exposure  of a
    biological component to stress, or the amount of change
    in condition (after Hunsaker and Carpenter 1990, Olsen
    1992). "Any expression  of the environment"  includes
    abiotic  and  biotic  characteristics that can  provide
    quantitative information on ecological resources. (Revised
    definition 1993, 1994. Preferred term for environmental
    indicator, deleted 1993.) "In biology, an organism, species,
    or community whose characteristics  show the presence of
    specific environmental conditions,  good or bad" (EPA
    1992, 15).   (See related:  condition indicator,  stressor
    indicator, biotic, abiotic.)

indicator development:   The  process through which  an
    indicator is  identified, tested, and implemented.   A
    candidate indicator is identified and reviewed by peers
    before it is selected for further evaluation as a  research
    indicator. Existing data are analyzed, simulation studies
    are performed with realistic scenarios,  and limited field
    tests are conducted to evaluate the research indicator. In
    the  past,  this  research  indicator   was  called   a
    "probationary  core  indicator"  or   a  "development
    indicator" as it was evaluated in regional demonstration
    projects. An indicator is  considered  a core indicator
    when it is selected for long-term, ecological monitoring as
    a result of its acceptable performance, demonstrated ability
    to satisfy the data quality objectives.

integration: The formation, coordination, or blending of units
    or components into a functioning or unified whole.  In
    EMAP, integration refers to a coordinated approach to
    environmental monitoring, research, and assessment, both
    among  EMAP  resource  groups  and  with  other
    environmental monitoring  programs.    Integration  in
    EMAP also refers to  the technical processes involved in
    normalizing and combining data for interpretation and
    assessment.
                                                         34

-------
                                                              The Environmental Monitoring and Assessment Program
lake:  Li EMAP, a standing body of water greater than 1
    hectare (about 2.5 acre) that has at least 1000 m2 (about
    0.25 acre) of open water and is at least 1 meter (about 3
    feet) deep at its deepest point (See related:  surface
    waters, wetlands.)

landscape:   The  set of traits,  patterns, and structure of a
    specific   geographic   area,  including  its  biological
    composition,   its  physical   environment,   and   its
    anthropogenic  patterns.    An  area  where  interacting
    ecosystems are grouped and repeated in similar form. In
    EMAP, Landscapes is the name of a resource group.

landscape characterization: Documentation of the traits and
    patterns  of the  essential  elements  of the  landscape,
    including  attributes  of  the physical   environment,
    biological composition, and anthropogenic patterns.  In
    EMAP, landscape characterization emphasizes the process
    of describing land use or land cover, but it also includes
    gathering  data  on  attributes  such as  elevation,
    demographics, soils, physiographic regions, and others.

landscape ecology:   The  study of distribution patterns of
    communities and ecosystems, the ecological processes that
    affect those patterns, and changes in pattern and process
    over time (Forman and Godron  1986).
  M
marginal condition:  The state that exists when the nominal
    and subnominal criteria are not contiguous.

measurement:  A quantifiable attribute that is tied to an
    indicator.

measurement  endpoint:    A   measurable   ecological
    characteristic that is related  to the valued characteristic
    chosen as the assessment endpoint (Suter 1990). RAF
    added to  Suter:  "Measurement endpoints  are  often
    expressed as the statistical or arithmetic summaries of the
    observations that  comprise [sic] the measurement" (RAF
    1992, 38). (See related:  assessment endpoint.)

modeling:   "Development of a mathematical  or physical
    representation of a system or theory that accounts for all
    or some of its known properties. Models are often used
    to test the effect of changes of components on the overall
    performance of the system" (EPA 1992,18).
monitoring: Li EMAP, the periodic collection of data that is
    used to determine the condition of ecological resources.
    "Periodic  or  continuous  surveillance  or  testing  .to
    determine   the  level  of  compliance  with  statutory
    requirements and/or pollutant levels in various media (air,
    soil, water) or in humans, plants, and animals" (EPA 1993,
    18).
  N
National Academy of  Sciences  (NAS):   The National
    Academy of Sciences/National Research Council (NRC)
    performs level 2  peer  review  to  determine if EMAP
    projects have overall scientific merit and integrate both
    internally   and   with   other   government-sponsored
    monitoring programs. Two commissions of the NRC—the
    Commission on Geosciences, Environment, andResources
    (specifically, its Water Science and Technology Board)
    and the Commission on Life Sciences—jointly organized
    the  Committee   to Review  EPA's  Environmental
    Monitoring and Assessment Program in  1991.   This
    NAS/NRC  committee  holds about 12 meetings  and
    produces two or three reports every two years; its primary
    purpose is to consider the scientific and technical aspects
    of EMAP as  designed as well as  considering  ways to
    increase EMAP's usefulness in monitoring conditions and
    trends  in six representative types  of ecosystems.  The
    Committee also reviews me overall design objectives of
    the program, the indicator strategies, data collection
    methods, data analysis interpretation, and communication
    plans. Preparation  for NAS/NRC reviews is coordinated
    by the Director of OMMSQA,  EPA-ORD, who is  also
    responsible for funding.

nominal:   Referring  to  the state  of  having  desirable or
    acceptable ecological condition: The quantified standard
    established  for a  condition indicator to  represent the
    desirable or acceptable condition  is called a  nominal
    assessment endpoint. (See related: assessment endpoint,
    marginal, subnominal.)

NRC (National Research Council, see National Academy of
    Sciences.)
  o
Office of Modeling,  Monitoring  Systems, and  Quality
    Assurance (OMMSQA): The office within EPA's Office
    of Research and Development responsible for EMAP
    management within the Agency.
                                                       35

-------
 Assessment Framework
parameter:   "Any quantity such as a mean or a standard
    deviation characterizing a population. Commonly misused
    for 'variable,'  'characteristic,'  or 'property'" (QAMS
    1993,15).

pattern: In EMAP, the location, distribution, and composition
    of structural landscape components within a particular
    geographic area or in a spatial context.

peer review: In EMAP, peer review means written, critical
    response provided by scientists  and  other technically
    qualified participants in the process.  EMAP documents
    are subject to formal peer review procedures at laboratory
    and program levels. In EMAP, Level 1 peer reviews are
    performed by EPA's Science Advisory Board, level 2 by
    the NAS National Research Council, level 3 by specialist
    panel  peer  reviews,  and  level 4  by  internal  EPA
    respondents.   (See  related:   National Academy of
    Sciences, Science Advisory Board.)

population:  "A group of interbreeding organisms occupying
    a particular space; the number of humans or other living
    creatures in a designated area" (EPA 1992,22 after Odum
    [1953] 1959,6). In statistics and sampling design, the total
    universe addressed in a sampling effort; an assemblage of
    units of a particular resource, or any subset of extensive
    resources, about which inferences are desired or made.
    RAF  defines   population  to  be  "an  aggregate  of
    individuals of a species within a specified location in
    space and time" (RAF 1992, 38).

precision: The degree to which replicate measurements of the
    same attribute agree or are exact "The degree to which
    a set of observations or measurements  of the  same
    property,  usually  obtained  under  similar  conditions,
    conform to themselves; a data quality indicator" (QAMS
    1993,16).  (See related: accuracy, bias.)
quality assurance (QA): "An integrated system of activities
    involving planning, quality control, quality assessment,
    reporting and quality improvement to ensure that a product
    or service meets defined standards of quality with a stated
    level of confidence" (QAMS 1993, 17).
          Li EMAP, quality assurance consists of multiple
    steps taken to ensure that all data quality objectives are
    achieved.  (See related: data quality objectives, quality
    control.)
quality control (QC):   "The overall  system of technical
    activities whose purpose is to measure and control the
    quality of a product or service so that it meets the needs
    of users.  The aim is to provide quality that is satisfactory,
    adequate, dependable, and economical" (QAMS 1993,17).
          In EMAP, quality control consists of specific steps
    taken during  the data collection process  to ensure that
    equipment and procedures are operating as intended and
    that  they will  allow data  quality objectives  to  be
    achieved. (See related: data quality objectives, quality
    assessment, quality assurance, QA/QC.)

QA/QC: Quality Assurance/Quality Control. "A system of
    procedures, checks, audits, and corrective actions to ensure
    that   all  EPA  research  design  and  performance,
    environmental  monitoring  and  sampling,   and  other
    technical and reporting activities  are of the  highest
    achievable quality" (EPA 1992, 23).
reference condition:   The  set of  attributes of ecological
    resources that assist in identifying  the location of a
    portion of the resource population along a condition
    continuum from worst possible condition  to  the best
    possible condition given the prevailing topography, soil,
    geology, potential vegetation, and general land use of the
    region. Reference condition typically refers to the best
    resource condition, but it is used more broadly in EMAP
    (Term added in 1993).

region: Any explicitly defined geographic area. In the EMAP
    objectives, region refers  to  the  ten  standard Federal
    regions (OMB 1974).

relation: The concept of function, correlation, or association
    between or among attributes, which may be qualitative as
    well as quantitative.

representativeness: "The degree to which data accurately and
    precisely represent the frequency distribution of a specific
    variable in the population; a data quality indicator"
    (QAMS 2993, 20).

resource: In EMAP, an ecological entity that is identified as
    a target of sampling and is  a group of general, broad
    ecosystem types or ecological entities sharing certain basic
    characteristics.   Eight  such categories  currently  are
    identified within EMAP: estuaries, Great Lakes, surface
    waters, wetlands, forests, arid ecosystems, agroecosystems,
    and  landscapes.  These  eight  categories  define  the
    organizational  structure  of  resource  monitoring
    groups—called resource groups—in EMAP and are the
    resources addressed by EMAP assessments. A resource
                                                        36

-------
                                                                The Environmental Monitoring and Assessment Program
    can be characterized as belonging to one of two types,
    discrete and extensive, that pose different problems of
    sampling and representation.

resource  class:  A subdivision of a resource; examples
    include  small  lakes,  oak-hickory  forests,  emergent
    estuarine wetlands, field cropland, small estuaries,  and
    sagebrush dominated desert scrub.

resource domain:  The areal extent of a resource; the region
    occupied by a resource.

resource group:   A group of scientific  and administrative
    personnel, headed by a technical director, responsible for
    research, monitoring, and assessments for a given EMAP
    resource.   There  are  eight such groups in  EMAP:
    Estuaries, Great Lakes, Surface Waters, Wetlands, Forests,
    Arid Ecosystems, Agroecosystems, and Landscapes. (See
    related cross-cutting group.)

resource unit:  A unit of a discrete resource, for example, a
    lake. A population of such a resource will be an explicit
    set of resource units.

risk:   "A measure of the  probability that damage to life,
    health, property, and/or the environment will occur as a
    result of a given hazard" (EPA 1992, 25). In statistics,
    "the expected  loss due to the use of a given decision
    procedure" (QAMS 1993, 20).

risk assessment: "Qualitative and quantitative evaluation of
    the risk posed to human health and/or the environment by
    the actual  or potential presence and/or use of  specific
    pollutants" (EPA 1992, 25).

risk characterization:  Determination of the nature of a given
    risk and quantifying of the potential for adverse change to
    the environment from that risk.  "A phase of ecological
    risk assessment that integrates the results of the exposure
    and ecological effects analyses to evaluate the likelihood
    of adverse ecological effects associated  with exposure to
    a stressor.  The ecological  significance of the  adverse
    effects is discussed, including consideration of the types
    and magnitudes of the effects, their spatial and temporal
    patterns, and the likelihood of recovery" (RAF 1992,38).

risk communication: "The exchange of information about
    environmental risks among risk assessors, risk managers,
    the general public, news media, special interest groups,
    and others" (EPA 1992, 25).

risk management: "The process of evaluating and selecting
    alternative regulatory and non-regulatory responses to risk.
    The   selection  process    necessarily  requires    the
    consideration of scientific, legal, economic, and behavioral
    factors" (EPA 1992, 25).
sample:  A subset of the units from a frame. A sample may
    also be a subset of resource units from a population or
    a set of sampling units.

sampling strategy:  A sampling design, together with a plan
    of analysis  and estimation.  The  design  consists  of a
    frame, either explicit or implicit, together with a protocol
    for selection of sampling units.

sampling unit:   An entity that is  subject to  selection and
    characterization  under a  sampling design.   A sample
    consists of a set of sampling units or sites that will  be
    characterized. Sampling units are defined by the frame;
    they may correspond to resource units, or they may  be
    artificial units constructed for the sole purpose  of the
    sampling design.

Science Advisory Board (SAB): A peer review panel internal
    to EPA.  The Ecological Effects Committee of the SAB
    conducts reviews of EMAP's  overall program and the
    conceptual  framework for integrating EMAP with the
    Ecological Risk Assessment program. Preparation for SAB
    reviews is coordinated by the Director of the Office of
    Modeling, Monitoring systems,  and  Quality Assurance
    (OMMSQA); the Assistant Administrator for ORD is
    responsible for funding. SAB review is considered level
    1 peer review.

status:  The distribution of scores for condition indicators
    with relation to  the reference condition associated  with
    specific social values or desired uses  for a specific  time
    period. (See related: change, condition, trends.)

stressor:  "Any  physical, chemical, or biological entity that
    can induce an adverse response" (RAF 1992, 38).

stressor indicator:  A characteristic of the environment that
    is suspected to elicit a change in the state of an ecological
    resource, and they include both  natural and human-
    induced stressors.  Selected stressor  indicators will be
    monitored in EMAP only when a  relationship between
    specific condition and stressor indicators is known, or a
    testable hypothesis   can  be formulated.   (See related:
    indicator, condition indicator.)

subnominal:  Having undesirable or unacceptable ecological
    condition.   The quantified standard  established for a
    condition  indicator  to   represent   unacceptable   or
    undesirable ecological condition is called the subnominal
    assessment endpoint. (See related: assessment endpoint,
    marginal, nominal.)

surface waters:  The inland surface waters consisting of all
    the Nation's lakes (other than the Great Lakes),  rivers,
    and streams.  Lakes are distinguished from wetlands by
    depth and by size. Streams (and rivers) will be identified
                                                        37

-------
 Assessment Framework
    from stream traces on maps and confirmed in field visits.
    Streams are operationally defined as any first or higher
    order stream that is represented as a blue line on a USGS
    1:100,000 topographic map.
          "All water naturally open to the atmosphere (rivers,
    lakes, reservoirs, ponds,  streams, impoundments,  seas,
    estuaries, etc.) and all springs, wells, or other collectors
    directly influenced by surface water" (EPA  1992, 28).
    (See related:  lake, wetlands.)
target population:  A specific resource set that is the object
    of target of investigation.

technical coordinator (TC): The individual responsible for
    directing the activities of an  individual  cross-cutting
    group.

technical director  (TD):   The  individual responsible for
    directing the activities of an individual resource group.

total  quality management (TQM):    A  system  that is
    implemented in every aspect of an organization with the
    focus of providing quality; that is, highly valued products.
    The  system  provides  a  framework  for  planning,
    documentation,   communication,  etc.   and  strongly
    emphasizes  a client-oriented perspective.  "The process
    whereby an entire organization, led by senior management,
    commits to focusing on quality as a first priority in every
    activity.  TQM implementation creates a culture in which
    everyone in the organization shares the responsibility for
    continuously improving the quality of  products  and
    services in order to satisfy the customer" (QAMS 1993,
    26).

trends:  The change in the distribution of scores for condition
    indicators  over  multiple  time periods.  (See related:
    change, status.)
value: A characteristic of the environment that is desired. In
    the past, the term "environmental value" was defined to
    mean characteristic of the environment that contributes to
    the quality of life provided to an area's inhabitants, for
    example, the ability of an area to provide desired functions
    such as food, clean water and air, aesthetic experience,
    recreation, and desired  animal  and  plant  species.
    Biodiversity, sustainability, and aesthetics are examples of
    environmental  values   (Suter  1990).    A  quantity's
    magnitude.
  w
watershed: "The terrestrial area of the landscape contributing
    to flow at a given stream location.  The land area that
    drains into a stream" (EPA 1992, 31).
 v.  WMnlWd
   N Boundfcy
  _ Drecflon of
  — BuneH
                                                  \ _^
                                                             Watershed
  u
universe: The total entity of interest in a sampling program,
    often together with some structural features.  The EMAP
    universe  is  the entire United  States, together with
    adjoining waters. (See related: population.)
wetlands:  Lands transitional between terrestrial and aquatic
    systems where the water table is usually at or near the
    surface or where shallow water covers the land and where
    at least one of the following attributes holds:  (1) at least
    periodically,   the   land  supports   aquatic   plants
    predominantly;  (2)  undrained  hydric  soils  are  the
    predominant substrate; and (3) at some  time during the
    growing season, the substrate is  saturated with water or
    covered by shallow water (Cowardin et al. 1979). An area
    that  is saturated  by  surface or ground  water with
    vegetation adapted for life under those soil conditions, as
    swamps, bogs, fens, marshes, and estuaries (EPA 1992,
    31).
                                                         38

-------
                                                             The Environmental Monitoring and Assessment Program
                                                 References
Baker, L.A., P.R. Kaufinann, A.T. Herlihy, and J.M. Eilers.
   1990.  Current  Status  of  Surface  Water  Acid-Base
   Chemistry.   National  Acid  Precipitation Assessment
   Program Acidic Deposition:    State  of Science and
   Technology. Report 9. Washington, DC:  Office of the
   Director, National Acid Precipitation Assessment Program.

Bartell,  S.M., R.H.  Gardner,  and R.V.  O'Neill.  1992.
   Ecological  Risk Estimation.  Ann  Arbor,  MI:  Lewis
   Publishers.

Bardwell,  L.V.  1991. Problem-framing:  A  perspective on
   environmental problem-solving. Environmental Management
   15:603-612.
        T
CEQ   (Council   on   Environmental  Quality).   1990.
   Environmental   Quality.  Twentieth   Annual  Report.
   Washington, DC: Executive Office of the President

Charles, D.F., R.W. Battarbee, L Renberg, H. van Dam, and
   JP.  Smol.  1989.    Paleoecological  analysis of  lake
   acidification trends in North America and Europe using
   diatoms and chrysophytes. In Acid Precipitation, 207-276.
   New York, NY: Springer-Verlag.

Church, M.R., K.W. Thornton,  P.W. Shaffer, D.L. Stevens,
   BP. Rochelle, R.G. Holdren, M.G. Johnson, JJ. Lee, R.S.
   Turner, D.L. Cassell, D.A. Lammers, W.G. Campbell, C.I.
   Liff, C.C.  Brandt,  L.H. Liegel,  G.D.  Bishop,  D.C.
   Mortenson, and S.M. Pierson. 1989. Future Effects of Long-
   Term Sulfur Deposition on Surface Water Chemistry in the
   Northeast and Southern Blue Ridge  Province (Results of
   the Direct/Delayed Response Project). EPA600389061 a, b,
   c, d. Corvallis, OR: U.S. Environmental Protection Agency,
   Environmental Research Laboratory.

Costanza,  R., E.G. Norton, and B.D.  Haskell  (eds). 1992.
   Ecosystem  Health:    New   Goals for  Environmental
   Management. Washington, DC:  Island Press.

Cowling, E.B. 1992. The performance and legacy of NAPAP.
   Ecological Applications 2:111-116.

Cowardin, L.M., V. Carter, F.C.  Golet, and E.T. LaRoe. 1979.
   Classification of Wetlands and Deepwater Habitats of the
   United States. FSW/OSB-79/31. Washington, DC:  U.S.
   Department of the Interior, Fish and Wildlife Service.

Deuel, JJC., and J. D'Aloia, Jr. 1989. Defining our vocabulary.
   Environmental Auditor 1: 33-38.

Ehrlich, P. 1980. An ecologist standing up among seated social
   scientists. The Co-Evolution Quarterly. Fall: 24-35.

EMAP (Environmental Monitoring and Assessment Program).
   1990. Near Coastal  Program Plan for 1990: Estuaries,
   A.F.  Holland   and  J.F.  Paul,  eds.  EPA600490033
   (PB93116176). Naragansett, RI:   U.S.  Environmental
   Protection Agency, Office of Research and Development,
   Environmental Research Laboratory.

EPA  (U.S.  Environmental  Protection   Agency).   1976.
   Guidance for conducting cancer assessments. 41 Federal
   Register 21402.

	. 1986.  Guidelines for conducting human health risk
   assessments 51 Federal Register 33992-34054.
	. 1987. Unfinished Business: A Comparative Assessment
   of   Environmental  Problems.   Overview   Report.
   EPA230287025A [5 report set: PC-E19, OPB88-127030J
   Washington, DC: U.S. Environmental Protection Agency,
   Office of Policy, Planning, and Evaluation.

	. 1992. Terms of Environment: Glossary, Abbreviations
   And Acronyms.  EPA175B92001. Washington, DC: U.S.
   Environmental  Protection  Agency,  Communications,
   Education, And Public Affairs.

	. 1993. Terms of Environment: Glossary, Abbreviations
   And Acronyms. EPA175B93001. Washington, DC: U.S.
   Environmental  Protection  Agency,  Communications,
   Education, And Public Affairs.

Finkel, A. 1990. Confronting Uncertainty in Risk Management.
   Washington, DC: Resources for the Future.

Forman, R.T.T., and M. Godron. 1986. Landscape Ecology.
   New York, NY:  John Wiley & Sons.
                                                       39

-------
 Assessment Framework
Powells, H.A. 1965. Silvics of Forest Trees of the United
   States. Agricultural Handbook No. 271. Washington, DC:
   U.S. Department of Agriculture, Forest Service.

GAO (United States General Accounting Office).    1988.
   Environmental Protection Agency.  Protecting  Human
   Health  and  the  Environment.    GAO/RCED-880-101.
   Washington, DC:  U.S. General Accounting Office.

Hill, A.B.  1965.  The environment and disease: Association
   or causation? Proceedings of the Royal Society of Medicine
   58:295-300.

Holling, C.S., ed. 1978. Adaptive Environmental Assessment
   and Management. New York, NY: John Wiley and Sons.

Hughes, RJM. 1989. Ecoregional biological criteria In Water
   Quality  Standards  for the  21st  Century,  147-151.
   Proceedings of a National Conference, March 1-3,  1989,
   Dallas,  TX.   Washington, DC: t. U.S. Environmental
   Protection Agency, Office of Water.

Hunsaker, C.T., and DJ3. Carpenter,  eds. 1990. Ecological
   Indicators for  the  Environmental   Monitoring   and
   Assessment Program.  EPA 600390060. Research Triangle
   Park, NC: U.S. Environmental Protection Agency, Office of
   Research and Development.

Karr, J.R. 1991. Biological integrity: A long-neglected aspect
   of water resource management Ecological Applications
   l(l):66-84.

	.  1987.  Assessment of biotic integrity using fish
   communities. Fisheries 6(6):21-27.

	.  1981. Biological  monitoring  and  environmental
   assessment   A conceptual framework.  Environmental
   Management ll(2):249-256.

Karr, J.R., K.D. Fausch, PJL. Angermeier, P.R. Yant, and IJ.
   Schlosser. 1986. Assessing Biological Integrity In Running
   Waters: A Method and Its Rationale. Special Publication 5.
   Champaign, IL: Illinois Natural History Survey.

Kelly, J.R., and MA. Harwell. 1990. Indicators of ecosystem
   recovery.  In  Recovery  of  Lotic   Communities  and
   Ecosystems  Following  Disturbances:    Theory  and
   Application, ed. JJX Yount and GJ. Niemi, 527-546.
   Special  analytical series of Environmental Management
   14(5): 515-572.

Landers, DJL, J.M. Eilers, D.F. Brakke, W.S. Overton, P.E.
   Keller, M.E. Silverstein, R.D. Schonbrod, R.E. Croe, R.A.
   Linthurst, J.M. Omernik, S.A. Teagure, and  EJ?. Meier.
   1987.  Characteristics of Lakes in the Western United
   States. Volume I: Population Descriptions and Physico-
   Chemical Relationships.  EPA600386054a.  Washington,
   DC:  U.S.  Environmental Protection Agency, Office of
   Research and Development.

Linthurst, R.A., D.H. Landers, J.M. Eilers, P.E. Kellar, D.F.
   Brakke, W.S. Overton, E.P. Meier, and R.E. Crow.  1986.
   Characteristics of Lakes in  the Eastern United States.
   Volume I: Population Descriptions and Physico-Chemical
   Relationships.  EPA600486007a. Washington, DC: U.S.
   Environmental Protection Agency, Office of Research and
   Development.

Morgan, M.G., B. Fischhoff, A.  Bostrom, L. Lave, and C.J.
   Atman.  1992.  Communicating  risk  to  the  public.
   Environmental Science and Technology 26(11): 2048-2056.

Mosteller, F., and J. W. Tukey.   1977.  Data Analysis and
   Regression: A Second Course in Statistics. Reading, MA:
   Addison-Wesley.

NADP (National Atmospheric Deposition Program).  1988.
   NADP/NTN  Annual  Data   Summary:  Precipitation
   Chemistry in the United States for 1987. Fort Collins, CO:
   NADP Coordinator's Office,  Natural Resource Ecology
   Laboratory, Colorado State University.

NAPAP (National Acid Precipitation Assessment Program).
   1991.1990 Integrated Assessment Report. Washington, DC:
   National Acid Precipitation Assessment Program, Office of
   the Director.

NEPA (National Environmental Policy Act). 1969.42 U.S.C.
   4321 et. seq.

Norton, B.G. 1991. Ecological health and sustainable resource
   management In Ecological Economics: The Science and
   Management of Sustainability, ed. R. Costanza, 102-117.
   New York, NY: Columbia University Press.

NRC  (National Research Council). 1983. Risk Assessment in
   the  Federal  Government:  Managing  the  Process.
   Washington, DC: National Academy Press.

	.  1984.   Acid  Deposition:    Processes  of Lake
   Acidification. Washington, DC: National Research Council,
   Panel on Processes of Lake Acidification, Environmental
   Studies  Board,  Commission  on  Physical  Sciences,
   Mathematics, and Resources.  (Available  from NTIS  as
   PB84-216175, 11 p.)
  	.   1986.   Acid  Deposition   Long-Term
   Washington, DC: National Academy Press.
Trends.
                                                       40

-------
                                                             The Environmental Monitoring and Assessment Program
National Research Council (NRC) continued.
	. 1990. Managing Troubled Waters: Role of Marine
   Environmental Monitoring.  Washington,  DC: National
   Academy Press.
	. 1991. Environmental Epidemiology: Public Health and
   Hazardous Wastes.  Washington, DC: National Academy
   Press.

Odum, EP.  1959.  Fundamentals of Ecology. 2nd Edition.
   Philadelphia, PA: W.B. Saunders Co.

Olsen, A.R., ed. 1992. The Indicator Development Strategy for
   the Environmental Monitoring and Assessment Program.
   EPA600391023.  Corvallis, OR:  U.S.   Environmental
   Protection Agency, Environmental Research Laboratory.

OMB (Office of Management and Budget). 1974. Standard
   Federal Regions. Circular A-105. Washington, DC:  Office
   of Management and Budget. (April 4, 1974)

O'Neill, R.V., D.L. DeAngelis, J.B. Waide, and T.FJL Allen.
   1986. A Hierarchical Concept of Ecosystems. Princeton, NJ:
   Princeton University Press.

OTA (Office of Technology Assessment). 1987. Technologies
   to Maintain Biological Diversity. OTA-F-330 (contains
   OTA-F-331).  Washington, DC:  U.S Congress, Office of
   Technology  Assessment.   (Available   from  NITS   as
   PB87204494.)

ORB (Oversight Review Board). 1991. The Experience and
   Legacy of NAPAP.  Washington,  DC: National  Acid
   Precipitation Assessment Program.

QAMS (Quality Management Assurance Staff). 1993. Glossary
   of Quality  Assurance Terms.  Washington,  DC: U.S.
   Environmental Protection Agency, Office of Research and
   Development

Posner, M.L  1973. Cognition: An Introduction. Glenview, IL:
   Scott Foresman Publication.

RAF (Risk  Assessment Forum).   1992.  Framework for
   Ecological Risk Assessment. EPA630R92001. Washington,
   DC:    U.S.  Environmental  Protection  Agency,  Risk
   Assessment Forum.

Rapport, D.  J.  1992. Environmental assessment at  the
   ecosystem level: A  perspective.   Journal  of Aquatic
   Ecosystem Health 1:15-24.

Reining, P., ed. 1978. Handbook on Desertification Indicators.
   Washington,  DC:    American   Association  for   the
   Advancement of Science.
Roberts, L.  1990.  Counting  on science at EPA. Science
   249:616-618.
Rochelle, B.R., and M.R. Church. 1987. Regional patterns of
   sulfur retention in watersheds of the eastern U.S. Water,
   Air, Soil Pollution 36:61-73.

Rothman, KJ.  1986.  Modern Epidemiology. Boston, MA:
   Little, Brown and Company.

Rubin, E. S., L. B. Lave, and M. G. Morgan. 1992. Keeping
   climate  research  relevant    Issues  in  Science  and
   Technology Winter 1991-92:47-55.

Sampson, R.N., and D. Hair. 1990. Natural Resources for the
   21st Century. Washington, DC: Island Press.

Sala, O.E., WJ. Parton, L.A. Joyce, and W.K. Lauenroth.
   1988. Primary production of the central grassland region
   of the United States. Ecology 69:40-45.

SAB (Science Advisory Board). 1988. Future Risk: Research
   Strategies of the 1990's. SAB-EC-88-040. Washington, DC:
   U.S. Environmental Protection Agency.
	—. 1990. Reducing Risk: Setting Priorities and Strategies
   for  Environmental  Protection.   SAB-EC-90-021.
   Washington, DC: U.S. Environmental Protection Agency.

Schindler, D.W. 1988. The effects of acid rain on freshwater
   ecosystems. Science 239:149-157.

Schnaiberg, A. 1980. The Environment:  From Surplus to
   Scarcity. New York, NY: Oxford University Press.

SPA  (Science  and  Policy Associates). 1992.   Report of
   Findings:   Joint Climate Project  to  Address Decision
   Maker's Uncertainties.  TR-100772. Pleasant Hill,  CA:
   Electric Power Research Institute (EPRI).

Streets, D.G.  1989.  Integrated assessment:  Missing link in
   the  acid  rain  debate?  Environmental  Management
   13(4):393-399.

Sullivan, TJ., J.M. Eilers, MH. Church, D J. Blick, Jr., K.N.
   Eshleman,  D.H.  Landers, and M.S. DeHaan. 1988.
   Atmospheric  wet  sulphate deposition  and  lakewater
   chemistry. Nature 331:607-609.

Suter, G.W.  1989.  Ecological  endpoints.  In  Ecological
   Assessments of Hazardous Waste Sites: A Field  and
   Laboratory Reference Document, eds. W. Warren-Hicks,
   B.R.  Parkhurst and S.S. Baker, Jr., 2-1 through 2-28.
   EPA600389013. Washington,  DC:   U.S.  Environmental
   Protection Agency.

	.  1990.    Endpoints  for  regional  ecological  risk
   assessments. Environmental Management 14:9-23.
                                                      41

-------
Assessment Framework
Suter continued.
	. 1993. Ecological Risk Assessment. Ann Arbor, MI:
   Lewis Publishers.

Suter, G.W., L.W.  Barnthouse, and R.V. O'Neill.  1987.
   Treatment of risk in environmental  impact assessment.
   Environmental Management 11:295-303.

Thornton, K.W. 1993. Effects of acidic deposition on aquatic
   ecosystems—a regional problem. In A Review of Ecological
   Assessment   Case  Studies  from  a  Risk  Assessment
   Perspective, W. Van der Schalie, R. Landy, and C. Menzie,
   eds.,  n.p.    EPA630R925.  Washington,  DC:  U.S.
   Environmental Protection Agency, Risk Assessment Forum.

UNEP (United  Nations  Environmental Programme).  1992.
   World Atlas of Desertification. London: Edward Arnold.

USDAFS (U.S. Department of Agriculture Forest Service).
   1989. Interim Resource Inventory Glossary. Washington,
   DC:  U.S. Government Printing Office.

Waide, J.B., K.W. Thornton,  G.E. Saul, and CM. Knapp.
   l992.Nominal/subnominal classification: Issues associated
   with classifying and assessing the condition of ecological
   resources in the Environmental Monitoring and Assessment
   Program.  [EPA  in  review.]   Corvallis,  OR:  U.S.
   Environmental Protection Agency, Environmental Research
   Laboratory.

Webster, M.  1991. Ninth  New  Collegiate  Dictionary.
   Springfield, MA: Merriam-Webster, Inc.

Westman, WJB. 1985.  Ecology,  Impact Assessment and
   Environmental Planning. New York, NY: Academic Press.
                                                       42

-------
                                                              The Environmental Monitoring and Assessment Program
                                                     Index
abiotic 7, 14,15,31,32,34
accuracy 33, 36
acid deposition  31,40
agroecosystem  14, 31
ancillary data 20, 31
annual statistical summary  27, 31
area frame  34
arid ecosystem  16
assessment ii, iii, iv, 1-16, 18-20,24-37, 39-42
assessment endpoint 31, 35, 37
attribute  31-33,35,36
auxiliary data 9,20,31
bias 32,33,36
biodiversity  2, 14, 32, 38
biogeographic province 28
biotic 7, 14, 15, 20, 31, 32, 34,40
candidate indicator 34
c# 32, 33
changes 7, 8, 12, 15, 19, 20, 22-24, 35
characterization 2, 4, 5, 8, 9, 19, 32, 35, 37
classification  24, 32, 39,42
community iii, 9,14, 22, 23, 26, 27, 32-34
comparability 32, 33
completeness  32, 33
conceptual model 2, 5, 16, 17, 25, 32
condition  ii, 1, 3,  5, 7-9, 11-16, 18-20, 23, 24, 26-28,
        30-32, 34-38,42
condition indicator 15,  19, 23, 31, 32, 34,35, 37
continuous 9,31-33,35
cross-cutting group 9, 32, 37, 38
cumulative distribution  19, 32, 33
data quality  32-34, 36
data quality indicators 33
data quality objective (DQO) 33
discrete 31,33,37
domain 33,37
environmental assessment 2, 33,40, 41
estuary  33
forest 8, 14, 19, 27, 28, 32, 34,40,42
fragmented 14,33,34
frame 27,34,37
geographic information system (GIS) 34
Great Lakes  8, 9, 14, 34, 36, 37
grid 29,34
grid, triangular  34
habitat 2, 33, 34
index 15,20,34,43
indicator 8, 9, 15,16,19,20,23,27, 31, 32, 34-37,41
indicator development 8, 9, 27, 32, 34,41
integration  iv, 8, 9, 34
lake 14, 15, 20, 32-35, 37-40
landscape ii, 8, 9, 14, 20, 27, 32, 35, 36, 38, 39
landscape characterization 8, 9, 32, 35
landscape ecology 35, 39
marginal condition 24, 35
measurement ii, 15,16, 25, 31, 32, 35
measurement endpoint 31, 35
modeling  iii, 25, 35, 37
monitoring ii, iii, iv, 1-3, 5, 7-9, 11,  12, 14, 18-20, 25,
        27-31, 34-37, 39-42
National Academy of Sciences (NAS) 35, 36
National Research Council (NRC)  3, 4, 7,  10, 20, 21,
        35, 36,40,41
nominal  13, 19, 24,26,27, 31, 35, 37, 42
Ecological Effects Committee  (See Science Advisory
        Board.)
ecological health (ecosystem health)  2, (4, 20)
ecological risk assessment ii, iii, 2-5,25,30,33, 37,41,
        42
ecology ii, 32,  33, 35, 39-42
ecosystem 4,14,16, 20, 32, 33, 36, 39-41
entire 9,20, 33, 34, 38
environment  1, 2, 7, 12, 15, 25, 28, 31-35, 37-41
Office of Modeling, Monitoring Systems, and Quality
        Assurance (OMMSQA) 35, 37
parameter 25, 36
pattern 2,35,36
peer review  27,  35-37
population  1, 8,  14, 20-24, 31-34, 36-38, 40
precision 33, 34, 36
                                                       43

-------
 Assessment Framework	

QA/QC 19,25,36
quality assessment  36
quality assurance (QA)  iii, 8, 9, 19, 20, 25, 31, 32,
        35-37,41
quality control (QC)  19, 25, 36
recovery 22,37,40
reference condition 8, 36, 37
region ii, 9, 13, 24, 27,28, 30, 32, 33, 36, 37,41
relation iii, 8,24, 36,37
representativeness 32, 33, 36
resource ii, 1, 5,7-9,12-16,18-20, 24, 27-38,40,42
resource class 16,19, 37
resource domain  37
resource group 8,16, 24, 32, 35, 37, 38
resource unit  31,32,37
risk  ii, iii, 2-6, 9, 13, 18, 25,29-33, 37, 39-42
risk assessment  ii, iii, 2-6, 9, 18,25, 30-33, 37, 40-42
risk characterization 2,4, 5, 37
risk communication ii, 25, 37
risk management ii, 37, 39
sample  32,33,37
sampling strategy  34, 37
sampling unit 31,34,37
Science Advisory Board (SAB) 2, 36, 37,41
status 2,7-10,14, 16, 19,20, 24, 27, 28, 32, 37-39
stressor  1, 37
stressor indicator 15, 32, 34, 37
subnominal  19, 24, 26,27, 31, 35, 37,42
surface waters  8,9,21,24,35-37
target population 23, 38
technical coordinator (TC)  iv, 32, 38
technical director (TD)  iv, 37, 38
total quality management (TQM) 38
trends  ii, 2, 5,7-10,14-16, 19, 20, 24, 32, 35, 37-40
universe  34,36,38
value 1, 2,14,15, 19, 23, 24, 28, 31, 32, 38
watershed iv, 20-23, 38
wetlands 8, 9,14,15, 24, 33-39
                                                       44

-------
                        For Additional Information


To be put on the distribution list for EMAPs Monitor, a newsletter, send your name,
affiliation, and mailing address to:
         Dorothy Williams
         EMAP Monitor
         U.S. Environmental Protection Agency (G-72)
         26 West Martin Luther King Drive
         Cincinnati, OH 45268

For more Information on EMAP databases or information systems contact:
         Technical Coordinator
         EMAP-lnformation Management
         U.S. Environmental Protection Agency (Mail Code 8205)
         401 M Street, S.W.
         Washington, DC  10460
         (202) 260-3255
         FAX: (202) 260-4346

For more Information on EMAP assessments and reports contact:
         Technical Coordinator
         EMAP-Assessment and Reporting
         EMAP Center
         U.S. Environmental Protection Agency (MD-75)
         Research Triangle Park, NC 27711
         (919)541-0673

To order EPA brochures, posters, magazines, and non-technical reports, contact:
         U.S. Environmental Protection Agency
         Information Access Branch- Public Information Center (PIC)
         401 M Street, SW PM-211B
         Washington, DC 20460
         (202) 260-7751
         FAX: (202) 260-6257
         (Have the "EPA" number or title or subject of inquiry ready when ordering.)

To order EPA technical reports, Including EMAP technical reports or videos, contact:
         National Technical Information Service (NTIS)
         U.S. Department of Commerce
         Springfield, VA 22161
         NTIS QuikService (703) 487-4650 and ask for PR-846/827
         Orders: (800) 553-NTIS
         (Have the "PB" number ready when ordering.)

Numbers for EMAP  publications can be located  through the Government Printing
Office  and NTIS bibliographic databases as well as In the  EPA  Publications
Bibliography.
                                 45
                                             •if U.S. GOVERNMENT PRINTING OFFICE: 1994 - 5SO-OOI/S0395

-------

-------