EPA/620/R-94/022
March 1994
Environmental Monitoring
and Assessment Program
Indicator Development Strategy
Edited By
M. Craig Barber
Environmental Research Laboratory
U.S. Environmental Protection Agency
Athens, Georgia
EMAP Center
Environmental Monitoring and Assessment Program
Office of Research and development
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711
Printed on Recycled Paper
-------
Notice
The research described in this document was funded by the U.S. Environmental Protection
Agency. The document was prepared at the EPA Environmental Research Laboratory in Athens,
Georgia with technical contributions through contract #68-CO-0021 with Technical Resources, Inc.,
purchase order OBO 387 NTTA to Arizona State University, cooperative agreement CR817489 with
Indiana University, and contract #68-08-0006 with ManTech Environmental Technology, Inc.
Mention of trade names or commercial products does not constitute endorsement or recommendation
for use.
Abstract
This document outlines the strategy of the Environmental Monitoring and Assessment
Program (EMAP) for indicator development. This strategy consists of general approaches, criteria,
and procedures for selecting and evaluating indicators in the context of a long-term monitoring
program. In practice, EMAP's indicator development is a multiphase process to identify and then
evaluate indicators that can estimate the condition of ecological resources using synoptic survey
monitoring methods defined over large geographic areas. The strategy described in this document
outlines this process from the identification of potentially useful indicators through the adoption of a
set of core indicators for use in EMAP. This strategy is intended to promote internal consistency of
indicator development across ecological resources and to provide a basis for internal and external
review of EMAP indicators. The implementation and review of this process will continue throughout
the existence of EMAP.
Key words: indicators (biology), environmental indicators, ecological monitoring, environmental
monitoring, research, United States, environmental condition, U.S.EPA-EMAP.
Preferred citation:
Barber, M.C., ed. 1994. Environmental Monitoring and Assessment Program: Indicator Development
Strategy. EPA/620/R-94/XXX. Athens, GA: U.S. Environmental Protection Agency, Office of
Research and Development, Environmental Research Laboratory.
Acknowledgments
Many individuals contributed directly and indirectly to the development of this document.
Foremost among these are Mel Knapp, Dave Marmorek, Joan Baker, Kent Thornton, Jeff Klopatek,
and Don Charles who co-authored EMAP's original Indicator Development Strategy. That technical
report was revised by Tony Olsen and was used as the basis for the current document. Many of the
initial elements of EMAP's Indicator Development Strategy were developed during the EMAP
Indicator Strategy Development Workshop that was held in Las Vegas, Nevada, in June 1990. In
addition to the original authors, key people who participated in that workshop included: John Baker,
Steve Bromberg, Dean Carpenter, John Eaton, Chris Elvidge, Jerry Filbin, Sue Franson, Luis
Hernandez, Bruce Jones, Bill Kepner, Bev Law, Nancy Leibowitz, Chris Maser, Dan McKenzie, Jay
Messer, Tom Moser, Dave Mouat, Tony Olsen, Steve Paulsen, Jim Pollard, John Scott, WooIIcott
Smith, Louisa Squires, Renee Stang, Gary Turner, Steve Weisberg, and Jamie Wyant. In revising
the report, the editor benefitted from numerous technical discussions with many EMAP scientists
including Hal Kibby, Dan McKenzie, Tony Olsen, Eric Hyatt, Rick Linthurst, Dave Bradford, Walt
Heck, Lee Campbell, Bill Kepner, Sam Alexander, Dick Latimer, Kevin Summers, Steve Hedtke,
Steve Lozano, Phil Larsen, Steve Paulsen, Spence Peterson, and Bruce Jones. Technical editing
was provided by Cynthia B. Chapman ELS, ManTech Environmental Technology, Inc.
EMAP Indicator Development Strategy
ii
-------
Contents
1. Introduction , ... . 1
2. Background . , 3
2.1 EMAP Overview .'....'.- 3
2.2 Program Objectives 3
2.3 Classification of Ecological Resources and Assessment Regions . . . 4
2.4 Values and Assessment Questions as Program Foundations 5
2.5 Quantifying Resource Condition '. . 11
2.5.1 Ecological Indicators and Risk Assessment Endpoints 11
2.5.2 Developing Indicators for Ecological Monitoring 12
2.5.3 Resource Condition and Monitoring Design 15
3. Framework for Indicator Development .;...,. 17
3.1 Overview of EMAP's Indicator Development Strategy •..,.... 17
3.2 Coordinating the Development Process 19
3.2.1 Internal Integration . 19
3.2.2. External Integration 20
4. Indicator Selection 22
4.1 Identifying Environmental Values and Assessment Questions . 22
4.2 Identifying Expected Stressors 23
4.3 Identifying Potential Indicators 26
4.3.1 Developing Conceptual Models of Resources' Structure and Function ... 27
4.3.2 Selecting Research Indicators 34
5. Indicator Evaluation '. 36
5.1 Indicator Performance Criteria 36
5.2 Phases of Indicator Evaluation 38
5.2.1 Conceptual Considerations ; 42
5.2.2 Operational Monitoring Considerations • . 43
5.2.3 Statistical Evaluation of Indicators 43
5.2.4 Evaluation of Indicators for Resource Assessments . . . 44
5.2.4.1 Identification of Nominal-Subnominal Criteria . 46
5.2.4.2 Condition-Stressor Associations 49
5.2.4.3 Example Assessments 50
5.3 Evaluation Methods . . . 51
5.3.1 Existing Data and Desk-Top Studies 51
5.3.2 Pilot Research Projects 53
5.3.3 Regional Demonstration Projects . 56
5.4 Identification of Indicators for Implementation 57
...'...' . . . . . . ; '..:. ..:...: 59
6. Indicator Implementation 60
7. Indicator Reevaluation 61
7.1 Reevaluation Procedures . , 62
8. Integration Among Resource Groups : 64
8.1 Framework for Indicator Integration :.'•.. . . 64
8.2 Types of Indicators that Integrate Across Ecological Resources . 65
8.2.1 Indicators Linking Resource Groups 66
EMAP Indicator Development Strategy
iii
-------
8.2.2 Indicators Shared by Resource Groups 66
8.3 Use of Conceptual Models to Facilitate Integration 67
8.4 Coordination of Indicator Development Among Resource Groups 67
8.5 Problems Associated with Differences in Spatial and Temporal Scales 68
9. Concluding Remarks . 70
9.1 Planned Reviews 70
9.2 Evolving Process 70
10. References 72
List of Figures
Figure 2-1. Standard Federal Regions as defined by OMB Circular A-105 '. 6
Figure 2-2. Cumulative distribution of an indicator's score over the resource population.
The proportion of the population less than or equal to a particular score can
be determined with the associated confidence interval. 9
Rgure 2-3. Conceptual model of the estuarine ecosystem. Solid lines indicate material
flows and dashed lines indicate interaction. Redrawn from Holland (1990) 14
Rgure 4-1. Conceptual model for the trophic structure of arid ecosystems. Redrawn from
Evenari et at. (1986) 29
Figure 4-2. Conceptual models for the trophic structure of the soil community in
agroecosystems. Adapted from Moore and de Ruiter (1991). 30
Figure 4-3. Conceptual model for the vegetative succession in arid ecosystems. Adapted
from Graver and Musick (1990) 31
Figure 4-4. Conceptual model for functional biodiversity of macroinvertebrates in stream
as a function of stream order and associated landscape. Redrawn from
Minshali et al. (1985) 32
Figure 5-1. Illustration of the relationship between power and magnitude of trends
detectable and years of monitoring. From left to right, curves are for trends of
approximately 2%/yr, 1.5%/yr, 1%/yr, and 0.5%/yr 52
Figure 5-2. Example of a pilot research project design for evaluating indicator
responsiveness to major stressors in Virginian Province estuaries (Holland
1990) 55
Figure 5-3. Cumulative distribution for Index of Biotic Integrity in streams in Ohio during
four months of 1986 (after Paulsen et al. 1991) 58
List of Tables
Table 4-1. Environmental Values Selected by Different EMAP Resource Groups 24
Table 4-2. Definitions of Selected Environmental Values by EMAP Resource Monitoring
and Research Groups. These values reflect Resource Group thinking as of
early 1993. Definitions have evolved since that time 25
Table 4-3. Conceptual model for the relationships between fish assemblages and trophic
conditions in the Great Lakes. Adapted from Ryder and Kerr (1978) 33
Table 5-1. Indicator Evaluation Criteria 37
Table 5-2. Example of the Early Stages of an Indicator's Evaluation 39
Table 5-3. Example of the Later Stages of an Indicator's Evaluation 40
Table 5-4. Indicator Status Sheet that Tracks Moving Condition Indicators from Research
to Implementation Mode 41
EMAP Indicator Development Strategy
iv
-------
Table 5-5. Summary of Estimates of Components of Variance for Secchi Disk
Transparency (SD), Chlorophyll-a (Chl-a), and Total Phosphorus (TP) Derived
from the Vermont Lake Monitoring Database 45
Table 5-6. Great Lakes Resource Group examples of nominal - subnominal criteria
(ranges) established for the Laurentian Great Lakes 48
EMAP Indicator Development Strategy
v
-------
-------
1. Introduction
In 1989, the U.S. Environmental Protection Agency's (EPA) Office of Research and Development
initiated the Environmental Monitoring and Assessment Program (EMAP), an interagency research,
monitoring, and assessment program. When fully implemented, EMAP will be an integrated,
multi-resource program that will estimate the condition of the Nation's ecological resources at various
geographic scales over long periods of time and will provide possible explanations for regional and
national trends in these estimates (Messer et al. 1991). EMAP has been designed to provide
information needed to conduct "top-down" or effects-driven risk assessments (Messer 1990). In
"top-down" risk assessments, the observation of an effect stimulates efforts to identify plausible
stressors that might have caused the effect by focusing on trends in resource condition and seeking
associations between indicators of resource condition and stress. This approach is designed to
detect cumulative impacts of natural and anthropogenic influences on the condition of ecological
resources rather than effects caused by a limited set of individual stressors.
EMAP's success depends on its ability to characterize the condition of ecological resources.
Because of the intrinsic difficulty in defining what is meant by ecological "condition" of any ecological
resource at large, EMAP will monitor suites of environmental indicators that collectively describe the
condition of the resource of concern. Within EMAP, an indicator is defined to be any environmental
measurement that can be used to quantitatively estimate the condition of ecological resources, the
magnitude of stress, the exposure of biological components to stress, or the amount of change in
condition. I
This document outlines EMAP's strategy for indicator development. This strategy presents general
approaches, criteria, and procedures for selecting and evaluating indicators in the context of a long-
term monitoring program. EMAP's indicator development is a multiphase process to identify and :
evaluate indicators that can estimate the condition of ecological resources defined over large
geographic areas using synoptic survey monitoring methods. The procedures described in this
document outline this process from the identification of potentially useful indicators (research
indicators) through the adoption of a set of core indicators for use in a national monitoring program.
This strategy is intended to promote a consistent approach to indicator development across
ecological resources and to provide a basis for internal and external review of EMAP indicators.
The implementation of the strategy and refinement of the process will continue throughout the
existence of EMAP.
EMAP Indicator Development Strategy
1
-------
Although this document focuses on the development and use of indicators within EMAP, many of the
issues discussed and the general strategy have broader applications. Detailed, process-oriented
research on indicators will be conducted by other research programs; ongoing and planned
monitoring programs will be collecting information on indicators, and adding new indicators in their
monitoring networks. EMAP's objectives willbe best served through a close linking of the EMAP
indicator development process to these ongoing and planned research and monitoring programs. As
these programs mature, EMAP's indicator development process likewise will evolve and improve, and
this strategy will be revised and updated.
This document is a revision of two previous technical reports (Knapp et al. 1991; Olsen 1992). Much
of the text is taken directly from these reports, portions of the text are slight modifications, and new
text has been added to reflect EMAP's current stage in indicator development. Some of the
background material presented in this report relies on Hunsaker and Carpenter (1990) and Knapp et
al. (1991). In some cases, text from these sources have been slightly modified to reflect recent
changes in EMAP or to alter its emphasis to address indicator development issues. The document is
organized as follows
Section 2 provides background on EMAP and the role of indicators in ecological
monitoring.
Section 3 presents an overview of the indicator development strategy.
• Section 4 discusses procedures for identifying potential EMAP indicators.
• Section 5 discusses procedures for evaluating potential EMAP indicators.
• Section 6 discusses procedures for implementing core EMAP indicators.
• Section 7 discusses procedures for periodic review of core EMAP indicators.
Section 8 discusses procedures for coordination and integration of indicator
development across resource categories.
• Section 9 provides concluding remarks on the strategy.
Section 10 lists the references cited.
For additional information regarding the use of indicators in EMAP resource groups, readers should
refer to the reports and documentation from each EMAP resource group.
EMAP Indicator Development Strategy
2
-------
2. Background
2.1 EMAP Overview
In 1988, EPA's Science Advisory Board recommended implementing a program to monitor ecological
status and trends for identifying emerging environmental problems before they reach crisis
proportions. In the following year, the Administrator of EPA established an Agency priority for the
1990s to confirm that the Nation's annual expenditure on environmental issues is producing
significant results toward maintaining and improving environmental quality (Reilly 1989). In an effort
to identify environmental problems before they become irreversible and to allow the evaluation of
regional and national progress toward achieving environmental goals, EPA's Office of Research and
Development (ORD) began planning the Environmental Monitoring and Assessment Program
(EMAP). Initiated in 1989, EMAP was created to monitor and assess the condition of the Nation's
ecological resources, thereby contributing to decisions on environmental protection and management.
Consequently, EMAP represents a critical element in the overall development of EPA's Ecological
Risk Assessment methodologies.
2.2 Program Objectives ,
EMAP is designed to provide information needed to quantify the current status and trends in the
condition of the Nation's ecological resources and information needed to diagnose why those
conditions might exist. Several key questions that have guided the development of EMAP's specific
programmatic objectives include:
What is the current extent of our ecological resources and how are
they distributed geographically?
What proportions of the resources are currently in acceptable
ecological condition? ;
What proportions are degrading or improving, in what regions, and at
what rates?
Are these changes correlated with patterns and trends in
environmental stresses?
Are adversely affected resources improving in response to control and
mitigation programs?
EMAP Indicator Development Strategy
3
-------
Based on these questions, EMAP designed an interdisciplinary research, monitoring, and
assessment program with the following objectives
Estimate the current status, trends, and changes in selected
indicators of the condition of the Nation's ecological resources on a
regional basis with known confidence.
Estimate the geographical coverage and extent of the Nation's
ecological resources with known confidence.
Seek associations between selected indicators of natural and
anthropogenic stresses and indicators of condition of ecological
resources.
Provide annual statistical summaries and periodic assessments of the
Nation's ecological resources.
To meet these objectives, EMAP must develop indicators of the condition of ecological resources and
of natural and anthropogenic stressors influencing this condition. The indicators must be compatible
with the monitoring design to permit quantitative and unbiased estimates of ecological status and
trends, analysis of associations between indicators of ecological condition and environmental stress,
and sufficient flexibility to accommodate sampling of multiple resource types.
2.3 Classification of Ecological Resources and Assessment Regions
EMAP's programmatic goal is to monitor and assess the condition of the Nation's natural resources,
thereby contributing to decisions on environmental protection and management. To do so, EMAP will
provide regional and national assessments of the condition and extent of ecological resources that
are defined hierarchically based on specific assessment needs. In most cases, the condition of
specific ecosystems or landscapes will be the object of concern. In some cases, however, the
condition of specific populations or communities also could be the focus. At the lowest level of
resolution, EMAP partitions all of the Nation's ecological resources into seven broad ecological
resource categories, i.e., agroecosystems, arid ecosystems, estuaries, forests, the Great Lakes,
surface waters (lakes and streams), and wetlands. These resource categories are, in essence,
aggregations of similar ecosystem types. EMAP also recognizes landscapes, i.e., heterogeneous
land areas composed of clusters of interacting ecosystems that are repeated in similar form
throughout the area, as a resource. Each resource category, in turn, can be subdivided into distinct
resource classes (e.g., oak-hickory forest). Resource classes can be further subdivided, as needed,
EMAP Indicator Development Strategy
4
-------
into resource subclasses that may correspond to specific community assemblages or to specific
structural components of the ecosystem of concern (e.g.; bird assemblages of oak-hickory forest).
The development, monitoring, assessment, and reporting of indicators for each of these resource
categories is the responsibility of an EMAP resource group that is named after the resource
categories. For example, indicator development, monitoring, and assessment for agroecosystems is
the responsibility of the EMAP-Agroecosystems resource group.
The regions for which EMAP will estimate the condition and extent of ecological resources are
likewise hierarchically defined based on specific assessment needs. At a minimum, however, EMAP
is committed to report on the condition and extent of ecological resources within each of the 10
standard Federal regions defined by Circular A-105 (OMB 1974; see Figure 2-1). EMAP resource
groups, however, are not constrained to report only on these regional resources. In many instances,
EMAP data may convey more meaningful information to decision makers and environmental
professionals if they are reported for specific biogeographical regions within the standard Federal
regions. Decisions to report at such finer scales of resolution, and therefore to satisfy EMAP's data
quality objectives (DQOs), are the responsibility of each EMAP resource group.
2.4 Values and Assessment Questions as Program Foundations
In order for EMAP data to be useful to decision makers, EMAP's indicator development and
implementation must be designed to answer specific assessment questions that are related to values
that society bestows on the resources of concern. These values and assessment questions are
therefore the basis for EMAP's indicator development, monitoring, and assessment.
Identification of the values that society places on the Nation's ecological resources is the initial step
for developing EMAP indicators. The word, value, has multiple definitions including the monetary
worth of something (i.e., its marketable price), the desired use of something or its aesthetic quality
(e.g., beauty). Value, as defined in EMAP, are those characteristics of the environment desired from
an ecological resource. These ecological resource values are those characteristics of the
environment that contribute to society's quality of life including the ability of the resource to provide
food, fiber, clean water and air, recreation, desired plant and animal communities, and an aesthetic
experience. Values for the ecological resource by one segment of society can conflict with values for
the resource desired by another societal segment. However, indicators selected for monitoring
should permit both segments to determine if the resource values are being achieved. In general,
EMAP Indicator Development Strategy
5
-------
Figure 2-1. Standard Federal Regions as defined by OMB Circular A-105.
EMAP Indicator Development Strategy
6
-------
EMAP recognizes three broad categories of environmental values, i.e., ecological structure and
function, consumptive uses, and non-consumptive uses.
After values of a resource have been identified, then specific assessment questions related to those
values must be formulated. This is an iterative process that requires an understanding of the
relationships among policy and science, assessment practicality and utility, and monitoring logistics.
Formulating the assessment questions that guide EMAP's programmatic endeavors is not a trivial
exercise. These questions must be of long-term interest to the regulatory and decision-making
community and to environmental scientists and ecologists. For many values pertaining to a
resource's consumptive and non-consumptive uses, assessment questions often can be formulated
to meet the needs of specific legislative mandates, agency policies, or management regulations. For
values related to ecological structure and function, however, it often will be necessary or desirable to
develop conceptual models of the resource of concern in order to formulate the most critical and
encompassing assessment questions associated with those values and the resource of interest.
EMAP is concerned with three basic types of assessment questions:
What is the current status of the resource?
What are the trends in the resource's status over time?
What are the associations between the resource's current condition
and the occurrence of selected stressors?
Assessment questions regarding the resource's status are the most basic. Such assessment
questions identify the assessment endpoints, i.e., formal expressions of the environmental value
that is be protected, with which EMAP will be concerned. These assessment endpoints must have
unambiguous operational definitions, be biologically and socially relevant, be accessible to
measurement, estimation or prediction, and be susceptible to environmental stressors of concern
(Suter 1990). A complete operational definition of an assessment endpoint must identify both a
subject and a characteristic of that subject. Generally speaking, assessment endpoints are the
specific ecological components (i.e., populations, species, guilds, communities, etc.) or processes
(e.g., primary and secondary production, nutrient assimilation and cycling, etc.) that can be related
conceptually to the environmental value of concern (EPA 1992). General formats for EMAP's
assessment questions regarding a resource's current status for a single period of time include:
What proportion (by area, length, or number) of resource R (i.e.,
subject) in region X is in condition C (i.e., characteristic) ?
EMAP Indicator Development Strategy
7
-------
• What proportion (by area, length, or number) of resource R (i.e.,
subject) in region X has attribute A (i.e., characteristic) ?
• What is the extent (i.e., characteristic by area, length, or number) of
resource R (i.e., subject) in region X ?
• What is the extent (i.e., characteristic by area, length, or number) of
resource R with attribute A (i.e., subject) in region X ?
The information used to answer these questions are EMAP's condition indicators (see Section 2.5.1).
It is important to note that the first two generic EMAP assessment questions above focus on the
cumulative distribution of condition indicators rather than measures of central tendency and variation
per se (Figure 2-2). This perspective provides more information than does the latter since
it allows for analysis not only of the central tendency of the endpoint but also of its limiting behavior
in its extremes. In many cases, it may be more important to know either that the proportion of a
resource in poor or subnominal condition is increasing or that the proportion of a resource in optimal
or minimally distributed condition is decreasing, rather than knowing the average condition of the
resource at large. Such information can be assessed with known confidence only by estimating the
cumulative distribution of the resource's condition indicators.
While condition indicators alone do not provide enough information to conduct a risk assessment,
summarizing the results as a cumulative distribution is consistent with the current risk assessment
framework (EPA 1992, EPA 1994). Suter (1990) asserted that the product of any risk assessment
should be either an estimated probability of a dichotomous assessment endpoint or the probability
that an assessment endpoint will be greater than or less than some scalar value. For example, in
the first case the risk assessment might estimate the probability for a species' regional extinction,
whereas for the latter case, the risk assessment might estimate the probability that the number of
fishless lakes within a region is greater than some value X. An assessment endpoint is therefore
inextricably linked to a formal mathematical expression describing the likelihood of that endpoint.
Because the proportion of a regional resource that exhibits a particular attribute is conceptually
equivalent to the probability that any individual resource within that region exhibits that characteristic,
it follows that answers to EMAP's assessment questions should provide environmental decision
makers with information consistent with current ecological risk assessment methodologies and
philosophy.
EMAP Indicator Development Strategy
8
-------
Figure 2-2. Cumulative distribution of an indicator's score over the resource population.
The proportion of the population less than or equal to a particular score can be
determined with the associated confidence interval.
1.0
0.75 —
a.
£
•5
jo
t
S.
0.50 —
0.25 -
Upper Confidence
Interval
30
Indicator Score
EMAP Indicator Development Strategy
9
-------
In most cases, the resource with which EMAP is most concerned is the entire resource category or
resource class, that is, resources at the ecosystem or community level of organization. Specific
examples of possible EMAP assessment questions at this scale of concern include:
What proportion of agroecosystem units in the Southeast has erosion
exceeding tolerable limits?
• What is the extent of riparian communities in southwestern arid
ecosystems?
• What proportion of forest land in the Pacific Northwest has visual
crown ratings that are expected to impact their nominal growth
potential?
• What proportion of the Great Lakes has forage fish populations
capable of supporting lake trout or walleye populations?
• What proportion of lakes in the Northeast is hypereutrophic?
• What proportion of wetlands in the prairie pothole region provides
habitat suitable for breeding or over-wintering waterfowl?
Many resources, however, are valued not only as a component of a larger ecological entity but also
as an entity in their own right. Consequently, there may be cases where the resource of concern is a
specific species assemblage or population rather than the ecosystem itself. Two specific examples
of potential EMAP assessment questions at this scale of concern might be
• What proportion of forest bird communities in the Southeast has its
expected biodiversity?
• What proportion of salmonid smolts in the Pacific Northwest is
maintaining expected growth rates?
Both of these assessment questions have their counterparts at the ecosystem or landscape scale of
resolution, for example:
What proportion (by area) of forests in the Southeast has its expected
avian biodiversity?
• What proportion (by length) of rivers and streams in the Pacific
Northwest supports salmonid smolts with expected growth rates?
EMAP Indicator Development Strategy
10
-------
It is extremely important to recognize, however, that although these assessment questions are
related, they are not identical. These questions specify very different statistical populations and may
require different indicators or design considerations. Thus, when confronted with such dichotomies,
EMAP resource groups must carefully consider the implications and ramifications of their decision to
deal with one question or the other.
2.5 Quantifying Resource Condition
2.5.1 Ecological Indicators and Risk Assessment Endpoints
EMAP defines an indicator to be any expression of the environment that quantitatively estimates the
condition of ecological resources, the magnitude of stress, the exposure of biological components to
stress, or the amount of change in condition.' In the past, EMAP has recognized many types of
indicators (Hunsaker and Carpenter 1990; Knapp et al. 1991; Olsen 1992). However, in a effort to
clarify and simplify its conceptual foundations and terminology, EMAP now considers only two
generic types of indicators: ',
• condition indicator— any characteristic of the environment that provides quantitative
information on the state of ecological resources and is conceptually tied to a value. If
necessary, condition indicators can be subdivided into
biotic indicator—any characteristic of the environment that estimates
the condition of a biological component of the resource, and
abiotic indicator—any characteristic of the environment that estimates
the condition of physical or chemical components of the resource.
Some indicators might be both condition and stress indicators (e.g.,
dissolved oxygen).
stressor indicator—any characteristic of the environment that is suspected to elicit a
change in the condition of an ecological resource.
EMAP's definition of a condition indicator is conceptually equivalent to what the ecological risk
assessment literature calls a measurement endpoint, i.e., "a measurable environmental
characteristic that is related to the valued characteristic chosen as the assessment endpoint" (Suter
1990, 10). For additional discussion, see RAF (1992) and Norton et al. (1992).
Using the term "indicator" rather than "endpoint," however, reenforces EMAP's program objective to
estimate the condition of the Nation's ecological resource's at multiple scales. No monitoring
EMAP Indicator Development Strategy
11
-------
program could possibly monitor and assess all aspects of a resource's condition. However, a
program could monitor and assess selected indicators of the resource's condition that address
specific environmental values and assessment questions that are also indicative of the condition of
other non-monitored resource components.
An indicator may be a single field measurement, an index based on multiple measurements, or the
output of a mathematical simulation model that has been parameterized using field measurements.
Karr et al. (1986) developed the Index of Biotic Integrity (IBI) to describe conditions in freshwater
streams. Properly developed, indices of resource condition can be compared more easily across
regions than can the individual measurements from which they are derived (Hughes 1989; Karr
1991). The aggregation process, however, can be highly controversial and mathematically complex.
Although results tend to be extremely dependent on the data and the aggregation procedures used
(Westman 1985), indices afford a valuable methodology for assessing resource condition.
Consequently, the development of indices for assessing resource condition will be pursued as an
important concept of EMAP's indicator development program. Simulation results from mathematical
models also can be used as condition indicators of the resource of concern. Barnthouse (1992)
discusses the utility of mathematical models to assess environmental concerns on local, regional and
global scales. Models that simulate important ecological processes such as primary or secondary
productivity, nutrient cycling, or inputs to other resource categories may be particularly useful in this
regard. Although presently EMAP has developed few indicators of this type, such indicators might
provide the most effective means to address the interactions between resource categories (e.g.,
sediment transport from agroecosystems to surface waters).
2.5.2 Developing Indicators for Ecological Monitoring
Having identified environmental values and assessment questions of concern, potential indicators are
identified using a variety, of conceptual models. These models may be based either on current
understanding of the structure and function of ecological resources in good condition or on the
responses and recuperative capacities of stressed resources. The resulting set of research
indicators then are evaluated systematically to confirm their relationships to EMAP assessment
questions and their utility toward meeting EMAP's programmatic objectives. The product of this
evaluation process is a set of core indicators that will be used for EMAP's routine monitoring and
assessment activities.
Developing and documenting conceptual models that describe a resource's structure and function is
an essential part of EMAP's indicator development process. Not only do such models provide a
EMAP Indicator Development Strategy
12
-------
focus to select indicators for previously defined assessment questions, but they also can be used to
identify assessment questions that might be overlooked if considering only the assessment needs of
current environmental regulation and management. Conceptual models should identify, at some level
of detail, all significant components and processes of a resource that contribute to its ecological
organization and operation. These models then can provide strategic frameworks to identify and
develop indicators. In particular, conceptual models can be used to
link indicators to their identified value(s),
identify gaps and redundancies in the indicators needed to address
assessment questions of concern, ',
identify indicators that can be used as surrogates for resource
elements of primary concern but which cannot be addressed
effectively due to data collection and measurement constraints, and
suggest schema to construct indices or other quantitative models for
evaluating a resource's condition (see Section 4.3.1).
i
Conceptual models also can provide frameworks for resource assessments and reporting by
identifying anticipated stressed and unstressed behaviors,
identifying anticipated relationships between stressor and condition
indicators, thus providing a data analysis strategy for diagnosing
plausible causes of subnominal conditions, and
identifying linkages to other ecological resources
Conceptual models are important representations of EMAP's scientific understanding of the
ecological resource for monitoring purposes. They should be descriptive and clearly demonstrate the
relationships between indicators and among condition indicators, assessment endpoints, and
resource stressors. An example of such a model developed for the estuarine environment is
presented in Figure 2-3.
In addition to estimating resource condition, it is also important to suggest or associate plausible
causes of poor or degrading conditions. Therefore, condition indicators must be selected because of
their likely responses to the major anthropogenic and natural stressors that are likely to impact the
resource. If information about these stressors is not being compiled by other programs, EMAP will
not necessarily develop stressor indicators for routine monitoring. EMAP will monitor selected
stressor indicators when a relationship between a specific condition and stressor is known or if a
EMAP Indicator Development Strategy
13
-------
I
p
1
1
•O
CD
I
ro
£
ro
I
I
o
CO
I
i~
|i
oC.
§1
"5 2
f!
II
Mo _
o g
"S g
1*
ED:
ra c
3 p
o g
C» flj
a*-*
=
1
q>
5T
o>
Q-
^
0)
I
LLJ
14
-------
testable hypothesis can be formulated. EMAP also may develop stressor indicators to confirm and
evaluate the responsiveness of research condition indicators with no intention of long-term
implementation. When stressor indicators are developed, EMAP will use them primarily to examine
their statistical association, on a regional scale, with indicators of resource condition. Although these
correlative analyses cannot establish causality, they can serve to narrow the range of probable
causes for observed regional patterns and trends in resource status through process of elimination
procedures. More detailed monitoring and research efforts to determine cause-and-effect
relationships can then be focused on those geographical areas, stressors, and resource classes of
greatest concern. ;
Indicator development and program implementation in each major resource category are the
responsibility of an EMAP resource group. The indicator development program of each EMAP
resource group will be tracked through the use of research plans, peer reviews, and interaction with
other resource groups. Each resource group will produce research and monitoring plans that are
subject to written peer review. These plans will describe the evidence and rationale for selecting
each research indicator, identify the specific types of information needed to evaluate these indicators,
and discuss important implications of design and assessment associated with each indicator. The
current status of all indicators and rationale for all decisions made during the development process
will be documented annually in an indicator data base. It is envisioned that this data base will be
available as part of the EMAP Information Management: system, and thus could be used to facilitate
rapid review of both the current state and the evolution of all indicators used by each of the resource
groups (see section 3.2.1). i
2.5.3 Resource Condition and Monitoring Design ,
EMAP's monitoring design focuses on implementing sampling designs that enable status and trend
estimates to be made for the resource classes or subclasses specified by an EMAP resource group.
Each resource class or subclass is a statistical population for which estimates of ecological condition
are made. Indicators of resource condition are monitored on sampling units that constitute a
probability sample of the population (i.e., resource class or subclass). These indicators are
developed in such a way that they will apply to entire resource classes or subclasses and will
produce population estimates of their ecological condition.
In order to provide reliable estimates of the status and trends in the condition of our Nation's
ecological resources, indicators must be monitored regionally for long periods of time (i.e., years to
decades) using a statistically designed network for probability-based sampling of explicitly defined
EMAP Indicator Development Strategy
15
-------
ecological resources (or populations in statistical sampling terminology). Moreover, indicators must
be monitored on sampling units that are consistent with their level of ecological organization. These
indicators also should provide meaningful information across the entire resource. EMAP resources
are sampled during index periods. An index period is a period of the year when measurement of an
indicator provides meaningful information about the condition of the resource. The index periods are
generally resource specific. For example, the lake index period is during late summer while the
stream index periods during the spring.
EMAP's sampling design builds on experience gained from previous surveys, incorporates their key
features, and uses a systematic grid (the EMAP grid) to ensure random selection and appropriate
sampling distribution (White et al. 1992). Other agencies, including the U.S. Departments of
Agriculture, Commerce, Energy, and Interior, have active, ongoing monitoring programs that address
some of EMAP's needs for broader monitoring data. EMAP will develop procedures for directly
integrating data and components from these monitoring programs into the EMAP grid, where the form
and nature of the data are appropriate. In cooperation with other agencies, EMAP will supplement
existing networks to fill critical data gaps. For additional detail on the EMAP design, see Overton et
al. (1990), Messer et al. (1991), White et al. (1992), and Urquhart et al. (1993).
EMAP Indicator Development Strategy
16
-------
3. Framework for Indicator Development
3.1 Overview of EMAP's Indicator Development Strategy
Although EMAP's original indicator development strategy recognized six distinct phases in the
development of indicators, activities outlined for these phases are usually iterative and do not
represent a step-by-step schedule for indicator development. Conceptually, all indicator development
activities are intended to
define the purpose, focus, and scope of EMAP's indicators,
• evaluate the utility of proposed research indicators,
• implement national monitoring of selected core indicators that will
satisfy EMAP's programmatic objectives, and
• reevaluate core indicators periodically and develop new indicators as
needed.
These four general categories of efforts simply are designated as indicator selection, evaluation,
implementation, and reevaluation.
During indicator selection, EMAP resource groups
identify the environmental values of their resource,
document or formulate assessment questions that motivated or
followed from identified values,
identify major stressors that are expected to be most important for
interpreting the status and trends of condition indicators in tentative
diagnostic assessments, '
• develop conceptual models that depict the resource's structure and
function and likely responses to stressors of concern, and
select indicators for research and evaluation.
Because EMAP's indicator development process is an assessment-driven activity, environmental
values and long-term assessment questions are the catalysts for the development of EMAP
indicators. EMAP's assessment questions must be of long term interest (e.g., 10- to 20-year time
frame) and be related to well defined environmental values.
EMAP Indicator Development Strategy
17
-------
During indicator evaluation, EMAP resource groups use best professional judgement, existing data,
and results of pilot research and regional demonstration projects to
evaluate the logistics of monitoring each indicator on regional and
national scales,
• characterize each indicator relative to its temporal and spatial
variability, ecological responsiveness, and overall interpretability,. ' -
• identify or develop nominal-subnominal criteria for condition indicators,
• prepare example statistical summaries and resource assessments,
• determine sampling density needed to estimate regional status and
trends and assess associations between'condition indicators and
selected stressors, and r
• select core indicators for program implementation.
' . • . • :;> .
In EMAP's original Indicator Development Strategy (Knapp et al. 1991), the primary focus of
indicator evaluation was the characterization of the indicator's logistic feasibility and statistical
behavior. Identification of nominal-marginal-subnominal condition thresholds was a task implicitly
relegated to the assessment process. If EMAP indicators are to add value to the decision-making
process, they must be able to address "so what" questions. For example, is the majority of the
resource in nominal, marginal or subnominal condition? Consequently, indicators must be evaluated
explicitly on their ability to address this fundamental assessment need.
After receiving approval through formal programmatic or peer review, core indicators are moved
forward to the indicator implementation stage of the development process. During this stage, EMAP
resource groups
monitor core indicators nationally, '
• prepare annual statistical summaries, and - '
• prepare periodic resource assessments in'conjunction with the'EMAP-
Assessment and Reporting coordination group:
During the final stage of the indicator development process (indicator reevaluation), EMAP resource
groups " • -
periodically reevaluate core indicator performance,
identify emerging assessment questions, and
EMAP-. Indicator Development Strategy
18
-------
conduct research on new indicators.
Although this strategy is written to address the development of individual indicators, it should be used
by each EMAP resource group to assess its full suite of indicators. Often multiple indicators may be
under development at the same time. The objective of this process is to develop a comprehensive
suite of indicators that complement each other and provide a clear picture of the status and trends in
the ecological condition of resources of interest through time. It is anticipated that, due to limited
financial or human resources, time, or scientific knowledge, EMAP resource groups will be
developing indicators at different rates.
3.2 Coordinating the Development Process
Integration among EMAP resource groups, State governments, Federal agencies, and
nongovernmental organizations is critical to the success of EMAP. Some of the primary concerns
related to internal and external integration are discussed below.
3.2.1 Internal Integration
Although implementation of the indicator development process is the primary responsibility of the
individual EMAP resource groups, to fully achieve EMAP's goals these activities must be integrated
and coordinated. Important issues that must be addressed include: 1) consistency in terminology
regarding environmental values, assessment questions, and conceptual models, 2) consistency in
collecting and applying stressor information, 3) inclusion of indicators that link or integrate resource
categories (e.g., landscape pattern, soil carbon, mobile wildlife, etc.), 4) use of common indicators,
compatible sampling, and analytical methods, and 5) co-locating sampling units for special studies.
Thus, it is essential that all EMAP resource groups formally communicate on a regular basis to
facilitate intergroup information exchange, to avoid redundant data collection efforts, and to improve
the amount of information available for each EMAP resource group to use in assessing status and
trends in ecological resource condition. Because program .implementation for each EMAP resource
group will proceed at different rates, intergroup integration will foster and improve the efficiency and
effectiveness of the overall program.
Each EMAP resource group will compile and add timely information about its indicator development
activities to the EMAP indicator database. The indicator database will store up-to-date information
about each indicator being evaluated or considered by each EMAP resource group. This data base
should contain retrievable records of the indicator status sheets (Table 5-4) that contains all pertinent
EMAP Indicator Development Strategy
19
-------
information about each indicator ever considered by the resource group, including at least the level of
detail presented in the appendices to EMAP's original report on ecological indicators (Hunsaker and
Carpenter 1990). Once an indicator is listed, it should never be deleted from the data base even
though it can be deleted from further consideration (status: rejected) or revised and refined during
later stages of indicator development.
As information is developed during the process of evaluating indicators in each of the four phases of
indicator development, it should be added promptly to the EMAP indicator data base. Data base
listings for each indicator should be initiated during the identification of research indicators and
updated during each of the subsequent phases of the indicator development process. Indicator
information entered into the data base should be kept simple and short to make the data base easy
to update. It is extremely important that each resource group's indicator data base be updated
frequently. Not all information categories accommodated by the data base will be evaluated during
the first phases of indicator development; therefore, not all data base entries will be completed while
updating records for each indicator. The appropriate results from each phase of the development
process should be entered into the data base following completion of that phase of development.
Although the design of the data base has not been completed, the structure and format of the
indicator data base will be consistent across all EMAP resource groups. This consistent structure will
allow for information exchange and synthesis.
3.2.2. External Integration
The Science Advisory Board's Ecological Monitoring Subcommittee has stressed the importance of
interagency coordination and integration to the success of EMAP (SAB 1990). Although integrating
results from other monitoring efforts into EMAP is both efficient and essential, interagency
cooperation should also include sharing information and expertise. For example, in addition to
valuable data that can be obtained from the USDA Forest Service's Forest Inventory and Analysis
program, Forest Service personnel can be active participants in the indicator development process
for the EMAP-Forests resource group. Similarly, the U.S. Fish and Wildlife Service, National
Biological Survey, the Bureau of Land Management, the Soil Conservation Service, the National
Oceanic and Atmospheric Administration, and many other Federal and State agencies can contribute
to the indicator development efforts of EMAP resource groups.
Formal arrangements, such as Memorandums of Understanding, should be established by EMAP
resource groups as necessary to assist the indicator development process and to ensure that EMAP
EMAP Indicator Development Strategy
20
-------
develops appropriate tools to monitor the condition of ecological resources. For example, it may be
appropriate for EMAP resource groups to obtain indicator information from sources that could include
State agencies (e.g., for regional resource management actions), other Federal agencies (e.g., USDA
for soil erosion rates and crop production data), and other EPA programs (e.g., Office of Water for
pollutant discharge information).
Interest in the use of indicators and indices extends beyond EMAP. The National Academy of
Sciences (1975) discussed the need for ecological indicators to monitor the environment (ecosystem
condition) and to judge the effectiveness of environmental protection programs. Indices of
ecosystem condition also can be used to
prioritize funding for dealing with environmental problems,
rank locations (regional comparisons),
conduct environmental trend analysis,
provide public information,
condense and focus scientific research, and
enforce standards (Ott 1978). ,
EMAP Indicator Development Strategy
21
-------
4. Indicator Selection
The first phase of the indicator development process establishes a framework for indicator
interpretation by identifying the environmental values, assessment questions, primary environmental
stressors, and critical ecosystem components and processes for each resource of concern. This
phase defines the context of EMAP's programmatic concerns and establishes the conceptual and
functional relationships among environmental values, assessment questions, condition indicators, and
selected indicators of stress.
4.1 Identifying Environmental Values and Assessment Questions
Ecological resources have both intrinsic and extrinsic worth ranging from the societal value placed on
the protection of pristine ecosystems and their inherent ecological structure and function (e.g.,
biodiversity and nutrient cycling) to the direct economic benefits derived from resource harvests (e.g.,
agricultural and timber production and commercial fisheries). The first step toward developing
indicators that allow meaningful assessments of the condition of the Nation's ecological resources is
to identify the major environmental values associated with each EMAP resource category.
Each of the resource groups, depending on their needs and experience, initially identified
environmental values from a variety of sources, which include:
• legislative mandates (e.g., Endangered Species Act, Wetlands
Protection Act, Clean Water Act, etc.),
• agency policies,
• management regulations,
• literature review - including articles in newspapers and popular
magazines as well as articles published in scientific journals,
• government reports,
• conceptual models,
• workshops to determine expert opinion,
EMAP management and guidance, and
• contributions of peer reviewers.
The process of identifying environmental values and associated assessment questions requires a
broad perspective regarding both the desired structure and function of resources of concern (as
expressed by resource managers, scientists, private industry, legislators, and the general public) and
EMAP Indicator Development Strategy
22
-------
stressors that are likely to impact the resource (which may occur on local to global spatial scales,
and over short- to long-term temporal scales). In some cases, environmental values can be identified
\
from existing regulatory assessment questions. In other cases, the values themselves motivate the
assessment questions on which EMAP resource groups will focus their indicator development. In
general, these values concern structural or functional aspects of the resource that contribute to
society's quality of life by providing food, fiber, clean water and air, aesthetic experience, recreation,
and desired animal and plant communities. Forest ecosystems, for example, are valued for timber
production, wildlife habitat, food, water storage, erosion control, and aesthetics. Wetland ecosystems
moderate downstream flooding, improve water quality, control erosion, and provide breeding, shelter,
and feeding habitat for aquatic and terrestrial wildlife.
Assessment questions that will drive a resource group's indicator development usually should be
formulated in terms of the resource's status since questions about associated trends follow
immediately. Specific examples of potential EMAP status questions already have been discussed in
Section 2.4. It is important that these questions are the actual assessment questions that are driving
the resource group's indicator development, and not simply questions that can be addressed using
EMAP data. For example, many fundamental EMAP assessment questions may involve multivariate
indicators of resource condition. Although assessment questions could be formulated for each of the
component variables of these indices, such questions per se are not driving the resource group's
indicator development process. Depending on the sophistication of the resource group's intended
clients, initial assessment questions may be reasonably generic. Such assessment questions,
however, eventually must be refined to identify assessment endpoints clearly. Table 4-1 summarizes
the environmental values that have been identified by each EMAP resource group to date, and
definitions for these values are provided in Table 4-2. These values show a strong overlap, reflecting
commonality in the perceptions of key issues among these groups. This commonality highlights the
need for integration and coordination among the EMAP resource groups to ensure that all important
information is collected efficiently and in a form that will facilitate completion of multiresource
assessments, as discussed in Section 8. This initial list of societal values will change over time and
be refined as EMAP evolves. In addition, values common to multiple resource groups will be
identified, which will facilitate the integration of information across resource groups.
4.2 Identifying Expected Stressors
In addition to monitoring status and trends of ecological condition, EMAP's data also will be used in a
diagnostic mode to identify plausible causes of adverse or subnominal conditions. When adverse
conditions are detected, the question that will inevitably follow is why? Although EMAP is not
EMAP Indicator Development Strategy
23 ,
-------
Table 4-1. Environmental Values Selected by Different EMAP Resource Groups.
EMAP Resource Group
Environmental Values3
Agroecosystems
Arid Ecosystems
Estuaries
Forests
Great Lakes
Surface Waters
Wetlands
Landscapes
Sustainability of agricultural commodities, quality of air, water
and soil, biological integrity
Biological integrity, harvestable productivity, sustainability,
aesthetics
Ecological integrity, harvestable productivity, fishability, trophic
condition, aesthetics
Biological integrity, harvestable productivity, sustainability,
aesthetics
Biological integrity, fishability, trophic condition, aesthetics
Biological integrity, fishability, trophic condition
Biological integrity, harvestable productivity, hydrologic
function, water quality improvement
Distribution of patterns of communities and ecosystems,
sustainability, biological integrity
* These values direct the selection of condition indicators.
EMAP Indicator Development Strategy
24
-------
Table 4-2. Definitions of Selected Environmental Values by EMAP Resource Monitoring and
Research Groups. These values reflect Resource Group thinking as of early
1993. Definitions have evolved since that time.
Environmental Values3 Definition (EMAP Resource Group)
Biological Integrity
Ecological Integrity
Sustainability
Air, Soil, and
Water Quality
Trophic Condition
Water Quality
Improvement
Hydrologic Function
Supply of
Commodities
Fishability
Harvestable
Productivity
Aesthetics
The composition, structure, and functional organization of the biotic components
of the ecosystem at the genetic, species, community, ecosystem, and landscape
levels of organization that constitutes an integrated and adaptive community of
organisms. (Agroecosystems, Arid Ecosystems)
The ability to support and maintain a balanced, integrated, adaptive community of
organisms having a species composition, diversity, and functional organization
comparable to that of natural habitats in the region. (Great Lakes)
"A balanced, integrated, adaptive community of organisms having a species
composition, diversity and functional organization comparable to that of natural
habitat in the region" (Karr and Dudley 1981). (Surface Waters, Wetlands)
A balanced, integrated, adaptive, ecosystem of communities, populations, and
organisms having a composition, diversity, functional organization, functional
attributes, and physical environments comparable to that of natural ecosystems in
the region. (Estuaries)
The ability to maintain the desired biological integrity. Components of
sustainability include the extent of the resource (at multiple levels of resolution),
successional dynamics in biological integrity, and maintenance of forcing
functions. (Arid Ecosystems)
The physical and chemical condition of these natural resources.
(Agroecosystems) .
A classification based on actual or potential primary production, dependent on the
ambient levels and turnover rates of nutrient salts. (Great Lakes)
Algal and macrophyte abundance and water clarity comparable to natural systems
of the region. (Surface Waters)
Ability to assimilate nutrients, trap sediments, or otherwise reduce downstream
pollutant loads. (Wetlands)
The natural water flow pattern necessary for the sustainability of the wetland and
its function, e.g., flood conveyance, water storage capacity, and shoreline
protection. (Wetlands)
The ability to produce adequate amounts of food and fiber for human needs of
sufficient quality to satisfy market standards. (Agroecosystems)
The presence of catchable fish that are safe to eat. (Estuaries, Great Lakes,
Surface Waters)
Quantity or quality of any service or product that the resource provides society
(e.g., commercial timber, wildlife, recreation, and food production). (Arid
Ecosystems, Estuaries, Forests, Wetlands)
Attributes that affect the public's perception of their environment. (Arid
Ecosystems) :
"These values direct the selection of condition indicators.
bSome resource groups have not yet adopted the definitions provided in this table.
EMAP Indicator Development Strategy
25
-------
designed as a cause and effect research program, it must anticipate how major stressors are likely to
affect the indicators that it will monitor (e.g., Rapport et al. 1985). The starting point for assembling
stressor information is the listing of the major problems and stressors currently impacting or
threatening the resource. Major environmental stressors that should be considered by EMAP
resource groups when developing indicators include:
reduction, loss, or fragmentation of critical habitat,
• reduced food resources due to pest control programs or other causes,
• introduction of exotic species (including domesticated livestock and
recreational wildlife, e.g., fishery stocking programs),
• global warming,
• alterations to regional hydrologic cycles.
• chemical pollution of air, soil/sediment, and water,
• increased sediment and nutrients loadings to surface waters (including
wetlands and estuaries), and
• over-harvesting regional flora or fauna.
Certain indicators reflect external stresses or pressures that affect ecological resource condition.
Generally, these indicators are measured or estimated by other programs or agencies rather than by
the EMAP resource groups. Collating and analyzing these auxiliary data also will be used to develop
indicators for EMAP; however, considerable effort may be needed to assemble these data in the
proper format. These stressor indicators are most often anthropogenic (e.g., pesticide applications,
human population densities, livestock grazing pressures, atmospheric deposition, emissions of
atmospheric pollutants, applications of fertilizers or other nutrients, numbers of fishing and hunting
permits, and numbers of discharge permits), but they also include natural forcing functions, such as
precipitation or solar radiation, that in turn may be affected by anthropogenic factors (e.g., global
climate change) or indirectly by the ecological resources themselves.
Explicitly defining potential stressors serves to increase the relevance of the selected condition
indicators to current and future environmental concerns.
4.3 Identifying Potential Indicators
The process of identifying potential condition indicators that can be conceptually and functionally
linked to the resource's environmental values requires a broad scientific perspective obtained through
both detailed literature reviews and interactions with scientists conducting relevant research. This is
EMAP Indicator Development Strategy
26
-------
a continuing process, however, since scientific and technological advances will generate new
research indicators or improve the feasibility of previously rejected indicators .(see Section 5.).
For consumptive and non-consumptive use values, the very process of formulating explicit
assessment questions often identifies the potential indicators with which the resource group should
be concerned. In terms of the risk assessment paradigm, this situation is equivalent to the
assessment endpoint and the measurement endpoint being the same. Consider, for example, the
following fisheries assessment questions
What proportion of lakes in the Northeast United, States have game fish
present (i.e., > 0)?
What proportion of streams and rivers in Pacific Northwest currently
support reproducing salmonid populations?
What proportion of lakes in the Northeast United States show evidence of
degraded fish habitat?
What proportion of the nearshore Great Lakes do not support game fish
populations whose median body size is greater than or equal to legal size
limits?
What proportion of the nearshore Great Lakes do not support forage fish
populations of acceptable quality to desired game fish?
What proportion of estuarine area in the Virginian Province demonstrate
high incidence (e.g., > 1%) of external fish pathologies?
What proportion of estuarine area in the Gulf of Mexico has fish with body
burdens of contaminant X that exceed FDA action limits?
For these particular questions, it is fairly obvious what appropriate condition indicators might be.
When identifying potential indicators for values related to ecological structure and function, however,
objective identification of potential condition indicators generally requires well-developed conceptual
models of the resource of concern.
4.3.1 Developing Conceptual Models of Resources' Structure and Function
Each resource group should identify or develop a suite of conceptual resource models to guide,
corroborate, and document their indicator selection for values pertaining to the" resource's ecological
structure and function; in some cases, these models even guide and document the formulation of
EMAP Indicator Development Strategy
27
-------
assessment questions. These models should identify the structural components of the resource, the
interactions between these components, inputs from and outputs to surrounding resources, and the
important factors and stressors that determine the resource's ecological operation and sustainability.
Conceptual resource models can be constructed at many scales, ranging from simple population or
community models to complex ecosystem models identifying full complements of ecosystem
structural and functional attributes. Each of these approaches may be useful, and should be
developed or reviewed as appropriate. Conceptual resource models can be formulated and .
documented in a variety of different formats including: "box and arrow" diagrams (Figures 4-1 and 4-
2), kinematic graphs (Figure 4-3), and graphic and matrix representations (Figure 4-4 and Table 4-3).
Components or processes that fulfill-key or central functions in these models obviously might be
better indicators of the resource's overall condition than those components or processes that occupy
peripheral positions. Figure 4-1 presents one possible conceptual model for identifying key indicators
(and assessment questions) regarding the ecological organization of arid ecosystems. Figure 4-2
displays one of the conceptual models that EMAP-Agroecosystems considered in deciding that
nematodes were an appropriate indicator of the biological integrity of agroecosystem soil community.
The predominance of nematodes in the structure and function of this model corroborates the decision
made by EMAP-Agroecosystems to' pursue evaluation of this indicator.
Conceptual resource models also should consider the temporal and spatial dynamics of the resource
at multiple scales because information from different scales can result in different conclusions about
resource condition (Wiens 1989). Developing such models is an extremely important exercise that is
required to substantiate the choice of a particular indicator. For example, annual wood increment
can be linked directly to forest productivity and can be incorporated into a conceptual model. Data
can be readily obtained at the temporal scale appropriate to EMAP. Soil microbial respiration, on the
other hand, is more difficult to link to forest productivity, and is fraught with interpretation problems at
differing temporal and spatial scales of interest in EMAP. Figure 4-3 displays a conceptual model of
successional dynamics that EMAP-Arid Ecosystems has used to organize its process for selecting
indicators of ecological sustainability and for expected trends in the resource's vegetative biodiversity.
On the other hand, Figure 4-4 presents a graphical conceptual model of the functional biodiversity of
aquatic macroinvertebrates as function of stream order and associated landscapes. Clearly, this
model would have significant ramifications to potential sample designs, data interpretation, and index
development. '•"•••- •
Conceptual resource models also should identify how major stressors of the resource are expected to
Impact its structure and function. For example, Table 4.3 presents a conceptual model for the Great
Lakes' fish communities in matrix format. The EMAP-Great Lakes group believes this matrix shows
EMAP Indicator Development Strategy
28
-------
Figure 4-1. Conceptual model for the trophic structure of arid ecosystems. Redrawn from
Evenari et al. (1986).
POOR RELIABLE RESOURCES
RICH EPISODIC RESOURCES
arido-acttv8 arthropod
predators (scorpions,
ants & spiders)
detrithrorous Invertebrates
(isapods, beetles, termitos,
snails)
detritus
large vertebrate predator
(raptors, snakes &
carnivorous mammals)
t
small vertebrate
omntvores (rodents
& lizards)
ephemeral & migratory
iruecthrores (birds etc.)
(ants)
lichens & algae
DEW & FOG
arido-acttve plants
(xerophytes & succulents)
ephemeral & migratory
folivorous insecte
(caterpillars & locust)
ephemaral plants
(annuals & geophytes)
TOPSOIL MOISTURE
RAIN
EMAP Indicator Development Strategy
29
-------
Figure 4-2. Conceptual models for the trophic structure of the soil community in
agroecosystems. Adapted from Moore and de Ruiter (1991).
EMAP Indicator Development Strategy
'; 30
-------
Figure 4-3. Conceptual model for the vegetative succession in arid ecosystems. Adapted
from Grover and Musick (1990).
Perennial Grass
i
t
Perennial
Grass/
Mesqulto
1
Mesqutta/
Perennial
Grasses
i
Coppice Dune
Formation with
Deflation zones
Aridlfloatton
Livestock Grazing
Seed source
Shrub Competition
Select Livestock Grazing
Fire Suppression
Rodent/Rabbit Activity
ecus Plant
Nutrient Distribution
Wind Erosion
Local/Regional Climate
Deep Coarse Soils No CaCO
Shallow Coarse Soils
CaCO near surface
EMAP Indicator Development Strategy
31
-------
I
ra
.CM O>
i
o
o
1
a
ra
$
£
to
I
o
CO
o —^
« «
S«-i
0
1
g
I
O ra
3'
§
CD
8
§ § 5 OJ
(%) indui aiBinoRjBd
.CM O>
co
-CM
o>
1
w
o>
a.
jo
LJU
32
-------
(0
0)
ra
O
I
c
w
o
S
•o
o
o
_o
Q.
2
4^
T3
re
(0
CO
CD
re
£
(0
I
a .
22
E
O 0.
C (Q
O T3
O <
co
•<*
JQ
re
"o
Q)
Q.
CO CD
CL >,
"0 CD
•i 5!
Q- >
'•§
3
•a
2
a.
CD
CD"
1
CO
CO
CO
E
0
ha
CD
S?-
1
*£•
•a i
O c
o fc
CD O
CO 0
I
E
o
O
CO
CL
c:
.O
'•*-•
•a
o
0
o
0.
2
1-
'c
"O C
*C "^
0 §
12
.0
r"
a.
2
0
.£?
2
-i—*
3
„
§
- ., **5
I 1 1 ^
o>.*: S3 o .co
CD O S -Q CO
1
1
T3
'CJ
CD
Q.
'E
0
"CO
CO
O
a.
2
0
O)
O
"3
o ™
-b 2
CD
to en -t;
.8 1 el
flllftl
a. o 73 O Q. Ill CD
0)
2
CD
T3
0
JZ
.D)
^
12^ "o
11
1 8
T3
2
CD
a.
.0
Q.
2
0
CO
CD
5
„
CO
centrarchid:
cyprinids
D)
,
D)
ic
2
1
T3
'c
'o.
o
'2
0)
Q.
!
2
1
8
.0
,0.
2
i3
"c
8
D)
T3
]c
Q.
O
Q.
2
^-i
2
CD
CL
>>
X
TO
.2
2
•4-*
CO
.»-«
CD
Q.
JO
I
Q
1
o
T3
LLJ
33
-------
how the resource's biodiversity and fishability is related to eutrophication, which is one of the major
stressors of Great Lakes ecosystems.
Developing conceptual models that describe a resources's ecological components is one of the most
important aspects of the indicator development process; it also may be one of the most difficult. The
process of developing models and of determining the degree of complexity and the model focus (e.g.,
ecological structure or function) is the responsibility of each EMAP resource group. Conceptual
models should serve as reference points both for identifying indicators needed to assess the
condition of ecological resources and for guiding data analyses and construction of multivariate
indicator indices.
4.3.2 Selecting Research Indicators
Indicators selected for development may be either indicators that were previously suspended pending
new findings or completely new indicators that will augment or substitute for existing indicators. New
indicators should consider recent advances in environmental sciences and monitoring technologies
(e.g., new methods of remote sensing) and consolidate insights gained through analysis of data
collected by EMAP and other research programs. Several approaches may be used to identify
research indicators for further evaluation and development. These include workshops, systematic
literature reviews to identify potential improvements to the current suite of indicators, attendance at
major conferences, solicitation of involvement of the scientific community through presentations and
published articles, and continued personal contact with leading scientists researching relevant topics.
Whatever the mechanism, the key to continual replenishment of indicators used in EMAP is active
and effective communication with the scientific community.
Periodic technical workshops should be conducted by each EMAP resource group to update their
suite of research indicators and to assess existing indicators for completeness with respect to the
environmental values, assessment questions, and conceptual resource models that have been
previously identified for their resource. Particular emphasis should be devoted to identifying gaps in
their suite of indicators and to generating ideas for new indicators. To facilitate effective workshops,
the Technical Director of each EMAP resource group should prepare information summarizing the
status of indicator development for distribution prior to the workshop. At a minimum, this information
should include: 1) EMAP's current indicator development strategy, 2) the resource group's current
research plan, 3) examples of results to date, and 4) the current research plans of other EMAP
resource groups working in similar media.
EMAP Indicator Development Strategy
.34
-------
Although some discrimination at this stage may be appropriate, it is better to identify too many
research indicators than to overlook potentially useful ones. It is not necessary to provide substantial
amounts of evidence about indicator behavior nor is it necessary to conduct peer reviews of the
selection process and documentation. Critical evaluation of selected indicators during the second
phase of the development process will eliminate inappropriate indicators or suspend their evaluation.
The main issue is whether each indicator appears to address environmental values and assessment
questions that have been identified by the resource groups and whether the indicator appears to be
quantifiable and applicable for a national monitoring program.
EMAP Indicator Development Strategy
35
-------
5. Indicator Evaluation
EMAP's indicator evaluation is a multistage process designed to compile and analyze a body of data
and information needed to select a set of core indicators for EMAP's national implementation from a
host of research indicators identified by the indicator formulation process. Each step in this process
involves two stages: 1) an assessment of the sufficiency of the available data to support the
indicator's evaluation and 2) screening the indicator based on the selection criteria. This process
results in one of three outcomes: 1) acceptance for consideration at the next stage of evaluation, 2)
temporary suspension of consideration due to insufficient data, technology, time or resources for
proper evaluation, or 3) rejection for failure to satisfy one or more of the selection criteria. The latter
two outcomes may lead to new approaches for collecting, synthesizing, and analyzing data. Many
research indicators are expected to be placed in a state of suspended evaluation, to be revived at
some future date when evidence, time, and resources are sufficient to thoroughly evaluate them.
During the initial stages of evaluation, an indicator will be rejected only if it fails certain critical criteria,
and the indicator's performance can not be expected to improve over the next decade. In contrast,
during the final stages of evaluation, it is likely that a much higher proportion of indicators will either
advance or be rejected, rather than suspended, because sufficient data regarding their overall
performance will be available to make a firm decision.
5.1 Indicator Performance Criteria
For EMAP's indicator development process to be scientifically defensible, each resource group must
use a consistent procedure and set of criteria to judge each potential indicator. The use of clearly
defined criteria not only increases the objectivity and consistency of indicator evaluations, but also
provides a framework for documenting EMAP's indicator evaluation process. Although certain
decisions made in the evaluation process will be subjective, the goal of the evaluation procedure is to
provide an appropriate amount of information so that an independent evaluation by peer reviewers
could follow the logic of the original decisions.
EMAP's indicator evaluation is guided by a set of criteria that initially were identified by Messer
(1990). Later the same year, these criteria were modified based on an EMAP Indicator Strategy
Workshop (Hunsaker and Carpenter 1990). These modifications were incorporated into EMAP's
initial indicator development strategy (Knapp et al. 1991; Olsen 1992). These criteria address a
variety of issues and are identified as being either critical or desirable with -respect to EMAP's overall
programmatic objectives (see Table 5-1). Kelly and Harwell (1990) proposed similar criteria for
indicators of ecosystem recovery. The type and amount of data needed to evaluate any specific
EMAP Indicator Development Strategy
36
-------
Table 5-1.
Indicator Evaluation Criteria
Criteria
Definition of Criteria
Essential Criteria
Unambiguously Interpretable
Ecologically responsive
Index period stability
Amendable to synoptic survey
High signal-to-noise ratio
Nominal-subnominal criteria
Minimal environmental impact
Desirable Criteria
Available method
Historical record
Retrospective
Anticipatory
Cost effective
New information
Relates unambiguously to a recognized environmental value or
assessment question and quantitatively conveys the same information for
most resource sampling units within a regional resource class.
Responds to stressors and to changes in the condition of the resource
across most pertinent habitats within a regional resource class.
Exhibits low measurement error and temporal variation during an index
period.
Can be quantified by synoptic monitoring or by cost-effective automated
monitoring.
Possesses sufficiently high signal strength (when compared to natural
annual or seasonal variation) to allow detection of ecologically significant
changes within a reasonable time frame.
Possesses documented or identifiable thresholds or patterns of trends
that identifies the nominal or subnominal condition of the resource.
Sampling produces minimal environmental impact.
Possesses a generally accepted and standardized measurement method
that can be applied on a regional scale.
Has existing historical data base or one can be generated from accessible
data sources.
Relates to past conditions by way of retrospective analyses.
Provides an early warning of widespread changes in ecological condition
or processes.
Has low incremental cost relative to its information.
Provides new information; does not merely duplicate data already
collected by cooperating agencies.
EMAP Indicator Development1 Strategy
37
-------
indicator with respect to these criteria vary greatly. In some cases, best professional judgement
based on theoretical and logistical considerations can be used to evaluate an indicator's
programmatic utility. In other cases, existing data from the literature and other sources can be used.
If such data are not available, EMAP resource groups will conduct pilot research and regional
demonstration projects to generate the information needed to evaluate the indicators in question.
In general, the critical criteria are those considered essential for satisfying EMAP's programmatic
objectives. The first two critical criteria (unambiguous interpretability and ecological responsiveness)
should be the initial focus of all indicator evaluations. Although these two criteria continue to be
important in the evaluation at subsequent phases, other critical criteria relating to the utility and
feasibility of sampling the indicator (index period stability, amenable to synoptic survey, low year-to-
year variation, minimal environmental impact) increase in importance. Tables 5-2 and 5-3 exemplify
the use of these critical criteria during the early and later stages of an indicator's evaluation. During
this evaluation process,'it is not crucial that each indicator satisfies each criterion completely; rather,
there should be a reason to believe that each criterion can be satisfied when the appropriate data
and models for detailed analyses are assembled in later stages of the evaluation process.
Indicators that fulfill some or all of the desirable criteria have obvious advantages over those that do
not. These advantages may include an improved assessment of associations between stresses and
ecological conditions (the third objective of EMAP), an increased timespan over which the indicator
can be quantified, higher information value per unit cost, greater ease of implementation, or special .
value for early detection of widespread ecological changes. Desirable criteria should be applied to
assist in distinguishing among alternative indicators for the same assessment question.
5.2 Phases of Indicator Evaluation
There are four types of evaluations that must be performed to evaluate a research indicator:
• evaluation of the indicator's conceptual soundness,
• evaluation of the feasibility of implementing the indicator in a routine monitoring and
assessment program (current and future),
• evaluation of the indicator's statistical behavior, and
evaluation of the indicator's utility in resource assessments.
Although all evaluations of EMAP indicators begin with a thorough evaluation of their conceptual
soundness (Section 5.2.1), the remaining phases of the process may be conducted either in the
sequence indicated above or with overlap among phases. For example, in pilot research projects
discussed below (Section 5.3.2), resource groups may simultaneously evaluate both the feasibility of
EMAP Indicator Development Strategy
38
-------
.C CD T3 .£ ^
Zl '-a S '.. to
(0
o
te
co).>^-.0c2 °
O
1
I
UI
to
ca
re
•s.
0)
1
"wESwwSScc^
sllgs&lgsil
period.
Small to
(O
E
Q.
C1
O
'co
•e
I
^—•
CO
0)
D.
_O
, a>
I?
£
re
ill
ii
1
li
g e
O
111
o m
g.co
CD
"5.
CO
2
'S en
'-5 .c
5T
o:
•
g
I
>. .
a)T3o
s
CD
CM
CO
1O
tO
39
-------
on
ra
&
t
*
o
(0
03
?
i
2
0)
01
I
CO
f
-s
II
Q -Q
a
I 8
8g£
•
^ °
•g £3
2 «» ~°
o
*=
<1)
• - ai
If l
llll
co"
Of
S -
w
: -o
I-Sl
O CO
£ to
.£ "o
j s
U}
c
c ^
is i -|
s as
.
.2ro^'jr!S^ta)(B:5o—
'
8
e
Q.
C
O
'eo
"8
U)
73
3
O)
0)
2
•~
0)
Q.
-------
Table 5-4. Indicator Status Sheet that Tracks Moving Condition Indicators from Research
to Implementation Mode.
Name of indicator:
Developed by: EMAP resource group •
Status: Core, Research, Suspended, or Rejected
Date:
Formulation Criteria '
Associated environmental value(s)
Associated assessment questions
• Principal stressors
Conceptual link(s) to value(s); for values related to
ecological structure and function, conceptual
resource models are available
References: "•"'*.'••••
Research Evaluation Criteria - Pilot Level ' • •"
Me.thods available and fairly standardized
Measurable on all sample units
• Responsive, to stressors or to changes in the
condition of the resource at large
• Literature reviewed for data appropriate for
characterizing variance components
Analysis of intra-annual variety sufficient to define
index period
Analysis of within and among site variability
sufficient to define the number of sample units.
References:
Research Evaluation Criteria - Demonstration Level
• ' Statistical power analyses for trend detection
• Correlates with regional patterns of stressors
• Analysis of logistical and economical feasibility for
regional monitoring
Analysis within the context of annual statistical
report and regional assessment
References:
Core Implementation Criteria
• Quantitative relationship to assessment endpoints
or values established
•• Correlates with regional patterns of stressors and
can be used for diagnostic assessments
Responds quantitatively to changes in the condition
of the resource category at large
• Nominal-subnominal criteria established
EMAP DQO's satisfied
Standardized methods established
References:
' References may be direct documentation, internal EMAP reports, or published papers that indicate how and when each
criterion is satisfied.
EMAP Indicator' Development Strategy
41 !
-------
indicators and important aspects of their statistical variability (e.g., within- and between-site variance).
Resource groups should document the current status of each indicator using the indicator status
sheet presented in Table 5-4.
5.2.1 Conceptual Considerations
The most important criteria in evaluating a potential EMAP indicator is its ability to provide
information that addresses environmental values and assessment questions identified during the
indicator selection phase. Each potential indicator must be evaluated in terms of how it contributes in
forming a complete suite of indicators that can address all environmental values and assessment
questions that have been identified by the resource group. Particular attention must be paid to avoid
developing multiple indicators that provide redundant information on the same environmental values
and assessment questions. Multiple indicators for the same fundamental assessment question
should be evaluated or developed only when there is no a priori reason to assume that one indicator
will perform better than one or more possible alternative indicators or when a single indicator is not
expected to be responsive to all major stressors that might impact the resource. Resource groups
also should evaluate potential indicators in terms of how they link or integrate with the needs of other
EMAP resource groups.
For an indicator to be useful in regional assessments, it must convey the same quantitative
information concerning the condition of the resource regardless of where it was measured within the
region. For example, if a particular score for an indicator of forest primary productivity corresponds
to a subnominal condition in one locality, then, for the purpose of interpreting the indicator's regional
cumulative distribution, that score should correspond to a subnominal condition elsewhere in the
region. To ensure this characteristic, indicators often will need to be normalized or otherwise
adjusted with respect to underlying environmental variables that determine limits for their expected
values. For example, forests that occur at different elevations and latitudes and on different soil
types generally would be expected to demonstrate different levels of nominal primary productivity. If
such an indicator is not adjusted for this expectation, its regional cumulative distribution would be of
limited value for assessing the condition of forest resources. Resource groups should identify and
evaluate these normalizing variables as an integral part of evaluating an indicator's conceptual
soundness.
These considerations can be evaluated best by critical analysis of each resource group's conceptual
models and of those conceptual models that link several resource classes together. Decisions made
EMAP Indicator Development Strategy
42
-------
during this phase are based on best professional judgment; no field activities or data analyses are
needed for this stage of the evaluation process.
5.2.2 Operational Monitoring Considerations
Having confirmed the conceptual linkage between a potential indicator and one or more of the its
formal assessment questions, the resource group must consider practical implications that affect an
indicator's possible utility. These considerations include:
Are available field methods appropriate to the anticipated skill level of field crews?
What field sampling methods should be used for sample collection, and are there
operational constraints on the use of these methods?
What are the best analytical techniques for measuring the indicator?
How precisely can the indicator be measured?
What are the beginning and ending times for site access and for the indicator's
expected index period?
What time of the year should measurements be made (definition of the index period)?
Within what location of the resource sampling unit should the indicator be measured
(identification of the index sampling site/area)?
Can sufficient measurements of the indicator (e.g., numbers of target organisms) be
collected, given the sampling design, operational constraints, and the need to
minimize the environmental impacts of the sampling process?
What are expected sampling costs for transportation, equipment, and personnel?
What are the data-processing demands in terms of sample identification, shipping,
and archiving?
• What are the data-processing and reporting demands in terms of data management
arid statistical support?
Many operational monitoring questions regarding sampling methods and analytical techniques can be
evaluated based on the literature. However, others can be evaluated only by conducting pilot
research or regional demonstration projects.
5.2.3 Statistical Evaluation of Indicators
To evaluate an indicator's statistical behavior, resource groups must first explicitly identify those
sources of variance that must be quantified to evaluate its performance. In general, these sources qf
EMAP Indicator Development Strategy
43
-------
variance include the indicator's 1) temporal variability within and between potential index periods, 2)
spatial variability within and between sampling units, 3) annual variability between index periods, and
4) response variability to stressors. Specific research questions that should be answered for each
indicator include:
What is the indicator's spatial variability among concurrent samples collected at
different locations within a resource sampling unit?
What is the indicator's spatial variability among non-concurrent samples collected
within the index window but from different ecological resource sampling units within
the same region?
What is the indicator's spatial variability among samples collected within the index
period in different regions during the same year?
What is the indicator's temporal variability among samples collected from the same
ecological resource sampling unit, during the same year and at the same locations
but during different potential index periods?
What is the indicator's temporal variability among samples collected within the same
region, during the same index period, but spanning a number of years?
• What is the indicator's variability among concurrent samples collected at different
sampling units that are impacted to differing degrees by stressors identified during the
indicator formulation phase?
Similar questions and similar information are required for indices composed of these indicators.
These questions are intended to ascertain whether the condition of the resource of concern can be
estimated with the precision required to meet data quality objectives using indicators measured
during an index period and whether the indicator's responses to major stressors or to changing
ecological condition of the resource can be detected and quantified within a reasonable time frame.
An example of partitioning the variance into its component parts for Vermont lakes is shown in Table
5-5. Both spatial (population - among lakes, index - within lake) and temporal (year - among years)
variability are considered in evaluating an indicator.
5.2.4 Evaluation of Indicators for Resource Assessments
EMAP will not implement regional or national monitoring of any indicator unless it is expected to
convey meaningful and useful information to decision makers, environmental scientists, or the public
at large (see acceptance criteria in Table 5-3). In order to evaluate an indicator's ability to convey
information, each resource group should address a number of specific research questions including:
EMAP Indicator Development Strategy
44
-------
Table 5-5. Summary of Estimates of Components of Variance for Secchi Disk
Transparency (SD), Chlorophyll-a (Chl-a), and Total Phosphorus (TP) Derived
from the Vermont Lake Monitoring Database3.
VARIANCE SOURCE
a2 lake (Population Variance)
a2 year (Year Variance)
a4 ,ake.year (Lake-Year Interaction)
o* ,ndex (Index Variance)
a2 meas (Measurement Variance, a component of
Index Variance)
VARIANCE ESTIMATES
SD
4.203
0.007
0.628
0.660
0.191
Chl-a
0.229
0.002
0.178
0.230
0.111
TP
0.412
0.018
0.089"
a Grand means: SD - 4.82 m, Chl-a = 10.0 //g/L, and TP = 11.7 jug/L. Both Chl-a and TP were
transformed before calculating variance components.
b Only the aggregate of a1 ,ake.year and a* index could be calculated because single samples were obtained each
year.
EMAP Indicator Development Strategy
45
-------
How will field measurements be standardized with respect to underlying
environmental constants that determine limits for their numerical values to ensue that
indicators convey the same quantitative information concerning the condition of all
resource units within a region?
How will multiple field measurements be combined into indicator indices that estimate
the ecological condition of resources of concern in regional assessments?
How will the indicator contribute to defining the proportion of the ecological resource
considered to be degrading, improving, or impacted?
How will the indicators aid in identifying the likely causes for patterns and trends in
ecological condition?
• How will the monitoring results be summarized in annual statistical summaries?
How will the monitoring results from the indicator be used in EMAP's annual
statistical summaries?
5.2.4.1 Identification of Nominal-Subnominal Criteria
To achieve its goal, EMAP must present its results and assessments in a form that is readily
understood and meaningful to intended users. The concept of nominal-marginal-subnominal is at the
crux of the assessment process and the use of EMAP information in environmental decision making
and management. Nominal refers to the state of having desirable or acceptable ecological condition.
Subnominal is its antonym or the state of having undesirable or unacceptable ecological condition.
Marginal condition exists when the nominal and subnominal criteria are not contiguous. Although
many other terms could be used to describe this concept (i.e., desired-undesired, acceptable-
unacceptable, attainment-nonattainment, impaired-unimpaired, degraded-undegraded, stressed-
unstressed, polluted-pristine, etc.), EMAP has adopted "nominal-subnominal" as its preferred
terminology.
EMAP is committed to developing nominal-subnominal criteria for each of its condition indicators in
order to add value to the decision making process and to address the "so what question" associated
with interpreting the cumulative distribution that will be reported by EMAP resource groups in
statistical summaries. The fact that 40 percent of the lakes in the northeastern United States have
an Index of Biotic Integrity (IBl) score less than 35 is not significant to decision makers. What is
important is knowing that 40 percent of the lakes in the northeastern United States have an IBl score
less than 35 and that such a score implies subnominal biological integrity.
EMAP Indicator Development Strategy
46
-------
There are at least two different approaches for identifying nominal or subnominal conditions of
ecological resources. The first approach involves identifying critical scores or thresholds along an
indicator's cumulative distribution that delineate different conditions of the resource. Depending on
the amount of information available and the assessment needs of potential clients, resource groups
may choose to identify either a single nominal-subnominal threshold or multiple thresholds. In the
latter case, one threshold could delineate resources in subnominal condition, whereas a second
threshold could identify resources in optimal condition. Presumably, resources having indicator
scores in between these two values would be considered to be in marginal condition. When
developing thresholds, however, it is important to remember that criteria are value specific. For
example, one would expect indicator scores for trophic condition of inland surface waters, estuaries,
or the Great Lakes to have three distinct nominal-subnominal thresholds depending on whether the
value of concern is the resource's biological integrity, fishery productivity, or aesthetic features.
For certain indicators (e.g., certain indices of biodiversity and landscape indicators), there may be
little consensus of what constitutes a nominal-subnominal | threshold, but there might be a general
consensus that a negative or positive trend over time is indicative of change in a resource's
condition. In such cases, it may be sufficient to ^stablish a reasonable criteria for status estimates.
Nominal-subnominal criteria indicate an indicator can satisfy EMAP's trends DQO (i.e., detect a 20
percent change per decade) and convey relevant information to decision makers and environmental
managers regarding the resource's nominal-subnominal condition. The establishment of reasonable
criteria, however, should not preclude an attempt to establish nominal-subnominal criteria for the
status estimates of EMAP condition indicators. Examples of nominal-subnominal criteria for various
biotic and abiotic condition indicators for the Great Lakes are listed in Table 5-6.
Expectations or definitions of nominal-subnominal condition change as the resource itself changes
and as more is learned about natural processes and system functioning. Five years ago, for
example, it would have been easier to characterize the condition of rangeland and rangeland
management practices than it is today because the paradigm previously used in rangeland
management is changing. As more information on the factors influencing rangeland condition
becomes available, the classification of range condition indicators into nominal-subnominal categories
will change. This evolution of scientific knowledge and societal expectations must be factored into
the specification of nominal-subnominal scores.
Rapport (1989) lists three approaches or criteria commonly used to assess ecosystem health: 1)
identification of systematic indicators of ecosystem functional and structural integrity, 2) measurement
of ecological sustainability or resiliency (i.e., the ability of the system to handle stress loadings, either
natural or anthropogenic), and 3) an absence of detectable symptoms of ecosystem disease or
EMAP Indicator Development Strategy
47
-------
Table 5-6. Great Lakes Resource Group examples of nominal - subnominal criteria (ranges)
established for the Laurentian Great Lakes.
Condition Indicator
Nominal Threshold
Source
Remarks
Fish Populations ' ' ' ~
Internal Pathology
External Pathology
Recruitment
< 2-4 % of fish
< 6-8% of fish
> 80-90%
AOC Listings, Literature
USFWS, Pers. Communication
L. Superior Objectives
Some Published
Little Info Published
100% ultimate goal
Trophic Status . ,_,--., , ,
Total Phosphorus
Chlorophyll a
TN/TP
SJO2/SRP
< 6.3 - 15 ug/L
< 2.5 - 4.3 ug/L
>29
> 93
ULRG, Literature
ULRG, Literature
Smith, Literature
Holm/Armstrong, Literature,
Well Documented
Well Documented
Lacks verification
Lacks verification
Fish Contaminants "
Aldrin/Dieldrin
DDT+metabolites
Endrin
Heptachlor+Epoxide
Lindane
Mirex
PCBs
Mercury
TCDD
0.3 ug/g1
1.0 ug/g2
0.3 ug/g1
0.3 ug/g1
0.3 ug/g3
absent4
0.1 ug/g5
0.5 ug/g6
absent3
IJC 1972-1987
IJC 1972-1987
IJC 1972-1987
IJC 1972-1987
IJC 1972-1987
IJC 1972-1987
IJC 1972-1987
IJC 1972-1987
IJC 1972-1987
edible portions
whole fish
edible portions
edible portions
edible portions
all aquatic organisms
whole fish
whole fish
all biota
1 Based on wet weight; for the protection of human consumers of fish
2 Based on wet weight; for the protection of fish-consuming aquatic birds
1 Based on wet weight; for the protection of human consumers of fish
1 Based on wet weight; for the protection of human consumers of fish
3 Based wet weight; for the protection of human consumers of fish; recommended change - total BHC
isomers should not exceed 0.3 ug/g.
4 For the protection of fish-consuming birds and animals
5 For the protection of fish-consuming birds and animals
6 Based on wet weight; for the protection of aquatic life and fish-consuming birds
3 For the protection of fish-consuming birds and animals
EMAP Indicator Development Strategy
48
-------
stress. Thus, he defines ecological health as both the occurrence of certain attributes deemed to be present
in a sustainable resource and the absence of conditions that result from known stressors or problems
affecting the resource. Some of the approaches that can be used effectively by EMAP resource groups to
identify or develop nominal-subnominal classifications include:
Field calibration and hypothesis testing in sampling units that have been identified a priori
as "good" and "bad" using criteria that are independent of the indicators being evaluated.
Criteria developed using reference sites, e.g., the long term ecological research sites
(LTERs) supported by the National Science Foundation.
• Laboratory methods such as bioassays for toxicity.
Legislative mandates such as Food and Drug Administration (FDA) action limits for fish
contamination.
External criteria such as regulatory mandates, goals and scientifically defensible criteria.
Common sense and professional judgment. Public meetings, hearings, focus groups,
special interest groups, and other outreach approaches can be used to determine what
the public considers to be nominal and subnominal. Professional and scientific
perspectives concerning what constitutes nominal or subnominal conditions often can be
obtained by Delphi methods or Kepner-Tregoe analysis (Kepner and Tregoe 1981).
Model based criteria. For example, to evaluate the condition of soils with respect to
credibility, calculated soil indicators, S, could be compared to T scores of the Universal
Soil Loss Equation (USLE) (Wischmeier and Smith 1978), i.e.,
A ^ T = Nominal
A > T = Subnominal '
• Similarly Habitat Suitability Indices (HSI) (e.g., Fausch et al. 1988) could be used to
identify nominal-subnominal conditions for selected wildlife species as individual objects
of concern and as surrogates for other biota. Successional models could be used to
identify expected trends in biodiversity under nominal and subnominal conditions.
Retrospective analyses and paleoecological analyses can be used to define a historical
background or baseline condition that can compared with current and future conditions.
Specific experimental research directed at determining nominal-subnominal scores and
ranges.
5.2.4.2 Condition-Stressor Associations :
In addition to knowing what proportion of the resource is in subnominal condition, it is also desirable to know
that stressors are correlated with the same condition. Although such correlations do not prove causality, they
EMAP Indicator Development Strategy
49
-------
are an important diagnostic tool toward establishing such relationships and for initiating detailed, cause-and-
effect research. Resource groups must plan these statistical analyses for each selected condition indicator
by 1) identifying non-EMAP stressor data bases that could be used for this purpose, 2) developing selected
stressor indicators concomitant with condition indicators based on specific testable hypotheses or
documented conceptual models, and 3) using landscape indicators as stressor indicators for certain classes
of stressors.
Although the utility of the first two approaches is intuitively obvious, the utilization of landscape indicators for
condition-stressor associations may be less clear. The EMAP-Landscapes resource group will be developing
a number of landscape indicators to estimate the condition of spatially defined complexes of ecosystems and
other resource categories. Initial analyses, however, suggest that several of these landscape condition
indicators can be potential indicators of specific types of stress. Besides the obvious stressor of habitat
reduction and fragmentation, it appears to be possible to associate certain landscape indicators with such
stressors as regional hydrologic alterations, increased sediment loadings, and point and nonpoint chemical
pollution sources. The utility of landscape condition indicators as surrogate stressor indicators for other
EMAP resource groups is a major area of research planned by EMAP-Landscapes. If successful, this
approach affords a very cost-effective way to perform certain analyses of condition-stressor associations.
5.2.4.3 Example Assessments
The annual statistical summary is a primary mechanism for reporting on resource condition. Results from
each demonstration study should be analyzed as proposed by its EMAP resource group and reported in a
preliminary annual statistical summary to confirm the utility of the data. Although the degree of evaluation
possible is extremely limited during the first year of data collection, this evaluation will allow confirmation of
the basic rationale for including each indicator. Subsequent annual statistical summaries will be increasingly
important for evaluating the ability of each indicator to identify changes and trends in the status of ecological
resources.
Example assessments explore the types of data analysis, data presentation, indicator responsiveness, and
indicator variability that can be expected. These assessments may be conducted using either simulated or
real data. Conducting an example assessment is intended to assist in selecting among alternative indicators,
identifying redundant indicators, identifying gaps in the suite of indicators, deciding how data from each
indicator would be used in EMAP for assessing status and trends, exploring the ability of the indicator suite
to ascribe plausible causes to observed patterns and trends in the region's subnominal areas and refining the
conceptual model. An EMAP resource group's conceptual model, which links stressors with assessment
endpoints, should be used throughout the example assessment to guide the analysis and interpretation of
EMAP Indicator Development Strategy
50
-------
both empirical and simulated data. Example assessments should be conducted for multiresource
assessments to assist in the selection of indicators that aid integration of information across resources.
5.3 Evaluation Methods
One of the first steps in indicator evaluation is to review and analyze existing data. Existing data can permit
a preliminary assessment of an indicator's characteristics, such as stability during the index period, and
identify additional data needs for further evaluation.
In general, there are three principal sources of data used to quantitatively evaluate EMAP's indicators, i.e.,
1) existing data from the literature and compiled data bases,
2) data generated by limited-scale field pilot research projects, and
3) data generated by regional demonstration projects.
Because the acquisition of these data involve increasing costs, resource groups should utifize them in the
sequence indicated. Although much of the existing data may not be appropriate to the temporal and spatial
scales required by EMAP, to the greatest degree possible, an indicator's evaluation, should be undertaken
using such data. . , '• " < "' =
5.3.1 Existing Data and Desk-Top Studies •
Ideally, a large portion of an indicator's statistical behavior could be characterized and evaluated using
existing data (e.g., Table 5-5). Consequently, before initiating any field research, resource groups should
determine what types of data currently exist that can be, used to address each of the research questions
defined in Section 5.2.3. Data sets from field studies or surveys that have sampled resources more
intensively than is being considered for EMAP, for example, can be extremely valuable for evaluating an
indicator's intra-annual and spatial variability and, consequently, would be valuable for identifying potential
indicator index periods and index locations, respectively. Data from long-term plot experiments can be useful
to assess inter-annual and local spatial variability (Franklin et al. 1990). Although such data would be of
obvious value for corroborating potential index periods and index locations for an indicator, these data also
could be used to establish the indicator's responses to changes in the condition of the resource (i.e., plot) at
large. Retrospective data also may be useful to characterize the natural temporal variability of related
indicators. For example, sediment cores and tree ring chronologies could be useful for evaluating research
indicators of biological integrity or water chemistry (using diatom assemblages) and of forest productivity,
respectively. Finally, data on indicator scores at regional reference sites or data on indicator responses
EMAP Indicator Development Strategy
51
-------
Figure 5-1. Illustration of the relationship between power and magnitude of trends detectable and
years of monitoring. From left to right, curves are for trends of approximately 2%/yr,
1.5%/yr, 1%/yr, and 0.5%/yr.
POWER
1.00
0.80
0.60
0.40
0.20
0.00
2 4 6 8 10 12 14 16 18 20
YEARS
EMAP Indicator Development Strategy
52
-------
across stressor gradients can be useful in characterizing an indicator's variability under prescribed
environmental conditions.
Having obtained variance estimates for an indicator, simulation models of its statistical behavior should be
constructed and analyzed. These investigations should be used to estimate an indicator's preferred index
period for sampling, the time needed for an indicator to detect changes of a specified magnitude, and other
aspects of the indicator's cumulative distribution such as changes in its cumulative distribution during
different index periods and over longer time frames of several years. Simulation results for longer time
periods can be input to statistical programs to determine minimum detectable trends in condition indicators
(e.g., 2 percent change per year in indicator score over 10 years). For example, statistical power analysis
can be conducted to determine the time or duration required to detect different rates of change in an
indicator for a given variance structure (Figure 5-1). If only limited data are available, it may be possible to
use Monte Carlo simulation techniques or simple process models to generate hypothetical data with
reasonable spatial and temporal variability that can be employed for the same purposes.
5.3.2 Pilot Research Projects
Pilot research projects are designed to evaluate any aspect of an indicator's overall performance and
programmatic utility except for the indicator's ability to provide preliminary regional estimates of the
resource's condition. Consequently, sampling sites for pilot research projects need not be selected
according to probability sample protocols unless pilot project objectives require it. The objectives might be
achieved more efficiently using alternative research designs.
One of the primary purposes of these investigations is to corroborate the regional monitoring feasibility of the
research indicators. These studies answer the when, where, how, and what questions associated with the
indicators that appear to be tractable for a regional monitoring program. These projects also can be used to
evaluate the relative merits of alternative sampling and analytical methods (including considerations of
sampling and analytical error). Finally, from a logistical point of view, pilot research projects are used to
develop general protocols for sample acquisition, handling, and archiving.
The second principal purpose of pilot research projects is to generate data needed to answer the questions
that were identified in Section 5.2.3 regarding the indicators' statistical behavior. In general, data from these
studies are used to begin the characterization of any or all of an indicator's components of variance with the
exception of its regional spatial variability that must be addressed via a regional demonstration project
(described below in Section 5.3.3). In some cases, questions identified in Section 5.2.3 should be refined to
EMAP Indicator Development Strategy
53
-------
address specifics of the indicator's statistical behavior. Examples of subjects to emphasize in a pilot
research project include:
Intensive temporal sampling to define the best boundaries for the index period (e.g., fall turn-over in
lakes, seasonal low water in wetlands) and to quantify the within-index period sampling variability.
• Extensive spatial sampling within a regional resource class to determine the value of data collected
at index sampling sites relative to more intensively or randomly located sampling sites and to
quantify indicator variability within the index sampling area (e.g., the central basin of the lake)
where permanently fixed monitoring sites cannot be established.
• Sampling along gradients from polluted to unpolluted or impacted to natural sites to 1) evaluate
the responsiveness of the indicator to stress, 2) aid in defining nominal and subnominal
classifications, and 3) evaluate the specificity of the indicator to particular types of stress or change
and the repeatability of the indicator response in different regions or ecological resource classes.
The optimal pilot research project design will depend on the specific questions to be addressed. Two
examples from the EMAP-Estuaries resource group illustrate the types of studies that may be useful:
Definition of the index period. Levels of dissolved oxygen (DO) in estuaries are highly variable,
yet DO also serves as an important condition and stressor indicator for assessing estuarine
condition. Therefore, field studies were conducted in 1990 to determine 1) the optimal boundaries
for the summer index sampling period and 2) the utility of point-in-time measurements of DO. At
about 100 sites in the Virginian Province, three point-in-time measurements of DO were collected
during three sampling intervals (early, mid-, and late summer). Comparison of the DO cumulative
distributions for the three periods provides information of the regional stability of the DO indicator.
In addition, DO was measured continuously at a subset of 30 sites, selected by experts as sites
expected to experience problems with low DO. These continuous records will be used to both
refine the index period and to evaluate the utility of point-in-time measurements as an indicator of
the frequency, severity, and extent of low DO episodes.
Indicator responsiveness to stressors. Using expert judgment, 24 sampling sites were selected
to reflect important gradients of both pollutant exposure (DO gradient) and habitat (salinity) within
two geographic regions (latitudinal gradient). A variety of indicators (e.g., benthic biomass, species
abundance) were sampled at each site, three times during the summer index period. Condition
indicators that consistently reflect the effects of pollutant gradients across a range of habitats and
regions are obviously preferred and fulfill the prime criterion for acceptance as EMAP core
indicators (see Figure 5-2). In choosing sampling sites and regions for the pilot research project,
resource groups should attempt to include the full range of conditions that are expected to be
EMAP Indicator: Development Strategy
54
-------
I
I
o.
S>
>
1N3IQVUO
0)
o
'«*
o
w
§
Q.
re
o
0)
O)
1
o
o
S"
Q.
H .
Sf
CO O>
0) 0>
re o
1-
O)
I
OT
I
1
Q
3
S
111
V)
05
iZ
55
-------
encountered during national implementation. Answers to some of the above questions (e.g., best index!
period) may vary from region to region or among ecological resource classes or types. If so, it may bel
necessary to include multiple regions in the pilot research projects or to evaluate the indicator in regions thai
are expected to represent the extreme conditions for the indicator. Multiregional pilot research projects!
should be limited to investigations of issues that cannot be resolved within a single region and should be|
designed to obtain only the minimum information needed to complete the assessment.
Table 5-2 illustrates decisions that resource groups should be able to make using the results of pilot research!
projects. To fulfill the second criterion in the center ("Accept if:") of Table 5-2, there must be quantitative]
evidence that an indicator 1) responds to changing stressor levels, 2) responds to changes in the conditior
of most resource classes, and 3) has signal-to-noise ratios both temporally and spatially stable enough I
during the index period not to mask its responsiveness. To fulfill criteria #3 and #6, data on the costs and j
logistical constraints associated with sampling must be compiled and evaluated.
5.3.3 Regional Demonstration Projects
The primary objective of regional demonstration projects is to provide preliminary estimates of a resource's
condition over a standard Federal region for one or more resource classes. Although these projects are I
used to confirm the results of site-specific pilot research projects on regional scales, they are designed to|
evaluate whether data collected on these indicators can be interpreted regionally. These projects confirm the
feasibility and utility of research indicators at regional scales over a broad range of conditions. These I
projects focus only on indicators whose interpretability (i.e., conceptual or mathematical relationship to
identified assessment questions), sampling procedures, and methods have been well established. Regional
demonstration projects are designed to further characterize the indicator's among-site and between-year
variability. Data obtained from regional demonstration projects are used to evaluate the indicator's ability 1
• be broadly applicable (e.g., are different indicators needed for large lake, small lakes,
rivers, and streams?),
• detect regional trends,
• integrate spatial and temporal influences and stressors,
• be used in diagnostic assessments of resource condition, and
• convey meaningful results to decision-makers in the form of annual statistical summaries
and periodic assessments.
A key function of these studies is the determination of the sampling intensity needed to estimate regional
status and trends in resource condition and to detect associations among regional patterns of ecological
EMAP Indicator Development Strategy
56
-------
condition and anthropogenic stresses. From a logistical viewpoint, these projects establish EMAP's
infrastructure for conducting regional monitoring activities through the field implementation activities that are
necessary to conduct regional demonstrations.
Table 5-3 illustrates key decisions that should be able to be made using results of regional demonstration
projects. Criteria #1, -#3, and #4 in the acceptance column of Table 5-3 are absolutely critical tests for any
indicator to attain core status. Figure 5-3 illustrates the temporal dynamics of a regional Index of Biotic
Integrity. These data suggest that although a summer index period would appear appropriate for this
indicator, a late spring index period would not. Although criterion #2 (responsiveness to changing ecological
condition of the resource class) is critical for an indicator's long-term utility, in many cases full evaluation of
this criterion may require regional monitoring of a suite of indicators for several years. In such cases, an
indicator could be accepted and nominated for core status based on well documented theoretical grounds.
Full evaluation of this criteria then would be completed during its reevaluation as a core indicator.
Each indicator must be evaluated for all pertinent resources classes. Although rejection of an indicator for
one resource class should not affect decisions regarding the utility of that indicator for other resource
classes, indicators that are effective for multiple resource classes are desirable and will always be sought
(e.g., failure of nutrient concentration indicators to adequately characterize the condition of one agro-
ecosystem resource class should not result in rejection of this indicator in other agroecosystem resource
classes).
5.4 Identification of Indicators for Implementation
The final phase in the indicator evaluation process is the selection of core indicators for routine regional and
national monitoring. In this stage of the indicator development process, EMAP resource groups nominate
selected research indicators for long-term (i.e., 10-15 years minimum) core implementation. These indicators
are then subjected to a formal programmatic and peer review process. There must be substantial and
documented evidence that each proposed core indicator satisfies all of the critical indicator criteria
summarized in Table 5-1 and that it does not provide merely redundant information to that provided by other
core indicators. Indicator status sheets (Table 5-4) should be maintained and updated to summarize the
results of the evaluation of each indicator and to document all sources of information used or generated in
the evaluation of research indicators. These indicator status sheets should be included as an integral part of
the annual research plans and the more comprehensive 5-year research and monitoring plans of each
resource group.
Although multiple research indicators of the same or closely related assessment endpoints may have been
evaluated by a resource group, in general, only one of such multiple indicators will be implemented as a core
EMAP Indicator Development Strategy
57
-------
Q)
M
m
i
(O
CO
en
1
i
»
p
-------
indicator. There are, however, two situations when multiple indicators of the same assessment endpoint
could be elevated to core status. The first of these occurs when the indicators under consideration are
F ' *
multivariate or index indicators that depend on the same set ;of field measurements. The second situation
occurs when no one indicator is expected to respond to all of the resource's major stressors. In this case,
however, it must be remembered that each core indicator also must be responsive to changes in the
condition of the resource at large. The fact that an indicator may not appear to be directly responsive to
particular stressors may be irrelevant since it should be indirectly responsive to those stressors via its
responses to changes in the condition of other components of the resource.
EMAP Indicator Development Strategy
59
-------
6. Indicator Implementation
The third phase in EMAP's indicator development process, is the national implementation of the core
indicators that have undergone formal programmatic review and approval. During this stage, EMAP resource
groups
• monitor core indicators nationally,
prepare annual statistical summaries, and
prepare periodic resource assessment in conjunction with the EMAP-Assessment and
Reporting coordination group.
In the implementation phase, there are explicit environment values and assessment questions associated
with each resource. Conceptual models relate the condition indicators or indices to assessment endpoints
and to associated stressors. Indicator variances have been quantified and partitioned into spatial and
temporal components. Based on these variance estimates, both status estimation and trend detection data
quality objectives have been satisfied for each core indicator. Regional demonstration projects have been
conducted to document the feasibility of monitoring these indicators at regional and national scales.
Nominal-subnominal ranges have been established for at least selected environmental values and the
process is continuing for other segments of society and societal values.
The annual statistical summary is the primary product of each resource group during this stage of the
program. These reports should summarize the current regional cumulative distribution of each core indicator
and identify the indicator's associated nominal-subnominal thresholds. Results of trends analyses for some
appropriate time window should also be summarized in these reports. If stressor indicators also have been
implemented, results of correlation analyses between condition and stressor indicator scores likewise should
be presented. EMAP information will be shared with other monitoring programs at the federal, regional, and
state levels.
EMAP Indicator Development Strategy
60
-------
7. Indicator Reevaluation
This phase of the indicator development process is actually an ongoing activity that begins upon initial
implementation of the monitoring of core indicators at regional and national spatial scales. Core indicators
are periodically reevaluated to confirm their ability to detect changes and identify trends in the condition of
ecological resources and to ensure that appropriate advances in technology and information have been
considered for incorporation. In this phase, it is important that EMAP balances continuity of methods to
maximize its capability to detect trends with procedures for refining or replacing indicators that fail to perform
optimally. This phase is designed to critically review the performance of core indicators through time,
evaluate alternative indicators to address emerging issues and; inadequate core indicator performance, add
new indicators as deemed desirable, and substitute superior indicators for less adequate core indicators.
Scientific advances and technological innovations obviously will occur during EMAP's implementation, arid
these advances may improve the precision, accuracy, representation, cost effectiveness, and overall
applicability of EMAP indicators. These advances may necessitate modifying specific indicators, replacing
indicators with others that provide improved or equivalent information at reduced cost, or adding indicators
that address important emerging issues. To accommodate these changes, it will be necessary to specify
appropriate procedures to reexamine indicator performance and usefulness. This section presents a
preliminary outline of a systematic approach to indicator reevaluation that will ensure the use of the best
possible set of indicators for achieving EMAP's objectives. This approach should be revised and expanded
as EMAP matures and as modification of the core indicators is considered.
An EMAP core indicator should be revised only after a thorough reexamination indicates that it is clearly
necessary (i.e., when revision results in a significant improvement in the quality of the assessment of status
and trends of ecological condition, without diminishing the continuity of the assessment record). Once the
current EMAP program has been evaluated and modification of a core indicator is appropriate, a plan for
transition must be developed for programmatic peer review. Proposed changes will not be implemented
unless approved by the peer review process. Having established the need to modify the set of indicators,
the primary objective is to accomplish a smooth transition. Continuity of EMAP's data bases is extremely
important for ensuring that trends or changes in condition can be detected. Situations that may require
reexamination of the core indicators include the following scenarios:
A newly identified indicator appears to be superior to an indicator currently in use for measuring an
assessment endpoint or stressor. The decision to replace the current indicator with the new one
and to discontinue monitoring the current indicator must be made after obtaining adequate ;
EMAP Indicator Development Strategy
61
-------
information to ensure continuity of the monitoring record and comparability of the new assessments
with those that have relied on the old indicator.
• The environmental conditions have changed such that underlying mechanisms are altered and
previous linkages between the indicators and environmental values may not be representative of
the existing situation.
• An improved method promises to provide similar quality data at lower cost or higher quality data at
a similar cost. The impact of using the improved method to monitor condition and to detect trends
must be evaluated before replacing the original method to ensure that data quality actually equals
or exceeds that available using the current method.
Evolution of assessment methods may also result in changes for data analysis, presentation, or evaluation in
annual statistical summaries (see Paulsen et al. 1991) and assessments. For example, a revised conceptual
model of the resource might suggest a new index or a revised nominal-subnominal threshold for assessing
ecological condition for selected resource classes. Although these changes will not directly affect the set of
indicators, they may result in an opportunity for modifying or adding core indicators. Therefore, the impact of
these changes on the assessment process should be evaluated as early as possible to increase the ability of
the indicators to provide needed information.
7.1 Reevaluation Procedures
The primary approaches for evaluating core indicators include routine review, evaluation of assessment
products, and searches for new ideas for indicators. Such approaches call for continual tracking of the
published literature and ongoing research programs to identify promising new information.
Once a new or revised indicator or method is identified, the process to evaluate that idea should proceed as
outlined in Sections 4 and 5 above. Implementation of this review process ensures that indicators or
methods cannot be revised or replaced without 1) carefully conducting the evaluations needed to ensure
that the new indicator or method provides a meaningful improvement in EMAP's monitoring or assessment
capabilities and 2) quantifying the relationship between the new and old indicators or methods (i.e.,
calibrating the new indicator).
Evaluation of proposed changes to core indicators must be conducted in such a way to allow not only the
evaluation of the relative merits of the new and old indicators or methods but also the mathematical or
statistical relationship between the two. This evaluation requires that pilot and regional demonstration
projects be designed and conducted to test for comparability and relative responsiveness of the two
indicators or methods under a range of conditions. Field demonstrations should be conducted to test
EMAP Indicator Development Strategy
62
-------
alternative indicators or methods in one to several regions for a number of years to verify the consistency of
that relationship. Pilot and regional demonstration projects should be conducted to calibrate the relationship
between the old and new indicators (or measurement techniques), and both the old and new (or modified)
indicators will be monitored long enough to ensure comparability of the data sets from both indicators before
phasing out the old indicator. Once the spatial and short-term temporal relationships between the
alternatives are well established, simultaneous collection of data for both indicators may be desirable for an
extended period of time at a limited number of sites to ensure the similarity of the relationships over an
extended time period.
Revision of core indicators requires the conduct of all assessments described in Section 5.2.4 and that all
criteria for adoption of the changes are satisfied. The advantages of new indicators or methods must be
significant and must represent improvements over existing indicators. These advantages must be well
documented, and the documentation of the research efforts (laboratory, pilot research, and regional
demonstration projects) must include the following information:
Quantification of the calibration between the old and new indicators or methods under the full range
of conditions observed during EMAP's monitoring to date.
• Evaluation of how the proposed change in indicators or methods would affect the annual statistical
summary and data interpretation and integration (may require recalculation of indices).
EMAP Indicator Development Strategy
63
-------
8. Integration Among Resource Groups
As discussed in Section 2.3, eight broad ecological resource categories have been defined within EMAP:
agroecosystems, arid ecosystems, forests, estuaries, Great Lakes, surface waters, wetlands, and
landscapes. Indicator development and implementation within each of these resource categories is the
responsibility of an individual EMAP resource group. All ecological communities, ecosystems, and biomes
are open thermodynamic and material flow systems that can be sustained only by maintaining critical
intersystem connections and interactions. If EMAP is to achieve its programmatic goals, its indicator
development process, implementation, and assessments must reflect and embody this principle.
Consequently, indicator development and implementation of individual EMAP resource groups must be
integrated and coordinated.
This integration occurs at two levels: 1) during indicator selection to ensure that all important intergroup
linkages are considered and 2) during data interpretation. The second level of integration is beyond the
scope of this document. Tasks relating to an integrated interpretation of EMAP's monitoring results are the
responsibility of the EMAP-Assessment and Reporting coordination group. This group recently published an
EMAP Assessment Framework document that discusses this data interpretation process (EPA 1994).
However, the utility of each indicator for interpreting resource status and trends is an important consideration
in the indicator selection process. Thus, close cooperation between resource groups and EMAP-Assessment
and Reporting is essential. Integrating monitoring results among ecological resource categories will enable
EMAP to address a wide range of issues including:
• source apportionment and diagnostic analyses across resource boundaries (e.g., nonpoint sources
to surface waters),
• the status at a regional scale, encompassing all ecosystem types,
the extent and magnitude of environmental problems that impact multiple ecological resource
categories,
• the effectiveness of regulatory actions, and
emerging environmental problems or new assessment questions that EMAP can address.
The primary approach to achieving an integrated set of indicators across all resource groups is through
communication and information exchange.
8.1 Framework for Indicator Integration
EMAP Indicator Development Strategy
64
-------
Integration of indicator development and application of indicators across all EMAP resource groups involves
a number of factors including 1) maintaining inter-group communication and interaction to foster development
of indicators that will integrate ecosystem level information among different resource groups, 2) assimilating
new knowledge, 3) consistent interpretation of critical indicator development concepts, 4) consistent
collection and use of stressor indicator data, 5) collaboration in identifying special condition indicators that
integrate across EMAP resource groups (e.g., wide ranging or migratory organisms that use multiple habitats,
landscape pattern metrics, soil carbon, etc.), and 6) co-locating sampling units for special studies.
EMAP's ability to diagnose plausible causes of changing trends will be facilitated if the pathways of
interaction between ecological resources are explicitly identified. EMAP needs to examine the stresses that
each ecological resource receives as a result of processes or conditions in other resource categories. For
example, the nutrient balance of a lake may be highly dependent on nutrient fluxes in the surrounding forest.
Nutrient flux from the forest may be measured as a condition indicator in the forest, but as a stressor
indicator for the lake. Such identification will help clarify the auxiliary stressor data requirements of each ;
EMAP resource group and the level of mutual assistance that is needed to acquire such information.
Consistency in using auxiliary information will improve the abilities of all EMAP resource groups to detect
spatial and temporal associations among condition and stressor indicators and the natural and anthropogenic
stressors affecting them, particularly intersystem problems and issues.
Substantial consistency already exists among EMAP resource groups in their definitions of environmental
values, formulation of assessment questions, and identification of major stressors. This parallelism provides
EMAP with opportunities for identifying plausible causal relationships on large regional scales. For example,
if EMAP-Surface Waters detects that nutrients are significantly increasing in streams across a broad region,
but data from EMAP-Agroecosystems indicate no increase in nutrient export from agricultural lands, then
other nonpoint sources (e.g., atmospheric loadings) or point sources (e.g., discharges) may be responsible
for the observed trends in aquatic nutrients. Conversely, noncomplementary data may indicate that the
conceptual models being used are inappropriate or incomplete and should be reexamined.
EMAP resource groups proceed at different rates to implement the indicator development strategy. The
differences in phasing among EMAP resource groups contributes to the overall indicator development
because it allows the pioneering groups to pass on lessons learned to other groups. This process began at
ah indicator strategy workshop in Las Vegas, Nevada, in June 1990, and it should be maintained and
fostered in EMAP. Circulation of annual research plans, communication of lessons learned, and informal
intergroup discussions provides important information for efficient indicator development.
8.2 Types of Indicators that Integrate Across Ecological Resources
EMAP Indicator Development Strategy
65
-------
EMAP has two general types of indicators: condition and stressor. Although all indicators are one of these
two indicator types, many indicators also contribute to an integrative understanding of status and trends and
contribute to diagnostic evaluations.
8.2.1 Indicators Linking Resource Groups
Some indicators are an output from one resource category and an input to another (e.g., nitrogen in fertilizer
used in agroecosystems and as subsequent runoff to surface waters and wetlands) and, therefore, interface
one EMAP resource group with another. Such indicators measured using the EMAP monitoring design
generate data that can be used by more than one EMAP resource group. For example, an index of soil
erosion measured by EMAP-Forests or EMAP-Agroecosystems would provide EMAP-Wetlands, EMAP-
Surface Waters, and EMAP-Estuaries with an indicator of the potential inputs of sediment, nutrients, or
pesticides. Because soil and sediments represent sinks for chemical contaminants in all ecosystems, soil
and sediment contaminant data can be used as important links between ecological resources. For example,
soil and sediment contaminant measurements would be of importance not only in evaluating forest status but
also in assessing potential effects on aquatic receiving systems.
8.2.2 Indicators Shared by Resource Groups
Common or shared indicators are measured in multiple resource categories using similar techniques.
Examples include wildlife biomarkers, landscape attributes, and commonly used metrics of population or
community status, such as relative species abundance. By using consistent sampling and analysis tech-
niques in all resource categories, interpretation of multiresource patterns in ecosystem status and trends is
facilitated. Landscape-level indicators (e.g., mosaic diversity, patch fractal dimensions) may be applicable for
many or all resource categories as measures of habitat quality or as surrogates for other indicators that are
more difficult to measure (e.g., wildlife density). Biomarkers (e.g., DNA alterations, cholinesterase levels) are
common indicators that can be used as a metric of exposure to metals or organic constituents, whether the
organism is a plant, fish, or mammal.
A second type of shared indicator are migratory indicators. Migratory indicators provide quantitative
information on organisms that move across resource boundaries, that is, from one resource category to
another and back again (e.g., honey bees, migratory birds, white-tailed deer). These indicators would be
expected to reflect changes in exposure or habitat in one or more ecological resources, and in some cases
they might indicate cumulative impacts in several resource classes or categories within or outside a region.
Indicators that integrate the effects of ecological resource conditions in multiple regions, resource classes, or
resource subclasses (e.g., some birds, amphibians, top carnivores) might be particularly useful in detecting
EMAP Indicator Development Strategy
66
-------
the cumulative effects of changes in more than one resource category. Observing such indicators may lead
to the detection of stress pathways that had not previously been recognized (e.g., DDT and reduced
reproduction in raptors due to eggshell thinness).
8.3 Use of Conceptual Models to Facilitate Integration
As discussed in Sections 3 and 4, conceptual models are an important tool for formalizing possible relations
among indicators, assessment endpoints, and stressors and for identifying data or knowledge gaps that could
be filled through the selection and development of additional indicators. In a similar manner, conceptual
models also play a key role in identifying indicator-endpoint-stressor relationships and interactions across
resource groups.
Each EMAP resource group should develop one or more conceptual models that emphasizes the major
inputs, outputs, and structural and functional attributes of interest for the resource class. These resource-
specific conceptual models then provide the basis for integrating needs and results, (see Section 3 for the
framework).
As a first step toward integration among resource groups, the individual models developed by each group will
be compared. This process will 1) help to formalize expected relationships among indicators in different
resource groups, 2) establish consistency in the ways that conceptual resource models are developed and
used to aid in indicator selection, 3) encourage the identification and use of linking, common, and migratory
indicators, 4) identify commonalities in methods and indicator use among resource groups, and 5) ensure
that important processes and linkages are considered within the EMAP monitoring network. Developing,
updating, and revising the structural aspects and the inputs and outputs of the individual conceptual models
will form the framework for coordinating EMAP indicator development, as outlined in Section 3.
8.4 Coordination of Indicator Development Among Resource Groups
Five major tasks are planned to facilitate coordination and integration of the indicator development process
among EMAP's resource groups:
1. Compile and cross reference lists of assessment endpoints, environmental values, stressors, and
indicators proposed by each resource group to identify areas of similarity or commonality.
Assessment endpoints and environmental problems in different resource categories are, in general,
highly interdependent and linked to common stressors. Foliar damage, fish loss, and estuarine
eutrophication, for example, can all be related to atmospheric deposition. Compilation of the
EMAP Indicator Development Strategy
67
-------
proposed endpoints, stressors, and indicators serves as the first step towards identifying areas of
overlap or inconsistencies in approach among resource groups. A preliminary listing of the
environmental values identified by each resource group is provided in Table 3-1.
2. Conduct workshops that include non-EMAP scientists to identify linking and stressor indicators that
EMAP deems necessary or has overlooked. Interaction matrices are commonly used to develop
and chart linkages in computer or simulation models. This same method can be used to identify
possible linkages among resource groups for indicator development. The conceptual models
identify major inputs and outputs for each resource group and provide a starting point for
discussions.
3. Develop conceptual models that identify cross-resource linkages and relationships.
4. As appropriate, propose alternate assessment endpoints and indicators that would provide
information similar to that provided by the resource group but improve comparability with endpoints
and indicators being monitored by other resource groups. For example, sustaining biological
integrity is an environmental value common to all resource groups. Thus, to the degree possible,
biodiversity should be assessed using similar assessment endpoints and indicators in each group.
Greater compatibility of indicators, endpoints, and stressor information used in the different
resource groups results in easier and more direct program integration and cross-resource analyses.
Direct comparison of responses and effects among resource categories allows weight of evidence
and process of elimination approaches for diagnosing possible causal factors and mechanisms,
thereby providing greater confidence in EMAP's results.
5. For indicators selected by more than one group, examine and compare the proposed field sampling
and measurement methods and suggest modifications needed to improve comparability among
groups. Comparable methods and units are also important if comparisons are to be made across
resource groups. In some cases, further research may be needed to develop methods applicable
to several resource categories. For example, nutrient and pesticide exports from terrestrial
systems are typically measured using different techniques and expressed in different units
compared to estimates of inputs of these constituents to aquatic systems. Selection of the optimal
approach for satisfying the needs of both the terrestrial and aquatic resource groups may require
additional simulation analyses or field testing.
Efforts related to each of the above tasks will evolve continually, and the lists, matrices, and models will be
updated as needed.
8.5 Problems Associated with Differences in Spatial and Temporal Scales
EMAP Indicator Development Strategy
68
-------
For some stressor and exposure indicators, temporal displacement might be desirable; the observed
response may be displaced in time from the perturbation that caused the response. For example, soil
erosion indices measured during the spring period in agroecosystems and forests, combined with nutrient
export coefficients, might correspond better with estimates of summer chlorophyll concentrations in lakes
than would export estimates for the summer period. Selection of the optimal index period for indicator
measurements must consider, therefore, hypothesized stress-response relationships and the potential
displacement that might be required to associate a dependent response variable with an independent
stressor or exposure indicator. Expert opinion, obtained during workshops and peer review (see Sections
4.3.2, 7.1, and 8.1), will aid in evaluating the importance and effects of temporal displacement.
Spatial displacement is more difficult to evaluate and address. Paired comparisons are generally used for
association, and regression analysis is used to relate dependent and independent variables. Indicators
linking multiple resource categories, however, may riot be collocated. Data analysis techniques are being
developed, therefore, to deal with non-collocated data, relying largely on aggregation of regional or
subregional EMAP results before conducting diagnostic analyses. For example, aggregation of data by
subregions was used during the National Acid Precipitation Assessment Program to identify a linear
relationship between sulfate deposition and surface water sulfate concentrations.
Analysis of associations at regional or subregional scales is consistent with EMAP's design objectives for
determining regional patterns and trends in the status of ecological resources. Most environmental analyses
to date, however, have focused on causal relationships at local or site-specific scales. Extension of these
techniques to larger spatial scales will require new perspectives and approaches for data aggregation and
analysis and, where possible, the development of new indicators better suited for application and inter-
pretation on regional scales. The potential utility of regional-level analyses also hinges, in part, on the
degree of intra- versus inter-regional indicator variability, and the spatial scale over which indicator values
and variability are relatively homogeneous. Indicator integration both within and across ecological resource
categories represents a major challenge; however, the benefits resulting from a successfully integrated
program are substantial. The process of integration will require considerable amounts of cooperation,
communication, and coordination.
EMAP Indicator Development Strategy
69
-------
9. Concluding Remarks
9.1 Planned Reviews
Each EMAP resource group will formally reevaluate its indicator suite every five years. Proposed revisions to
the EMAP core indicators and their associated justifications should be included in the 5-year plan. These
proposed revisions will be peer reviewed along with the rest of the program.
The research plan, indicator status sheets, and indicator data base will be updated at each evaluation stage
of proposed new indicators and methods. It will also be necessary to ensure that the documentation for
each of the new indicators identifies the reasons for the investigation (e.g., identified gaps, inadequate
precision of current indicators). All information developed through comparison of alternative methods should
also be summarized in these documents.
9.2 Evolving Process
Lists of environmental values, assessment questions, and major stressors are not static; they each must be
reevaiuated periodically as new issues emerge, environmental values shift, and experience is gained with the
use and interpretation of EMAP monitoring data. In addition, unforeseen stressors may begin to operate on
ecological resources, or ecosystem relationships may change. Either of these circumstances could require
alterations to the suite of indicators in order for adequate monitoring of changes in the status and trends in
resource condition to continue.
Like the lists of values, assessment questions, and stressors, conceptual models linking these components
should not be viewed as static. Lists of environmental values and assessment questions should be reviewed
both internally and externally via workshops, annual budget plans, and 3- to 5-year research plans.
Likewise, conceptual models that are used to identify and interpret research indicators for these assessment
needs should be reevaiuated periodically for their utility, validity, and completeness.
The periodic review of core indicators must incorporate new and evolving environmental issues and values,
questions, emerging stressors and new knowledge about linkages among indicators to ensure EMAP
continually provides information useful in making decisions about environmental protection and management.
This includes identifying and developing indicators that relate directly to those variables or factors used in the
decision-making and resource management process.
EMAP Indicator Development Strategy
70
-------
Sharing of information, including information on EMAP indicators, with other programs will continue to expand
as EMAP evolves. Improved information management, technology transfer and reporting linkages among
EMAP and other agency/organization monitoring efforts will facilitate this sharing of information.
EMAP Indicator Development Strategy
71
-------
10. References
Barnthouse, L.W. 1992. The role of models in ecological risk assessment: A 1990's perspective. Environmental
Toxicology and Chemistry 11:1751-1760.
Evenari, M., I. Noy-Meir, and D.W. Goodall. 1986. Ecosystems of the World: Hot Deserts and Arid Shrublands.
12B. Amsterdam, Netherlands: Elsevier.
Fausch, K.D., C.L. Hawkes, and M.G. Parsons. 1988. Models That Predict Standing Stock of Stream Fish From
Habitat Variables. PNW-GTR-213: U.S. Department of Agriculture, U.S. Forest Service, Pacific Northwest
Research Station.
Franklin, J.F., C.S. Bledsoe, and J.T. Callahan. 1990. Contributions of the long-term ecological research
program. Bioscience 40:509-523.
Grover, H.D., and H.B. Musick. 1990. Shrubland encroachment in southern New Mexico, U.S.A.: an analysis of
desertification processes in the American Southwest. Climate Change 16:165-190.
Hughes, R.M. 1989. Ecoregional biological criteria. In: Water Quality Standards for the 21st Century, 147-151.
EPA68C80006, EPA/600/D-89/167. Washington, DC: U.S. Environmental Protection Agency, Office of
Water.
Holland, A.F. 1990. Near Coastal Program Plan for 1990: Estuaries. EPA/600/4-90/033. Narragensett, Rl: U.S.
Environmental Protection Agency, Office of Reasearch and Development, Environmental Research
Laboratory.
Hunsaker, C.T., and D.E. Carpenter, eds. 1990. Ecological Indicators for the Environmental Monitoring and
Assessment Program. EPA/600/3-90/060. Research Triangle Park, NC: U.S. Environmental Protection
Agency, Office of Reasearch and Development, Atmospheric Research and Exposure Assessment
Laboratory.
Karr, J.R., D.D. Fausch, P.L. Angermeir, P.R. Yant, and I.J. Schlosser. 1986. Assessing Biological Integrity in
Running Waters: A Method and Its Rationale. Special Publication No. 5. Champaign, IL: Illinois Natural
History Survey.
Karr, J.R. 1991. Biological integrity: A long-neglected aspect of water resource management. Ecological
Applications 1:66-84.
Karr, J.R., and D.R. Dudley. 1981. Ecological perspective on water quality goals. Environmental Management
5:55-68.
Kelly, J.R., and M.A. Harwell. 1990. Indicators of ecosystem recovery. Environmental Management 14:527-545.
Kepner, C.H., and B.B. Tregoe. 1981. The New Rational Manager. Princeton, NJ: Princeton Research Press.
Knapp, C.M., D.R. Marmorek, J.P. Baker, K.W. Thornton, J.M. Klopatek, and D.F. Charles. 1991. The Indicator
Development Strategy for the Environmental Monitoring and Assessment Program. EPA/600/3-91/023.
Corvallis, OR: U.S. Environmental Protection Agency, Environmental Research Laboratory.
McKenzie, D.H., D.E. Hyatt, and V.J. McDonald, eds. 1992. Ecological Indicators. Vol. 1 & 2. New York, NY:
Elsevier Applied Science.
EMAP Indicator Development Strategy
72
-------
Messer, J.J. 1990. EMAP indicator concepts.. In: Ecological Indicators for the Environmental Monitoring and
Assessment Program, C.T. Hunsaker and D.E. Carpenter, eds., 2-1 through 2-26. EPA/600/3-90/060.
Research Triangle Park, NC: U.S. Environmental Protection Agency, Atmospheric Research and
Exposure Assessment Laboratory.
Messer, J.J., R.A. Linthurst, and W.S. Overton. 1991. An EPA Program for monitoring ecological status and
trends. Environmental Monitoring and Assessment 17:67-78.
Minshall, G.W., K.W. Cummins, R.C. Petersen, C.E. Gushing, D.A. Burns, J.R. Sedell, and R.L. Vannote. 1985.
Developments in stream ecosystem theory. Canadian Journal of Fisheries and Aquatic Sciences
42:1045-1055.
Moore, J.C., and P.C. de Ruiter. 1991. Temporal and spatial heterogeneity of trophic interactions within below-
ground food webs. Agriculture, Ecosystems, and Environment 34:371-397.
NAS (National Academy of Sciences). 1975. Planning for Environmental Indices. Washington, DC.: Planning
Committee on Environmental Indices to the Environmental Studies Board.
Norton, S.B., D.J. Rodier, J.H. Gentile, W.H. van der Schalie, W.P. Wood, and M.W. Slimak. 1992. A framework
for ecological risk assessment at the EPA. Environmental Toxicology and Chemistry 11:1663-1672.
Olsen, A.R., ed. 1992. The Indicator Development Strategy for the Environmental Monitoring and Assessment
Program. EPA/600/3-91/023. Corvallis, OR: U.S. Environmental Protection Agency, Office of Research
and Development, Environmental Research Laboratory.
OMB (Office of Management and Budget). 1974. Standard Federal Regions. OMB Circular A-105. Washington,
DC: OMB. [April 4]
Ott, W.R. 1978. Environmental Indices: Theory and Practice. Ann Arbor, Ml: Ann Arbor Science Publications.
Overton, W.S., D. White, and D.L. Stevens, Jr. 1990. Design Report for EMAP (Environmental Monitoring and
Assessment Program). EPA/600/3-91/053. Corvallis, OR: U.S. Environmental Protection Agency,
Environmental Research Laboratory.
Paulsen, S.G., D.P. Larsen, P.R. Kaufmann, T.R. Whittier, J.R. Baker, D.B. Peck, J. McGue, D. Stevens, J.L
Stoddard, R.M. Hughes, D. McMullen, J. Lazorchak, and W. Kinney (with contributions by: W.S.
Overton, J. Pollard, D. Heggem, G. Collins, A. Selle, M. Morrison, C.B. Johnson, S. Thiele, R. Hjort, S.
Tallent-Halsell, K. Peres, S. Christie, and J. Mello). 1991. Surface Waters Monitoring and Research
Strategy—Fiscal Year 1991. EPA/600/3-91/022. Corvallis, OR: U.S. Environmental Protection Agency,
Environmental Research Laboratory.
RAF (Risk Assessment Forum). 1992. Framework for Ecological Risk Assessment. EPA/630/R-92/001.
Washington, DC: U.S. Environmental Protection Agency.
Rapport, D.J., H.A. Regier, and T.C. Hutchiinson. 1985. Ecosystem behavior under stress. American Naturalist
125:617-640.
Rapport, D.J. 1989. What constitutes ecosystem health? Perspectives in Biology and Medicine 33(1): 120-132.
Reilly, W.J. 1989. Measuring for environmental results. EPA Journal May/June:2-4.
Ryder, R.A., and S.R. Kerr. 1978. The adult walleye in the percid community—A niche definition based on feeding
behavior and food specificity. In: Selected Coolwater Fishes of North America, R.L. Kendall, ed.,
American Fisheries Society Special Publication 11:39-51.
EMAP Indicator Development Strategy
73
-------
SAB (Science Advisory Board). 1990. Evaluation of the Ecological Indicator Report for EMAP: Report of the
Ecological Monitoring Subcommittee of the Ecological Processes and Effects Committee. EPA-SAB-
EPEC-91-001. Washington, DC: U.S. Environmental Protection Agency.
Suter, G.W. 1990. Endpoints for regional ecological risk assessments. Environmental Management 14:9-23.
Urquhart, N.S., W.S. Overton, and D.S. Birkes. 1993. Comparing sampling designs for monitoring ecological
status and trends: Impact of temporal patterns. In: Statistics and the Environment, V. Barnett and K.F.
Turkman, eds., 71-85. New York, NY: John Wiley and Sons.
U.S. Environmental Protection Agency, 1992. Framework for Ecological Risk Assessment. EPA/630/R-93/001.
Risk Assessment Forum. U.S. Environmental Protection Agency. Washington, DC 20460.
Westman, W.E. 1985. Ecology, Impact Assessment, and Environmental Planning. New York, NY: Academic
Press.
White, D., A.J. Kimerling, and W.S. Overton. 1992. Cartographic and geometric components of a global sampling
design for environmental monitoring. Cartography and Geographic Information Systems 19:5-22.
Wiens, J. 1989. Spatial scaling in ecology. Functional Ecology 3:385-387.
Wischmeier, W.H., and D.D. Smith. 1978. Predicating Rainfall Erosion Losses -A Guide To Conservation
Planning. Agriculture Handbook No. 537, Washington, DC: U.S. Department of Agriculture.
EMAP Indicator Development Strategy
74
&U.S. GOVERNMENT PRINTING OFFICE: 1994 - 550-OOI/OOWZ
------- |