EPA-SAB-EPEC-LTR-94-004
Novembers, 1993
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401M Street, S.W.
Washington, D.C. 20460
Subject: Review of the Environmental Monitoring and Assessment Program's Draft
Assessment Framework
Dear Ms. Browner:
On June 21-23, 1993, the Ecological Processes and Effects Committee (EPEC) of the
Science Advisory Board (SAB) met to review the draft Assessment Framework (dated May 14,
1993) for the Environmental Monitoring and Assessment Program (EMAP). EPEC has reviewed
strategic aspects of the EMAP Program since its inception: evaluation of EMAP ecological
indicators (EPA-SAB-EPEC-91-001), and the EMAP program plan (EPA-SAB-EPEC-91-011,
EPA-SAB-EPEC-LTR-92-008). EMAP's Assessment Framework is a critical component of the
program since it provides the framework for interpreting and evaluating EMAP results to answer
policy-relevant questions about ecological resources.
In Future Risk: Research Strategies for the 1990's (EPA-SAB-EC-88-040) and in
Reducing Risk: Setting Priorities and Strategies for Environmental Protection (EPA-SAB-EC-90-
021), the SAB recommended that EPA plan, implement, and sustain a long-term monitoring and
research program and report on the status and trends in environmental quality. Several years ago,
partially in response to this recommendation, the Office of Research and Development (ORD)
initiated EMAP. EMAP's goal is to monitor and assess the condition of the Nation's ecological
resources and to contribute to decisions on environmental protection and management.
Specifically, EMAP is intended to evaluate the Nation's ecological resources in terms of status
and trends, geographic coverage and extent, and associations between stresses and ecological
condition.
-------
In June 1993, EPEC was asked by ORD to review the draft EMAP Assessment
Framework with respect to the following questions:
a) Is the proposed assessment approach consistent with EMAP objectives?
b) Is the EMAP Assessment Framework appropriate to guide assessments in EMAP?
c) Are the contributions and limitations of the proposed EMAP assessment approach
to the Ecological Risk Assessment process clear?
We have organized our comments and recommendations under these three broad
questions.
1. Is the proposed assessment approach consistent with EMAP objectives?
a) Importance of Assessment
We continue to feel that a greater priority should be placed on Assessment and Reporting
within EMAP. While development of the Assessment Framework and the creation of an
assessment team are positive steps, resources allocated to this component have been limited
(approximately $600,000 in Fiscal Years 1991 and 1992, and approximately $700,000 in Fiscal
Year 1993). Assessment and reporting, which entails the aggregation, interpretation, and
communication of data from different resource types, locations, and scales, are the heart of
EMAP and is the process which turns monitoring data into information. We urge the Agency
to allocate a greater portion of the EMAP budget to the Assessment and Reporting component of
the program.
b) Role of Research in EMAP
We acknowledge that the integration and assessment component of EMAP will probably
require research to develop new techniques to ensure that EMAP assessment goals can be met.
However, we feel that the research component of EMAP should be carefully delineated and
separated from the data collection process. If the program is to deliver timely assessments of
resource conditions, program managers will need to rely primarily on established indicators and
analytical methods. Research into new methods and procedures at all steps of EMAP is
unnecessary. EMAP should take advantage of existing state-of-the-science methods and
approaches, recognize and identify the associated levels of uncertainty, and proceed with the
development and analysis of monitoring data. Program managers
-------
should, however, retain sufficient flexibility to incorporate new monitoring methods and indicators
that may be developed in the future.
Nonetheless, research on methods for conducting regional assessments will be a major
contribution from the EMAP Assessment and should be mentioned under Assessment Tools and
Guidelines in the Assessment Framework.
c) Stressor Monitoring
Although the primary emphasis of EMAP's monitoring effort should be on effects, we
agree with the incorporation of some stressor monitoring into the program. The EMAP
Assessment Framework should more explicitly indicate the approach that will be used to
incorporate stressor information into the assessment process. The relation of resource
characteristics to potential stress is an important aspect of EMAP that can help to explain
observed changes in ecological resources. Therefore, inclusion of stress indicators in the database
is important and valuable. However, these data should be largely derived from other non-EMAP
sources, where available.
In cases where stressor information is deficient or non-existent, carefully selected
indicators of stress should be monitored by EMAP. In the case of forests, for example, it may be
appropriate for EMAP to include some wet and dry deposition monitoring for atmospheric
contaminants. Forest floors can provide extended term records of persistent chemicals such as
heavy metals and chlorinated organic compounds. In addition, EMAP monitoring of rural/forest
ozone may be the only means to obtain remote ozone exposure values.
2. Is the EMAP Assessment Framework appropriate to guide assessments?
Overall, the draft EMAP Assessment Framework clearly communicates the directions of
the EMAP program and provides an appropriate mechanism for information management and
evaluation. We commend the authors for the critical thinking that is evident in the document and
for the progress this represents. We recommend, however, that the Agency revise the document
to address the following issues:
a) Diagnosis: The Great Lakes Example
The Assessment Framework indicates that EMAP will utilize "weight of evidence" and
"process of elimination" approaches to identify the most likely associations between effects and
causal agents. The weight of evidence criteria listed in Table 6 of the document are appropriate
and well-tested factors. Careful application of these criteria will allow consideration of a wide
array of data analyses, both parametric and non-parametric, when characterizing temporal and
-------
spatial associations. However, the results from a process of elimination are highly dependent on
the scale at which temporal and/or spatial associations are sought. For example, the Assessment
Framework includes a case study where the process of elimination is used to narrow the list of
factors responsible for the widespread biotic changes in the Great Lakes over the last four
decades. The choice of scale (the entire Great Lakes system), and the fact that toxic chemicals
are broadly distributed across all six of the Great Lakes, led to the conclusion that impacts of
toxic chemicals should be a dominant factor in the observed degradation of the biotic
communities. However, this spatial association analysis does not consider concentration,
exposure, effects, etc.
In fact, we know from direct observation that the ecological impacts to the Great Lakes
were caused by different factors in different lakes, e.g.: demise of the Atlantic salmon in Lake
Ontario was caused by habitat destruction; the perturbation of Lake Erie was due to
eutrophication caused by phosphorus loadings; and the declines of the fish stocks in Lakes Huron,
Michigan and Superior were due to overfishing and introduction of exotic species (e.g., sea
lamprey and alewives). None of these events was caused by the impacts of xenobiotic chemicals.
The lesson learned from this example is that temporal and spatial associations should be
considered preliminary hypotheses of causal relationships. Empirical validation, through
literature surveys, professional interviews, and historical case studies, should be established before
further analyses and/or final conclusions are drawn. Rigorous application of the criteria for
inferring causality (particularly criteria four through eight in Table 6 of the Framework) would
have drastically modified the conclusions presented in the Great Lakes case study.
Another concern with the Great Lakes assessment is that different ecological resources
(e.g., fish, mammals, and birds) were assessed in relation to a common set of stressors. This
approach ignores the fact that different ecological resources can be (are) at risk due to very
different kinds of stresses.
b) Nominal vs. Subnominal Condition
The draft EMAP Assessment Framework discusses classification of resource condition
into nominal and subnominal categories (pp 33-34). This section of the document should be
revised to reflect the role of both societal values and ecological variability in this classification.
For example, the definition of nominal vs. subnominal condition is based on a decision as to the
desired state for ecological resources. The Assessment Framework should describe how the
definition of nominal/subnominal conditions in EMAP relates to similar decisions under other
Agency programs (e.g., definition of reference condition under the Biocriteria Program).
In addition, the Framework should discuss the effect of natural variability on the definition
4
-------
of nominal/subnominal conditions. The terms "nominal condition" and "subnominal condition", as
defined in the draft Framework, do not necessarily equate with "natural" and "anthropogenically
altered" conditions, respectively. In other words, many natural conditions can result in
uninhabitable, or "subnominal", environments. Such environments are not undesirable, but rather
are an integral part of the natural world.
Finally, statistical methods that have been developed to estimate expected values are not
necessarily applicable for estimating the extreme values used to define subnominal conditions of
EMAP indicators. The Assessment Framework should reference statistical methods (e.g.,
extreme event analysis) designed for this purpose.
c) Consideration of Uncertainty
We feel strongly that the Assessment Framework should include a discussion of the
sources and nature of uncertainties inherent in EMAP assessments. A critical component of
ecological risk assessment that differentiates this activity from traditional assessment is the explicit
consideration of uncertainties. These uncertainties are propagated through the assessment
towards an ultimate expression of risk. Assumptions and uncertainty issues will affect the
accuracy of predictions and correlations made in all phases of EMAP, and will ultimately have a
major influence on the ability of the program to achieve its objectives. Readers, user groups, and
researchers should all be made aware of the limitations inherent in the process. An early and full
discussion of these issues should be included in future drafts of the Framework.
d) Integration Across Resource Groups
We agree that the monitoring and assessment should be applied across the basic resource
groups (agroecosystems, arid ecosystems, etc.), as well as across landscapes. (Note, however,
that the preferred term for the broad scale group is "landscapes" rather than "landscape ecology,"
as contained in Figure 5 of the draft Framework.) In keeping with this emphasis, we urge that the
integration aspects of EMAP assessment be given sufficient visibility and resources as a distinct
task. There are cross-cutting issues and integration methodologies that are of sufficient
importance to warrant the focus of a specific unit within EMAP. Currently, much of the
integration and assessment activity is occurring within the individual resource groups. We
recommend that EMAP establish a centralized group,
-------
including representatives from each resource group, with the mission of integration and
assessment.
3. Is the relation between the proposed EMAP Assessment Approach and EPA's Ecological
Risk Assessment Framework clear?
EMAP is an example of the application of the Framework for Ecological Risk Assessment
(Ecorisk Framework, EPA/630/R-92/001) at regional and national scales. As such, the EMAP
Assessment Framework should incorporate the terminology and diagrams used in the Ecorisk
Framework, rather than portraying the EMAP assessment approach as an alternative to the
Ecorisk Framework. For example, the steps in EMAP assessments (assessment questions, data
analysis, and interpretation and communication) correspond directly to the components of ecorisk
assessment (problem formulation, analysis and risk characterization), the selection of
nominal/subnominal conditions corresponds to the selection of endpoints, etc. Since the Ecorisk
Framework will lead to the over-arching Ecological Risk Assessment Guidelines, all Agency
programs should adopt the Ecorisk Framework nomenclature when possible. The use of different
terminology in the EMAP Assessment Framework is confusing and unnecessary, and should be
corrected in future drafts.
In order to further clarify the relationship between the EMAP Assessment Framework and
the Ecorisk Framework, we recommend the following changes to the Assessment Framework:
a) Delete references to the Ecorisk Framework as being predictive rather than
retrospective. In fact, the Ecorisk Framework is designed for both predictive and
retrospective ecological risk assessments. Thus, EMAP's emphasis on
retrospective assessment fits within the Ecorisk Framework.
b) Indicate that EMAP assessments are not external to the ecorisk assessment
process, and include all phases of ecorisk assessment. This point might best be
made by revising Figure 4 in the Assessment Framework, and substituting Ecorisk
Framework terminology for that used in Figures 6, 7, 10 and 14 of the Assessment
Framework.
These changes are more than cosmetic, for they emphasize the relation between the two efforts.
-------
Final Comments
We appreciate the opportunity to review the draft EMAP Assessment Framework. As the
conceptual guide to assessments within EMAP, we feel the Framework document is important
and well-conceived. We understand that work is now underway to develop an EMAP
Assessment Methods Manual. The development of EMAP assessment tools and guidelines will
provide an important opportunity to couple the EMAP Assessment Framework with other related
EPA efforts (e.g., biocriteria, bioindicators, ecorisk issues, and habitat characterization) which are
focusing on operational techniques. Developments from these related programs must be
incorporated into the EMAP assessment guidelines, either directly or by reference, to provide a
consistent and compatible approach to ecological assessment for the Agency. We hope to have
an opportunity to review the EMAP Assessment Methods Manual which is being developed, and
we look forward to your response to the issues raised in this letter.
Sincerely,
/signed/ /signed/
Dr. Raymond C. Loehr, Chair Dr. Kenneth L. Dickson, Chair
Executive Committee Ecological Processes and
Science Advisory Board Effects Committee
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
ECOLOGICAL PROCESSES AND EFFECTS COMMITTEE
ROSTER
June 21-23, 1993
CHAIR
Dr. Kenneth L. Dickson, Institute of Applied Sciences, University of North Texas, Denton,
Texas
MEMBERS/CONSULTANTS
Dr. Steven M. Bartell, SENES Oak Ridge, Inc., Center for Risk Analysis, Oak Ridge, Tennessee
Dr. Edwin L. Cooper, Department of Anatomy and Cell Biology, School of Medicine, UCLA,
Los Angeles, California
Dr. William E. Cooper, Zoology Department, Michigan State University, East Lansing,
Michigan
Dr. Virginia Dale, Environmental Sciences Division, Oak Ridge National Laboratory, Oak
Ridge, Tennessee
Dr. Mark A. Harwell, Rosenstiel School of Marine and Atmospheric Science, University of
Miami, Miami, Florida
Dr. Robert J. Huggett, Virginia Institute of Marine Science, College of William and Mary,
Gloucester Point, Virginia
Dr. Alan W. Maki, Exxon Company, USA, Houston, Texas
Dr. Frederic K. Pfaender, Institute for Environmental Studies, University of North Carolina,
Chapel Hill, North Carolina
Dr. Anne McElroy, SUNY at Stoney Brook, Stoney Brook, New York
Dr. William H. Smith, Professor of Forest Biology, School of Forestry and Environmental
Studies, Yale University, New Haven, Connecticut
-------
Dr. Terry F. Young, Environmental Defense Fund, Oakland, California
SCIENCE ADVISORY BOARD STAFF
Ms. Stephanie Sanzone, Designated Federal Officer, Science Advisory Board (1400F), U.S.
EPA, 401 M Street, S.W., Washington, DC 20460
Mrs. Marcia K. Jolly, Staff Secretary, Science Advisory Board (1400F), U.S. EPA, 401 M
Street, S.W., Washington, DC 20460
-------
NOTICE
This report has been written as part of the activities of the Science Advisory Board, a
public advisory group providing extramural scientific information and advice to the Administrator
and other officials of the Environmental Protection Agency. The Board is structured to provide
balanced, expert assessment of scientific matters related to problems facing the Agency. This
report has not been reviewed for approval by the Agency and, hence, the contents of this report
do not necessarily represent the views and policies of the Environmental Protection Agency, nor
of other agencies in the Executive Branch of the Federal government, nor does mention of trade
names or commercial products constitute a recommendation for use.
-------
DISTRIBUTION LIST
Administrator
Deputy Administrator
Assistant Administrators
Deputy Assistant Administrator for Research and Development
EPA Regional Administrators
EPA Laboratory Directors
EPA Regional Libraries
EPA Laboratory Libraries
------- |