UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
June 22, 2005
EPA-SAB-05-011
The Honorable Stephen L. Johnson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: SAB Review of the EPA Region 5 Critical Ecosystem Assessment Model
Dear Administrator Johnson:
The EPA Region 5 Office of Strategic Environmental Assessment requested that the
Science Advisory Board (SAB) review the methodology and conceptual framework used in the
Region's Critical Ecosystem Assessment Model (CrEAM). The CrEAM was developed to
identify ecologically significant areas in Region 5 in order to quantify and track ecosystem
quality, target areas for protection, prioritize protection activities, and provide information to
conduct National Environmental Policy Act reviews. A panel of the SAB Ecological Processes
and Effects Committee augmented by experts in ecology and the use geographic information
systems has reviewed the CrEAM. The enclosed SAB report addresses EPA's charge questions
to the Panel and provides recommendations for improvements in future versions of the CrEAM
to make the model more useful to EPA.
Addressing regional issues is a critical concern for the Nation. To date, environmental
information is typically not available until after key decisions are made. Tools like the CrEAM
will facilitate access to environmental information early in the decision-making process at an
appropriate spatial scale. Therefore, the SAB enthusiastically supports the development of
regional tools like the CrEAM. In developing the CrEAM, EPA Region 5 has made an important
initial effort to incorporate an understanding of ecological condition in the environmental
decision making process at EPA. The SAB notes, however, that there are limitations associated
with the methodological approach presently used in the CrEAM to identify areas of ecological
importance. These limitations restrict the usefulness of the CrEAM and must be considered in
any application of the model.
-------
SUMMARY OF RECOMMENDATIONS
The SAB finds that:
The CrEAM offers great promise as a regional screening level approach to identifying
critical landscapes. The CrEAM, as presented, can be an appropriate regional tool for
allocating internal EPA resources for site inspection activities, tracking general trends in
the regional landscape condition, and reviewing grant proposals to the Agency. The
CrEAM is also an appropriate framework to foster further communication and dialogue
between other federal and state agencies on the use of regional and spatial data in
environmental decision-making.
EPA's proposed uses of the CrEAM are not all fully supported by the science underlying
the model. The CrEAM, as presented, is not reliable for use in regulatory processes such
as issuing or reviewing air and/or water quality permits; use as a basis for federal or state
agency determination in National Environmental Policy Act (NEPA) reviews; use as a
basis for setting compliance, enforcement or cleanup actions; or for establishing reference
context for ecological protection and restoration. Such uses could, however, be
supported by later versions of the CrEAM.
In summary, the SAB finds that CrEAM holds great promise as a tool for use in
identifying critical landscapes. Although limitations restrict the usefulness of the current version
of the CrEAM, the SAB has provided recommendations for improvements in the model. The
SAB believes that for CrEAM to be an important tool, computational limits and validity issues
must and can be overcome. It will be necessary to invest resources to upgrade CrEAM with the
most recent versions of Arc View and Spatial Analyst and also to devote personnel to continued
development of the model. Enhancing the predictive validity of the model from both an
ecological and a statistical perspective will continue to be important.
Sincerely,
/signed/ /signed/
Dr. M. Granger Morgan, Chair Dr. Virginia Dale, Chair
EPA Science Advisory Board EPA Science Advisory Board
Ecological Processes and
Effects Committee and
Chair of the Review Panel
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board,
a public advisory group providing extramural scientific information and advice to the
Administrator and other officials of the Environmental Protection Agency. The Board is
structured to provide balanced, expert assessment of scientific matters related to the problems
facing the Agency. This report has not been reviewed for approval by the Agency and, hence,
the contents of this report do not necessarily represent the views and policies of the
Environmental Protection Agency, nor of other agencies in the Executive Branch of the Federal
government, nor does mention of trade names or commercial products constitute a
recommendation for use. Reports of the EPA Science Advisory Board are posted on the EPA
Web site at http://www.epa.gov/sab.
-------
U.S. Environmental Protection Agency
Science Advisory Board
Critical Ecosystem Assessment Model Review Panel
CHAIR
Dr. Virginia Dale, Corporate Fellow, Environmental Sciences Division, Oak Ridge National
Laboratory, Oak Ridge, TN (Chair of SAB Ecological Processes and Effects Committee)
MEMBERS
Mr. DeWitt Braud, Director Academic Area, Coastal Studies Institute, Louisiana State
University, Baton Rouge, LA
Dr. Peter Curtis, Professor, Department of Evolution, Ecology, and Organismal Biology, Ohio
State University, Columbus, OH
Dr. Ivan Fernandez, Professor and Chair, Department of Plant, Soil and Environmental
Sciences, University of Maine, Orono, ME (Member of SAB Ecological Processes and Effects
Committee)
Dr. Judith Meyer, Distinguished Professor, Institute of Ecology, University of Georgia, Athens,
GA (Member of SAB Ecological Processes and Effects Committee)
Dr. Thomas Mueller, Professor, Department of Plant Sciences, University of Tennessee,
Knoxville, TN (Member of SAB Ecological Processes and Effects Committee)
Dr. Michael Newman, Professor of Marine Science, School of Marine Sciences, Virginia
Institute of Marine Science, College of William and Mary, Gloucester Point, VA (Member of
SAB Ecological Processes and Effects Committee)
Dr. Charles Pittinger, Managing Scientist, Exponent, Cincinnati, OH (Member of SAB
Ecological Processes and Effects Committee)
Dr. Amanda Rodewald, Assistant Professor, School of Natural Resources, Ohio State
University, Columbus, OH
Dr. James Sanders, Director, Skidaway Institute of Oceanography, Savannah, GA (Member of
SAB Ecological Processes and Effects Committee)
Mr. Timothy Thompson, Engineer/Scientist, the RETEC Group, Seattle, WA (Member of
SAB Ecological Processes and Effects Committee)
Ms. Sandra Williams, Senior Environmental Specialist, President, Blueskies Environmental
Associates, Inc., Richmond, VA
-------
SCIENCE ADVISORY BOARD STAFF
Dr. Thomas Armitage, Designated Federal Officer, U.S. Environmental Protection Agency,
Washington, DC
in
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 5
3. CHARGE TO THE REVIEW PANEL 6
4. REVIEW PROCESS 7
5. RESPONSE TO THE CHARGE QUESTIONS 7
5.1 Charge question 1.1 7
5.2 Charge question 1.2 8
5.3 Charge Question 1.3 16
5.4 Charge Question 2.1 17
5.6 Charge Question 3.1 31
6. REFERENCES 33
APPENDIX A: SPECIFIC COMMENTS FROM INDIVIDUAL COMMITTEE
MEMBERS AND TECHNICAL CORRECTIONS A-l
IV
-------
1. EXECUTIVE SUMMARY
The Science Advisory Board Critical Ecosystem Assessment Model Review Panel was
charged with reviewing the Critical Ecosystem Assessment Model (CrEAM) developed by the
EPA Region 5 Office of Strategic Environmental Assessment. The CrEAM is a spatially explicit
model for predicting the ecological condition of undeveloped land using ecological theory,
existing data sets, and geographic information system (GIS) technology. The CrEAM was
developed to identify ecologically significant areas in EPA Region 5 in order to: quantify and
track ecosystem quality, target areas for protection, prioritize protection activities, and provide
information to conduct National Environmental Policy Act reviews.
EPA Region 5 has provided several examples illustrating how the CrEAM could be used.
The CrEAM could be used to identify areas that are high in diversity and low in sustainability.
Permitted discharges to such ecologically rich but threatened areas might be reduced from usual
levels, reduced during breeding seasons, or conditioned on additional ambient monitoring. The
CrEAM could also be used to provide ecosystem scale trend analysis for tracking environmental
improvements due to restoration and protection efforts as well as documenting degradation of
environmental quality across the Region at a landscape scale. In inspection, enforcement, or
granting activities, CrEAM scores could be used to prioritize workloads or grant awards by
identifying areas that are potentially ecologically rich or threatened. For example, the
Underground Injection Control Program for the state of Michigan has expressed interest in using
the CrEAM to help prioritize well inspections. EPA Region 5 has also indicated that analysts
reviewing National Environmental Policy Act (NEPA) Environmental Impact Statements (EIS)
could benefit from knowing the relative ecological significance of various options being
proposed. In addition, the CrEAM could be used to help identify areas that might be restored in
enforcement settlement agreements where permit violators voluntarily agree to Supplemental
Environmental Projects.
EPA Region 5 sought the SAB's comments on the scientific validity of the conceptual
framework and methodology used to identify ecologically significant ecosystems and on the
scientific defensibility of the results generated from CrEAM queries. EPA Region 5 gave the
following charge questions to the EPEC.
Question 1. Conceptual Framework
1.1 Is EPA use of the term "ecological significance" appropriate as EPA has defined it? Is there
a better term for what is being rated?
1.2 Is it scientifically defensible to use spatial data as indicators of the three ecological criteria?
(diversity, sustainability, and rarity) and to generate ratings of the criteria by compositing these
indicators?
1.3 Is the nesting and compositing of multiple indicator data sets a scientifically valid
framework to rate ecosystems?
-------
Question 2. Methodology
2.1 Are the three criteria sufficient and reasonable for rating ecological significance as defined?
2.2 Are the indicators sufficient and reasonable for rating the ecological diversity, self
sustainability, and biological and land-cover rarity as defined?
2.3 Are there any relevant data sets consistently collected across the 6-state area of EPA Region
5 that should have been used but were not? If one or more such data sets exist, is the value they
add to the CrEAM likely to exceed the cost of adding them to the model?
Question 3. Application of the CrEAM to Environmental Decision-Making
3.1 Please comment on the scientific defensibility of the use of CrEAM results to support broad
based strategic planning and priority setting activities (e.g., identifying locations for geographic
initiatives and EPA/State joint efforts) and program activities such as:
Inspection
Permitting
Enforcement and cleanup
Reviewing grant proposals
Establishing reference context for ecological protection and restoration
The SAB strongly supports the efforts of EPA Region 5 to develop the CrEAM and
encourages EPA to continue to improve the model. In developing the CrEAM, EPA Region 5
has made a good initial effort to strengthen ecological engagement in the environmental
decision-making process at EPA. The SAB notes, however, that there are a number of
limitations associated with the methodological approach used in the CrEAM to identify areas of
ecological importance. These limitations are surmountable, but additional resources will be
necessary. The work accomplished to date has contributed substantially toward the development
of a vital database of information, but the modeling task is not yet complete. The SAB provides
specific comments and recommendations in response to the EPA's charge questions.
It is the strong opinion of the SAB that the term "ecological significance" does not
optimally reflect the nature of the CrEAM methodology. Consideration of ecological
processes and functions were not part of the CrEAM. It is the recommendation of the
SAB that EPA should instead use a neutral term to describe what is being rated in the
CrEAM. This term should emphasize the technical nature of the CrEAM. The SAB
recommends using terms such as: "the CrEAM ecological metric", "CrEAM ecological
condition", or "biotic and landscape condition."
The SAB finds that it is scientifically defensible to use spatial data as indicators of the
three ecological criteria used in the CrEAM (diversity, sustainability, and rarity). Spatial
indicators in the CrEAM can be composited to generate ratings of landscape condition.
However, the SAB has identified significant limitations associated with the
methodological approach used in the CrEAM. The SAB notes that the data layers used in
-------
the CrEAM are currently assigned equal weights in the analysis. This interim reliance on
even weighting is a basic assumption of the CrEAM that is a source of concern about the
current model. The current CrEAM model cannot be used to generate statistically
significant results for decision making because it has not been "ground truthed" using
empirical data to validate the weights used to combine indicator variables into a single
CrEAM ecological metric. It would be preferable to derive the data layer weights used in
the CrEAM from a statistical analysis (e.g., regression analysis) that determines how
variations in the underlying data layers actually contribute to differences in the types of
direct ecological assessments for which the CrEAM ecological metric is intended to serve
as a proxy. These limitations, and others discussed below, restrict the usefulness of the
model and must be considered in any application of the current model. As discussed
below, in order to add credence to the CrEAM, the SAB encourages EPA to perform a
robust validation of the model.
The SAB finds that that nesting and compositing of multiple indicator data sets is a
scientifically valid approach for rating "CrEAM ecological condition." However, the
SAB notes that, as currently developed, the CrEAM fails to completely characterize and
rate areas of ecological importance. This is because the scale and dimensions of the
CrEAM and data layers used in the model do not provide the level of detail required to
accurately assess exposure resulting from ecosystem stressors (including their sources,
intensity, proximity, and frequency). The SAB also notes that the methodological
approach used in the current version of CrEAM does not appear to be applicable to
several key components of ecological systems. Aquatic systems are not adequately
considered, and connectivity resulting from water flowpaths has been ignored. In
addition, small but potentially keystone systems are not a part of the analysis.
The SAB finds that the three fundamental criteria developed in the current version of the
CrEAM offer great promise for use in a regional screening level approach to identifying
critical landscapes. However, as a means to characterize landscape stressors for
management or permitting purposes, the SAB finds that the CrEAM is incomplete,
inadequate, and unreliable. In order to more clearly and precisely articulate the key
landscape criteria and data layers used in the CrEAM, the SAB recommends that the
three criteria used in the model be renamed. The use of the ecological diversity criterion
is conceptually appropriate. However, because the CrEAM deals with landscapes, the
SAB recommends that the "ecological diversity" criterion in the model might be more
accurately titled "landscape diversity." It is recommended that "persistence,"
"resistance," or "vulnerability" would be better terms to reflect the self-sustainability
metric developed in the CrEAM. The SAB supports the use of the "rarity" criterion
developed in the CrEAM. However, it is recommended that the "rarity" criterion used in
the model be renamed "landscape rarity" to distinguish it from species, community, or
ecosystem rarity.
The SAB finds that the indicators used in the CrEAM for rating ecological diversity and
biological and land-cover rarity are generally supported by underlying ecological
principles. However, the indicators used to rate the "self-sustainability" criterion in the
model are more problematic in scope and content. The SAB notes that a number of
-------
limitations must be considered when using some CrEAM indicator data sets to rate
ecological diversity, self-sustainability, and rarity. The SAB has identified limitations
associated with selected indicator data layers. In some cases additional indicator data are
identified for use in the 6-state area of EPA Region 5.
Although the CrEAM documentation briefly describes proposed uses of the model, the
SAB recommends that EPA Region 5 include additional information to further describe
EPA's goals in developing the CrEAM. The SAB finds that the CrEAM index, as
presented, can be an appropriate regional tool for the allocation of internal EPA resources
for site inspection activities, tracking general trends in the regional landscape quality, and
reviewing grant proposals to the Agency. The CrEAM is also an appropriate framework
to foster further communication and dialogue between other federal and state agencies on
the use of regional and spatial data in environmental decision-making. The SAB endorses
the Region's validation process for the CrEAM index. The SAB also urges that future
developments of the CrEAM be consistent with the principles embodied in EPA's
Guidance on the Development, Evaluation, and Application of Regulatory Environmental
Models (U.S. EPA Office of Science Policy, 2003).
The SAB finds that underlying science does not support the use of the current version of
the CrEAM in any environmental decision-making or regulatory processes. This would
include, but is not exclusive to, issuing or reviewing air and/or water quality permits, as a
basis for the EPA or any other federal or state agency's determination in National
Environmental Policy Act (NEPA) reviews, as a basis for setting compliance,
enforcement or cleanup actions, or for establishing reference context for ecological
protection and restoration. While these are functions that the SAB envisions could
eventually be supported by later versions of the CrEAM index, application of CrEAM in
its current iteration to environmental decision-making is not scientifically defensible.
The SAB further stresses the need for EPA to make it clear that CrEAM is only one tool
and should only be used in conjunction with other tools and factors that affect internal
resource allocation in the near-term or for broader decision or policy related issues in the
future.
In summary, the SAB finds that CrEAM holds great promise as a tool for use in
identifying critical landscapes. Although limitations restrict the usefulness of the current
version of the CrEAM, the SAB has provided recommendations for improvements in the
model. The SAB believes that for CrEAM to be an important tool, computational limits
and validity issues must and can be overcome. It will be necessary to invest resources to
upgrade CrEAM with the most recent versions of Arc View and Spatial Analyst and also
to devote personnel to continued development of the model. Enhancing the predictive
validity of the model, from both an ecological and a statistical perspective, will continue
to be important.
-------
SAB Review of the EPA Region 5 Critical Ecosystem Assessment Model
2. INTRODUCTION
This report transmits the advice of the U.S. Environmental Protection Agency (EPA)
Science Advisory Board (SAB) Critical Ecosystem Assessment Model Review Panel. The Panel
met on June 29-30, 2004 to review the Critical Ecosystem Assessment Model (CrEAM)
developed by the Critical Ecosystems Team in the EPA Region 5 Office of Strategic
Environmental Assessment. The CrEAM was developed to identify ecologically important areas
in Region 5 in order to: quantify and track ecosystem quality, target areas for protection,
prioritize protection activities, and provide information to conduct National Environmental
Policy Act reviews.
The CrEAM is a spatially explicit model for predicting the ecological condition of
undeveloped land using ecological theory, existing data sets, and geographic information system
(GIS) technology. The model has been used to predict the locations of ecosystems of high
ecological condition in the Region. Twenty data sets were used in the CrEAM. These data sets
were developed from existing data, entered into a geographic information system, and converted
into twenty spatially explicit GIS data layers with associated attributes. The twenty data sets
were used as indicators for three criteria that were used to define ecological condition. These
three ecological condition criteria are the potential for: 1) ecological diversity, 2) self-
sustainability, and 3) biological and land-cover rarity. Of the twenty indicator data sets used in
the model, four provided an indication of diversity, twelve indicated sustainability, and four
indicated biological and land-cover rarity. Indicators for each of the three ecological condition
criteria were combined by summing their values at a scale of 300 m x 300 m. In this way, three
composite GIS layers were generated to predict spatially explicit ratings for the ecological
condition criteria in undeveloped areas of EPA Region 5 (EPA Region 5 covers the states of
Illinois, Indiana, Michigan, Minnesota, Ohio, and Wisconsin). The CrEAM thus provided
ratings for each of the three ecological condition criteria in 300 m x 300 m cells in undeveloped
land within EPA Region 5.
The CrEAM documentation states that because the ratings for each of the ecological
condition criteria were statistically independent, the composite data layers for the criteria in the
CrEAM can be used individually or in combination to predict ecological condition of an area. If,
for example, it is important to use summary information solely about diversity, sustainability, or
rarity, each composite data layer could be used individually. The CrEAM documentation further
states that if it is important to combine two or three of these criteria ratings, they could be
summed for each 300 m x 300 m cell. As discussed below, the SAB notes that even if the ratings
for each of the CrEAM ecological criteria are statistically independent, summing the criteria may
not result in a meaningful ecological metric. Additional analysis could provide empirical support
for the CrEAM metric.
The SAB strongly supports the efforts of EPA Region 5 to develop the CrEAM. In
developing the CrEAM, EPA Region 5 has made a good initial effort to introduce ecological
perspective into an environmental decision-making tool. The SAB notes, however, that there are
-------
a number of limitations associated with the methodological approach used in the CrEAM to
identify areas of ecological importance. These limitations restrict the usefulness of the CrEAM
and must be considered in any application of the model. The SAB provides recommendations
for improvements in the CrEAM and encourages EPA to continue development of the model.
The SAB also wishes to recognize the Regional staff that developed the CrEAM. The wealth of
information and extensive knowledge of the subject matter, as well as the professionalism
displayed by the authors and their colleagues, were invaluable to the SAB as it conducted this
review.
3. CHARGE TO THE REVIEW PANEL
EPA Region 5 sought the SAB's comments on the scientific validity of the conceptual
framework and methodology used to identify ecologically important ecosystems and on the
scientific defensibility of the results generated from CrEAM queries. EPA Region 5 gave the
following charge questions to the SAB panel.
Question 1. Conceptual Framework
1.1 Is EPA use of the term "ecological significance" appropriate as EPA has defined it? Is there
a better term for what is being rated?
1.2 Is it scientifically defensible to use spatial data as indicators of the three ecological criteria?
(diversity, sustainability, and rarity) and to generate ratings of the criteria by compositing
these indicators?
1.3 Is the nesting and compositing of multiple indicator data sets a scientifically valid
framework to rate ecosystems?
Question 2. Methodology
2.1 Are the three criteria sufficient and reasonable for rating ecological significance as defined?
2.2 Are the indicators sufficient and reasonable for rating the ecological diversity, self
sustainability, and biological and land-cover rarity as defined?
2.3 Are there any relevant data sets consistently collected across the 6-state area of EPA Region
5 that should have been used but were not? If one or more such data sets exist, is the value
they add to the CrEAM likely to exceed the cost of adding them to the model?
Question 3. Application of the CrEAM to Environmental Decision-Making
3.1 Please comment on the scientific defensibility of the use of CrEAM results to support
broad based strategic planning and priority setting activities (e.g., identifying locations for
geographic initiatives and EPA/State joint efforts) and program activities such as:
Inspection
-------
Permitting
Enforcement and cleanup
Reviewing grant proposals
Establishing reference context for ecological protection and restoration
4. REVIEW PROCESS
To establish the CrEAM review panel, the EPA Science Advisory Board Staff Office
published a Federal Register notice requesting nominations to augment the expertise of members
on the SAB's Ecological Processes and Effects Committee (EPEC). The SAB Staff Office then
identified a subset of the nominees for consideration as panelists. The final panel was selected
after requesting public comments on the nominees and further evaluating them against EPA
Science Advisory Board selection criteria. The members of the review panel included ecologists
on the Ecological Processes and Effects Committee as well as additional members with expertise
in ecology and the use geographic information systems.
The review was conducted in a two-day face-to-face public meeting. At the public
meeting, the review panel heard presentations from EPA Region 5 staff on: 1) the conceptual
approach and proposed uses of the CrEAM, 2) the architecture of the CrEAM, 3) the indicator
data layers and criteria measures in the CrEAM, and 4) model validation and results. The panel
then deliberated on each of the charge questions and developed the final SAB report.
5. RESPONSE TO THE CHARGE QUESTIONS
5.1 Charge question 1.1. Is EPA use of the term "ecological significance" appropriate
as EPA has defined it? Is there a better term for what is being rated?
It is the strong opinion of the SAB that the term "ecological significance" does not
optimally reflect the nature of the CrEAM methodology. CrEAM is a regional spatial model
resulting in an index. Consideration of ecological processes and functions were not part of the
CrEAM. Because of this and other model limitations discussed below, it is the recommendation
of the SAB that EPA should instead use a neutral term to describe what is being rated in the
CrEAM. This term should emphasize the technical nature of the CrEAM. The SAB
recommends using terms such as: "the CrEAM ecological metric," "the CrEAM index,"
"CrEAM ecological condition," or as discussed below, "biotic and landscape condition". The
SAB notes that self-sustainability, one of the three criteria used in the CrEAM to rate areas of
"ecological significance," provides an assessment of environmental vulnerability. In this regard
the CrEAM shares a similar purpose with EPA's Regional Vulnerability Assessment (ReVA)
approach (U.S. EPA, 2004b). ReVA was developed by EPA's Office of Research and
Development to inform decision-makers about anticipated environmental vulnerabilities within a
geographic region.
-------
5.2 Charge question 1.2. Is it scientifically defensible to use spatial data as indicators of
the three ecological criteria (diversity, sustainability, and rarity) and to generate
ratings of the criteria by compositing these indicators?
The SAB finds that it is scientifically defensible to use spatial data as indicators of the
three ecological criteria used in the CrEAM (diversity, sustainability, and rarity). Spatial
indicators in the CrEAM can be composited to generate ratings of landscape condition.
However, there are a number of significant limitations associated with the methodological
approach used in the CrEAM. The following limitations of the CrEAM restrict the usefulness of
the model and must be considered in any application of the model. The SAB also notes that the
data layers used in the CrEAM have not been weighted (the parameters are all weighted equally)
in the analysis. As discussed below, this lack of a weighting may further limit the usefulness of
the CrEAM because it is not always valid to assume that factors used in the analysis are equally
significant. In order to add credence to the CrEAM, the SAB also encourages EPA to perform a
robust validation of the model.
Limitations of Model Approach
Lack of applicability of methodological approach. The SAB notes that the
methodological approach used in the current version of the CrEAM does not appear to
fully address several key components of ecological systems. For example, aquatic
ecological systems are not adequately represented or considered, and connectivity
resulting from water flowpaths has been ignored. The SAB also notes that hydraulic and
hydrologic conditions, nutrient loads, and contaminant loads are important factors to
consider in determining ecological condition, but are not surrogates for ecological
condition. In addition, small potentially keystone systems are not a part of the analysis.
These systems are not considered because the cell size applied in the model is 300 meters
by 300 meters, and any patch occupying an area less than 10 hectares was eliminated
from consideration. Small wetlands or vernal ponds are an example of an ecosystem type
that would be overlooked in this analysis. Furthermore, consideration of ecological
processes and functions and their corresponding goods and services were not a part of
CrEAM approach. The CrEAM analysis is also temporally confined since it only uses
1990's data. This implies that the model cannot deal with major events such as changes
in climate, recent disturbances such as storms and fires, or changes in land use.
Ecological principles are not set forth clearly in the CrEAM. The ecological principles
underlying the use of each data set in the CrEAM are not clearly articulated in the current
documentation. The SAB recommends that the ecological principles and limitations
associated with the use of each data set be clearly articulated. In addition, the rationale
for selecting data manipulation approaches should be fully documented.
The current approach to normalization of the data layer scores and weights in the CrEAM
does not preserve the validity of the underlying statistical relationships between
dependent and explanatory variables in the model. As stated previously, the weights
associated with each of the underlying data layers in the CrEAM should be estimated by
conducting an analysis to determine how variations in the data layers contribute to
-------
differences in the CrEAM ecological metric. The approach currently used in the CrEAM
is to normalize the scores assigned to the indicator data layers to values from 0-100, and
to assign equal weights to the data layers. The SAB notes that this approach does not
preserve the fundamental statistical relationships between the underlying variables and
the ecological condition criteria.
Normalization of data layer scores is inconsistent. The SAB notes that there is some
inconsistency in the approach used to normalize the scores assigned to the indicator data
layers in the CrEAM. In all of the data layers and the resultant criteria layers, scores
were normalized to values ranging from 0 - 100. However, some of the data layers were
normalized using continuous metrics, others were normalized using binomial metrics, and
in some cases scores were normalized by assigning values to frequency distribution
groupings. Combining continuous metrics with binomial metrics results in
disproportionate weightings of certain data layers in the aggregate criteria score (an
example of this is data layer C2.9 watershed obstructions). The SAB notes that this
approach has introduced some bias into the model and recommends that EPA look for
alternatives to normalization using binomial metrics. The SAB also notes that text
describing each data layer in the CrEAM should indicate how the data layer was scored
or scaled from 0-100; this is not done in every case.
CrEAM assessments are influenced by availability of data. The SAB notes that the
usefulness of the CrEAM is limited by the paucity of region-wide data in the model to
reflect ecological processes and natural disturbance regimes. Table 10 of the draft
CrEAM methodology provides a crosswalk between the data layers used in the model
and the essential ecological attributes identified by the Science Advisory Board for use in
assessing and reporting ecological condition (U.S. EPA Science Advisory Board, 2002).
An examination of Table 10 shows that there are no data layers in the CrEAM related to
ecological processes. One CrEAM data layer, temperature and precipitation maxima, is
used in the model to relate natural disturbance regimes to landscape diversity. However,
the SAB finds that it is not appropriate to use temperature and precipitation data as input
in this context. Hence there are no data layers in the CrEAM reflecting the two essential
ecological attributes of ecological processes and natural disturbance regimes. Moreover,
the SAB is not aware of any systematic, region-wide data that could fill this gap. This
data gap effectively restricts the scope of the CrEAM from the original goal of predicting
"ecological significance" (which would at a minimum require data input for essential
ecological attributes) to a more narrowly defined assessment of biotic and landscape
condition. The SAB emphasizes, however, that biotic condition and landscape condition
are two important ecological attributes identified by the SAB's Ecological Processes and
Effects Committee. The CrEAM does incorporate adequate data layers to represent these
attributes. Therefore, the SAB finds that a more appropriate title for the CrEAM might
be, "CrEAM: a Method to Assess Regional Biotic and Landscape Condition."
The lack of available data in a number of CrEAM data layers is also problematic. The
SAB notes the following sources of uncertainty introduced into the model as a result of
lack of available data.
-------
1. Data representing the abundance of rare species or higher taxonomic units can be
provided by all states within EPA Region 5 through their Natural Heritage Programs.
However, the legal agreement reached by EPA Region 5 with the six state Natural
Heritage Programs requires that these data be summarized only at the 7.5 minute USGS
quadrangle scale. This requirement presents a basic mismatch with both the predominant
scale of analysis of landscape condition in the CrEAM (the "cell" or .9 hectares) and the
scale used in the model for analysis of biotic condition (the quadrangle, or 10 hectares).
The SAB emphasizes that it is very important to continue using the biotic data in the
CrEAM. However, to address the scale problem, the SAB recommends placing a high
priority on obtaining measures of species diversity that can be mapped at a finer scale.
2. The paucity of relevant hydrological data in the CrEAM limits its use in assessment of
aquatic ecosystems and the vital hydrologic connections that occur on the landscape.
3. The CrEAM relies very heavily on the Kuchler map of potential vegetation to
characterize the temporal continuity of land-cover type (data layer C1.4) and land-cover
suitability (data layer C2.12). The SAB recognizes that this map was used because all
states do not have good data on pre-settlement vegetation. However, the SAB notes that
over reliance on the Kuchler map introduces uncertainty into model assessments. The
SAB also notes that data layers C1.4 (temporal continuity of land cover type) and C2.12
(land cover suitability) are exactly the same, one should probably be eliminated.
4. The CrEAM relies upon measures of water quality stressors (ambient concentrations of
dissolved oxygen, nitrate and nitrite-nitrogen, and total suspended solids data obtained
from EPA's Storage and Retrieval, STORET, database). EPA's Better Assessment
Science Integrating Point and Nonpoint Sources (BASINS) software was used in CrEAM
assessments to derive average concentrations of these water quality parameters across
USGS hydrologic cataloging units. The SAB notes that uncertainty is introduced into
CrEAM assessments because available water quality data in STORET may not be
representative of undeveloped areas where few water quality samples are collected. No
information on water quality contaminants such as metals (e.g., mercury) or persistent
organics (e.g., PCBs) is included in the CrEAM water quality summary data layer. The
CrEAM assessments also rely upon predicted ambient air pollution concentrations and
human health benchmarks for air toxics. Further uncertainty is introduced into CrEAM
assessments because, although these benchmarks may represent reasonable proxies for
assessing stress on ecological endpoints, the benchmarks are not quantitatively
appropriate for "non-human" stress assessment.
5. The accuracy of the National Land Cover Database (NLCD) land-cover data is generally
very poor. Since so many layers rely on these data, it is obvious that the results would be
substantially improved by orders of magnitude if a better land-cover database were
developed.
The SAB recommends that these sources of uncertainty be considered in any application
of the CrEAM and addressed when improvements are made to the model.
10
-------
Chemical contamination data used in the CrEAM ambiguously reflect ecological
exposure. The current information in the CrEAM on National Priority List (NPL)
Superfund sites and Resource Conservation and Recovery Act (RCRA) corrective action
sites is ambiguously reflective of ecological exposure, and the ecological effects
associated with contaminants at these sites are likely to be very local. More pervasive
toxicant effects in EPA Region 5 are likely to be associated with atmospheric deposition
of mercury and persistent organic pollutants (POPs). State fish tissue monitoring
programs have relevant data on these contaminants. The SAB recommends that these
important data be used in the CrEAM in addition to the NPL Superfund site and RCRA
corrective action site data. The SAB also notes that pesticides and herbicides are likely to
be important stressors in EPA Region 5 and recommends that efforts be undertaken to
obtain usage data for widely used and pervasive pesticides and herbicides. The SAB also
recommends that EPA Region 5 determine whether fertilizer use data could be used in
the CrEAM as a potential source of information about stresses on local systems. It
should be noted, however, that pesticide and fertilizer use is quite variable over time.
Point sources, such as sewage treatment plant discharges and confined animal feeding
operations may also be more important pollutant sources to be considered in the CrEAM.
Undeveloped land-cover categories in the CrEAM are not well supported by land-cover
characteristics. The SAB notes that focusing EPA efforts on ecological resources at risk
by using "undeveloped" land-cover categories from the National Land Cover Database
(NLCD) is a meritorious objective. There is no question that ecological valuation often
gives way to the pressures of limited resources and to the clarity and passion behind the
identification of human health concerns. However, the SAB finds the "undeveloped"
land-cover category to be largely an artificial distinction that is not well supported by the
characteristics of the land-cover categories. All of the land-cover categories in EPA
Region 5 are influenced by human endeavors through global effects on the chemical and
physical character of the atmosphere and by the historical effects of humans through
agriculture, fire management, and modern multi-use management of forested, wetland,
and aquatic resources. The CrEAM makes no distinction between abandoned farmland
now in plantation forestry and areas growing native forests, nor is there distinction
between natural lakes and reservoirs created by dams. It is possible that excluding the
"developed" land-cover units in the CrEAM limits the integrity of various metrics in the
overall model.
The SAB therefore recommends that EPA reconsider the distinction between
"developed" and "undeveloped" land-cover units and include more or even all of the
NLCD land-cover categories in the CrEAM. The SAB notes that this would not appear
to be a significant task in light of the availability of the data. Having the entire region
represented in the CrEAM could improve the model by making identification of buffer
zones and issues of remoteness much more spatially explicit. EPA should consider using
different terminology to define categories of land-cover units and developing an approach
that can utilize more or even all of the land-cover units by relying on the system of
metrics to eliminate units that are not suitable. This is preferable to using arbitrary
distinctions of a whole group of NLCD categories. Developing a future version of
CrEAM that utilizes all of the NLCD categories should improve linkages between
11
-------
ecological condition and the sources of stressors on the landscape such as high intensity
development. The SAB also notes that NLCD satellite imagery should be referred to in
the CrEAM as NLCD classified satellite imagery.
Spatial linkages of hydrologic systems are not incorporated into the CrEAM. The
CrEAM does not adequately incorporate hydrologic linkages into the methodological
approach. The model is therefore of limited use for characterizing aquatic ecosystems.
Hydrologic linkages impact ecosystem condition in many ways. Active groundwater
recharge areas can impact distant ecosystems, particularly in EPA Region 5 where
wetlands fed by groundwater are important features of the landscape. These patches of
the landscape are critical areas where contaminants can be introduced into aquatic
ecosystems or where disruptions of hydrologic connectivity can have profound impacts.
Another illustration of the importance of hydrologic linkages rests on the observation that
aquatic ecosystems are sensitive to alterations of hydrologic regime. For example, two
stream reaches, each flowing through a forested landscape, could be profoundly different
if the headwaters of one are in catchments with 40% impervious surface cover and the
headwaters of the other are in catchments with no impervious surfaces. Because
hydrologic linkages have not been incorporated into the CrEAM analysis, these
significant differences would not be detected. The SAB recommends that EPA
incorporate data into the CrEAM to represent hydrologic linkages. Databases on
groundwater recharge areas should be available for EPA Region 5, and it is
recommended that these data be used in the CrEAM.
Scale is a major determinant that is operative on several levels in the CrEAM. The
following issues should be acknowledged and discussed in the model documentation.
Data scale must be appropriate to capture variation in data (spatial frequency). Data scale
must be appropriate for decision making (e.g., it is not possible to make a decision
regarding one acre of land when 10 hectares are filtered out). Scale issues related to data
aggregation, resampling and reseating functions should be discussed. The SAB notes that
there is a large mixture of scales in the CrEAM. This is probably unavoidable, but it does
cause a concern and should at least be acknowledged. Some explanation of the large
difference in sizes of squares should be provided.
Weighting Spatial Data Layers in the CrEAM
Data layers used in the CrEAM have been equally weighted in the analysis. The SAB
recognizes that weighting the data layers is a difficult task, and that weighting can create serious
problems if not expertly and accurately implemented. However, there are situations in which it
is desirable to provide weights to data layers that are being summed because it is not always
valid to assume that factors are equally significant in an analysis. This is particularly true when
many data layers are used and when subsets of these data may tend (to some degree) to measure
the same underlying factors, so that the data layers are correlated. Therefore, the SAB
recommends that EPA conduct additional analyses to determine how a more appropriate
weighting scheme can be applied to the individual data layers used in the CrEAM. When a
dependent variable is available, regression-type analyses are typically used to infer the
appropriate weights on each of the data layers. In the absence of a dependent variable, however,
12
-------
other methods will have to be devised to support whatever weighting scheme is chosen. Equal
weights are as arbitrary as any other weights in the present version of the CrEAM. The SAB
also notes that EPA's Regional Vulnerability Assessment Program has explored the use of a
number of statistical methods for integrating spatial data sets across a region (Smith, Iran &
O'Neill, 2003). The SAB provides the following advice to EPA for weighting spatial data layers
in the CrEAM.
Consider the number of factors used in the analysis. The SAB notes that, in the present
formulas, as the number of factors (variables, layers, or elements) in the analysis
increases, the accuracy of output results can change. In addition, the significance and
thus impact of truly important factors is diluted as the number of factors increases. For
example, the equal weighting of the three final criteria in the CrEAM makes the
significance of each of the layers in criterion C-2 (self-sustainability) much less than the
data layers in the other criteria. This is because there are so many more layers in this
criterion. On the surface, it might seem that this problem could be alleviated by
eliminating insignificant data layers. However, it is never a good idea to throw away
information, however minimally it might contribute to one's understanding of variations
in the ecological metric. If a regression method could be used to infer the weights on the
different data layers, there would be a statistical basis for concluding which layers should
be kept and which have no discernible effect on ecological status. With random sampling
and statistical analysis, formal hypothesis tests can be conducted to determine for which
data layers a weight of zero cannot be rejected.
Be clear about assumptions involved in assigning weights arbitrarily. If the current equal
weights on the data layers reflect the true relationships in the data, it would only be a
result of a remarkable coincidence. Incorrect weights produce biased predictions about
how a higher value along one dimension (data layer) can make up for a lower value along
another dimension (data layer). When assigning weights, it is important to question
whether the same scores in different data layers are truly equal.
In lieu of moving to a regression based method to infer the proper weights, some
members of the SAB have suggested considering signed (positive and negative) scales for
scores rating data layers. Weighting can potentially introduce unreliable results when it
is based on a simple linear numerically positive scale of unsigned values. Applying
weights to the low end of a positive scale may unintentionally increase the importance of
a variable when it should be decreased. For example, if a low positive value can be
interpreted as an undesirable (negative) attribute, and if a weight greater than one is
applied, the poor rating will improve when in fact it should get worse (smaller), not better
(larger). Use of a signed (±) scale can reduce this problem.
Other members of the SAB point out that if a regression perspective is adopted, it is
completely unnecessary to manipulate the scales whereby each data layer is measured. If
the variable is continuously measured, it can be entered directly, in its natural units.
Alternately, transformations can be explored (e.g. logarithms, quadratic forms,
generalized power transformations such as the Box-Cox, etc.). If it is binary, it can be
represented as a binary (0,1) "dummy" variable. If it is categorical variable, it can be
13
-------
captured by a set of "dummy" (0,1) variables. If data are available for only a subset of
plots, then an indicator for data availability can be activated, and for those plots where
the data layer is available, the incremental effect of that information can be exploited.
Interactions among the different variables can also be explored, to determine whether the
effect of a one-unit difference in one variable depends upon the current level of another
variable. These effects are not merely asserted, but estimated. Hypotheses about the
signs and sizes of these effects are statistically refutable.
Similarly, consider the erstwhile dependent variable for the relevant regression-type
model. It would be a directly assessed ecological rating (or the central tendency of a set
of ratings) for each of a sample of plots drawn at random from the population of interest.
Nothing requires that this variable be continuous. Modern statistical methods can
accommodate ratings as dependent variables, and maximum likelihood methods can be
used to explain the probabilities that particular out-of-sample plots would get ordinal
ratings at each different level. If multiple ratings are solicited for the estimating sample
of plots, there will be valuable information in the extent of any disagreement between the
individual ratings. This "noise" can also be exploited in the estimation phase.
It is important to consider data layer interactions or interrelationships. A model that is
linear and additively separable in the layers (let alone one with equal weights on all
layers) does not account for the interaction or interrelationships among data layers.
Arbitrary weights can be assigned to imply such relationships, but often the structure of
interrelationships among data layers is difficult to identify and quantify. If a sample of
directly assessed ecological ratings could be used to estimate the weights, inferring the
interrelationships among data layers would be a natural part of the analysis.
Arbitrary weighting implies assumptions about relationships among data layers, and
these relationships are usually scale-dependent. One problem with the current approach
is that the absence of any independently assessed dependent variable means there is no
viable source of information to use in calibrating the model. But even within a regression
context, changes in the scale of measurement of the data layers will change the sizes of
the weights. Provided there are no interaction effects, desired weights could be attained
by scaling the explanatory variables so that they are consistent with equal weights.
Alternately, the explanatory variables can be measured on any arbitrary scales and the
estimated weights take up the slack. But it is impossible to specify both the weights and
the scaling, yet to preserve the real relationships between the dependent variable and the
data layers being used to explain variations in this dependent variable.
Representation of landscape features is affected by scale, resolution, cell size,
aggregation of data, and filtering. Landscape features and relationships among features
can change or become lost at certain scales. Values representing variation in attributes
can be significantly modified by cell size, resolution and aggregation based on
boundaries, categorization, or grouping. When landscape features are rescaled or
aggregated, the Modifiable Areal Unit Problem (MAUP) (Openshaw, 1984) is
introduced, and this affects weights. In regression analysis, this concern corresponds to
the problem of measurement error in the explanatory variablesthe "errors-in-variables"
14
-------
problem. This problem tends to produce attenuation in the estimated weight on the
variable that is measured with error and, unfortunately, indeterminate biases in the other
weights in the model. However, the problem is one of degree. (Note that it does not
matter what analytical method is used to deal with such data deficiencies; the
qualifications that measurement error necessitates will be present for all methods.)
Model Validation
In order to add credence to the CrEAM, the SAB strongly encourages EPA Region 5 to
perform as complete and robust a validation of the model as possible. Additional sensitivity
analyses could be completed to understand the influence and/or weight of the underlying model
layers on the model output. One technical issue concerning validation of the CrEAM is that the
model output is a unitless parameter, which is the composite of several other scaled and non-
scaled factors. Therefore, it is impossible to validate the model by directly measuring a given
pixel for the value of the model output. For example, if a given pixel or cell has a model output
of 240, there is no way to directly measure that value of the cell. There is not presently a
dependent variable in the CrEAM that can be used to statistically calibrate the data layer weights
in the model. Therefore, it is not possible to measure the substitutability between one feature of
the CrEAM and another in producing the same overall level in any of the three criteria used to
derive the ecological condition metric. As discussed below, on-the-ground inspection of a
random sample of plots resulting in qualitative ratings along the dimensions targeted by the
CrEAM is an approach that can be used for statistical calibration of the model. The SAB
recommends that EPA recruit statistical expertise to collaborate with ecologists in conducting
such an analysis. Model output can be labeled in a number of different ways including: discrete
numerical values, percentile rankings, letter groupings, and placing various groupings into
"bins." The SAB recommends that EPA consider the advantages and disadvantages of
alternative approaches to labeling model output and include a strong section on model limitations
in the CrEAM in order to avoid misuse of the model.
The SAB also urges EPA to pretest the application of CrEAM results to decision making
in an explicit situation and with input on the way decision makers use the model. There is a need
for dialogue with decision makers about how to use the CrEAM. In addition, the CrEAM should
be subjected to the standard EPA guidance for models as set forth in the Agency's Guidance on
the Development, Evaluation, and Application of Regulatory Environmental Models (U.S. EPA
Office of Science Policy, 2003).
A long-term goal for the use of regional models in decision making is to have well
documented statistical models of cause and effect. The current CrEAM model cannot be used to
generate statistically defensible results for decision making because it has not been "ground
truthed" using empirical data to validate the weights used to combine indicator variables into a
single CrEAM ecological metric. The current weights represent untested hypotheses. As such,
the limitations on the data, model, and general approach need to be clearly set forth. The SAB
notes that regression analysis might be an appropriate basis upon which to determine the correct
indicator weights and provide empirical support for the CrEAM metric. Such an analysis might
be completed by having experts conduct on-the-ground inspections of a sample of plots to
develop subjective but quantitative ratings that could form the dependent variables in the
15
-------
CrEAM. Observed rating values could be combined in regression-type models with the set of
explanatory indicator variables (continuous or discrete) to estimate appropriate and defensible
weights for use in the model.
If it is not possible to directly assess a dependent variable for a sample of plots that can
be used to calibrate the CrEAM model, then it is also the case that there will be no way to
adequately validate the output of the CrEAM model and it is questionable whether the numbers
produced by the CrEAM formula can be used for any purpose. As they stand, the weights that
are being used to combine the data layers can be viewed only as hypotheses about the "true"
weights that could be derived by regression analysis. These hypotheses are, as yet, untested.
If this is truly the case that no dependent variable can be mustered, either for regression
or simply for validation, the EPA may wish to back away from the goal of producing a model
that is in any way predictive of something. Instead, the EPA might consider using factor analytic
methods that reduce the array of available data layers to a smaller set of factors (formed by linear
combinations of the raw data) that span approximately the same space. The loadings that
produce these factor scores can illuminate the subsets of variables that tend to be correlated
across plots and can help clarify the number of unique dimensions of variation in the data. Just
as this kind of analysis has been applied to reduce the dimensionality of the demographic
characteristics of census tracts (e.g. Cameron and Crawford, 2003), it could be used to reduce the
dimensionality of the characteristics of different plots of land. However, this is a very different
undertaking from the goals of the CrEAM at present.
5.3 Charge Question 1.3. Is the nesting and compositing of multiple indicator data sets
a scientifically valid framework to rate ecosystems?
The SAB finds that that nesting and compositing of multiple indicator data sets is a
scientifically valid approach for rating ecological condition, although there are advantages and
disadvantages associated with such an approach. The SAB notes that, as currently developed,
the CrEAM fails to completely characterize and rate areas of importance. This is because the
scale and dimensions of the CrEAM and data layers used in the model do not provide the level of
detail required to accurately assess exposure resulting from ecosystem stressors (including their
sources, intensity, proximity, and frequency). The SAB also notes that the methodological
approach used in the current version of CrEAM does not appear to be applicable to several key
components of ecological systems. Aquatic systems are not adequately considered, and
connectivity resulting from water flowpaths has been ignored. In addition, small potentially
keystone systems are not a part of the analysis.
The principal advantage of nesting and compositing multiple indicator data sets is that
this methodology provides a single metric for describing the critical uniqueness of a landscape,
and thus establishes a common comparative basis upon which many landscapes can be ranked.
EPA is using such an approach in the Agency's Regional Vulnerability Assessment (ReVA)
Program to conduct comprehensive integrated regional assessments, quantify regional ecological
vulnerabilities, and target and prioritize risk management activities (U.S. EPA, 2004). The SAB
recognizes the appeal of a simple ranking or scoring system for broad program development and
organizational planning. Inevitably, the question of where to invest resources must be addressed.
16
-------
The SAB suggests that the CrEAM could be one useful tool for informing such a decision, but
other tools and criteria (e.g., characterizing stressors, economics, perceived value to the public,
etc.) must also be considered.
There are several disadvantages of composite scoring systems. First, they tend to mask
potentially useful information that may underscore key aspects of a unique landscape. For this
reason, some environmental assessment approaches have adopted a "score-card approach" in
which a number of discrete descriptors are developed and maintained for independent
consideration. Such an approach might provide composite scores for the three ecological
condition criteria, or even subsets of criteria based on underlying data layers. The second
disadvantage of composite scoring systems is that single scores used to rate or rank landscapes
can be misconstrued or misapplied in resource management decisions. As discussed above, the
uncertainty and variability of the CrEAM scoring system has not been determined, and it is not
clear what minimum difference in scores is environmentally significant. For this reason, the
SAB suggests that an alternative approach might be to avoid continuous quantitative scoring
systems by adopting categories or "bins" which link similar characteristics of landscapes into
logical groupings. The third disadvantage of composite scoring is that it implicitly requires some
form of weighting of various attributes, often based on the subjective perceptions of the user or
developer. In the current model, the assignment of scores evenly for each of the data layers
represents a weighting approach based on an assumption that each data layer or criterion is
equally important to identification of a critical landscape. While this may be true or
questionable, there is no clear basis for making such an assumption.
5.4 Charge Question 2.1. Are the three criteria sufficient and reasonable for rating
ecological significance as defined?
The CrEAM model, as currently developed, is based on three fundamental criteria:
ecological diversity, self-sustainability (consisting of landscape fragmentation and stressor
presence), and rarity. Within each of these criteria are discrete data layers that describe the
criteria. The SAB finds that use of the three fundamental criteria to rate the CrEAM ecological
metric is reasonable but, as discussed above in section 5.2, renaming the criteria is
recommended. The SAB also notes that there are limitations associated with the use of CrEAM
indicator data sets. These limitations are discussed in the response to charge question 2.2 below.
Calculation of three discrete criterion categories, rather than lumping all indicator data
sets, is advantageous because it allows separate examination of diversity/rarity and
risks/stressors. This is useful in identifying areas that need additional protection or regulation.
As discussed below, the three criteria in the CrEAM did not represent all of the essential
ecological attributes identified in the SAB's "Framework for Reporting on Ecological
Condition" (U.S. EPA Science Advisory Board, 2002). Only landscape condition and biotic
condition were well represented by all three criteria. Physical/chemical characteristics and
hydrology/geomorphology were addressed in the sustainability criterion. Natural disturbance
regimes and ecological processes were virtually absent from the criteria in the CrEAM. The
SAB acknowledges that it might be difficult if not impossible to represent ecological processes
and disturbance regimes in the CrEAM. Instead of "retooling" the model to represent these
ecological attributes, the SAB recommends that EPA consider including more explicit language
17
-------
in the model documentation to describe what the criteria are rating (i.e., the criteria rate
"landscape and biotic attributes" rather than "ecological significance").
Alternative Terms for CrEAM Criteria
The SAB finds that the three fundamental criteria developed in the current version of the
CrEAM offer great promise for use in a regional screening level approach to identifying critical
landscapes. However, as a means to characterize landscape stressors for management or
permitting purposes, the SAB finds that the CrEAM is incomplete, inadequate, and unreliable.
In order to more clearly and precisely articulate the key landscape criteria and data layers used in
the CrEAM, the SAB recommends that the three criteria used in the model be renamed. The use
of the ecological diversity criterion is conceptually appropriate. However, because the CrEAM
deals with landscapes, the SAB recommends that the "ecological diversity" criterion in the
model might be more accurately titled "landscape diversity." This terminology will avoid
confusion with other levels of biological organization.
The SAB finds that use of the "self-sustainability" criterion in the model is problematic in
several respects, both in naming conventions and more importantly in scope and content. The
SAB notes that the term "sustainability" carries a number of different connotations to diverse
audiences and can easily be misconstrued. The modified term, "self-sustainability," implies a
mechanism for landscapes themselves to foster their own preservation. This is somewhat vague
and illogical. The SAB is also concerned that higher self-sustainability rankings are assigned to
systems that can persist for 100 years, preferably without external management. The SAB notes
that almost all ecosystems within the Till Plains are historically disturbance-maintained (e.g.,
grassland and oak-savannah). These systems now exist in landscapes with altered disturbance
regimes (e.g., fire suppression) that render them non self-sustaining. Nevertheless, their
ecological importance is still great. The SAB also notes that the indicator data sets in the
CrEAM do not include measures of processes, which are probably the most important elements
of self-sustainability. In addition, the concept and valuation of self-sustainability as developed in
the CrEAM seems to bias the metric against early serai stages, yet these are important ecological
systems in a landscape mosaic. The SAB also notes that most of the data sets supporting the
self-sustainability metric describe fragmentation that may make a system less likely to persist. It
is recommended that "persistence," "resistance," or "vulnerability" would be better terms to
reflect the self-sustainability metric developed in the CrEAM. However, the appropriateness of
any terms adopted for the criteria ultimately depend on the larger question of their scope, content
and intent.
The SAB supports the use of the "rarity" criterion developed in the CrEAM. Use of
rarity may provide the only opportunity to account for local or unique areas. However, it is
possible that accelerated declines in ecological condition and biodiversity in EPA Region 5 could
lead to reclassification of species by heritage databases, and this might lead to increased values
for rarity (i.e., at some point, rarity will decrease because once-common species become rare). It
is recommended that the "rarity" criterion used in the model be renamed "landscape rarity" to
distinguish it from species, community, or ecosystem rarity.
18
-------
Organization, Scope, and Content of the Criteria
The SAB finds that EPA's proposed uses of the CrEAM are not all fully supported by the
science underlying the model. This has compromised the chief merits of the model as it
presently exists. As noted above, the CrEAM offers great promise as a regional screening level
approach to identifying critical landscapes, but the model is not reliable for characterizing
landscape stressors for management or permitting purposes. The current list of stressor data
layers grouped in the "self sustainability" criterion is incomplete and sketchy. For example,
many key chemical contaminants are not represented under "water quality" or "air quality." The
arbitrary distinction between "developed" and "undeveloped" lands in the model also excludes
the majority of key stressors and their sources from consideration (e.g., habitat loss from
urbanization). Inadequate information is currently available for characterizing stressors in a
useful management context (e.g., identifying the sources of the stressors and a management plan
to mitigate or preclude additional stress on those systems). The absence of hydrologic linkages
renders the model unable to consider downstream effects of upstream stressors.
Although EPA has developed approaches for stressor identification and mitigation (U.S.
EPA, 2000b), the Agency has not developed similar approaches for critical landscape
identification at a regional level. The CrEAM can be very useful for critical landscape
identification, and the SAB notes that the model should be recognized for its merits and not its
liabilities. Achieving this will require revision of the "self-sustainability" criterion. EPA may
wish to consider limiting the use of the CrEAM to critical landscape identification and excluding
the broad subject of ecological stressors from the model. If stressors were removed from the
model, the "self-sustainability" criterion could be renamed "landscape pattern." This term
encompasses the unique data layers comprising the criterion (i.e., perimeter to area analysis,
patch size by land-cover type, weighted road density, waterway impoundment, and land-cover
suitability). This approach would also parallel the guidance provided by the Ecological
Processes and Effects Committee of the SAB in the document "Framework for Reporting on
Ecological Condition" (U.S. EPA Science Advisory Board, 2002). In that guidance document,
this group of metrics is referred to as "landscape pattern and structure."
Comparison of the CrEAM to the SAB EPEC Framework for Reporting on Ecological Condition
The SAB notes that one element in the SAB EPEC's "Framework for Reporting on
Ecological Condition" is not included among the criteria used in the CrEAM to rate areas on the
basis of the CrEAM ecological metric. This element could be termed "landscape condition," and
it includes descriptors of the landscape's health or integrity that may be used to define an
ecosystem as critical. The SAB notes that landscape condition can be evaluated using a number
of existing assessment and management tools, so it may not be necessary to expand the CrEAM
to include this element. As noted above, there are numerous tools, many of which have been
developed by EPA, to assess the condition of ecological systems and the stressors impinging on
them. The Stressor Identification Process developed by the National Center for Environmental
Assessment in EPA's Office of Research and Development is one such tool, and it has been
applied to Darby Creek near Columbus, Ohio (within EPA Region 5) as a case study (U.S. EPA,
2000b). The Risk Screening Environmental Indicator Model is another tool developed to
identify the distribution of chemical contaminants across the United States (U.S. EPA, 2003).
19
-------
If EPA chooses to characterize landscape stressors in the CrEAM for management or
permitting purposes, it would be advantageous to consider the approach developed by the EPEC
in the Framework for Reporting on Ecological Condition. In developing this framework, the
SAB EPEC chose to distinguish condition indicators ("essential ecosystem attributes") from
stressor indicators, and did so by describing a "parallel universe" of stressor indicators with a
"cross-walk" to condition indicators. Recognizing the importance of certain "natural stressors"
(e.g., fire, flood, storms, etc.), the SAB EPEC included natural stressors within the attribute of
"natural disturbance regimes." The SAB notes that the approach of distinguishing
anthropogenic stressor indicators from ecological condition indicators has the following
advantages.
It more clearly distinguishes natural variations from human-induced variations in a
manner that facilitates environmental remediation and natural resource management.
Defining reference conditions and criteria for determining deleterious effects may be
contextual, depending upon local management or conservation goals. Societal
institutions may choose to alleviate or mitigate anthropogenic stress but would have little
control (and may be ill-advised) to alter natural background conditions and variation. In
cases where this has occurred (e.g., restricting the frequency of forest fires; altering the
course of rivers), serious consequences have been observed.
Presenting a "separate universe" of anthropogenic stressors enables more logical and
systematic relationships to be drawn between these stressors and the mechanisms through
which they impact ecosystems. Anthropogenic stressors may impact ecosystems at a
number of levels, and through both direct and indirect effects upon one or more Essential
Ecosystem Attributes. A separate presentation of anthropogenic stressors can help to
highlight the causal mechanisms underlying compromised conditions.
This approach encourages indicator selection criteria to be based upon fundamental
environmental attributes and processes rather than mere data availability. Reports on
ecosystem condition often focus primarily or exclusively on anthropogenic stressors
because data (e.g., on emissions, exceedances, incidents, etc.) are more readily collected
through conventional regulatory processes. This creates potential for overlooking
important ecosystem characteristics and prioritizing environmental risks and protection
needs inappropriately.
Distinguishing condition and stressor indicators can be helpful in allocating management
responsibilities among public and private institutions, depending upon their charter and
regulatory domain. A framework that separates yet clearly links stressor and condition
measures may lead to more comprehensive, cross-agency and cross-media coordination
of environmental management functions.
20
-------
5.5 Charge Question 2.2. Are the indicators sufficient and reasonable for rating the
ecological diversity, self-sustainability, and biological and land-cover rarity as
defined?
Charge Question 2.3. Are there any relevant data sets consistently collected across
the 6-state area of EPA Region 5 that should have been used but were not? If one or
more such data sets exist, is the value they add to the CrEAM likely to exceed the
cost of adding them to the model?
The SAB finds that the indicators used in the CrEAM for rating ecological diversity and
biological and land-cover rarity are generally supported by underlying ecological principles. As
discussed above, the SAB has identified a number of problems associated with indicators used to
rate the self-sustainability criterion. Moreover, the SAB notes that limitations must be
considered when using some CrEAM indicator data sets to rate ecological diversity, self-
sustainability, and rarity. Limitations associated with selected indicator data layers are discussed
below. In some cases additional indicator data are identified for use in the 6-state area of EPA
Region 5.
Data Layer C.I.I. Patch Sizes of Undeveloped Land
Use of the CrEAM data layer describing patch sizes of undeveloped land is partially
supported by Island Biogeography Theory (IBT). Although the use of patch size data is
emphasized in the CrEAM, there are no data included in the model to describe isolation (distance
from mainland). Application of IBT to terrestrial fragmented landscapes has received much
criticism because terrestrial systems do not fit the oceanic island model well (Anderson & Wait,
2001; Davies, Melbourne, & Margules, 2001; Gascon & Lovejoy, 1998; Harrison, 1999; Holt,
1997). In particular, the landscape matrix is not uniformly inhospitable. Permeability of the
matrix and its effects on adjacent patches depends strongly on land-use type.
The following limitations are associated with use of CrEAM data layer C. 1.1:
The landscape matrix is not explicitly considered in this data layer or in the diversity
index (Cl) as a whole. A greater value for larger sized patches does not necessarily
indicate greater ecological value. Ecological condition is highly dependent upon
landscape context. Grassland systems are an interesting example of a case where
adjacent developed (agricultural) land actually increases the value of an area to grassland
specialist species compared to the surrounding forest land (Herkert, Sample & Warner,
1996). When the entire CrEAM metric is computed, the lack of matrix effect on the
diversity index (Cl) will be ameliorated by the stressors included in the self-sustainability
index (C2). However, if the ecological diversity measure (Cl) is viewed alone, the
diversity measure may be misleading because of the presumed lack of matrix effect (i.e.,
greater area = greater diversity irrespective of landscape). If stressor measures are
included in the self-sustainability index, a caveat should be added to the model indicating
that the indices Cl and C2 should be viewed together. If stressor measures are not
21
-------
included in the self-sustainability index, matrix effects should be considered in the
diversity index.
Omission of patches of less than 10 hectares in size introduces bias into the model
increasing uncertainty and limiting application. While this omission may be a
computational necessity, it could eliminate potentially important areas from the model.
For example, a landscape might have several patches smaller than 10 hectares in size in
close proximity and therefore have high preservation and/or restoration potential. The
CrEAM may not recognize this. Keystone habitats smaller than 10 hectares in size might
influence a large area of surrounding landscape. An example is the Carolina Bays in the
coastal plains from Virginia to Florida. These small wetlands are essential habitat to a
variety of species including amphibians. The absence of these habitats significantly
changes the ecological community.
Data Layer C. 1.2. Land Cover Diversity
Land-cover diversity is an appropriate and widely accepted metric for use in quantifying
biodiversity at landscape-scale levels of biologic organization (USGS, 2001). Estimating land-
cover diversity is an integral part of the National Land Cover Diversity Project (USGS, 2001).
Land-cover diversity is used as a key indicator in Minnesota's Regionally Significant Ecological
Area Program (Minnesota Department of Natural Resources, 2004) and is also used by the
European Community to assess the impacts of agricultural practices (European Commission,
2000). The ecological principle underlying the use of land-cover data in the CrEAM is that a
higher degree of habitat diversity yields a higher degree of species richness and diversity. In
practice, documentation of that connection is tenuous and not appropriate for all species.
Nevertheless, it is a commonly accepted and applied principle.
The CrEAM makes appropriate use of the National Land Cover Database (NLCD) and
follows accepted procedures for estimating landscape diversity. Having said that, the SAB notes
that there are differences in the way the CrEAM and the NLCD estimate diversity. For example,
CrEAM estimates diversity using the Shannon-Weiner Index, while the NLCD uses the Simpson
Index. Furthermore, NLCD has developed gradations of diversity index values that are different
from those used in the CrEAM. The SAB is not suggesting that the land-cover data are used
inappropriately in the CrEAM. The European Community land-cover program also uses the
Shannon-Weiner Index (European Commission, 2000). However, the SAB recommends that the
CrEAM documentation be expanded to provide additional information about how the model fits
within the context of the National Land Cover Diversity Project jointly managed by the U.S.
EPA and the USGS.
The following limitations are associated with the use of CrEAM data layer C1.2:
The application of the diversity landscape metric is appropriate for the general uses of the
CrEAM intended by EPA Region 5. However, the spatial scale of the metric and the
implicit assumption that the nine NLCD land-cover classes used to calculate the metric
are appropriate indicators of "habitat" are not likely to be appropriate for evaluations
pertinent to the National Environmental Policy Act. The CrEAM documentation
22
-------
describes the determination of diversity for a 1 km x 1 km square. Presumably, this was
done because of computational limits, but the CrEAM documentation does not provide a
sufficient explanation of why this spatial scale was used as opposed to the 300 m x 300 m
cells used for other metrics. Spatially, a 1 km x 1 km grid will overlook smaller but
potentially keystone habitat types.
The assumption that the nine NLCD land-cover classes represent true habitat diversity is
probably not valid. Richness and diversity of animal and bird species will be dependent
upon the richness and diversity of the plant communities within the CrEAM pixels, cells,
or squares. This habitat composition is likely to be the determining metric, but the
classification system used in the CrEAM is not structured to provide sufficient resolution
to enable this level of discrimination.
Calculation of the land-cover diversity index in the CrEAM appears to follow standard
practice used in landscape-level analyses. However, it is recommended that the CrEAM
documentation indicate why the diversity calculation used in the model is different from
the calculation used in the NLCD project. Cost and resource efficiency would be
achieved if the work completed for the NLCD Project were used in the CrEAM.
Data Layer C.I. 3. Temperature and Precipitation Maxima
The authors of the CrEAM correctly point out that there is a well-established pattern at
continental and larger scales of plant species diversity increasing from temperate to tropical
regions and hence along axes of temperature and moisture. This is true for many, but not all
animal species (exceptions include aphids and salamanders) (Levin, 2001). EPA should,
however, consider the following limitations associated with the use of these data.
It is not valid to apply a very large-scale temperature and precipitation maxima pattern to
predict diversity at the very much smaller scale of the Omernik Ecoregion. At the
Ecoregion scale, other factors such as disturbance regime, soil properties, and land-use
history are the primary drivers of vegetation diversity. The SAB notes that it would not
be surprising to find an inverse relationship between species diversity and temperature
and precipitation maxima within a particular region.
The temperature and precipitation maxima data used in the CrEAM might be applied as a
diversity indicator for the entire EPA Region rather than distributing these data among
Ecoregions. Given the span of EPA Region 5, from warm, moist southern Indiana and
Ohio to cold, dry northern Minnesota, there could be some predictive power in the use of
temperature and precipitation maxima. However, using these data in this way introduces
the risk of unwanted bias against local diversity in more northern states. Avoiding such
bias was the original intent of using Ecoregions in the CrEAM. The SAB finds this to be
a worthy goal and therefore does not recommend the Region-wide use of temperature and
precipitation maxima data in the model.
The SAB notes that there are no other available climate data that might be positively
related to species diversity at the scale of the Ecoregion. However, some available data may be
23
-------
negatively correlated with species diversity (i.e., climate stressor data). Within an Ecoregion,
winter temperature minima and growing season drought stress could both be negatively related
to species diversity. Drought stress is not necessarily well correlated with precipitation, since
there is an interaction with temperature, but this may not be a major problem at the scale of these
Ecoregions. More integrated measures of drought stress might be available, although these vary
significantly on an annual basis. Mean precipitation data could be an adequate proxy and the
SAB recommends that EPA explore the use of these data.
Data Layers C.2.1. Perimeter to Area Ratio and C. 2.2 Patch Size by Land Cover
Data Layers C.2.1 and C.2.2 are measures of patch fragmentation and are used in the
CrEAM to predict sustainability and landscape condition. The SAB finds that these data layers
have some validity for predicting sustainability, but a number of limitations associated with use
of these data layers in the CrEAM are noted.
Criticisms of the Island Biogeography Theory underlying use of this data layer to predict
sustainability in terrestrial landscapes are noted above.
The Perimeter to Area Ratio data are used in CrEAM to predict sustainability by
assigning higher scores to areas with low perimeter to area ratios and less "edge effects."
The SAB notes that "edge effects" may, in fact, be beneficial and contribute to the higher
value of an area.
The SAB notes that areas surrounding a patch may have a substantial impact on the
outcome of the ecological processes that dominate a given patch. Often such surrounding
areas are developed land. This "context" of the patch may be an important factor that is
not captured in the CrEAM analysis.
In the CrEAM analysis, patches under ten hectares in size are considered to be inclusions
or are otherwise ignored. As noted previously, the SAB finds that this approach is
improper because it can ignore keystone communities.
The SAB recommends that EPA explore the use of new data sets that may be available
for use in this data layer. Remote sensing is currently providing many new data sets that could
be used, including those provided by the U.S. Forest Service, although these data are not yet
publicly available.
Data Layer C. 2.4. Waterway Impoundment
EPA has stated that in the CrEAM analysis, all cells contained in any open water,
forested wetland, or emergent wetland patch touched by a 500 m buffer zone around a dam were
considered to be part of a fragmented hydrologic system. These cells received a lower score
regardless of the size of the patch. The ecological basis for this indicator is well established in
that dams are known to impede the movement of plants and animals, create sediment-starved
reaches, and alter physical and chemical characteristics of rivers both above and below them.
Dams fragment river networks. A more fragmented stream network is less sustainable in the
24
-------
sense that if a disturbance (e.g., an oil spill) were to wipe out a population of aquatic organisms
upstream of the dam, that population could not be re-established by natural processes of animal
migration. Migration and migratory pathways are blocked by the dam. The presence of a dam
also limits genetic exchange between populations above and below it. The SAB recommends
that EPA consider the following limitations associated with this data layer.
The choice of a 500 m buffer zone around a dam for determining impacts appears to be
arbitrary. The CrEAM documentation should be clarified to indicate that in the analysis,
an area greater than 500 m around the dam may be given a lower score. This is important
because the entire river network upstream of a dam could be affected by the presence of
the dam, particularly if migratory species are present. The SAB recommends that the
zone of impact be scaled to dam size.
The scoring of this indicator does not appear to be appropriate. This indicator has a
reported value of either 0 or 100 and appears to be redundant with and less valuable than
data layer C.2.9, watershed obstructions. Data layer C.2.9 is based on the same data set
and is a continuous metric rather than having a value of 0 or 100. Data layer C.2.9 also
expresses the number of dams in a river network and seems to be a better measure of
fragmentation than simply a 500 m zone around a dam. The SAB recommends that EPA
conduct a correlation analysis of indicators C.2.9 and C.2.4 to determine if they are
measuring different attributes of sustainability. The combining of continuous metrics
with binomial metrics results in disproportionate weighting in the aggregate score. Other
than recommending that this metric be dropped from the overall CrEAM index, the SAB
does not have a ready alternative to solve the binomial metric problem. One
consideration that might be applied to data layer C.2.4 is to weight the metric based upon
the number of miles between impoundments with longer reaches assigned a higher score
and shorter reaches assigned a lower score.
It is not clear whether CrEAM data layer C.2.4 contains only information about large
dams. The SAB recommends that EPA expand the description of this metric in the
CrEAM documentation to indicate the size of the dams included in the data layer. The
SAB notes that the states of Michigan and Wisconsin are developing databases of
information on small dams and these data could be used in the CrEAM analysis.
Data Layer C.2.5. Airport Buffers
The ecological principle supporting application of this data layer in the CrEAM is that
noise from airports is a well-known disturbance and stressor to wildlife. A number of limitations
associated with this data layer are noted.
Data layer C.2.5 is based solely upon airport runway length with no consideration given
to frequency of airport use. The SAB recommends that EPA present a justification for
the assumption that runway length is an appropriate indicator.
The data layer does not include any sources of noise other than airports. Noise from
other sources (e.g., roads) should be considered in the CrEAM.
25
-------
The scoring of this indicator does not appear to be appropriate. The application of
absolute values in data layer C.2.5, as opposed to using a scoring system based on a range
of values, is a concern.
It appears that EPA did not consult the Federal Aviation Administration (FAA)
concerning the availability of data on airports in EPA Region 5. Airports in the six states
in EPA Region 5 have been actively engaged in master planning and construction and
have assessed the environmental impacts of these activities in accordance with the
National Environmental Policy Act (NEPA). Environmental impact assessments have
identified noise impacts as well as others that may result from operational activities at
airport facilities.
The SAB notes that additional data sets for use in data layer C.2.5 are directly available
from the FAA. FAA Headquarters Offices in Washington, D.C. have been actively collecting
and analyzing data from airport facilities throughout the nation. The FAA uses a standard noise
model (as well as an air model) that has been applied consistently for a number of years. The
FAA has also developed guidance concerning wildlife management at airport facilities (Federal
Aviation Administration, 1997). This guidance identifies management strategies for airport
facilities as well as buffers and safety requirements. In addition, the EPA Office of Compliance
Sector Notebook on the Air Transportation Industry from 1998 (U.S. EPA, 1998) and the EPA
Preliminary Data Summary, Airport Deicing Operations (Revised) (U.S. EPA, 2000a) contain
relevant data for use in the CrEAM. These studies outlined the activities and concerns of the
Agency and contain large reference sections. Noise models are also available from the
Department of Defense Aberdeen Proving Ground. In addition, the Federal Aviation
Administration's (FAA) Office of Environment and Energy has developed the Integrated Noise
Model (INM) for evaluating aircraft noise impacts in the vicinity of airports (Federal Aviation
Administration, 2003).
Data Layer C. 2.6. National Priority List Superfund Sites
The ecological principle supporting application of this data layer is that Superfund Sites
listed for remediation will have stressors present that could impact wildlife, and that remediation
will disrupt associated systems. The SAB understands why EPA would want to explore
inclusion of this data layer in the CrEAM. EPA Region 5 has a high incidence of persistent
organic pollutants within their aquatic systems. These pollutants have a large impact upon the
Great Lakes. A number of substantive data sets related to National Priority List Superfund sites
in Region 5 are available (e.g., Saginaw River, Grand Calumet River, Waukegan Harbor, Fox
River, Sheboygan River, and Duluth Harbor). Available data from such sites should be
considered in the CrEAM analysis. The SAB notes, however, that the indicator described in data
layer 2.6 appears to be of limited use. The Geographic Information System data layer displaying
the sites and associated buffer areas suggests that these sites, as represented in data layer C2.6,
are not major features in the current version of the CrEAM. In addition, hydrologic linkages of
these sites with other parts of the landscape have not been included.
26
-------
Data Layer C.2.7. RCRA Corrective Action Sites
The ecological principle supporting application of this data layer is that sites listed as
having unacceptable human health risk, caused by exposure to contaminants in groundwater and
other media, will constitute a risk to ecological systems on these legally-defined sites.
Unacceptable risk to humans from noncarcinogenic contaminants is based on very conservative
metrics and specific pathways (i.e., imbibing contaminated groundwater from local wells) that
are not consistently linked to pathways relevant to ecological entities. However, RCRA
corrective action sites initially classified on the basis of risks to human health do have significant
ecological risk components. RCRA program sites include large refineries, manufactured gas
plants, creosote facilities, and other facilities. Available data from ecological risk assessments
completed at RCRA sites should be considered in the CrEAM analysis. However, The SAB
finds that the metric currently provided in data layer C2.7 of the CrEAM does not include
hydrologic linkages of sites with other parts of the landscape and is of limited use for reflecting
potential harm to ecological endpoints.
Data layers C.2.6 and C.2.7 essentially report the same condition: impaired, contaminated
sites. Available data from such sites should be included in the CrEAM analysis. However, as
currently developed, data layers C2.6 and C2.7 appear to be of limited use in the CrEAM. If
these data layers are retained in the model there is no good reason for treating them as separate
metrics. They should be combined into a single value. The SAB also finds that the presence of
fish consumption advisories is a useful landscape-level data set that could be applied as a metric
to represent aquatic stressors. Many of the rivers and lakes within EPA Region 5 have fish
consumption advisories, principally for PCBs and mercury that have been in place since the late
1970s. Each state and the EPA follow a standard protocol for sampling and testing Great Lakes
Fish (Great Lakes Commission, 2003). Waters with fish consumption advisories in EPA Region
5 include a large number of rivers and many lakes. Fish advisories are useful indicators of risk
to ecological systems because levels of PCBs and mercury in fish that would precipitate human
health risks are much higher than those known to cause reproductive and other adverse impacts
to piscivorous avifauna and mammals. Fish advisory information are available in EPA's
National Listing of Fish and Wildlife Consumption Advisories (U.S. EPA, 2004a). Additional
data are available from Michigan State University (Michigan State University, 2004).
Data Layer C.2.8. Water Quality Summary
Although data layer C.2.8 is based on water quality data, it is really an indictor of
watershed disturbance. Streams are integrators of landscape activities, and that is what the
metric in this data layer reflects. The metric can be assigned one of four values depending on
whether water quality thresholds (for dissolved oxygen, nitrate and nitrite-nitrogen, and total
suspended solids) are crossed. It is not clear why this data layer is considered by EPA to be a
measure of sustainability. The following limitations associated with this data layer are noted.
Phosphorus is acknowledged to be a limiting nutrient in many of the aquatic ecosystems
in EPA Region 5. The absence of a phosphorus water quality threshold in the CrEAM
analysis limits the usefulness of the water quality data layer, and the SAB recommends
that a phosphorus threshold be included in the analysis. Even consideration of
27
-------
phosphorus concentrations above or below a 100 jig/1 threshold would be a valuable
addition to this analysis.
No information on water quality contaminants such as metals (e.g., mercury) or persistent
organics (e.g., PCBs) is included in the water quality summary data layer. That limits its
usefulness for aquatic systems, and the SAB recommends that water quality data on
contaminants be included in the analysis.
The metric in the water quality data layer is dependent upon the values chosen for the
thresholds. The CrEAM developers have acknowledged that data are currently available
to refine the thresholds in the model to better incorporate regional variability. The SAB
recommends that such refinements be incorporated into the model.
New National Pollutant Discharge Elimination System requirements have resulted in the
collection of considerable water quality data that are reported to EPA's Office of Water. The
SAB notes that these data should be a valuable resource for future development of the CrEAM.
In addition, fish consumption advisory information and data on the mercury content of fishes are
widely available in EPA Region 5 and would provide a data layer for assessing contaminants.
Plots of total phosphorus concentrations by USGS hydrologic unit are also available in EPA
Region 5.
Data Layer C.2.9. Watershed Obstruction
Watershed obstructions are relevant to landscape evaluations as they pertain to
fragmented water systems but are largely used as a metric for free migration offish species
within a river reach. This is particularly important for anadromous species, and the metric in
data layer C.2.9 is a useful indicator for planning restoration activities. Data layers C.2.9 and
C.2.4 rely on USGS index maps, and are largely appropriate for the intended use. However, the
following limitations of the data layer are noted.
Given that the CrEAM index is focused principally on relative values of terrestrial
landscapes and does not include a large number of other hydrological metrics that would
be critical to evaluate aquatic habitat, the SAB notes that a separate data layer reflecting
"watershed obstruction" is not likely to provide a large amount of additional useful
information except perhaps for riparian habitats.
Given that the same data are used to indicate watershed obstructions and water
impoundments (data layer C.2.4), including watershed impoundments and watershed
obstructions as separate metrics essentially "double counts" the dataset.
The SAB recommends that EPA cross-reference the data layers C.2.4 and C.2.9 with data
available from the U.S. Army Corps of Engineers Detroit, Chicago, and St. Paul Districts. The
Corps of Engineers is responsible for dam maintenance and is likely to have more accurate
records of watershed obstructions.
28
-------
Data Layer C.2.10. Air Quality Summary
The EPA air quality model, Assessment System for Population Exposure Nationwide
(ASPEN), was used in CrEAM data layer C.2.10 to obtain predicted ambient air pollution
concentrations. Modeled outdoor air toxics concentrations and human health benchmarks were
used to provide a first approximation of general air degradation risk in areas defined by census
tracts. The underlying principle supporting application of this data layer in the CrEAM is that
using a human health risk assessment would be approximate to ecological risk. The following
limitations of the data layer are noted.
The SAB finds the use of a subset of the ASPEN data and human health benchmarks to
be problematic. As previously noted, human health benchmarks are not quantitatively
appropriate for "non-human" stress assessment. While the use of these data represents a
satisfactory first step, the approach does not provide an adequate estimate of exposure
and ecological risk. The SAB recommends that ASPEN-generated exposure levels be
used as part of a more comprehensive air quality index that could: a) utilize a different
spatial unit of resolution (using USGS hydrologic cataloging or watershed units instead
of census tracts); b) utilize information on ecological rather than human health risk in
developing the air quality summary metric; and c) utilize other available air quality data
from EPA Region 5 (e.g., National Atmospheric Deposition Program/National Trends
Network precipitation chemistry data, Mercury Deposition Network data, Clean Air
Status and Trends Network data, NOAA data, and AmeriFlux data). The SAB suggests
that EPA consider using the following data in data layer C.2.10: atmospheric nitrogen
deposition (wet), tropospheric ozone concentration, and atmospheric mercury inputs. It is
recommended that EPA consider weighting scores obtained from these data sets.
The air quality summary index is currently reported as a linear extrapolation of the
exceptions per census tract. A more robust metric would include a number of factors for
ecological risk, perhaps in a linear model, to provide a metric of exposure. For
"undeveloped" forests, this could include multipliers for exposure to account for forest
canopy interception, which can dramatically increase deposition inputs, and models that
are available to account for topographic influences on local wind and deposition patterns.
Data Layer C.2.11. Development Disturbance Buffer
Two ecological principles support application of data layer C.2.11 in the CrEAM. The
first principle is that land uses surrounding a patch can exert positive or negative influences on
ecological processes and biota within a patch. In this case, developed land is assumed to have a
negative effect on such processes. The SAB notes that, with the possible exception of grassland
systems, this is probably a correct assumption. The second principle is that the influence of land
use adjacent to a patch decreases with increasing distance from the edge of the habitat. A 300 m
buffer was used in the CrEAM as the limit for these edge effects. The following limitations
associated with use of this data layer are noted.
A uniform buffer size was used in data layer C.2.11 for all types of land development.
The authors of the CrEAM acknowledge that different types of development vary in the
29
-------
environmental pressure exerted on a patch, but the lack of quantitative data was cited as
the primary reason for using a uniform buffer width. Although available literature may
not explicitly provide recommended buffer widths or penetration distances for different
disturbances, there are numerous studies showing substantially greater edge and matrix
effects resulting from urban or residential land uses compared to some agricultural or
silvicultural land uses. The SAB notes that this would appear to argue for the use of
wider buffers for urban areas.
The data layers in the CrEAM do not currently discriminate among developed land uses.
Coding land uses would add much complexity to the model and reduce final bias in the
sustainability metric if land uses are applied in combination with other stressor data sets
such as road density. The SAB notes that EPA may want to provide more explicit
discussion in the CrEAM documentation concerning the use of metrics in combination
and alone.
The SAB notes that CrEAM data layer 2.11 metric is also an "all or nothing" measure
(i.e., the pixel is either within 300 m of an adjacent patch and is assigned a value of 0 or
beyond 300 m and assigned a value of 100). The limitations of such a binomial scoring
system have been discussed above. There may be some benefit gained by adding the
complexity of a step function to this data layer (e.g., assigning scores such as: 0-50 m =
0, 50-100 m = 10, 100-150 m = 20, etc.).
Data Layers C.3.2 - C. 3.4 Rarity of Individual Species of Taxa
Rarity of individual species or taxa was measured in three data layers (C.3.2 - Species
Rarity, C.3.3 - Rare Species Abundance, and C.3.4 - Rare Species Taxa and Abundance). The
ecological principle supporting application of all three layers is that rare species are of special
ecological interest. Rare species data may therefore identify landscapes that are either
ecologically different (unique) or under decline and therefore threatened. The following
limitations are noted.
The very large size of the squares that contain these data (USGS 7.5 minute quad)
relative to the standard CrEAM cell make the layers of less value. While this may be
inevitable based upon the source of the data, it limits the application of these layers for
desired planning and priority setting activities.
The data layers do not display continuous data; rather, each is broken into 5 groups.
While this categorization allows some differentiation between present and absent, it does
not reflect the gradient that is present in continuous data. Of these layers, both C3.3 and
C3.4 could be normalized to allow a continuous score.
It is hard to discern any difference between data layers C3.2, C3.3, and C3.4 in the maps
provided in the Appendix of the draft CrEAM report. Perhaps this is due to the
coloration of the maps, but it is difficult to understand why they are not more highly
correlated.
30
-------
5.6 Charge Question 3.1. Please comment on the scientific defensibility of the use of
CrEAM results to support broad based strategic planning and priority setting
activities (e.g., identifying locations for geographic initiatives and EPA/State
joint efforts) and program activities such as:
- Inspection
Permitting
Enforcement and cleanup
- Reviewing grant proposals
- Establishing reference context for ecological protection and restoration
The SAB notes that the CrEAM index, in its current form, lacks the "scientific
defensibility" to support broad based strategic planning and priority setting. The SAB has
provided recommendations for improvement of the CrEAM index. These recommendations,
taken together, could provide an adequate scientific basis for establishing a GIS-based decision
making and resource allocation tool. As noted above, the current CrEAM model cannot be used
to generate statistically significant results for decision making because it has not been "ground
truthed" using empirical data to validate the weights used to combine indicator variables into a
single CrEAM ecological metric. The limitations on the data, model, and general approach need
to be clearly set forth. However, the SAB finds that the CrEAM index, as presented, can be an
appropriate regional tool for the allocation of internal EPA resources for site inspection
activities, to track general trends in the regional landscape condition, and may be applicable for
reviewing grant proposals to the Agency. CrEAM is also an appropriate framework to foster
further communication and dialogue between other federal and state agencies on the use of
regional and spatial data in environmental decision making. The SAB endorses the Region's
validation process for the CrEAM index
The SAB, however, finds that underlying science does not support the use of CrEAM in
any environmental decision making or regulatory processes. This would include, but is not
exclusive to, issuing or reviewing air and/or water quality permits, as a basis for the EPA or any
other federal or state agency's determination in National Environmental Policy Act (NEPA)
reviews, as a basis for setting compliance, enforcement or cleanup actions, or for establishing
reference context for ecological protection and restoration. While these are ultimate functions
that the SAB envisions could be supported by later versions of the CrEAM index, application of
CrEAM in its current iteration to environmental decision making is not scientifically defensible.
The SAB further stresses the need for EPA to make it clear that CrEAM is only one tool, and
should only be used in conjunction with other tools and factors that affect internal resource
allocation in the near-term or for broader decision or policy related issues in the future.
The SAB would like to recognize that Region 5 has made a good initial effort to
strengthen incorporation of ecological understanding in the environmental decision-making
process at EPA. There are some very sophisticated techniques and methods used in the CrEAM,
and the authors should be complimented on application of these methods to extraordinarily
complex issues and difficult problems. A very good foundation has been established that can
hopefully be improved and developed into a functional and dynamic tool. The SAB recognizes
that the developers of the CrEAM index were required to balance the need to include the most
31
-------
detailed and readily accessible data/science against the reality of significant computational
burdens that addition of those data incurred. The developers were further constrained by their
lack of access to the most recent GIS computational resources. Additionally, validation of the
index, an important step in the scientific process, has been delayed. Recognizing that CrEAM is
an unfunded mandate within this Region, the development team made the best use of the
resources at its disposal. However, the SAB believes that for CrEAM to be an important tool,
the computational limits and validity issues must and can be overcome by investing resources for
upgrading CrEAM into the most recent versions of Arc View and Spatial Analyst and devoting
personnel to the effort.
32
-------
6. References
Anderson, W.B., and D.A. Wait. 2001. Subsidized island biogeography hypothesis: another
new twist on an old theory. Ecol. Lett.; 4, 289-291.
Cameron, T., and G. Crawford. 2003. Independent dimensions of sociodemographic variability
in neighborhood characteristics at the tract level of the 2000 census. Discussion Paper,
Department of Economics, University of Oregon, Eugene, OR. Available at
http://economics.uoregon.edu/papers/UO-2004-10_Cameron_Crawford_Census_F actors.pdf
(May 19, 2005)
Davies, K.F., B.A. Melbourne, and C.R. Margules. 2001. Effects of within-and between-patch
processes on community dynamics in a fragmentation experiment. Ecology; 82, 1830-1846.
European Commission. 2000. From Land Cover to Diversity in the European Union.
http://europa.eu.int/comm/agriculture/publi/landscape/ (December 9, 2004)
Federal Aviation Administration. 2003. Integrated Noise Model.
http://www.aee.faa.gov/noise/inm/ (December 10, 2004)
Federal Aviation Administration. 2000. Advisory Circular, Hazardous Wildlife Attractions On
or Near Airports. Circular 150/5200-33. Federal Aviation Administration, Washington,
D.C.
Gascon, C. and I.E. Lovejoy. 1998. Ecological impacts of forest fragmentation in central
Amazonia. Zoology; 101, 273-280.
Great Lakes Commission, 2003. Fish Consumption in the Great Lakes
http://www.great-lakes.net/humanhealth/fish/advisories.html - IL (October 14, 2004)
Harrison, S. 1999. Local and regional diversity in a patchy landscape: Native, alien, and
endemic herbs on serpentine. Ecology; 80, 70-80.
Herkert, J.R., D.W. Sample, and R.E. Warner. 1996. Management of midwestern grassland
landscapes for the conservation of migratory birds. In: Managing Mid-western Landscape for
the Conservation of Neotropical Migratory Birds (ed Thompson, F.R., III), pp 89-116. U.S.
Department of Agriculture Forest Service, North Central Forest Experiment Station, St. Paul,
MN. Technical Report GTR-NC-187
Holt, R.D. 1997. From metapopulation dynamics to community structure: some consequences
of spatial heterogeneity. In: Metapopulation Biology: Ecology, Genetics, and Evolution (eds
Hanski, I. and Gilpin, M.E.) Academic Press, San Diego, CA, pp. 149-164.
Levin, S.A., ed. 2001. Encyclopedia of Biodiversity. San Diego: Academic. 5 Vols.
33
-------
Michigan State University. 2003. State Laws Regulations and Policies.
http://www.iet.msu.edu/regs/state.htm
Minnesota Department of Natural Resources, 2004. Regionally Significant Ecological Areas.
http://www.dnr.state.mn.us/rsea/index.html (October 14, 2004)
Openshaw, S. 1984. The modifiable areal unit problem. Concepts and Techniques in Modern
Geography, 38:41
Smith, E.R., L. T. Iran, and R.V. O'Neill. 2003. Regional Vulnerability Assessment for the
Mid-Atlantic Region: Evaluation of Integration Methods and Assessments Results. EPA-600-
R-03-082. EPA Office of Research and Development.
U.S. EPA. 2004a. National Listing of Fish and Wildlife Consumption Advisories.
http://www.epa.gov/waterscience/fish/ (October 14, 2004)
U.S. EPA. 2004b. Regional Vulnerability Assessment (ReVA) Program.
http://www.epa.gov/reva/ (December 7, 2004)
U.S. EPA 2003. Risk Screening Environmental Indicators.
http://www.epa.gov/opptintr/rsei/get_rsei.html (November 30, 2004)
U.S. EPA. 2000a. Preliminary Data Summary, Airport Deicing Operations (Revised). EPA-
821-R-00-016. U.S. Environmental Protection Agency, Washington, D.C.
U.S. EPA. 2000b. Stressor Identification Guidance Document. EPA-822-B-00-025. U.S.
EPA Office of Water and U.S. EPA Office of Research and Development, Washington, D.C.
Available at http://www.epa.gov/ost/biocriteria/stressors/stressorid.pdf
U.S. EPA. 2000c. Workshop Report on Characterizing Ecological Risk at the Watershed
Scale. EPA/600/R-99/111. U.S. EPA Office of Research and Development, National Center
for Environmental Assessment, Washington, D.C.
U.S. EPA. 1998. EPA Compliance Sector Notebook Project Air Transportation Industry.
EPA/310-R-97-001. U.S. Environmental Protection Agency. Washington, D.C.
U.S. EPA Office of Science Policy. 2003. Draft Guidance on the Development, Evaluation, and
Application of Regulatory Environmental Models. U.S. EPA Office of Science Policy, Office
of Research and Development, Washington, D.C. Available at
http://www.epa. gov/osp/crem/library/CREM%20Guidance%20Draft%2012 03 .pdf (May 19,
2005)
U.S. EPA Science Advisory Board. 2002. A Framework for Assessing and Reporting on
Ecological Condition: An SAB Report (eds Young, T.F. and Sanzone, S.). EPA-SAB-EPEC-02-
009. U.S. Environmental Protection Agency Science Advisory Board, Washington, D.C.
34
-------
U.S. Geological Survey. 2001. National Land Cover Characterization Project.
http://landcover.usgs.gov/nationallandcover.asp (October 14, 2004)
-------
-------
Appendix A: Specific Comments From Individual Committee Members and Technical
Corrections
Page 1, Introduction, 1st ]f: The use of the term "large scales" is technically applied
incorrectly. Scale refers to the map ratio such as 1:24,000, which is a large ratio and thus a
large scale when compared to 1:100,000, which is a smaller ratio and thus a small scale.
Page 4, 1st 1J: The following sentence is confusing: "In all cases, cells having no majority
initially, obtained mixed forest as the majority."
Page 4, 2nd ]f 1st sentence: "The user accuracy is reported by pixel " Shouldn't this mean
"reported by class..."?
Page 4, 2nd ]f: The aggregation method by majority described here will smooth out the smaller
frequency errors reported by the NLCD accuracy assessment. It should be argued that it
would make the cells more homogeneous or more accurate.
Page 5, 1st ]j, 6th sentence: Does this mean heterogeneity is amplified?
Page 7, Land-cover diversity (C1.2): Although it is not stated, was the Shannon-Weiner
Index calculated using only undeveloped pixels? It is not clear why 1km squares were used
for the calculation.
Page 7, Temperature and precipitation maxima (C1.3): There is no explanation of how the
temperature and precipitation values were combined or how the data were quantified or why
11km cells were used. This is certainly a questionable layer.
Page 7, Temporal continuity of land-cover type (C1.4): A range of compatibility could be
calculated based on the diversity of land-cover types in a cell or the % majority class. Cells
could then be weighted on an interval scale rather than 0,100.
Page 8: B. Ecological Self-sustainability heading should end with "Data Sets" to be
compatible with A. on page 6.
Page 9: There should be a heading for "Landscape Fragmentation" before the "Patch
perimeter to area analysis" (C2.1) paragraph.
Page 10, Weighted road density (C2.3): Some explanation of why 5 km squares were used
should be included. No indication in the text is provided on how the road density
calculations were scaled. Higher road densities were given a lower score, so there appears to
be an inverse relationship, which was not mentioned.
Page 11: There should be a Stressors heading preceding Airport buffers (C2.5).
Page 11, Airport buffers (C2.5): Frequency of use seems like it would be a significant factor.
A-l
-------
-------
Page 12, NPL Superfund sites (C2.6) and RCRA corrective actions sites (C2.7): How long
lasting are the effects from these sites? .
Page 12, RCRA corrective actions sites (C2.7), 5th Line: "other media" seems out of place.
Page 13, Watershed obstructions (C2.9): "The same data used in C2.5...." Shouldn't this be
C2.4? No mention was made of how the data were scaled.
Page 13, Air quality summary (C2.10): This is a difficult concept to grasp because it is
compiled by Census Tracts which are (by definition) mostly in developed areas, which were
excluded from the study. It should be mentioned that this is an inverse score.
Page 13, Land-cover suitability: This is listed under Landscape Fragmentation in Table 2,
perhaps it should be moved under Landscape Fragmentation heading in the text.
Tables 5, 6, and 7: One could question whether the scores in these tables are truly linear, but
without further evidence, it is difficult to say otherwise.
Page 20, Figure 9: The description for Figure 9 is displaced on the page.
Page 22, 1st ]j, 3rd sentence: Indicates that sensitivity depends on data quality. It also depends
on many other factors including methods used to quantify, score, and rate data layers, spatial
scale, categorization, aggregation, and cell size.
Page 23, first sentence: Indicates that Figure 11 is a plot of change in count vs. change in
score, but the x-axis is actually cumulative score, not a delta (A).
Page 25, 3rd ]f, 1st sentence: The minimum mapping unit (mmu) of the NLCD data is not truly
30m. That is the inherent resolution of the Landsat TM satellite imagery used for the
classification, but that differs from the mmu, which is not implicit and probably not specified
unless the data were filtered. Even though the resolution is 30m, it is unlikely that a feature
that size could be identified, so the mmu is greater than 30m. It is more correct to refer to this
as resolution.
Page 25, 3rd ]f, 4th sentence. There is a reference to Turner et al., which is not listed in the
References Section.
Page 25, 3rd ]f, 5th sentence: " data of different scales, and geographic measures "
Page 25, 3rd ]j, 8th sentence: subject/verb disagreement" majority of pixels in a cells is
developed "
Page 26, Table 9: Why is the error rate for aggregation by centroid not shown? How were the
error rates determined?
Page 26, 2nd ]f: The 3rd sentence states "Because no other data layer was aggregated, the
A-2
-------
-------
MAUP is not a consideration." Actually, the census tract itself introduces MAUP errors
because that is an arbitrary boundary by which data are categorized. Census boundaries can
change, and thus introduce aggregation differences, and so are subject to MAUP. It is
probably not correct to imply the MAUP has been circumvented.
Page 30, 3rd bullet: benefits should be benefit.
Page 30, Figure 14: The description indicates that the region of the graph marked "A"
implies that there are fewer lowest quality areas for 2000. It appears to be just the opposite,
there are actually more low quality areas for 2000.
Appendix A, Criterion 2B Table, C2.8 Water Quality Summary, Scoring Column: The text
on pages 12-13 does not describe a log distribution for the variable score, and it should be
mentioned that the score is inverted. Appendix A, Criterion 3 Table, C3.1 Land-cover rarity,
Scoring Column: The text mentions that this is a log scale, but that is not shown here.
Pages in Appendices of the CrEAM report need to be numbered.
A-3
------- |