y"""%
f I
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
February 27, 2015
EPA-SAB-15-007
The Honorable Gina McCarthy
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: Review of the EPA's draft Report on the Environment 2014
Dear Administrator McCarthy:
In April 2014, the EPA released a draft of its Report on the Environment (ROE) 2014, an online product
that presents status and trends for environmental and human health using a set of 86 indicators. The
Science Advisory Board (SAB) was asked to review the draft ROE with particular attention to its
adoption of a sustainability framework and new sustainability indicators, as well as the online format as
a tool to communicate to scientists, policy makers and public audiences.
The EPA first released a draft Report on the Environment in 2003, and since that time has continued to
revise the indicators as additional relevant data have become available and in response to SAB advice
and user feedback. Over the years, the SAB has supported the Agency's efforts on the ROE to inform
the public about the state of the environment and the role of the EPA's programs in making progress on
issues important to the public. In this review, the SAB again finds the ROE to be a greatly improved
and enhanced version of earlier products, and commends the Agency for its efforts. The current, online
version of the ROE increases its usability and the ability of users to link to and download underlying
data for individual indicators. This serves as a substantial benefit in regards to transparency of the
underlying scientific basis of the ROE. The SAB supports the timely release of the ROE to Agency users
and to the public as a whole.
In past reviews, a key SAB recommendation has been that the ROE adopt a conceptual framework that
places the individual indicators within a broader context and allows interpretation of the information
they provide. In the 2014 draft ROE, a sustainability framework is introduced to serve that important
purpose and a sustainability theme has been added, with an initial set of four sustainability indicators.
The SAB agrees that sustainability is an appropriate conceptual framework for the ROE, but finds that
the sustainability framework has not been sufficiently integrated throughout the ROE. The Sustainability
theme is presented separately from the other themes of Air, Water, Land, Human Exposure and Health,
and Ecological Condition, and sustainability implications of the indicators in these other themes is not
discussed. In addition, the new indicators presented under the Sustainability theme provide useful
information on trends in resource consumption (for example, of water and energy) but do not provide
direct insight into sustainability of that resource use.
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
-------
With regard to the Agency's decision to produce the ROE as an interactive, online product, the SAB
agrees that this should have a number of benefits, including making the information available to a wider
array of users, facilitating updates to indicators as new data are available, and allowing users to access
information at different levels of detail depending on their needs. Despite the progress made, the SAB
concludes that the online version of the ROE has not yet achieved its full potential as a tool for
communicating national environmental status and trends to both technical and general public audiences.
In the enclosed report, the SAB recommends a number of revisions to expand and strengthen the ROE.
These recommendations focus primarily on two key areas: better integration of sustainability concepts
into the ROE and possible approaches to make the ROE more accessible and useful to the general
public. Some of the recommendations should be easily accomplished in the short term (e.g., removing
broken or inappropriate links, updating indicators to include the most recent data, discussing the
sustainability implications of existing indicators), and others will require more time and resources (e.g.,
development of new sustainability indicators, use of focus groups to obtain feedback on the website
structure). The SAB recommends that priority be given to rapid implementation of those changes that
can be achieved within a short time frame (a few weeks or perhaps months), followed by the official
release of the ROE. None of the recommendations made by the SAB imply that this release should be
delayed to a significant extent.
A summary of the recommendations in these areas is presented here.
Strengthen the sustainability focus
• The Agency should ensure that the definition of sustainability is readily available and is
consistent with the ROE conceptual framework and with definitions of sustainability used
elsewhere by the Agency.
• Sustainability indicators and narratives should be integrated into each of the other themes of the
ROE by including indicators of sustainability within each theme, and by discussing the
sustainability implications of the existing indicators.
• The suite of ROE sustainability indicators should be improved and expanded to include measures
of sustainable environmental outcomes and, where possible, indicators of social and economic
sustainability; the SAB suggests categories of data and indicators that might guide the Agency's
efforts in this regard. In addition, the ROE should discuss the inherent trade-offs between
societal actions and choices in the quest for sustainability, the notion of life-cycle effects, and the
role of indicators in measuring progress toward sustainability.
• To ensure the credibility of the ROE, the most recent available data should be provided for the
indicators. The Agency also should continue to increase the statistical rigor of the ROE, and in
particular to discuss uncertainty associated with specific indicators in language that is accessible
to both technical and non-technical audiences.
-------
Enhance the ROE for public audiences
• The Agency should better define the target audiences for the ROE. The ROE is currently
attempting to satisfy the needs of both technical and non-technical user groups. Accordingly, the
SAB recommends that the Agency collect user feedback, and if possible, employ user focus
groups or other formal methods to better understand site usage, clarity, usability and
performance.
• The ROE could be restructured so it is easier to navigate and users can more easily find answers
to environmental and human health questions and concerns, including those that require
information and insights across multiple indicators.
• Now that the ROE is online, the Agency should improve accessibility of the ROE for
stakeholders, including environmental justice communities, who may have limited access to the
internet. Providing the ROE highlights, including a summary of the indicators and trends, in a
single PDF or some other downloadable format would help achieve this goal.
The SAB recognizes that resources may limit the EPA's ability to greatly expand the scope of the ROE
and recommends that the Agency give highest priority to identifying and implementing indicators that
provide insights into sustainability. Modest restructuring of the ROE website also could be
accomplished to enhance its utility for a broader public audience. We appreciate the opportunity to
provide advice on this important effort and look forward to your response.
Sincerely,
/s/ /s/
Dr. Peter S. Thorne Dr. Robert J. Johnston
Chair Acting Chair
Science Advisory Board Advisory Panel on the ROE 2014
Enclosure
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board (SAB), a public
advisory group providing extramural scientific information and advice to the Administrator and other
officials of the Environmental Protection Agency. The SAB is structured to provide balanced, expert
assessment of scientific matters related to problems facing the Agency. This report has not been
reviewed for approval by the Agency and, hence, the contents of this report do not represent the views
and policies of the EPA, nor of other agencies in the Executive Branch of the federal government, nor
does mention of trade names of commercial products constitute a recommendation for use. Reports of
the SAB are posted on the EPA website at http://www.epa.gov/sab.
1
-------
U.S. Environmental Protection Agency
Science Advisory Board
Advisory Panel on EPA's Report on the Environment 2014
CHAIR
Dr. James Sanders, Executive Director, Skidaway Institute of Oceanography, University of Georgia,
Savannah, GA
MEMBERS
Dr. Joseph Arvai, Svare Chair in Applied Decision Research, Department of Geography, University of
Calgary, Calgary, Alberta, Canada
Dr. Terry Daniel, Professor of Psychology and Natural Resources, Department of Psychology, School
of Natural Resources, University of Arizona, Tucson, AZ
Dr. H. Christopher Frey, Distinguished University Professor, Department of Civil, Construction and
Environmental Engineering, College of Engineering, North Carolina State University, Raleigh, NC
Dr. Lucinda Johnson, Center Director, Center for Water and the Environment, Natural Resources
Research Institute, University of Minnesota Duluth, Duluth, MN
Dr. Robert J. Johnston, Director of the George Perkins Marsh Institute and Professor, Economics,
Clark University, Worcester, MA
Dr. James R. Mihelcic, Professor, Civil and Environmental Engineering, University of South Florida,
Tampa, FL
Dr. H. Keith Moo-Young, Chancellor, Office of Chancellor, Washington State University, Tri-Cities,
Richland, WA
Dr. Eileen Murphy, Director of Research Development, Office of Research and Economic
Development, Rutgers University, Piscataway, NJ
Dr. James Opaluch, Professor and Chair, Department of Environmental and Natural Resource
Economics, College of the Environment and Life Sciences, University of Rhode Island, Kingston, RI
Dr. Amanda Rodewald, Director of Conservation Science, Cornell Lab of Ornithology and Associate
Professor, Department of Natural Resources, Department of Natural Resources, Cornell University,
Ithaca, NY
Dr. Sujoy Roy, Director, Research and Development, Tetra Tech Inc., Lafayette, CA
11
-------
CONSULTANTS
Dr. Sharan Campleman, Program Manager, Toxicology Investigators Consortium, American College
of Medical Toxicology, Campbell, CA
Dr. Allan Legge, President, Biosphere Solutions, Calgary, Alberta, Canada
Dr. Rebecca Parkin, Professorial Lecturer in Environmental and Occupational Health and in
Epidemiology and Biostatistics, Milken Institute School of Public Health, The George Washington
University, Washington, DC
Dr. Thomas L. Theis, Director, Institute for Environmental Science and Policy, University of Illinois at
Chicago, Chicago, IL
Dr. Stephen Weisberg, Executive Director, Southern California Water Research Project Authority,
Costa Mesa, CA
SCIENCE ADVISORY BOARD STAFF
Ms. Stephanie Sanzone, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board (1400R), 1200 Pennsylvania Avenue, NW, Washington, DC
111
-------
U.S. Environmental Protection Agency
Science Advisory Board
CHAIR
Dr. Peter S. Thorne, Professor and Head, Department of Occupational & Environmental Health,
University of Iowa, Iowa City, IA
MEMBERS
Dr. George Alexeeff, Director, Office of Environmental Health Hazard Assessment, California
Environmental Protection Agency, Oakland, CA
Dr. Joseph Arvai, Professor and Svare Chair in Applied Decision Research, Department of Geography,
University of Calgary, Calgary, Alberta, Canada
Dr. Sylvie M. Brouder, Professor and Wickersham Chair of Excellence in Agricultural Research,
Department of Agronomy, Purdue University, West Lafayette, IN
Dr. Thomas Burbacher, Professor, Department of Environmental and Occupational Health Sciences,
School of Public Health, University of Washington, Seattle, WA
Dr. Ingrid Burke, Director and Wyoming Excellence Chair, Haub School and Ruckelshaus Institute of
Environment and Natural Resources, University of Wyoming, Laramie, WY
Dr. George Daston, Victor Mills Society Research Fellow, Global Product Stewardship, The Procter &
Gamble Company, Mason, OH
Dr. Costel Denson, Managing Member, Costech Technologies, LLC, Hockessin, DE
Dr. Michael Dourson, President, Toxicology Excellence for Risk Assessment, Cincinnati, OH
Dr. Joel Ducoste, Professor, Department of Civil, Construction, and Environmental Engineering,
College of Engineering, North Carolina State University, Raleigh, NC
Dr. David A. Dzombak, Hamerschlag University Professor and Department Head, Department of Civil
and Environmental Engineering, College of Engineering, Carnegie Mellon University, Pittsburgh, PA
Dr. Elaine M. Faustman, Professor and Director, Environmental and Occupational Health Sciences,
University of Washington, Seattle, WA
Dr. R. William Field, Professor, Department of Occupational and Environmental Health, and
Department of Epidemiology, College of Public Health, University of Iowa, Iowa City, IA
Dr. H. Christopher Frey, Distinguished University Professor, Department of Civil, Construction and
Environmental Engineering, College of Engineering, North Carolina State University, Raleigh, NC
Dr. Steven Hamburg, Chief Scientist, Environmental Defense Fund, Boston, MA
iv
-------
Dr. Cynthia M. Harris, Director and Professor, Institute of Public Health, Florida A&M University,
Tallahassee, FL
Dr. Robert J. Johnston, Director of the George Perkins Marsh Institute and Professor, Economics,
Clark University, Worcester, MA
Dr. Kimberly L. Jones, Professor and Chair, Department of Civil and Environmental Engineering,
Howard University, Washington, DC
Dr. Catherine Karr, Associate Professor - Pediatrics and Environmental and Occupational Health
Sciences and Director - NW Pediatric Environmental Health Specialty Unit, University of Washington,
Seattle, WA
Dr. Madhu Khanna, ACES Distinguished Professor in Environmental Economics, Department of
Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, Urbana, IL
Dr. Nancy K. Kim, Independent Consultant, Independent Consultant, Albany, NY
Dr. Francine Laden, Mark and Catherine Winkler Associate Professor of Environmental
Epidemiology, Harvard School of Public Health, and Channing Division of Network Medicine, Brigham
and Women's Hospital and Harvard Medical School, Boston, MA
Dr. Lois Lehman-McKeeman, Distinguished Research Fellow, Discovery Toxicology, Bristol-Myers
Squibb, Princeton, NJ
Dr. Cecil Lue-Hing, President, Cecil Lue-Hing & Assoc. Inc., Burr Ridge, IL
Dr. Elizabeth Matsui, Associate Professor, Pediatrics, School of Medicine, Johns Hopkins University,
Baltimore, MD
Dr. Denise Mauzerall, Professor, Woodrow Wilson School of Public and International Affairs, and
Department of Civil and Environmental Engineering, Princeton University, Princeton, NJ
Dr. Kristina D. Mena, Associate Professor, Epidemiology, Human Genetics, and Environmental
Sciences, School of Public Health, University of Texas Health Science Center at Houston, El Paso, TX
Dr. Surabi Menon, Director of Research, ClimateWorks Foundation, San Francisco, CA
Dr. James R. Mihelcic, Professor, Civil and Environmental Engineering, University of South Florida,
Tampa, FL
Dr. H. Keith Moo-Young, Chancellor, Office of Chancellor, Washington State University, Tri-Cities,
Richland, WA
Dr. Eileen Murphy, Director of Research Development, Office of Research and Economic
Development, Rutgers University, Piscataway, NJ
v
-------
Dr. James Opaluch, Professor and Chair, Department of Environmental and Natural Resource
Economics, College of the Environment and Life Sciences, University of Rhode Island, Kingston, RI
Dr. Martin Philbert, Dean and Professor, Environmental Health Sciences, School of Public Health,
University of Michigan, Ann Arbor, MI
Mr. Richard L. Poirot, Air Quality Planning Chief, Air Quality and Climate Division, Vermont
Department of Environmental Conservation, Montpelier, VT
Dr. Stephen Polasky, Fesler-Lampert Professor of Ecological/Environmental Economics, Department
of Applied Economics, University of Minnesota, St. Paul, MN
Dr. David B. Richardson, Associate Professor, Department of Epidemiology, School of Public Health,
University of North Carolina, Chapel Hill, NC
Dr. Amanda D. Rodewald, Director of Conservation Science, Cornell Lab of Ornithology and
Associate Professor, Department of Natural Resources, Cornell University, Ithaca, NY
Dr. William Schlesinger, President Emeritus, Cary Institute of Ecosystem Studies, Millbrook, NY
Dr. Gina Solomon, Deputy Secretary for Science and Health, Office of the Secretary, California
Environmental Protection Agency, Sacramento, CA
Dr. Daniel O. Stram, Professor, Department of Preventive Medicine, Division of Biostatistics,
University of Southern California, Los Angeles, CA
Dr. Paige Tolbert, Professor and Chair, Department of Environmental Health, Rollins School of Public
Health, Emory University, Atlanta, GA
Dr. Jeanne VanBriesen, Professor, Department of Civil and Environmental Engineering, Carnegie
Mellon University, Pittsburgh, PA
Dr. John Vena, Professor and Founding Chair, Department of Public Health Sciences, Medical
University of South Carolina, Charleston, SC
Dr. Elke Weber, Jerome A. Chazen Professor of International Business, Columbia Business School,
New York, NY
Dr. Charles Werth, Professor and Bettie Margaret Smith Chair in Environmental Health Engineering,
Department of Civil, Architectural and Environmental Engineering, Cockrell School of Engineering,
University of Texas at Austin, Austin, TX
Dr. Peter J. Wilcoxen, Associate Professor, Economics and Public Administration, The Maxwell
School, Syracuse University, Syracuse, NY
Dr. Dawn J. Wright, Chief Scientist, Environmental Systems Research Institute (Esri), Redlands, CA
vi
-------
SCIENCE ADVISORY BOARD STAFF
Mr. Thomas Carpenter, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board (1400R), 1200 Pennsylvania Avenue, NW, Washington, DC
vii
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 4
2.1. Background 4
2.2. Charge to the SAB 4
3. RESPONSE TO CHARGE QUESTIONS 5
3.1. SUSTAINABILITY AS THE CONCEPTUAL FRAMEWORK 5
3.2. SUSTAINABILITY THEME AND INDICATORS 10
3.3. Incorporating Statistical Information 20
3.4. ROE as a Web-Based Product 22
3.5. Communicating to Technical and Non-Technical Audiences 27
3.6. Potential Indicators for Future Updates of the ROE 33
REFERENCES 35
APPENDIX A. EPA's Charge to the SAB for Review of the Report on the Environment (2014) A-l
APPENDIX B. Draft ROE 2014 Questions and Indicators B-l
viii
-------
1. EXECUTIVE SUMMARY
In 2003, the U.S. Environmental Protection Agency released its initial draft Report on the Environment
(ROE) as a first comprehensive effort to report on national indicators that could be used to measure and
track the state of the U.S. environment. The 2003 draft ROE used groups of indicators to answer
questions about the state of the nation's air, land and water as well as human health and ecological
condition. In 2007, the SAB reviewed a revised and updated version of the ROE, commenting on the
individual indicators, the need for a conceptual framework to link the indicators and the questions, and the
importance of providing interpretation of the data including conclusions supported by statistical analysis.
Along with many specific recommendations to improve the product, the SAB reiterated its support for
continuing the ROE effort. A further consultation was held with SAB panel members in 2009 to provide
individual member feedback on ROE future directions and conceptual frameworks to organize and enhance
the understanding of the status and trends for the ROE indicators.
For the current review, the SAB was asked to review the 2014 draft ROE with particular attention to the
enhancements made since the 2009 consultation, including the utility of the ROE as an online-only
product rather than a printed document. The charge to the SAB includes questions about the use of
sustainability as the ROE conceptual framework, an initial set of sustainability indicators, inclusion of
statistical information for some indicators, the ROE as a web-based product, the success of the ROE for
communicating to multiple audiences, and recommendations for additional indicators.
The transition to a web-based format of the ROE represents an improvement from earlier versions,
particularly in regards to usability and the ability to link to, and download, underlying data for individual
indicators. This format contributes substantially to the transparency of the underlying scientific basis of
the ROE. The move to an online format has improved user-friendliness of the ROE and the draft takes
advantage of the web format by incorporating graphic enhancement tools, active links to data or related
information that have improved the ability for users to examine the underlying source data. The ability
to move through the website makes it easier to reach answers from multiple entry points and enhances
the potential for the ROE to serve as a resource or gateway for integration or access to other related data
sources.
Another important advantage of an online format is that findings can be updated as new data become
available. Instead of waiting to produce an entire new version of the ROE, data for individual indicators
can be updated as new data emerge.
The EPA selected the concept of sustainability as a unifying conceptual framework for the 2014 draft
ROE and introduced sustainability into the ROE in several ways: (1) by adding a conceptual framework
that includes a sustainability framework diagram, (2) by developing example applications of the
framework to six issues of concern, and (3) by adding a new stand-alone sustainability "theme" with an
initial set of four indicators of resource consumption. The SAB agrees that sustainability is an
appropriate organizing framework for the ROE indicators, but finds that sustainability has not been
sufficiently integrated into the overall ROE. Many of the indicators provide insight regarding progress
toward sustainability but lack sufficient context from which to judge whether sustainability has been
achieved, or what conditions would be considered sustainable. Sustainability is often described in terms
of the three pillars of economy, society and environment, and yet the ROE does not include economic
and societal indicators of sustainability.
The ROE uses 24 questions to organize the information within the themes of Air, Water, Land, Human
Exposure and Health, Ecological Condition, and Sustainability (Appendix B). However, it appears that
1
-------
many of the overarching questions cannot be or are not ever answered in the ROE. Further, the ROE
does not illustrate the relationship of each of the 86 indicators to any specific and systematic treatment
of sustainability, or to how sustainability tradeoffs would be quantified. There are many missed
opportunities to develop cross-linkages within and across the thematic areas, for example by creating
"stories" about how progress is being or can be made towards achieving a sustainable state.
In the charge to the SAB and in the introduction to the ROE, the Agency states that the ROE is intended
to serve as a tool for EPA managers, scientists and researchers as well as to communicate to the public
on national environmental and human health trends. While the stated purpose for the ROE is
appropriate, the Agency does not appear to have determined and defined with any degree of precision
which of these audiences should be the primary audience and which should be secondary. The needs of
these different audiences—what questions they may have and their level of scientific sophistication—will
likely differ and the current ROE format does not seem optimal for any of the intended audiences.
In addition to the above findings, the SAB provides a number of recommendations to more fully
integrate sustainability throughout the ROE, to enhance the interpretation of the existing indicators, to
add select new indicators, and to improve the overall usability of the ROE as an online product. Key
points are highlighted below.
• The web-based, draft ROE is an improved and enhanced version of earlier products. While the SAB
has many suggestions for improvement, some of which should be considered immediately, we also
support a timely release to the public and Agency as a whole.
• The SAB supports the use of sustainability as an organizing framework for the ROE. However, the
SAB recommends that the EPA modify the ROE home page and structure for the ROE to
build/integrate sustainability into each of the tabs in a holistic fashion. The Sustainability tab, if
retained, should be renamed to better reflect the indicators it presents (i.e., resource consumption and
reuse) or be refocused to provide a sustainability context for the ROE indicators, or groups of
indicators, that underlie the 24 questions posed in the ROE. Many new or enhanced indicators to
improve the ability of the ROE to address issues of sustainability are recommended for
consideration.
• The Agency should ensure that the definition of sustainability is readily available and is consistent
with the ROE conceptual framework and with definitions of sustainability used elsewhere by the
Agency.
• The SAB recommends that indicators of both social and economic sustainability be added to the
ROE to complement the current focus on the environmental pillar of sustainability.
• The SAB is encouraged to see the incorporation of statistical information and some analysis tools
within the ROE. The SAB recommends that the Agency continue to strengthen this aspect, and
consider where relatively modest levels of effort to increase statistical information could be of
significant benefit to the overall utility of the ROE.
• To ensure the credibility of the ROE, the most recent available data should be provided for the
indicators. In addition, the "Where you live" section of the site likely will be of interest to many
users and should be substantially revised in future versions of the ROE to make better use of local
data bases and local data aggregation sites.
• The SAB encourages the EPA to better define the audiences for the ROE and to determine who
should be the primary audience, and who are secondary. As the Agency gains an understanding of
2
-------
the needs of various users (e.g., from focus groups or other approaches), the ROE content and
structure could be more closely tailored to the primary audience, perhaps with alternative navigation
pathways for secondary audiences.
• In addition, the SAB urges the Agency to make every effort to ensure that the product is widely
available to all potential user groups, including users (environmental justice communities and others)
who may lack robust internet access. A downloadable overview document with an integrated
summary of the ROE indicators and trends would help achieve this goal.
• Now that the product is web-based, the SAB notes that adaptations/revisions can be made routine,
and can occur more frequently. The SAB recommends that the Agency develop a mechanism to
ensure regular EPA assessments and revisions where new information is available.
• The SAB has recommendations for areas that the Agency should consider for the development of
new indicators. However, the SAB also recognizes that opportunities exist for a greater attention to
relationships/associations and comparisons between/among the current group of indicators.
Accordingly, the SAB recommends that the Agency give priority to examining its current indicators
and developing such comparisons, where appropriate.
The SAB recommends that priority be given to rapid implementation of those changes that can be
achieved within a short time frame (a few weeks or perhaps months), followed by the official release of
the ROE. None of the recommendations made by the SAB imply that this release should be delayed to a
significant extent. Most of the recommended changes can be accomplished subsequent to the ROE
website going live. In general, improvements that are likely achievable in the short-term (prior to or
soon after official release) include changes to clarify, update and better describe the current ROE
indicators and content. Improvements that will require greater time and effort to accomplish include
those that add new indicators to the ROE, or that alter the structure of the ROE. The SAB also realizes
that these anticipated time frames are estimates, and that some of the recommended improvements may
require more or less time than envisioned by the Board.
3
-------
2. INTRODUCTION
2.1. Background
In 2003, the U.S. Environmental Protection Agency released its initial draft Report on the Environment
(ROE) as a first comprehensive effort to report on national indicators that could be used to measure and
track the state of the U.S. environment. The 2003 draft ROE used groups of indicators to answer
questions about the state of the nation's air, land and water as well as human health and ecological
condition. Supporting the draft ROE 2003 was a technical document that provided details for each of the
indicators, including interpretation of the data ("What the Data Show"), discussion of gaps and
uncertainties, and the data sources for each indicator. In a 2004 review of the ROE 2003, the SAB
recommended that the Agency continue the effort with regular updates, facilitated by an online format,
and with enhancements such as inclusion of indicators of climate change and its impacts on human
health and ecosystems (U.S. EPA SAB 2004).
Based on feedback from the SAB and stakeholders, in 2007 the EPA released a revised and updated
ROE science document written for environmental professionals, which was reviewed by the SAB. In
addition, the Agency released a companion "Highlights Document" for the more general audience of
concerned citizens, and a web-based "e-ROE." The SAB's review of the ROE 2007 (U.S. EPA SAB 2008)
commented on the individual indicators, the need for a conceptual framework to link the indicators and the
questions, and the importance of providing interpretation of the data including conclusions supported by
statistical analysis. Along with many specific recommendations to improve the product, the SAB reiterated
its support for continuing the ROE effort. A further consultation was held with SAB panel members in 2009
to provide individual member feedback on ROE future directions and conceptual frameworks to organize and
enhance the understanding of the status and trends for the ROE indicators.
2.2. Charge to the SAB
For the current review, the SAB was asked to review the 2014 draft ROE with particular attention to the
enhancements made since the 2009 consultation, including the utility of the ROE as an online-only
product rather than a printed document. The charge to the SAB includes questions about the use of
sustainability as the ROE conceptual framework, an initial set of sustainability indicators, inclusion of
statistical information for some indicators, the ROE as a web-based product, the success of the ROE for
communicating to multiple audiences, and recommendations for additional indicators (for the full
Charge, see Appendix A). The SAB Advisory Panel for the ROE 2014 was first established in 2012 in
anticipation of the review, and the panel membership was updated in 2014 in light of the specific charge
questions and member availability. The panel met on July 30-31, 2014 to discuss its responses to the
charge questions and held a follow-up teleconference call on October 3, 2014. The chartered SAB met
on January 23, 2015, to discuss and approve the panel's report, with minor clarifications.
4
-------
3. RESPONSE TO CHARGE QUESTIONS
In response to the charge questions, the following sections make multiple recommendations for
improvements to the ROE. The SAB recommends that priority be given to rapid implementation of
those changes that can be achieved within a short time frame (a few weeks or perhaps months), followed
by the official release of the ROE. None of the recommendations made by the SAB imply that this
release should be delayed to a significant extent. The SAB recognizes that it is difficult to predict the
amount of time required to complete specific changes, and hence does not provide recommendation-
specific guidelines regarding expected time frame. Nonetheless, in general, recommendations that are
likely implementable in the short-term (prior to or soon after official release) include changes that
clarify, update and better describe the current ROE themes, indicators and content. Improvements that
will require greater time to accomplish include those that add new sustainability indicators to the ROE
or that would require significant changes to the structure of the website.
3.1. Sustainability as the Conceptual Framework
Charge Question 1: Please comment on the concept of sustainability as an overarching conceptual
framework for representing the relationships between indicators. Please also comment on the
clarity by which the framework is depicted and discussed in the draft ROE and provide any
recommendations to improve its description and intended purpose of representing the relationship
between indicators.
3.1.1. Conceptual Framework
The EPA selected the concept of sustainability as a unifying conceptual framework for the 2014 draft
ROE and introduced sustainability into the ROE in several ways: (1) by adding a conceptual framework
that includes a sustainability framework diagram, (2) by developing example applications of the
framework to six issues of concern, and (3) by adding a new sustainability "theme" accessed by clicking
on a tab at the top of the ROE home page. In evaluating the extent to which the sustainability
perspective has been incorporated in the ROE, the SAB reviewed the new pages that focus on
sustainability as well as the extent to which sustainability was woven into the presentations of the
individual indicators that make up the ROE. While there is some overlap in the charge questions
considered by the SAB, the discussion in this section focuses primarily on the conceptual framework
and the six examples that are accessed at the bottom of the conceptual framework page. Discussion of
the sustainability "theme" and the resource use indicators displayed under the theme is provided in
response to charge question 2 (section 3.2).
The ROE includes a sustainability framework (Figure 1) adapted from that of Fiksel (2012) that depicts
stocks and value flows between three types of systems: economy, society and environment. The SAB
finds that the sustainability framework, and the associated systems perspective, will be a worthwhile
addition to the ROE when more fully implemented. Sustainability is often described in terms of these
three pillars, with diagrams that depict economy, society and environment in intersecting circles to
emphasize the inter-relationships among the pillars. Indeed, this figure is used extensively by the
Agency. However, the framework diagram included within the ROE is overly simplistic and may even
be misleading. In a recent report on EPA Office of Research and Development's strategic research
planning (U.S. EPA SAB 2015), the SAB and the Board of Scientific Counselors recommended that
EPA clarify the "three-legged approach" to sustainability in light of critiques of this approach in the
more recent sustainability literature. At a minimum, the ROE framework diagram should be revised to
remove the pictures within the boxes (e.g., the picture within the economy box implies that all economic
activity is bad for the environment), leaving just the boxes with appropriate labels. The text describing
5
-------
the conceptual diagram should be changed to "This framework represents the world as three interrelated,
integrated, and interacting systems: economy, society, and environment." This will better reflect the fact
that the three pillars are not independent, but rather have considerable overlap. In addition, the diagram
could be better supported by explanatory text. In particular, the Agency should discuss the concepts of
trade-offs between societal actions and choices in the quest for sustainability, life-cycle effects, and the
differences between national sustainability and global sustainability (trade and other flows that impact
sustainability across national borders). Text supporting the framework also should discuss sustainability
as a goal and the use of indicators to measure progress toward that goal. Each of these topics is
discussed below.
Sustainability Framework
Environment
Economy
products and services
it
is"*! U
' I*
~ u labor & regulations
Society
runoff, waste,
emissions,
discharges, and land
cover change
ecological goods
and services
recycling
resource
depletion
human exposure
to waste
ecosystem
degradation
ecological goods
and services
restoration
and protection
Figure 1. The sustainability framework presented in the draft Report on the Environment (ROE) 2014.
Definition of Sustainability. Sustainability is not defined on the Conceptual Framework page and the
definitions provided on other pages of the ROE and elsewhere on the EPA website are not always
consistent. For example, the definition under the Sustainability theme tab ("the ability to maintain or
improve standards of living without damaging or depleting natural resources for present and future
generations") differs somewhat from that provided on the EPA website at www.epa.gov/sustainability
("Sustainability is based on a simple principle: Everything that we need for our survival and well-being
depends, either directly or indirectly, on our natural environment. Sustainability creates and maintains
the conditions under which humans and nature can exist in productive harmony that permit fulfilling the
social, economic and other requirements of present and future generations"). A consistent definition,
6
-------
presented and highlighted early in the ROE website, is critical to ensuring that the conceptual framework
is understood by the user.
When discussing sustainability in the ROE, the EPA should distinguish between sustainability as a
desired status or condition and progress toward sustainability, in which the current status or condition
may not be sustainable but is a transition toward sustainability. The SAB noted also that the ROE—in its
definition of sustainability, choice of questions, and selection of data sources to answer those
questions—does not draw upon the perspectives and value systems of First Nations and Tribes.
Traditional Environmental Knowledge (TEK) is another source of understanding about sustainability,
including relationships between human societies and the natural world and qualitative observations of
natural resource trends (Johnson 1992; Berkes 1999; Hawley et al. 2004).
Economic and Social Pillars. The S AB recognizes that the EPA's focus (and thus that of the ROE)
should largely be on the environmental pillar of sustainability; however, to the extent possible or
appropriate, the EPA should include the economic and social pillars. For example, the ROE should
acknowledge the importance of both market- and non-market values, and the fact that the economic
pillar of the sustainability framework includes production (by industry and households) and demand
(e.g., by households and individuals). Market externalities are one example of factors that inter-connect
social, economic and environment pillars of the framework. Even though the Agency may not have
resources to develop additional ROE indicators for economy and society in the short-term, there may be
some readily available metrics that could be used to raise awareness among users that personal,
consumer, business, and industrial choices have sustainability implications. An example would be a
metric regarding consumer choices of new car purchases in the United States (e.g., Wall Street Journal,
October 1, 2014: http://online.vvsi.eom/mdc/public/paue/2 3022-autosales.html).
Trade-offs and Synergies. Making progress in the quest for sustainability requires a balance among
environmental, social, and economic objectives. In communities where economic and social wellbeing
are tied to the development of natural resources, for example, it is impossible to fully protect ecological
integrity while also maximizing social and economic benefits. The same is true of sensitive ecosystems;
ensuring that environmental services are protected implies that they cannot be simultaneously exploited
in order to obtain certain kinds of economic or social benefits. Data sources, such as the ROE, can
inform decisions by individuals (and policy-makers) or communities that balance environmental, social,
and economic objectives in the service of sustainability. To inform these tradeoffs, and to help decision-
makers identify synergies among objectives related to sustainability, more will be needed—in terms of
information and data sources—than a limited discussion underlying vague notions of what desirable
outcomes may look like. To this end, the ROE could help to provide clarity about the relationships
among different dimensions of sustainability; for example, the relationship between growing the
economy, protecting the environment, and jobs. The report should clarify that these relationships are not
always in conflict. It should also explain that when environmental protection does involve trade-offs
between economic benefits and environmental protection, there is an important role for environmental
regulations to achieve sustainability. This discussion could be linked to the various flow charts in the
ROE that have arrows for environmental regulations.
The ROE should provide a systems framework that enables users to see the connections among the three
dimensions of sustainability. Examples should be included to illustrate the potential for trade-offs (e.g.,
cleaner technologies may be more expensive than conventional technologies) as well as synergies (e.g.,
if cleaner technologies create more net jobs). Examples could also be used to show that efforts to
improve one environmental indicator can affect other environmental indicators positively or negatively.
7
-------
Life-cycle analysis provides a useful framework for quantifying all the environmental impacts of a
product or process within well-specified boundaries. Additionally, a discussion should be included on
the role that different methods for valuing the protection of environmental systems and services may
play (e.g., as a means to help people understand and assess the net impacts across the various
dimensions of sustainability). Similarly, the conceptual framework should provide some discussion
about the usefulness of evaluating the costs and benefits of environmental regulations, as well as the
environmental impacts of various activities that may affect the sustainability of systems.
Life Cycle Effects. In addition, sustainability requires considerations of the life cycle impact of choices.
Life cycle impacts can transcend national boundaries. For example, demand for consumer goods
imported from other countries entails resource consumption, re-use, losses, discharges, etc. that may
have adverse implications for exposed populations. This flow of goods may lead to adverse global
impacts (e.g., deposition of black carbon emitted from ocean-going vessels on polar ice caps, or long-
range hemispheric transport of pollutants such as ozone and fine particulate matter).
Sustainability as a Concept. The general concept of sustainability is normative and has implications for
choices of desirable "sustainability transitions" or progress toward sustainability. Many of the indicators
in the current ROE provide insight regarding progress toward sustainability, but lack sufficient context
from which to judge whether the sustainability has been achieved, or what conditions would be
considered sustainable. The interpretive text for indicators should discuss these points, as appropriate.
Although relative trends may be informative regarding progress toward sustainability, such as for trends
in total energy use, per capita energy use, and energy use per unit of gross domestic product (GDP), they
are not informative when assessing whether sustainability has been achieved (or is achievable given
current choices of resource utilization, technology, etc.).
Recommendations
• Ensure that the definition of sustainability is readily available and is consistent with the ROE
conceptual framework and with definitions of sustainability used elsewhere by the Agency. Also
ensure that the definition is readily accessible to users of the ROE.
• The link between environment and society should be strengthened in the conceptual framework
because protecting public health is an integral part of EPA's mission, and improvement of
environmental quality leads to improved public health and community well-being.
• The suite of ROE indicators should include economic and societal indicators of sustainability. In the
short term, the EPA should provide hyperlinks to existing resources that demonstrate the economic
value of public health and environmental protection.
• Indicators, or at a minimum the discussion relating to the indicators, should address the trade-offs
that are inherent in the interpretation of sustainability for each indicator.
• EPA should determine whether, and if so how, to address life cycle issues in the context of ROE
indicators and how to take into account resource use outside of U.S. boundaries that is associated
with consumption of imported goods, or resource use in the U.S. associated with exported goods.
• The definition and discussion of sustainability should include information about sustainability "for
what," and "for whom." Further elaboration of life cycle concepts, material flows, ecosystem
resilience, etc. would provide further links to the actions and outcomes related to sustainability.
8
-------
• EPA should provide some additional context for the definition of sustainability to point out that
sustainability is a goal. (The additional context can be a hyperlinked window or page that provides
explanation and context.)
• EPA should distinguish indicators of progress toward sustainability as distinct from those that
indicate reaching the objective of sustainability. Initially, EPA should discuss sustainability as a
goal. Over a longer time period, the ROE should provide sufficient indicators and identify
relationships among them to support assessment of whether sustainability has been achieved.
• For future updates to the ROE, the EPA should work to include the views of tribes and indigenous
peoples on what questions should be posed in the ROE and what information, including TEK, could
provide answers to those questions.
3.1.2. The Framework as a Tool for Integration across the ROE Indicators
While agreeing that sustainability is an appropriate organizing framework for the ROE indicators, the
SAB found that the sustainability framework has not been sufficiently integrated into the overall ROE.
The sustainability framework itself is hard to find (e.g., the navigation bar calls it "conceptual
framework") and if users navigate away from one of the framework issue examples to view a specific
indicator there is no obvious button to return to the framework diagrams. Further, the content of the
Sustainability Theme tab does not refer back to the conceptual framework.
A simplified image of the conceptual framework should be located on the ROE home page, with a more
detailed framework (similar to the current framework) located within the content of the Sustainability
page. The simplified image could be a thematic graphical aid used throughout the ROE to emphasize the
theme of sustainability, and would serve as a launching point for more detailed exploration of the ROE
materials. In general, the front page of the ROE should be redesigned to convey the goal of the ROE,
including the emphasis on sustainability, and not just the content and mechanics of the site. It would be
helpful if the entire ROE site could have a look that is somewhat distinct from other EPA web content so
that it is obvious when the user is arriving at or leaving the ROE. More recommendations for enhancing
the communication potential of the ROE are provided in response to charge question 4 (section 3.4).
The six issue frameworks, which illustrate application of the conceptual framework to an issue of
interest (such as acid deposition, mercury in fish, and others1), are appropriate although the examples
themselves could be improved with further development. The six examples provide a limited overview
of the salient components and linkages between a subset of ROE indicators but the available indicators
might not be the best measures of sustainability. In addition, the relationships between the six issue
examples and the four sustainability indicators of natural resource consumption are not clearly
articulated. The issue examples are context-specific models that could be embedded within each of the
five thematic areas (i.e., air, water, land, human exposure and health, ecological condition).
The concept of "questions" as a way to organize the information (see section 2.1) is useful. However, it
appears that many of the overarching questions articulated in the Conceptual Framework section cannot
be or are not ever answered in the ROE. The focus of the ROE centers on these 24 questions, attached as
Appendix B, yet there is only one question related to sustainability, which relates to resource
consumption. The ROE does not illustrate the relationship of each indicator (also shown in Appendix B)
to any specific and systematic treatment of sustainability, or to how sustainability (or sustainability
tradeoffs) would be quantified. There are many missed opportunities to develop cross-linkages within
1 The draft 2014 ROE includes conceptual framework diagrams for six issues: Acid Deposition, Coastal Hypoxia, Fish
Mercury Contamination, Nutrient Impacts, Tropospheric Ozone and Wetland Loss.
9
-------
and across the different thematic (e.g., air, water, land) indicators to reflect components of sustainability.
The questions posed could be answered with "story lines" that address how progress is being or can be
made towards achieving a sustainable state. These stories can also illustrate cross-linkages within and
between thematic areas to show breadth of influence and links among the sustainability pillars. The
conceptual diagrams for acid deposition and other issues are an excellent starting point for such story
lines; these should be more visible to the users of the ROE. Further issue areas may be warranted.
Recommendations
• The ROE home page should include a sustainability framework diagram that serves as an entry point
to exploration of the underlying components of the ROE.
• Relationships among the six example issue areas and the sustainability indicators should be clearly
articulated. The rationale for choosing these six examples should be discussed and links between the
examples should be identified.
• Each issue diagram should be accompanied by a more complete explanation, with a consistent level
of detail developed for each diagram. Further, the issue areas could be linked more directly to
thematic storylines or motivating questions to increase their effectiveness.
• The concept of sustainability should be better integrated within the overarching questions and the
thematic area indicators. For example, explanatory text regarding progress toward sustainability or
achievement of sustainability with regard to individual indicators, groups of indicators within a
theme area, and clusters of indicators that cut across theme areas, would be helpful.
• Further issue areas might be identified and linked to the overarching questions to "tell stories" that
can be used to convey Agency activities and successes. Within this context, an attempt should be
made to bundle indicators to reflect inter-related trends across a number of different indicators or to
show how multiple indicators influence a particular end-use good or service, or combine to jointly
impact the environment, economy, or society. The issue areas need not cover all possible
combinations of indicators, but rather should provide some cogent examples to illustrate the thought
process.
3.2. Sustainability Theme and Indicators
Charge Question 2(a). Please comment on the adequacy by which sustainability has been
incorporated into the ROE. More specifically, please comment on the descriptions and
explanations for the sustainability theme, question, and the four associated indicators.
As discussed in the previous section, the SAB finds that sustainability has not yet been adequately
incorporated into the ROE. While sustainability is described as an overarching framework, it appears to
have had little impact on the selection, presentation and interpretation of indicator data across the
website. Overall, in concert with the adoption of the sustainability framework, six indicators have been
added and three indicators dropped from the ROE. Given this relatively modest change in the suite of
ROE indicators, the question arises as to what extent the sustainability framework has influenced the
structure or content of the ROE. The interpretation and discussion of each indicator neither provide
explicit references to sustainability (their use to characterize whether and how sustainability is being
achieved or approached), nor do they address the types of sustainability questions that are likely to be of
interest to many users.
10
-------
There are multiple bodies of scientific literature that could be used to help situate the ROE indicators
within a more formal treatment of sustainability. One example is the growing scientific literature on
sustainability transitions, which seeks to model and characterize movements towards and away from
sustainability within socio-technical systems (similar to those illustrated by the sustainability framework
diagrams within the ROE). See Markard et al. (2012) for a recent review and Kates and Parris (2003) for
a discussion of relationships to indicator trends. This literature could help the Agency establish cross-
indicator relationships among socio-economic, technical and environmental data in the ROE to identify
the extent to which transitions to sustainability are taking place, particularly as these are related to the
adoption of more sustainable practices.
In the sections that follow, the SAB provides comments and recommendations to integrate sustainability
within the ROE by expanding the scope of the existing themes to include sustainability questions and
implications and by considering additional indicators of sustainability. Most of these recommendations
should be considered medium-term priorities, rather than short-term changes that need be accomplished
prior to launch of the ROE.
3.2.1. Integrate Sustainability throughout the ROE
The current, stand-alone sustainability tab is insufficient to incorporate sustainability into the ROE.
First, this stand-alone section does not adequately reflect the fact that sustainability is a relevant issue
that crosses and integrates all of the ROE themes (i.e., air, water, land, human exposure and health, and
ecological condition). Second, the indicators included in this section do not provide direct insight into
sustainability. To more fully incorporate sustainability within the ROE, the SAB recommends that
sustainability indicators and narratives be integrated into each of the other ROE themes. The SAB panel
members had mixed opinions regarding whether sustainability also should be maintained as a separate,
stand-alone theme in the ROE. A separate Sustainability theme would provide a visible "home" for new
indicators, perhaps including some candidate indicators suggested by the SAB in section 3.2.7, as well
as a place to discuss sustainability trade-offs and interactions among multiple indicators presented in the
other ROE themes. However, a stand-alone treatment of sustainability, without discussion of the
sustainability implications of indicators under the other themes, would imply that sustainability is
somehow a separate consideration rather than the unifying framework for the entire ROE. Regardless of
whether the Agency decides to keep a separate Sustainability theme, all panel members agreed that the
metrics of resource consumption are not sustainability indicators and should not be labeled as such.
The SAB identified two primary approaches for the EPA to more holistically integrate sustainability
throughout the ROE. First, the ROE should include indicators within each theme to provide direct
insight into sustainability. This will also require more explicit relationships to be established across
different indicators, both within and across different sections of the ROE. For example, indicators
related to energy consumption from different sources could be more explicitly linked to indicators on
greenhouse gas emissions. Indicators of contamination and pollution under different themes could be
cross-referenced to related human health indicators. A related suggestion is to combine information on
stocks and flows to provide greater insight into the sustainability implications of the illustrated trends.
For example, when providing trends of freshwater consumption (a flow), additional information on
regional reservoir and aquifer levels (a stock) would provide greater insight into the sustainability of the
illustrated consumption patterns. The introductory text for each thematic area should identify not only
those factors that affect condition, but also those (such as resistance and resilience) that affect
sustainability and progress toward sustainability. Relevant questions include whether sustainability has
been achieved, the extent to which sustainability is being approached, and whether there is evidence of
the adoption of more sustainable behaviors among households, firms and other groups. The SAB notes
11
-------
also that the ROE indicators provide some insight regarding what has happened, but they do not
anticipate expected changes that may be significant, such as changes in energy resource recovery (e.g.,
hydraulic fracturing), agricultural productivity, etc. Specific suggestions for indicator additions are
provided in the response to question 2(c) (section 3.2.7).
Second, indicators should be accompanied by a more explicit discussion of sustainability implications,
considering environmental, economic and social aspects. The ROE currently offers little in the way of
EPA's judgment about what indicator trends are, or are not, sustainable. With respect to the energy
indicator, for example, EPA states "It is difficult to define exactly what constitutes 'sustainability' in a
complex realm such as energy...." However, it seems reasonable to make some statements about
progress toward sustainability (or lack thereof), that the current system is in fact not sustainable, that
there are opportunities for sustainability transitions to move closer to sustainability, or combinations of
these or other relevant points. In general, many of the indicators, or thematic groups of indicators, would
be more meaningful to the user if there were accompanying interpretation of their significance. In this
regard, there are key stories that can be told, but that are not mentioned; such stories would enhance the
information content and usefulness of the trend and status data, and simultaneously could be used to tell
a positive story about EPA's activities.
The current design of the website, with sustainability discussions segregated within separate sections,
does not promote an understanding of environmental, economic and social sustainability as tightly
linked and overlapping issues that pervade all indicators within the ROE. These discussions could be
included within a separate "sustainability implications" tab within each indicator, including links to
other related indicators.
Recommendations
• To more fully incorporate sustainability within the ROE, the SAB recommends that sustainability
indicators and narratives should be integrated into each of the other ROE themes. This integration
should take two forms. First, the ROE should include indicators within each section to provide direct
insight into sustainability. Second, indicators should be accompanied throughout the website with a
more explicit discussion of sustainability implications, considering environmental, economic and
social aspects.
• The introduction section for each indicator should highlight the links to sustainability and if possible
add context to the indicator, such as regarding implications for progress toward or achievement of
sustainability (also see section 3.5.3).
• To the extent that changes are underway or reasonably anticipated that may affect progress toward or
achievement of sustainability, such changes should be discussed in the explanatory text
accompanying a given indicator.
• The EPA should identify and develop "key stories" that enhance the information content and
usefulness of the trend and status data embodied in the indicators, and that simultaneously tell a
story about the motivation for and results of EPA's activities. If the EPA prefers to define the scope
of the ROE to exclude such stories (which the SAB does not recommend), then the Agency should
develop a companion site that provides interpretation, including synthesis.
3.2.2. Select Indicators that Answer Sustainability Questions of Interest to Users
The ROE neither asks nor answers questions related directly to sustainability (Appendix B). The only
question for the sustainability theme, "What are the trends in consumption of natural resources?" does
12
-------
not speak to whether the trends are sustainable. Moreover, the motivating questions in other themes of
the ROE also relate primarily to raw trends, not sustainability. It is unclear what sustainability questions
are addressed by different ROE indicators or the narrative about sustainability that each indicator or
group of indicators conveys. Nor does the ROE provide information sufficient to enable users to
evaluate whether environmental, economic or social sustainability is being approached within any of the
ROE indicator categories. To effectively convey sustainability implications, the ROE themes should
include a set of related sustainability questions likely to be of interest to the public (e.g., to what extent
is U.S. freshwater use sustainable or approaching sustainability?) and then select and organize sets of
relevant ROE indicators to provide insight into those questions. The ROE should include information on
the specific implications of each indicator to the individual sustainability pillars, transitions and
tradeoffs.
Recommendation
• To more effectively convey sustainability implications, the ROE themes should contain a set of
related sustainability questions of interest to the public (e.g., to what extent is U.S. freshwater use
sustainable or approaching sustainability?), and then select and organize the ROE indicators to
provide insight into those questions.
3.2.3. Include Relevant Sustainability Indicators
Taken as a whole, the current ROE indicators (Appendix B) are not well chosen to convey sustainability.
In addition, none of the four indicators included under the Sustainability theme (energy use, freshwater
withdrawals, hazardous waste, municipal solid waste) provide direct insight into sustainability. This
limitation is highlighted explicitly in the technical documentation for each indicator. For example, the
technical documentation for the energy use indicator notes that "There are no thresholds or values that
are considered "sustainable" on a national scale for energy consumption, energy consumption per capita,
or energy consumption per unit of real GDP. Rather, this indicator provides general insights on energy
consumption trends. The degree of "sustainability" in energy use depends on factors such as the source
(e.g., coal versus hydropower), the manner in which the source was produced (e.g., specific fossil fuel
extraction methods), and the manner in which the energy has been used (e.g., where emissions are
released and whether emissions control technology is used)." A long-term solution to this shortcoming
would be to include supplementary indicators that provide the information necessary to answer related
sustainability questions. In the short-term, however, modest improvements can be made to better
communicate existing indicator data. For example, the ROE could communicate the ratio of renewable
versus non-renewable energy consumption as a broad indicator of the extent to which sustainable energy
production is being approached. In addition, the Agency should consider how it can build sustainability
into the other themes as well.
A parallel lack of direct relevance to sustainability applies to other indicators within the sustainability
theme: freshwater withdrawals, municipal solid waste, and hazardous waste. In all of these cases, it
would be possible—at least in principle—to develop a more revealing set of indicators that provide
direct insight into sustainability associated with each topic area. Additional discussion of enhancements
to the four indicators of resource consumption is provided in section 3.2.6 and suggestions for additional
categories of sustainability indicators are provided in section 3.2.7.
13
-------
Recommendation
• The SAB recommends that the ROE include indicators that communicate sustainability implications
across all ROE themes. In the near-term, priority should be given to adding a limited number of
indicators that (a) effectively leverage current information in the ROE to provide greater insight into
sustainability implications, and (b) reflect mature and quality-controlled data sources already
collected by EPA or other federal agencies. In the longer term, indicators should be developed that
more directly answer sustainability questions (see section 3.2.7).
3.2.4. Incorporate Indicators to Communicate Holistic Sustainability Perspectives
Although the sustainability framework diagrams highlight the three pillars of sustainability, the
sustainability section of the ROE focuses almost exclusively on the environmental pillar. This provides
the potentially misleading perception that the most important aspects of sustainability are within the
environmental pillar. As a longer-term goal for the ROE, a more comprehensive treatment of
sustainability would incorporate indicators of both social and economic sustainability, as well as the
adoption of sustainable practices. Even within the environmental area (where the ROE indicators are
concentrated), the provided indicators are generally insufficient to understand whether illustrated trends
reflect sustainable or unsustainable patterns. Specific suggestions for indicator additions are provided
under the response to question 2(c) below (section 3.2.7). In the short term, given that it is likely
infeasible to add indicators reflecting non-environmental aspects of sustainability, the ROE should be
both explicit and transparent about the aspects of sustainability that are and are not addressed.
Recommendation
• The sustainability theme of the ROE focuses primarily on the environmental pillar. A more
comprehensive perspective would incorporate indicators of both social and economic sustainability.
The ROE should also be more explicit and transparent about the aspects of sustainability that are and
are not addressed.
3.2.5. Communicate Linkages across Indicators and Sustainability Trade-offs
If the ROE intends to help users understand sustainability aspects of environmental change, there is a
need for the website to better communicate relationships and linkages across different indicators and
themes. The written material within each section discusses some of these linkages in qualitative terms.
However, at present, each indicator must be viewed independently. There is no easy way within the
ROE to link or cross-reference data from different indicators. The EPA could estimate various types of
correlation, covariance, or other forms of dependence between indicators. Furthermore, as noted above,
linkage of resource use with indicators of environmental discharges, such as emissions, environmental
quality, and human exposure would be informative. Ideally this linkage would be implemented in a
manner that enables a user to look at specific sub-categories of related indicators, such as emissions
trends related to coal as separate from natural gas. It would be helpful to look for linkages between
indicators that address life cycle issues, particularly as related to broader environmental, social and
economic sustainability concerns.
These linkages should be structured to enable users to understand sustainability trade-offs. The 2011
National Academy of Sciences report, Sustainability and the U.S. EPA, recommends (p. 5) that the
Agency develop "specific processes for incorporating sustainability into decisions and actions. As part
of the framework, EPA should incorporate upfront consideration of sustainability options and analyses
that cover the three sustainability pillars (social, environmental, and economic), as well as trade-off
considerations into its decision making." Considering this recommendation, the ROE would be
14
-------
improved by a more substantive discussion of the relationship of each indicator to specific sustainability
trade-offs. This discussion would require a more transparent consideration of what each indicator
implies (or does not imply) about different facets of sustainability, and how these facets are related.
Recommendation
• To the extent that the ROE intends to help users understand sustainability aspects of environmental
change, there is a need for the website to better communicate relationships and linkages across
different indicators and themes. These linkages should be structured to enable users to easily access
relevant sets of indicators and to understand sustainability trade-offs.
3.2.6. Indicators of Resource Consumption
Charge Question 2(b). Please address the utility of the four new sustainability indicators for
informing the reader on the intensity of resource consumption and the relevance of these
intensity metrics.
The four indicators presented under the Sustainability theme tab provide useful but limited information
on the intensity of resource consumption. As discussed in response to question 2(a) above, they are not
sustainability indicators; without additional interpretation and supporting data these indicators provide
little insight into the sustainability of the illustrated consumption trends. For example, trends in resource
extraction or discharge to receptors or environmental media are not placed within the context of
limitations or availability of resources. Indicators that can easily be compared to a benchmark or
standard, such as water quality or air quality, provide critical contextual information about the state of
the environment (and thus its resilience in the face of a disturbance) and are relevant to the sustainability
of the environment and to public health. While the SAB understands that the ROE provides data at a
national scale where possible, some ROE indicators require a regional scale due to geographic
differences in the distribution of some limiting resources (e.g., fresh water, strategic metals, and
renewable energy sources). Too much aggregation of indicators can mask potentially important trends or
structural shifts. For example, consumer choices of passenger cars and passenger trucks can affect motor
vehicle fuel consumption, but this would be masked if motor vehicle fuel consumption is lumped with
all petroleum end uses.
The four new indicators do provide relevant insight into consumption, and they are a useful addition to
the website. However, the ROE would be improved by provision of information or linkages that clarify
the environmental, social and economic implications of the resource consumption information provided
by the four indicators. The utility of these indicators for informing the reader on the intensity of resource
consumption is uneven. Each indicator provides relevant information, but leads to other questions which
remain unanswered. Brief commentary on each of the four resource consumption indicators follows.
Energy Use. The illustrated energy consumption trends (Exhibit 1. Total U.S. Energy Consumption by
Source, 1949-2010) show a relevant pattern in which U.S. energy needs are still met by substantial fossil
fuel consumption. However, it is unclear the extent to which this continued reliance on fossil fuels has
been offset (e.g., in terms of pollution emissions of various types) by increased pollution control
technology. This limitation speaks to the need for enhanced linkages across indicators, as highlighted
above; for example, additional insights on the sustainability of trends in energy use, and the relationship
to economic activity, would be provided by including the most recent data on greenhouse gas emissions
(http://www.eia.gov/environment/). Also, the energy use per capita is relatively flat and energy use per
dollar of GDP is now declining (Exhibit 3. Intensity of U.S. Energy Consumption, 1949-2010); the
impact of these changes (along with population growth and development of renewable energy sources)
15
-------
for the overall sustainability of U.S. energy consumption is unclear.
An additional concern is that GDP alone is a partial and potentially misleading measure of the social
value provided by energy use, as it overlooks influences on human welfare that are not reflected in
market goods and services. Gross domestic product could grow for reasons that are not associated with a
sustainable, healthy economy or society. Examples include economic expenditures related to hazardous
waste remediation (versus pollution prevention and advances in green engineering), expansion of
regional cancer treatment facilities (versus prevention efforts), response to road congestion by expansion
of conventional road systems that lead to unhealthy communities; all of these items increase GDP but
are not part of what would be considered a sustainable community of the future. This concern applies to
all of the resource consumption indicators - GDP alone is an insufficient normalizing factor. At a
minimum, the limitations of GDP as a normalization factor should be given greater emphasis. Data
beyond 2010 would also be particularly relevant to energy consumption indicators, given the significant
increase in U.S. fossil fuel production since that time.
Freshwater Withdrawals. The relevance and interpretability of the freshwater withdrawals indicator is
hampered by the absence of data on water supply, stocks and other relevant factors. Based on the ROE
indicators provided, it appears that freshwater withdrawals have been constant or declining since 1980,
and that the efficiency of water use has increased (per person and per dollar of GDP). However, absent
data on water supplies and recharge (in different regions) and other relevant factors (including the return
rate from various types of water withdrawals), it is impossible to evaluate the implications of these data
for sustainable freshwater resources. However, the current national data on freshwater withdrawals
could be paired with region-specific data on river flows, and reservoir or aquifer levels, along with other
data that could help convey whether the illustrated withdrawals are indeed sustainable. Also, this is an
indicator that is primarily relevant from a regional perspective. National trends in freshwater
consumption are of relatively little relevance, given that freshwater scarcity concerns are almost
exclusively regional and local.
Hazardous Waste. Information on RCRA Hazardous Waste Generated and Managed also leaves
relevant questions unanswered. First, as noted above, the implications for sustainability are unclear.
Second, it is unclear how the illustrated trend data should be interpreted or what it implies in terms of
resource consumption. Hazardous waste is not a consumed resource, but rather the result of certain types
of resource consumption. A more revealing set of indicators would include information on the use and
generation of toxic materials (both in production and by end user) versus the simple quantity of RCRA
hazardous waste generated and managed.
Municipal Solid Waste. Beyond the limited implications for sustainability that apply to all of the
indicators in this section, the Municipal Solid Waste indicator provides estimates of waste production
annually using industry best practices. It is unclear to what extent these indicators account for relevant
factors such as the actual versus projected recycling rate of different types of materials. The indicator
could estimate the utilization of sustainable practices such as recycling, composting and energy recovery
compared to standard landfilling practices over time. Information on these practices could help illustrate
whether and how the nation is moving towards more sustainable waste disposal practices. As currently
presented, the user has no way of determining whether the data show sustainable or unsustainable
trends, or the sustainability transitions or tradeoffs that might be involved. The current municipal waste
characterization report does not provide a mechanism for regionalization of the data set. However, there
are studies that have occurred in other states that can be referenced to provide additional data on
16
-------
regional efforts. The EPA could use its Waste Reduction Model (WARM)2 in order to derive benefit
estimates from recycling. The WARM could help to further document the linkages that exist between
waste management, and its potential contributions to climate change and energy conservation. The
Municipal Solid Waste indicator is also one for which information on uncertainty and statistical
variability are particularly relevant - yet this information is not presented.
Recommendations
• The resource consumption indicators are not sustainability indicators per se. The EPA needs to
present them in a fashion that provides greater contextual clarity.
• Where possible, the ROE should include updated data for all resource consumption indicators. This
is particularly relevant for the energy consumption indicator, which has likely been affected by the
recent surge in nonconventional energy extraction.
• To provide context for the sustainability of natural resource use, the ROE should provide
information not just on rate of use but also size of the resource stock. Trends should be placed within
the context of limitations or availability of resources wherever possible, or with regard to
environmental, economic, or social impacts of their extraction and use.
• The spatial scale at which each indicator is relevant should be clearly stated. Some indicators may
not be relevant at national scales, due to regional distribution or underlying impacts, and therefore
should only be presented as regional in scope. Regional disaggregation is particularly important for
the freshwater consumption indicator because what constitutes sustainable water use varies greatly
from region to region. The Agency should balance the additional relevance of these regional-scale
indicators with the additional demands of data management for regional-scale data.
• A more informative set of hazardous waste indicators would include information on the use and
generation of toxic materials versus the simple quantity of hazardous waste generated and managed.
• Where possible and feasible, indices aggregated over categories, space or time should provide finer
level detail.
• The ROE should incorporate additional normalization measures that are not solely related to the
production of market goods and services, and that capture more comprehensive aspects of economic
and social well-being. For example, economic indicators based on economic practices that represent
progress toward sustainability or achievement of sustainability should be identified where possible.
At a minimum, the limitations of GDP as a normalization factor should be discussed.
3.2.7. Other sustainability questions and indicators
Charge Question 2(c). EPA is anticipating expanding the sustainability theme with additional
questions and indicators in future ROEs. Please provide any specific recommendations on
additional sustainability topics, indicators, and extant data sources that are important to pursue.
Please provide your rationale for prioritizing additional topics and indicators.
The SAB finds that the ROE sustainability indicators by themselves fall short of the comprehensive
nature of the sustainability paradigm, and are at best indirect sustainability indicators. As discussed in
the response to question 2(a) and 2(b) above, the sustainability focus of the ROE would be enhanced by
the combined use of indicators that represent status, trends and relevant sustainability considerations.
Further, to provide a full picture of the sustainability state of the nation and individual regions,
2 Described at http://epa.gov/epawaste/conserve/tools/warm/index.html
17
-------
appropriate social and economic indicators should be included. Accordingly the SAB recommends that
the suite of sustainability indicators be expanded in two important ways; first through the addition of
environmental quality measures that more directly reflect sustainable outcomes, and second through the
compilation and presentation of social and economic data that environmental policies and regulations
can reasonably be expected to influence. The SAB notes that some of these indicators will draw upon
data from the peer-reviewed literature, in addition to federal data sources.
The SAB suggests a tiered approach to identifying appropriate data sets and indicators to enhance the
ROE, as follows. First, reexamine existing data sets in the ROE with a view toward identifying those
with direct relevance to sustainability trends; for example, greenhouse gas emissions, ecological
biodiversity, human health, and land use indicators. Once identified, these indicators might be grouped
under the sustainability tab in appropriate subcategories (e.g., resource consumption, climate change,
ecological health) in addition to their current locations. This approach would have the benefit of
providing symmetry with the other tabs, all of which have multiple subcategories.
Second, identify additional data sets not currently in ROE that, collectively, leverage existing ROE data
and enrich the information conveyed about sustainability trends. The SAB has identified several
categories that might guide the search for these data:
a) Indicators of resource consumption, human use and sustainable practices. The current
sustainability indicators fall into this category, but additional ones might be identified. The Agency
should consider indicators that reflect the sustainability of high-profile natural resource uses,
particularly to the extent that the required data are already collected by federal agencies. Examples
include indicators and estimates of federally managed fishery stocks and harvests (e.g., from the
National Marine Fisheries Service regional fisheries science centers) and areas open/closed to
shellfishing as an indicator of the suitability of coastal and estuarine water quality for specific human
uses. The National Estuary Program (http://water.epa.gov/type/oceb/nep/) might be a source of
relevant, comparable data for high profile estuaries. Other use-related indicators include areas
affected by extreme weather and climate events such as floods and droughts, along with related use
and resource indicators (e.g., crop loss due to droughts/floods), and regional indicators related to
community- or state-level water restrictions or bans. The SAB recognizes that data on indicators
such as local water restrictions and bans may not be available nationwide in a readily accessible
format. However, to the extent that such data could be developed in the longer-term, they could
provide an informative perspective on water scarcity and community-level sustainability. Finally, the
ROE could incorporate indicators on the adoption of sustainable practices related to each ROE
theme, to help characterize transitions to sustainability. The selection of indicators in this area could
be guided by data on sustainable practices already collected by the Agency, as well as insights from
the sustainability transitions literature (Kates and Parris 2003; Markard et al. 2012).
b) Indicators of resistance and resilience. Metrics that encompass characteristics of resistance and
resilience should be identified as potential indicators (e.g., measures of ecosystem condition), or the
relevance of existing indicators to inferring resistance and/or resilience should be discussed in
explanatory text. Healthy ecosystems are able to tolerate some level of disturbance without
substantial change to their structure and function. Once disturbed, a healthy ecosystem is resilient
and can restore itself. Similar to the concept of "adaptive capacity," an ecosystem's capacity for
resistance and resilience is inherently a measure of the system's underlying health.
c) Indicators of energy consumption. While this category might be subsumed under resource
consumption, there is value in offering an expanded set of indicators for energy generation and use.
18
-------
Trends in the growth of "green" power, the ratio of fossil to total energy consumption, are examples.
d) Indicators of efficiency. This category might include manufacturing processes such as steel
production per unit of energy and agricultural yields, but could also include more familiar measures
such as vehicle mileage rates, waste-to-product ratios (for a given choice of products), and
manufacturing productivity.
e) Indicators of transportation. The transportation sector is a major part of the U.S. economy and has
many direct impacts on social, economic, and environmental quality. Measures such as vehicle miles
traveled (and normalized by population), air travel statistics, freight traffic, and mass transportation
ridership convey some of the trade-offs associated with personal choices.
f) Life cycle based indicators. Life cycle approaches measure sustainability trends and demonstrate
the impacts and trade-offs associated with consumer choices and human decisions. Chief among
these are ecological footprints, an accepted and science-based approach with a large literature base
for which the strengths and weaknesses have been identified and analyzed (Rees 2000). The
ecological footprint approach also has the advantage of being presented in regional formats; EPA
might explore the potential of adding this (or other direct measures of sustainability) as an indicator
that integrates information on multiple categories of natural resource use, including consumption of
energy, water, food and manufactured goods. A second life cycle indicator is Embodied Energy, an
accounting method that aims to find the sum total of the energy necessary for an entire product life-
cycle. The Agency might choose a suite of products, or adopt an existing product grouping such as
the Consumer Price Index "market basket" (www.bls.gov/cpi/ ), and illustrate trends over time of the
life cycle energy embodied therein. Software tools and calculators exist to facilitate the assembly of
such data sets.
g) Economic well-being indicators. The SAB notes that GDP is a poor measure of environmental
impacts or benefits, and encourages the Agency to seek better measures. The Green Regional (or
National) Product (GDP minus loss of value of human and natural capital) is in principle a more apt
indicator of the sustainability trade-offs associated with economic transactions (e.g., see Boyd 2006),
although there are currently no well-developed and widely accepted measures of GDP that account
adequately for the loss of natural capital. In addition an indicator such as "green jobs" (maintained
by the Bureau of Labor Statistics, www.bls.gov/green/home.htm) is a direct measure of the salutary
influence green industries can have on the economy.
h) Social and quality of life indicators. The SAB recognizes that this may present the greatest
challenge to the Agency as it tries to present the case that the sustainability paradigm can have
positive social impacts. But there are existing measures from the EPA's own brownfields
remediation and environmental justice programs that can be compiled and presented for their
positive social impacts (e.g., the number of contaminated sites undergoing reuse). Although each has
limitations, additional measures that relate environmental quality to social impacts are hospital visits
due to asthma, longevity as influenced by air quality, and the incidence of waterborne disease
transmission. Information may also be available from other areas of the Sustainable and Healthy
Communities program within EPA's Office of Research and Development (ORD) that could assist
in the development of suitable social indicators for inclusion in the ROE.
Recommendations
• The SAB recommends that the suite of sustainability indicators be expanded through the addition of
environmental quality measures that more directly reflect sustainable outcomes, and through the
19
-------
compilation and presentation of social and economic data that environmental policies and
regulations can reasonably be expected to influence.
• The SAB recommends a tiered approach to identifying appropriate data sets and indicators to
enhance the ROE. First, EPA should reexamine existing data sets already in the ROE with a view
toward identifying those with direct relevance to sustainability trends. Second, the Agency should
identify additional data sets not currently in the ROE that, collectively, leverage existing ROE data
and enrich the information conveyed about sustainability trends. The SAB provides a number of
examples to consider.
3.3. Incorporating Statistical Information
Charge Question 3. Please comment on the approach used to incorporate statistical information
into the 2014 ROE. Please provide any recommendations to enhance the presentation, including
the clarity in describing uncertainty.
The 2014 draft ROE takes a first step towards increasing the statistical rigor of the ROE by presenting
statistical information on a limited number of indicators. Statistical analysis was presented on 12 of the
86 indicators (14%), and the report includes confidence intervals on 21 indicators. Despite trends being
the focus of the 24 questions addressed by the ROE (Appendix B), formal trend analysis was seldom
used. The absence of de novo statistical analysis was deliberate, and the ROE relied instead on extant
statistics. Thus, the lack of statistical analysis in the ROE reflects the general lack of analysis in the
original studies and datasets.
Detailed statistical information and interpretation are presented in the ROE sections entitled "What the
Data Show," "Limitations" and "Technical Documentation." Uncertainty is discussed primarily in
qualitative terms in table footnotes and technical documentation for each indicator. The SAB finds that
the ROE would be improved if the statistical information were more visible and presented in a way that
is more accessible to a broad audience. In its current form, the supplemental information included in the
technical documentation and other sections is dense, somewhat hidden and requires the user to click to a
box to open and read the information. Many users will probably miss these links to key information,
especially casual and novice users. The EPA should consider a shortened version of this information up
front on each indicator page, immediately visible when looking at the indicator.
The important concept of uncertainty also can be explained in multiple ways, ranging from simple to
more complex characterizations, as would be appropriate for a product with a broad user group. The
"What the Data Show" section does a good job presenting and interpreting key uncertainties in a way
that is understandable to most users. While the scientific community will be comfortable with formal
statistical metrics and hypothesis tests, the general public is better served by less formal
characterizations of uncertainty. The ROE is an opportunity to help the public to understand uncertainty
in science; the ROE might draw upon the thoughtful approach used by the Intergovernmental Panel on
Climate Change (IPCC) (IPCC 2013) to relate quantitative statistical metrics to common language
characterization of uncertainty. For example, IPCC uses the terms "likely" to indicate probability levels
of 66% and "extremely likely" to indicate a probability level of 95% (IPCC 2013, p. 142). The SAB
suggests that the use of common language in this way provides a good model for how the ROE can
improve communication of uncertainty to a lay audience, and it can enhance the ability to develop a
shared language to communicate with both technical and non-technical audiences.
20
-------
Although the SAB understands that the ROE is not intended to examine causality nor to directly assess
the consequences of EPA programs or actions, additional interpretation and context of indicator trends
would make the report more useful. Currently, interpretation is largely left to the user. A broader,
summary discussion that synthesizes information from multiple indicators, and at a level similar to the
"What the Data Show," would be a useful addition. For example, do the majority of air indicators show
similar trends, or are indicators sending differing signals about status and trends for a resource?
Interpretation of results across multiple indicators might be done within the context of a new page that
summarizes the results.
Even without formal statistical analysis, the ROE might employ a weight-of-evidence approach (e.g., in
the sense of Suter and Cormier 2011). The existing approach of reporting only confidence intervals from
available studies means that metrics can only be used from individual statistically based studies. But key
questions within the ROE can best be answered when information is compiled from multiple studies. For
example, a question such as whether surface water quality is improving can be answered by providing a
series of indicators from different studies or from summary information compiled from a number of
studies. The latter approach is typically most useful, especially for non-technical audiences. However,
compiling information and summarizing uncertainty from multiple studies is challenging because
typically different methods are used, data quality are not uniform, etc. The IPCC reports provide a
thoughtful model on how to carry out such analyses and provide uncertainty information. However, the
SAB understands that such enhancements would require a substantial effort that may not be practical in
the near term.
Finally, the Agency should continue to increase the statistical rigor of the ROE. While extensive new
statistical analyses are not warranted within the context of the ROE, limited new analyses could provide
considerable value with a modest investment. For example, ambient annual average air quality
monitoring data of SO2 concentrations shown in Exhibit 1 of the draft ROE indicator Ambient
Concentrations of Sulfur Dioxide over the period 1980 through 2011 illustrates a decrease in average
concentrations from 11.94 to 1.95 ppb. The underlying data have been subject to rigorous EPA QA/QC
protocols, and trend analyses of data such as these could provide low-cost opportunities for the ROE to
extend statistical analyses beyond drawing existing statistical metrics from reports. Many of these time
series are important success stories that EPA could present, and would be an important answer to the key
question of whether environmental quality is improving. Likewise, it would be helpful to explore
relationships between indicators to get more insight with respect to factors that co-vary or that may be
independent of each other. That said, ultimately, it is really the role of the primary studies more than the
ROE to do a more rigorous analysis of uncertainty.
21
-------
The following are recommendations for medium- to long-term improvements in statistical information
presented within the ROE.
Recommendations
• The ROE should present statistical information in a way that is more visible and accessible to a
broad audience, including explaining and characterizing uncertainty in a variety of ways - from
simple to complex - for different audiences and providing additional interpretation and synthesis of
the patterns illustrated by the indicators.
• The EPA should consider a weight-of-evidence approach to data interpretation for single indicators
and groups of indicators as appropriate to the question being addressed.
• The EPA should continue to increase the statistical rigor of the ROE. When there is opportunity for
statistical analyses, focus efforts on key indicators as illustrated in the example above.
3.4. ROE as a Web-Based Product
Charge Question 4. (a) Please comment on the scientific rigor and clarity of the ROE content with
the transition from a printed document to an online presence, (b) Please provide suggestions on
other factors that could be considered concerning the overall content, format, credibility, user
friendliness and navigability of the site.
3.4.1. Transition to an Online Format
The transition to a web-based format of the ROE represents an improvement from earlier versions,
particularly in regard to usability and the ability to link to, and download, underlying data for individual
indicators. This serves as a substantial benefit in regard to transparency of the underlying scientific basis
of the ROE. The move to an online format has improved user-friendliness of the ROE and the draft takes
advantage of the web format by incorporating graphic enhancement tools including a subset of
interactive maps, graphic data "mouse-overs," active links to data or related information that have
improved the ability for users to examine the underlying source data. The ability to move through the
website makes it easier to reach answers from multiple entry points and enhances the potential for the
ROE to serve as a resource or gateway for integration or access to other related data sources.
Another important advantage of the move to an online format is that it allows the findings to be updated
as new data become available. Instead of waiting to produce an entire new version, data for individual
indicators can be updated as new data emerge. Most of the programs from which the ROE pulls data are
ongoing and most produce data on a quarterly to annual basis. Incorporation of these data on a
continuing basis should be more easily accomplished with the online format rather than a printed report.
However, the functionality of a web-based presence has not been fully realized in the 2014 ROE, with
many enhancements still possible. The Agency should consider improvements in summary presentation,
navigability within maps/graphics and between theme/indicator-level content. Two specific areas that
have not transitioned to the web-based format are (1) the inability to read offline, which may be a barrier
to some audiences with access or technical constraints (this is a particular concern among vulnerable or
susceptible groups with environmental justice or economic equity concerns); and, (2) the lack of a high
level, or executive summary highlighting the ROE content.
In addition, the move to an online format presents the opportunity to take full advantage of the explosion
in the use of social media by large portions of the public. The Agency now has the opportunity to
22
-------
provide the information in the ROE in many different, user-friendly snippets through social media sites,
and through the development of ROE-specific mobile applications (or "apps"). The SAB discusses
these, and other possible improvements below, in sections 3.4.2 to 3.4.3. The SAB recognizes that some
of the topics discussed are more important than others; in addition, some will be more easily
implemented while others will take considerable time and effort. Therefore, the SAB has included some
suggestions for priority of its recommended improvements and for the time needed to implement them.
Recommendations
• The SAB recommends that EPA take steps to ensure accessibility of the ROE for stakeholders who
have limited or no access to the internet or who for other reasons need to use the ROE off-line. Some
users, particularly those from disadvantaged communities, may only have brief windows of internet
access at public libraries. Providing the ROE highlights, including a summary of the indicators and
trends, in a single PDF or some other downloadable format would help achieve this goal in the near
term (see section 3.4.2).
• Over the longer term, the Agency could increase access to the ROE on mobile devices and use social
media and mobile apps to capture the attention of potential users of ROE information. For example,
the Agency could develop an app and use it to push out notifications with an "indicator of the day",
when indicators are updated, or to make a thematic announcement (e.g., the anniversary of a key day
in the history of an environmental issue that is addressed in the ROE).
3.4.2. Clarity of the ROE
The ROE text is clearly written, but the SAB suggests that the Agency begin to move past a "just the
facts" design to one that provides greater content to reach the intended audience(s). The text sections are
of the right length for a website and the subsections for each indicator are logical and contain
appropriate topics (Introduction, What the Data Show, Limitations, Data Sources, etc.). The SAB finds
that additional headings (What the Data Mean, Links to Other Indicators) would increase the value and
usability of the ROE by providing data interpretation and some discussion of the linkages among
indicators and their relationship to sustainability (see section 3.5 in response to question 5).
The ROE provides little background on the indicator selection process. During panel deliberations in
July 2014, the Agency described the indicator selection process, and some subset of the information
from that presentation should be made part of the background section of the ROE. The SAB found that
background information was also lacking about the 24 questions (Appendix B) that form the core around
which the indicators are presented. In each question there is reference to "effects on human health" or
"effects on human health and the environment," but the interpretation of the indicator data to these
endpoints is not always clear, especially to a general audience. The questions could be edited so that
they focus better on the data that are available, or the interpretation of the indicator values and trends
needs to be explicitly presented.
The opportunity for improving the drill-down capability is particularly large in the "Where You Live"
section. Presently, when trying to get to state-specific data, one is simply directed to the main page for
each ROE indicator - no state-specific indicator data are presented. For mapped indicator data, one is
directed to the main U.S. map, which then must be manually zoomed to locate the desired state. The
"Where You Live" functionality is extremely limited and could be enhanced to provide place-specific
data, above and beyond the basic nationwide indicator data currently in the ROE. Some combinations of
links are not meaningful and could result in a negative experience for a lay user. For example, when one
clicks on FL for ozone one gets U.S. ambient concentrations (not FL ambient concentrations or
23
-------
especially the part of Florida that was clicked). When clicked on FL for regional haze, one is provided
data on the visibility in selected national parks in the western United States.
The data download feature allows users to access additional data above and beyond those in the simple
charts and graphs, including data to address more sophisticated questions. Future versions of the ROE
would be enhanced by providing access to more spatially detailed data, beyond those shown in the
highlighted graphs. A more flexible and useful system could involve much larger and more sophisticated
datasets that could be viewed and analyzed in multiple ways. Developing such flexibility would be a
labor-intensive and difficult process, as it would require an ability to reconcile indicators collected for
different purposes and at different time scales. However, improvements in this direction would greatly
enhance the ROE as a science and research tool, rather than as a simple data illustration tool. Such
enhancements, however, would likely require a substantial increase in the resources available to develop
the ROE.
The website is structured in a manner such that it is easy to miss important content. For example, to
reach the discussion of greenhouse gases (http://cfpub. epa.gov/roe/chapter/air/ghgs. cfm), it appears that
the user must follow the path from the ROE Home, to Air, and then to the hyperlink to the Greenhouse
Gases page. If one instead jumps directly to one of the individual greenhouse gas indicators through the
"Air" theme tab on the ROE homepage, the only way to return to the Greenhouse Gases page is to notice
the small note at the bottom (i.e., "This indicator relates to: Greenhouse Gases"). The website could be
redesigned to be more user-friendly and to enable users to access information more consistently.
Finally, in terms of clarity, the sustainability diagrams are an interesting approach to illustrate the
linkages between different systems, the related processes, and the indicators being shown elsewhere in
the website. At the overview level, these diagrams are helpful, but as additional layers are added, or
when specific issues (such as acid rain or mercury in fish) are considered, the overall result is quite
complex. It is not clear that these graphical presentations have value for the non-technical user.
Feedback from users could be used to enhance the guidance on how to use the ROE. For example, an
animated tutorial using one of the six sustainability issues could be helpful in illustrating the systems-
based approach for linking the three figures for each issue (Overview, Attributes, ROE Indicators). The
site also could point the user to bundles or subgroups of the 86 indicators that relate to vulnerable
populations (children, elderly), to specific regions, or to particular aspects of sustainability. See also the
discussion that follows in section 3.5.1.
Recommendations
• The Agency should revise the ROE home page to summarize the goals and vision of the ROE and to
make it more user-friendly. Revisions to the text of the front page are needed in the near term to
clearly communicate the goals and vision of the ROE, and to relate the information to a general
audience. A high-level "state of the environment" summary should be presented with "highlights" of
the key environmental and public health conditions and trends that have important implications for
things people care about (e.g., the health of ecosystems and of individual humans and communities).
The highlights summary should be easily downloaded (e.g., an attractive pdf) and should provide
visible and helpful links to more detailed information in the ROE, including references that link
across media/themes where relevant.
• To inform future updates to the ROE, the EPA should employ user focus groups or other formal
approaches to better understand site clarity, usability and performance. An important first step to
24
-------
enhance the value of ROE to different user groups would be to interact with representatives of
potential user groups to evaluate how they use the ROE and enhancements that would increase the
utility of the website. EPA should engage with different types of users to assess what areas of the
report are being used and where users are being challenged. In addition to focus groups, other
potential approaches include one-on-one interviews and site testing with individual users, or surveys
of potential users to evaluate information needs that might be addressed by the ROE.
• Results from the focus groups or other approaches could be used to create user guidance about the
website. Additional guidance could be provided on how to use the ROE features beyond the current
textual explanation under the "Detailed Guide" page on "Explore an Indicator" and "Special Tool
Interactive Maps," perhaps via an online, step-by-step animation walk-through example. The
Agency could improve the introductory material on the website to increase the visibility of
sustainability indicators and to help highlight opportunities to understand the sustainability
framework both on the Sustainability tab and perhaps from the ROE home page (via
link/highlighting).
• The Agency should reframe the 24 questions so that they more clearly align with the data being
presented. The indicator interpretation needs to describe how the effects on health or the
environment are linked. In many cases this is not possible with the current ROE indicators and the
SAB recommends that the questions be reworded in the near term to better fit the indicator being
described. Thus, a question could be: "What are the trends in greenhouse gas emissions and
concentrations?" which is more direct and is answered through the data, rather than "What are the
trends in greenhouse gas emissions and concentrations and their impacts on human health and the
environment?"
• Over the longer term, the Agency should expand the opportunities for users to "drill down" to
underlying data. For example, the ROE could provide access to underlying data for a larger number
of the exhibits (charts/graphs). The ROE maps, while more informative than prior printed versions,
are still rudimentary compared to those present in other types of online decision-support tools. For
most if not all maps, the sole functionality is an ability to zoom in (or out) to different scales, or turn
off selected data layers. This stands in contrast to more sophisticated interfaces on other web-based
tools.
3.4.3. Scientific Rigor of the ROE
The indicators and data sets used in the ROE are all products from other programs that have undergone
some level of peer review, either programmatically or through scientific journal review. As such,
credibility of the information is high. Several reviewers, including a public commenter, noted nuance
level problems with a few indicators, such as wetlands, benthic infauna and greenhouse gases, but most
of the concerns focused more on interpretation than on problems with the underlying data. The EPA has
an opportunity to address those issues with a thorough discussion in the data limitations section for each
indicator.
A larger threat to credibility of the document lies with the "Where You Live" section. The concept of
personalizing the data by allowing people to see patterns in their local geography is a good one, but the
execution of this concept needs to be improved. The present approach is to provide the reader a subset of
data out of the national data base, but the national data sets are often not data-rich enough to retain
sufficient sample density for assertions when subset to a region. Moreover, many local sampling
programs are not included in the national data bases and the ROE would be more effective if they point
the reader to these local information sources. The ROE needs to embrace these local information
sources, perhaps through links, rather than providing alternative answers based on a less robust data set.
25
-------
Another shortcoming that lessens credibility of the ROE is that many of the data presentations do not
include the most recent data. Although the draft report is dated 2014, the most recent data for some of
the indicators are 10 years old even though more recent data are available. The SAB realizes that the
current version of the ROE 2014 is still a draft product, but the EPA is strongly encouraged to update
these data prior to official release of the product.
Credibility of the ROE 2014 also can be improved by adding more, or providing links to, relevant data
from other federal agencies. The report does this for several indicator groups but not for others. A more
thorough inclusion of data from other agencies will ensure that the document is perceived as a national
Report on the Environment, rather than as a report on EPA's activities.
Of the following recommendations, the highest priority should be given to updating the current ROE
indicators with the most recent data, and to improving the "Where You Live" section (although the SAB
recognizes that some of the latter improvements are more extensive and may require a greater amount of
time to implement). Other improvements, including the provision of links to data from other agencies,
should be considered medium- to long-term goals.
Recommendations
• The Agency should substantially revise the "Where you live" section of the site in future versions of
the ROE. Emphasis should be on making better use of local data bases and local data aggregation
sites. The revisions also should incorporate better graphical methods for zooming down to lesser
spatial scales. The SAB also noted that this section had the greatest concentration of broken or
inappropriate links in the report. These should be fixed in the short term.
• The ROE should incorporate more recent data for the included indicators, where available. One of
the strengths of the web format is that it allows new data to be added as they become available for
each indicator. EPA should take advantage of that feature to ensure that the information presented is
the most recent the Agency has. At a minimum, the ROE should contain the most recent data
available at other locations on EPA's website.
• The ROE should provide a more critical evaluation of each indicator in the data limitations section.
Several panel members, and a public commenter, identified issues with a few of the indicators with
respect to their limitations in addressing some of the questions posed in the report. The EPA should
re-examine each of the indicators in the report to ensure that they are appropriately stating the
interpretation of the data for each indicator and enhance the data limitations section where the
possibility of readers' misinterpretation exists.
• Over the longer term, the Agency should increase the use of relevant, non-EPA national data sets.
EPA should consult with leadership of other agencies (such as CDC, NOAA and USGS) to ensure
that the report includes appropriate indicators they collect that are relevant to addressing the 24
questions posed in the document. Indicators suggested by those agencies that are less relevant than
the ones already selected for the report should be included under the section "For More Information"
on the indicator pages.
• Prior to release of the draft ROE, the Agency should address navigation issues and broken links.
When clicking through to the "Where You Live" ROE data for each state, one is simply directed to
the main page for each ROE indicator - no state-specific indicator data are presented. For mapped
indicator data, one is directed to the main U.S. map, which then must be manually zoomed to locate
26
-------
the desired state. Individual members noted they encountered broken links; the entire site should be
scanned and broken links removed or replaced.
• Over the longer term, the Agency should highlight and create functionality to aid in interconnection
among indicators. Make the connection between the 24 questions and the related themes and
indicators more obvious beyond the clicking on the high level theme (air, land, etc.) or the link under
Basic Information. Perhaps these connections also could be highlighted at the indicator level (via
another narrative bar under the exhibit), and/or with a stronger summary description on the theme
home page.
3.5. Communicating to Technical and Non-Technical Audiences
Charge Question 5(a). Please provide feedback on the approaches used in the ROE to provide
information such that audiences with varying interests can efficiently and reasonably find
information concerning the status and trends of environmental conditions. Please provide any
recommendations to enhance the access to information.
Charge Question 5(b). Please comment on the accuracy of the ROE's presentations and their
effectiveness in communicating complex scientific information to a broad range of technical and
non-technical audiences. Please provide recommendations for specific components of the ROE as
appropriate.
The charge to the SAB and the introduction to the 2014 ROE draft state the purpose and goals of the
report as follows:
EPA's Report on the Environment (ROE) presents the best available indicators of national
trends in the environment and human health. EPA selected these indicators to address 24
questions that are critical to EPA's mission of protecting the environment and human health.
The ROE indicators serve as a tool for EPA decision-makers, program planners, scientists,
researchers, the public, and others interested in environmental science and policy to track
changes in environmental condition. They allow EPA and the public to assess whether the
Agency is succeeding in its mission, and they help alert EPA to new challenges that may
need attention and action.
The declared purpose of the ROE, therefore, is to communicate within EPA and out to various public
audiences and other interested parties so that they may better understand and assess whether the Agency
is succeeding in its mission to protect the environment and human health. At the same time, the ROE is
intended also to help alert EPA scientists and administrators to new challenges that need attention and
action. The SAB finds that these are appropriate and important purposes that would clearly serve the
goals of the EPA. As currently constructed, however, the draft ROE does not effectively address all of
these declared intentions or the various audiences that they imply. In particular, the draft ROE in its
current form is less likely to effectively engage and inform public audiences who are not already highly
motivated and who do not have relevant technical experience and abilities with both the web medium
and the ROE content. These and other concerns and recommendations of the SAB are presented below.
These are recommendations for medium- to long-term improvements, rather than updates that are
necessary prior to launch of the ROE.
27
-------
3.5.1. Defining the Target Audiences
While the stated purpose for the ROE is clear and appropriate, the Agency does not appear to have
determined and defined with any degree of precision who will comprise the various audiences for the
ROE, and which of these should be the primary audience and which should be secondary. Without this
clarity it is unlikely that the ROE can fully satisfy the needs of any of the intended audiences. It is
inappropriate for the ROE staff alone (or for the SAB) to assume what any of these audiences might
want; it is imperative that representatives of the intended audiences be queried systematically and
directly to determine, for example:
• What will attract and retain user engagement with the site?
• What expectations and questions do users have in mind when they come to the site?
• What do the various intended audiences want to find when they enter the ROE; i.e., just the
indicator data, interpretations of data (the "so what" implications), synthesis across indicators
and across media?
• What do potential users want to learn, to understand, to do?
Given the declared purpose of the ROE, there is an immediate need to better define the desired
audiences for the ROE and to revise the product to better meet their needs, especially for public
audiences. Revisions of both webpage design and report content and structure would best be achieved
through systematic interactions and testing with samples of targeted audience populations, as discussed
in section 3.4.2. The SAB understands that a substantial national survey of audiences is not within the
scope or the resources of the ROE. However, a careful and well-conducted review of who has been
accessing the currently released draft and how they have used (and not used) it should be possible and
would be very informative to ROE designers and authors. Technically sophisticated audiences and those
with very focused interests, especially those who are familiar with the prior ROE, will likely find
accessing the ROE straightforward and also will need little encouragement to make their concerns and
wishes known to the Agency. The greatest need, therefore, is to learn how the ROE could better engage
and serve the needs of more general public audiences. These audiences include people who are
concerned about the state of the environment and related public health issues, but who do not have the
technical expertise or experience to find their way to the particular indicators relevant to their concerns
or to discern for themselves the implications of the displayed trends in those indicators.
Recommendation
• The SAB encourages the EPA to better define the target audiences for the ROE and to determine
who should be the primary audience, and who are secondary. Future revisions of the ROE then
should be guided by the goal of engaging and informing these audiences about the state of the
environment and public health.
3.5.2. Structuring the Site to Meet Audience Needs
The new web-based version of the ROE has great potential for effectively addressing a variety of
audiences, including concerned members of the general public, educators and students, and scientists
both inside and outside the EPA. However, the needs and wants of these audiences may differ in
significant ways and therefore have important implications for the design and structure of the ROE
website. The current one-size-fits-all format does not seem to work optimally for any of the intended
audiences.
28
-------
Scientific and technical audiences. The current structure and content of the ROE primarily addresses
the needs of scientific and technical audiences who are seeking data about specific indicators related to a
narrow set of questions that are mostly organized by environmental media. There is a lack of adequate
context at the start of the website and this presents a substantial barrier for most public users. The "tiered
approach" of the ROE is intended to allow users with different interests to follow their own paths to the
information through the layers of technical information based on their own interests and expertise.
While this organization may be sufficient for the more sophisticated audiences of scientists and technical
experts in and outside the Agency, it is unlikely that many concerned citizens will be engaged by the
site, and then will be able to access information that is relevant to their particular interests and concerns.
Public audiences. For public audiences the current draft of the ROE does not provide adequate context
for users to determine which themes, questions and/or indicators are most relevant to the issues they care
about. Even if public/citizen users do find and access a relevant indicator they are unlikely to understand
what the depicted "trends" imply for the environmental/human health values that concern them—i.e., are
the things they care about getting better or getting worse and what has the EPA done (or is planning to
do) about it? It is also very difficult for the public or even for technically experienced users to look at or
understand how one indicator might relate to others, especially those indicators that are housed under
different themes in the ROE.
Integration across ROE themes. The tiered structure of the draft ROE is one-dimensional—there are
clear pathways (for technically sophisticated users at least) down through the various themes and
questions, but there is little or no linkage or opportunities to transition between these "vertical" paths or
to integrate and synthesize across several themes/questions/indicators. An important case in point is the
presentation of the six issues that are offered to illustrate how the sustainability framework integrates the
various themes/questions/indicators. After a user selects and explores one of the indicators from a
selected issue framework diagram, there is no clear path for returning to the issue diagram to consider
other indicators. This same "dead end" effect occurs when exploring an indicator accessed through one
of the 24 key questions. It does not help that the presentations between different indicators contain
numerous inconsistent definitions, muddled levels of technical terminology, and unclear or inconsistent
means for accessing desired information. The current structure prevents the site from being as efficient
and effective for multiple audiences.
Although the current predominantly "vertical" structure may serve EPA insiders and sophisticated,
experienced outside users, even these audiences would benefit from better "horizontal" integration.
Indeed, greater integration across themes and the illustrative issues would be more consistent with (and
would better support) the reorganization efforts in ORD which are driven by the changing nature of
environmental and health threats and by the associated transition away from EPA's traditional single-
threat/single-medium focus. As noted above, general public/citizen audiences will require a different
structure that provides better context at the top and clearer identification of easily followed "tracks" to
lead them to the information relevant to their concerns.
While some audiences will come to the ROE with concerns about specific environmental conditions and
risks (e.g., drinking water, air quality, toxics) others will come with concerns that cut across the existing
themes and traditional risk categories (e.g., college and high school students and teachers looking for
integrated information on environmental condition, caregivers of children looking for information on
risks to children, environmental justice communities looking for risks associated with places). When
more than one indicator is related to a given theme or question, key findings and implications based on
the combination of indicators should be summarized before directing the user to the more detailed
29
-------
presentation of the individual indicators. Further, it should be obvious and easy for the user to get back
to the relevant theme/question/issue and highlights summary from any level within each of the
respective indicator sections.
Recommendation
• The structure of the ROE, which is primarily vertical, would be improved by providing greater
freedom to move horizontally across themes. A more flexible structure would address the concerns
of all targeted users, lead them to relevant data and information, help them relate the information to
other associated sections of the report, and help them to understand and appreciate the implications
for the environmental and public health issues that they care about and the role of the EPA in
addressing those issues and concerns.
3.5.3. Structure of Individual Indicators
The ROE, as it is currently structured, provides a wealth of information under various themes, questions
and indicators. Beneath the main figure or table that characterizes each indicator and trend, the ROE
presents information under the headings Introduction (to the indicator), What the Data Show,
Limitations, Data Sources, View Technical Documentation, and View References (see Figure 2 for an
example). However, the SAB finds that there is room for improvement both in terms of sharpening the
contextual focus of the indicators and trends presented and in ensuring that the information
communicated to users of the ROE will be understood by non-technical user groups.
The level of detail varies from indicator to indicator, and often demands a technical sophistication that
would exceed the level of scientific literacy that can be expected for many of the audiences for the ROE.
In order to facilitate more effective communication about and—importantly—comprehension of the
implications of each indicator or trend, some revisions to the current structure are warranted. These
revisions should be aimed at providing a more straightforward contextual entry-point into each indicator
(e.g., from the top level links in the highlights section on the home/front page, as recommended in
section 3.4.1 and reinforced in several sections below), followed by increasing technical and scientific
detail as users click down through the various links in whatever track they have selected. Relevant
relationships among multiple indicators under individual themes/questions should also be pointed out
and made easily accessible and, where appropriate, some succinct summary/highlights integrating the
findings and implications across those indicators should be presented at the respective theme/question
level.
30
-------
You are here: EPA Home » ROE Home "Energy Use
Energy Use
~ Ecological Condition ~ Sustainabiiity
Note to reviewers of this draft revised ROE: This is a new indicator that reflects data through 2010. EPA anticipates
updating this indicator in 2014.
H Exhibit I. Total
U.S. energy
consumption by source,
1949-2010
a Exhibit 2. Total
U.S. energy
consumption by sector,
1949-2010
~ Exhibit 3.
Intensity of U.S.
energy consumption.
1949-2010
Exhibit 1. Total U.S. energy consumption by source, 1949-2010
I Other renewables
| Biomass
I Hydroelectric
Nuclear
Natural gas
I Petroleum
I Coal
This graph does not account for net imports of electricity and coal coke, which would change the total by
-0.08% to 0.2496, depending on the year.
Information on the statistical significance of the trends In this exhibit is not currently available. For
more information about uncertainty, variability, and statistical analysis, view the technical
documentation for this indicator.
Data source: EIA, 2011
3
~
BE
i Click the
legend to
turn layers
on or off.
Hover your
the display
to reveal
data.
Introduction ~
What the Data Show ~
Limitations ~
Data Sources ~
| View Technical Documentation |
| View References
For More Information
U.S. Energy Information Administration
Learn how this indicator fits into conceptual diagrams for: Acid Deposition,
Coastal Hypoxia, Fish Mercury Contamination, Nutrient Impacts, Tropospheric
Ozone
This indicator relates to: Outdoor Air Quality, Resource Consumption,
Greenhouse Cases
This page provides links to non-EPA websites that
provide additional information about this topic. You will
leave the EPA.gov domain, and EPA cannot attest to the
accuracy of information on that non-EPA page. Providing
links to a non-EPA website is not an endorsement of the
other site or the information it contains by EPA or any of
its employees. Also, be aware that the privacy protection
provided on the EPA.gov domain (see Privacy and
Security Notice) may not be available at the external link,
itxn Mwiiimw
You will need the free Adobe Reader to view some of the
files on this page. See EPA's PDF page to learn more.
ROE Home ?Top of Page
Figure 2. Layout of information presented in the ROE about energy use.
To provide an illustration of how the ROE might be restructured at the individual indicator level, the
SAB offers the modified layout below:
Level 1 - Introduction to the Indicator: At this top level, the ROE could provide a concise and
accessible description of what the indicator is, what it measures, and why it was chosen by the
Agency.
Level 2 — What the Data Show: At this level, the ROE could present a concise and accessible
description of the data or trend being displayed. At this second level, the text should not confuse
descriptive information about the data or trend with additional contextual information that might
warrant its own indicator. For example, in the What the Data Show section for the indicator
about Energy Use (Figure 2, above), the ROE provides information about economic growth as a
function of energy use, as well as additional information about energy use per capita. This
information might provide links to additional, potentially important indicators that are displayed
in other parts of the ROE. Flowever, care should be taken to ensure that this information is not
misleading; e.g., some users of the ROE might be led to believe that increasing energy use
overall is not a matter of concern to the EPA (as a result of its relationships to other indicators
31
-------
and areas of concern; see below) because per capita energy use has, in the words of the ROE,
"remained fairly steady for the last few decades."
Level 3 - What the Data Mean: At this third level, the ROE could provide a concise narrative
about the overall meaning, (statistical) significance, and implications of the presented trend. In
the case of Energy Use, for example, the narrative might provide additional information about
the implications of a tripling of energy use in the United States between 1949 and 2010; e.g.,
what does this trend mean with respect to the consumption and sustainability of natural resources
(like fossil fuels), CO2 emissions, air quality, climate change, etc.? It would be ideal if each of
the implications presented at this level could link to other indicators being used in the ROE; see
Level 4, below.
Level 4 - Links to Other Indicators: At this level, users of the ROE would be able to view
information about other indicators and trends linked to the current indicator (e.g., the
consumption of fossil fuels, CO2 emissions, air quality). Unlike Level 3, which would provide a
narrative description, Level 4 would offer access to other sections of the ROE (and/or other
relevant webpages) that focus in detail on associated indicators. In future versions of the ROE,
users may be able to create 'custom' pages that draw together information about multiple
indicators so that they may be able to make their own comparisons of specific trends.
Level 5 - Limitations: At this level, the ROE would provide a detailed accounting of the
technical limitations associated with the indicator or interest. The ROE would also provide
necessary warnings associated with how one might instinctively view the indicator or trend; e.g.,
that correlation does not necessarily imply causation.
Level 6 - Data Sources and References: As the title suggests, this level of the ROE would
provide an accounting of the data sources and references used to establish the indicator or trend.
This level of the ROE might also subsume information that is currently located in the text boxes
entitled For More Information.
Level 7 - Documentation: At this level, the most scientifically proficient users of the ROE would
have the option to view technical documentation, which is specific to the indicator, and which
might appear elsewhere on the EPA's website, or on external pages.
The SAB acknowledges that some of these levels may be combined in order to create a more
streamlined website experience for the user. However, we caution against mixing information geared
toward users of the ROE who might come with different levels of scientific literacy. As structured here,
the deeper one reads into the various levels, the level of technical detail increases.
For this reason, the SAB also suggests that the use of a visual aid, for example the color-coding
illustrated below, would convey to ROE users the level of detail and technical sophistication they would
encounter at different levels. In the example below, lighter colored tabs would be associated with
information most accessible to generalist users and darker tabs would provide increasingly detailed
information requiring a higher degree of scientific literacy (Figure 3).
32
-------
Introduction to the Indicator
What the Data Show
What the Data Mean
Links to Other Indicators (Sustainability Implications?)
Limitations
Data Sources and References
Documentation
Figure 3. Illustration of color-coding for the tiered presentation of each indicator as a guide to users about
the levels of technical sophistication at each level.
Finally, the SAB notes that some of these changes and reconfigurations will exceed the resources
currently available. However, the SAB provides these suggestions as a roadmap toward a future,
enhanced ROE.
Recommendation
• The discussion of each indicator should be restructured to begin with a straightforward description of
the indicator (e.g., accessed from the top level links in the recommended highlights section on the
ROE home page), followed by increasing technical and scientific detail as users click down through
the various links in whatever track they have selected. Relevant relationships among multiple
indicators under individual themes/questions should also be noted and, where appropriate, a succinct
summary of the findings and implications across those indicators should be presented at the
respective theme/question level.
3.6. Potential Indicators for Future Updates of the ROE
Question 6. Please provide suggestions concerning existing or potential future indicators so as to
more fully address the questions of interest to the Agency outlined in the ROE. Please provide
specific recommendations concerning approaches to an integrated understanding of the status and
trends for environmental and human health conditions related to the mission of the Agency.
The SAB notes that the EPA has a framework in place for the continued assessment and consideration of
indicators, and finds the Agency's use of this assessment framework to be appropriate. This framework
will be of value in the future as the Agency works to keep the ROE current and useful. To this end, the
SAB reaffirms that it is more important to strive to have indicators that are useful, effective, and
informative, rather than just adding new indicators. Therefore, the Agency should give priority to efforts
that make the current information more useful (through development/discussion of interrelationships, for
example, see below), or replace less effective indicators with others.
While the ROE contains many useful indicators, the SAB finds that there are gaps where additional
indicators may be of value. In particular, a number of possible indicators for sustainability have been
discussed in some detail in response to charge questions 1 and 2 (sections 3.1 and 3.2). Given the use of
sustainability as an overarching concept, first priority for the Agency should be to assemble indicators
that help to further strengthen and integrate this concept throughout ROE. This approach is likely to
require the modification and enhancement of the sustainability tab.
33
-------
The SAB also recognizes that opportunities exist for a greater attention to relationships, associations and
comparisons between and among the current group of indicators. The SAB notes that there are "threads"
of commonality that run through the current indicators and, as discussed in section 3.5 above, the
Agency is encouraged to explore these associations further. Accordingly, the Agency also should give
priority to examining its current indicators, consider clusters and associations, and develop contrasts and
comparisons, where appropriate.
The SAB noted that economic and social science indicators were almost entirely absent, and
recommends that an indicator or indicators be considered that address human dimensions. Possible
topics discussed by the SAB included information on recreational and commercial fisheries, outdoor
recreation, restrictions/bans on water use or shellfish harvests, and beach water quality/swimming
advisories. Another potential area would be to increase coverage concerning drinking water supplies,
and also the state of water and wastewater infrastructure. An important indicator of sustainability of the
health of delivered water is the condition of the infrastructure. The EPA receives data on infrastructure
needs (in dollars) and conditions (see report, clean Water and Drinking Water Infrastructure Gap
Analysis (http://nepis.epa.gov/Adobe/PDF/901R0200.pdf) that could be developed into a useable and
useful indicator. The gap analysis report is from 2002 and it is not known if there is a more recent
version. Between the needs survey and the gap analysis, there is data to develop trends - from 1980s
through the present. The Agency collects information on replacement of lead service lines from each
state. This information also could be included in an infrastructure indicator; e.g., how successful is the
EPA in encouraging systems to replace lead service lines, and is this replacement having an effect on
lead levels in drinking water? The Agency might also consider data from (or providing links to) other
sources, such as the report card on the state of U.S. infrastructure provided by the American Society of
Civil Engineers (http://vvvvvv.infrastructurereportcard.oru/a/#p/home). These topics are provided as
suggestions; the Agency will be able to build priorities among them as appropriate. In general, however,
the SAB considers these to be medium- to long-term priorities.
Recommendations
• The SAB recommends that the Agency continue the use of the indicator assessment framework, as it
considers new indicators and the possible retirement of existing indicators.
• The SAB recommends that the Agency give highest priority to identifying and implementing a series
of indicators that will best support sustainability (including relevant economic and social indicators)
as an overarching framework. Many specific suggestions were provided in earlier sections. In
addition, the Agency has the opportunity to examine where comparisons or associations between and
among existing indicators may provide additional insight. Where appropriate, priority should be
given to identifying such comparisons.
34
-------
REFERENCES
Berkes, F. 1999. Sacred Ecology: Traditional Ecological Development and Resource Development.
Taylor & Francis, Philadelphia, Pennsylvania. 217 pp.
Boyd, J. 2006. The Nonmarket Benefits of Nature: What Should Be Counted in Green GDP? Resources
for the Future, RFF DP 06-24, 17 pp. Available at http://www.rff.org/Documents/RFF-DP-06-
24.pdf.
Fiksel, J. 2012. A systems view of sustainability: The triple value model. Environmental Development
2:138-141.
Hawley, A.W.L., E.E. Sherry and C.J. Johnson. 2004. A biologists' perspective on amalgamating
traditional environmental knowledge and resource management. BC Journal of Ecosystems and
Management 5:36-50.
IPCC. 2013. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the
Fifth Assessment Report of the Intergovernmental Panel on Climate Change [Stocker, T.F., D.
Qin, G.-K. Plattner, M. Tignor, S.K. Allen, J. Boschung, A. Nauels, Y. Xia, V. Bex and P.M.
Midgley (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY,
USA, 1535 pp.
Johnson, M. 1992. Research on Traditional Environmental Knowledge: Its Development and its Role.
In: LORE, Capturing Traditional Environmental Knowledge, M. Johnson (Editor). Dene Cultural
Institute and International Development Research Centre (IDRC), Ottawa, Ontario, pp. 3-21.
Kates, R.W. and T.M. Parris. 2003. Long-term trends and a sustainability transition. Proceedings of the
National Academy of Sciences 100(14): 8062-8067.
Markard, J., R. Raven and B. Truffer. 2012. Sustainability transitions: An emerging field of research and
its prospects. Research Policy 41: 955- 967.
National Research Council. 2011. Sustainability and the U.S. EPA. Washington, DC: The National
Academies Press.
Rees, W.E. 2000. Eco-footprint analysis: merits and brickbats. Ecological Economics 32: 371-374.
Suter II, G.W. and S.M. Cormier. 2011. Why and how to combine evidence in environmental
assessments: Weighing evidence and building cases. Science of the Total Environment 409:
1406-1417.
U.S. EPA SAB (U.S. Environmental Protection Agency Science Advisory Board). 2004. Advisory on
EPA's Draft Report on the Environment 2003. EPA-SAB-05-04, December 7, 2004. Available at
www, epa. gov/ sab.
U.S. EPA SAB (U.S. Environmental Protection Agency Science Advisory Board). 2008. SAB Advisory
on EPA's Draft Report on the Environment 2007: Science Report. EPA-SAB-08-007, April 18,
2008. Available at www.epa.gov/sab.
35
-------
U.S. EPA SAB (U.S. Environmental Protection Agency Science Advisory Board). 2015. Strategic
Research Planning for 2016-2019: A Joint Report of the Science Advisory Board and Board of
Scientific Counselors. EPA-SAB-15-004, January 27, 2015. Available atwww.epa.gov/sab.
36
-------
APPENDIX A. EPA's Charge to the SAB for Review of the Report on the
Environment (2014)
INTRODUCTION
In 2001, EPA began work on an Agency-wide "environmental indicators initiative" to gather and
develop high quality information into a state of the environment report, thereby creating the Agency's
Report on the Environment (ROE) program. The purpose of the initiative was to compile and maintain
indicators on the status and trends for environmental and human health conditions related to the mission
of the Agency.
The Agency released a draft report in 2003 and in response to comments from the Science Advisory
Board (SAB)3 and others, EPA developed a revised draft in 2007. Following an extensive external peer
review of indicators and an SAB review of the entire draft,4 the Agency published "EPA's Report on the
Environment" in 20085 in hard copy and an online format.
Since 2008, EPA has maintained and updated the ROE indicator information online. EPA again
consulted the SAB in 2009 to obtain additional advice on continued development of future versions of
the ROE6. Panel members offered several suggestions to improve the ROE, including the need to:
• Develop an overarching and unifying framework to link and integrate the scientific elements of
the ROE
• Incorporate statistical information into future reports
• Develop a powerful and interactive web-based platform for the ROE with links to pertinent
websites for additional information
• Develop a clear mission statement for the ROE, originating from EPA leadership, in order to
define the objectives and intended audiences of the report
In response to these suggestions from the SAB, and additional suggestions from the users of the ROE,
EPA developed the draft ROE 2014 (www.epa. gov/draftroe). The 2014 ROE site represents the
transition to a completely online environment with improved navigation, interactive graphing and
mapping, and access to the data, where possible.
The draft ROE 2014 indicators address questions of critical importance to EPA's mission of protecting
human health and the environment. The ROE indicators are numerical values derived from actual
measurements of a driver, stressor, state or ambient condition, exposure, or human health or ecological
condition over a specified geographic area, whose changes over time represent or draw attention to
underlying trends in the condition of the environment. The indicators are based on data collected by the
EPA, other federal and state agencies, and non-governmental organizations and meet high standards for
data quality, obj ectivity, and utility.
The ROE 2014 does not, however, analyze or diagnose the reasons for, and the relationships between,
trends in stressors and environmental and health outcomes.
3 SAB Advisory on EPA's Draft Report on the Environment 2003
4 SAB Advisory on EPA's Draft Report on the Environment 2007: Science Report
5 EPA's Report on the Environment 2008
6 Consultation on Development of EPA's Report on the Environment (2009)
A-l
-------
In response to evolving Agency priorities, EPA has expanded the scope of the ROE with a new theme
on sustainability to complement the existing themes on air, water, land, human exposure and health, and
ecological condition. EPA has also adapted a sustainability-based conceptual framework for the ROE in
response to comments from previous SAB meetings.
EPA's Office of Research and Development (ORD) requests that the SAB review and comment on the
draft ROE website with particular attention to the new features added since the release of the 2008
report. Specifically, the Agency requests that the SAB comment on: the clarity of the ROE's objectives
for EPA and other governmental scientists and policy-makers, educators, and members of the public; the
overarching conceptual framework, based on a sustainability theme that integrates issue-specific
elements; the addition of statistical information; and the presentation of these features in an online
format. Specific charge questions are provided in the following section.
CHARGE QUESTIONS
Charge Question 1. Sustainability as the ROE 2014 Conceptual Framework
One of the SAB's key recommendations from its 2007 and 2009 reviews was that EPA should develop a
framework describing the scientific understanding of relationships between the ROE indicators and the
basis for including them in the report. The ROE conceptual framework and associated diagrams were
created in response to these recommendations.
EPA adapted a sustainability conceptual framework (http://cfpub.epa.gov/roe/frameworks.cfm) as an
overarching structure for the ROE to show how indicators relate to specific human health or
environmental outcomes or conditions that are consistent with the Agency's mission and goals.
Sustainability is based on the principle that everything that humans need for survival and well-being
depends, either directly or indirectly, on our natural environment. In recent years, EPA has advanced the
concepts of sustainability and sustainable development, and ORD has realigned its research programs
with a systems thinking approach to develop sustainable solutions to increasingly complex, global, and
interconnected environmental problems. Sustainability—commonly defined as the ability to maintain or
improve standards of living without damaging or depleting natural resources for present and future
generations—offers a framework for addressing such challenges.7
With the sustainability framework as the foundation, EPA developed six examples of condition or
outcome-specific diagrams to show how ROE indicators relate to:
• Tropospheric ozone
• Acid deposition
• Nutrient impacts
• Coastal hypoxia
• Wetland loss
• Fish mercury contamination
7 U.S. EPA Office of Research and Development. 2012. Science for a sustainable future: EPA research program overview
2012-2016. June. EPA 601/R-12/002. http://www.epa.gov/ordntrnt/QRD/docs/strap-overview2012.pdf
A-2
-------
1. Please comment on the concept of sustainability as an overarching conceptual framework for
representing the relationships between indicators. Please also comment on the clarity by which the
framework is depicted and discussed in the draft ROE and provide any recommendations to
improve its description and intended purpose of representing the relationship between indicators?
Charge Question 2. Sustainability Indicators
Within the Sustainability theme, EPA poses a new question, written in a style consistent with the other
ROE questions: What are the trends in consumption of natural resources'? This question focuses on
trends of the intensity of natural resource consumption in the U.S. to track progress in reducing
environmental pressures due to depletion of these resources.
The ROE presents four indicators to help answer the new resource efficiency question:
• Energy Use (http://cfpub.epa.gov/roe/indicator.cfm?i=93)
• Freshwater Withdrawals (http://cfpub.epa.gov/roe/indicator.cfm?i=94)
• Quantity of Municipal Solid Waste (MSW) Generated and Managed
(http://cfpub. epa.gov/roe/indicator. cfm?i=53)
• Quantity of RCRA Hazardous Waste Generated and Managed
(http://cfpub. epa.gov/roe/indicator. cfm?i=54)
Each of these indicators includes an intensity metric that shows trends over time with respect to the U.S.
population and the U.S. real (inflation-adjusted) gross domestic product (GDP). For example, the energy
indicator shows trends in energy use per capita and energy use per dollar of real GDP. The municipal
solid waste (MSW) and RCRA hazardous waste indicators are presented as surrogates for material use.
All four indicators are cross-referenced in other ROE themes where appropriate; for example, the MSW
and RCRA hazardous waste indicators are also used to help answer the ROE question on wastes within
the Land theme.
2(a). Please comment the on the adequacy by which sustainability has been incorporated into the
ROE. More specifically, please comment on the descriptions and explanations for the sustainability
theme, question, and the four associated indicators.
2(b). Please address the utility of the four new sustainability indicators for informing the reader on
the intensity of resource consumption and the relevance of these intensity metrics.
2(c). EPA is anticipating expanding the sustainability theme with additional questions and indicators
in future ROEs. Please provide any specific recommendations on additional sustainability topics,
indicators, and extant data sources that are important to pursue. Please provide your rationale for
prioritizing additional topics and indicators.
Charge Question 3. Statistical Information
The 2007 SAB review pointed out that the ROE was limited in its usefulness because it contained no
data interpretation and no conclusions supported by statistical analysis. The SAB cited the need for a
statistical approach to analyze and present the ROE indicator data to enhance the rigor to the report.
The SAB suggested that the results presentation for each indicator:
A-3
-------
• Include formal statistical analyses and/or additional information, such as error bars around
mean values.
• Report statistical limitations when insufficient data are available for robust quantitative
analyses.
EPA determined not to conduct de novo statistical analysis at this time, but instead to include extant
statistical information, typically analysis performed by the primary researchers or scientists that
collected and provided the data. Reflective of the current scope of the ROE, EPA did not provide an
interpretation of indicator trends for the draft ROE 2014.
The draft ROE 2014 has 21 indicators with error bars and four have trend data incorporated into the
display of the indicator graphics. This web enhancement provides additional information on the
uncertainty and variability of the indicators for the reader. Detailed descriptions of the available
statistical information can be found in the technical documentation for the individual indicators.
Additionally, each ROE exhibit has a footnote clarifying whether any trend testing was done, and if so,
whether the analysis was statistically significant or not.
3. Please comment on the approach used to incorporate statistical information into the 2014
ROE. Please provide any recommendations to enhance the presentation, including the clarity in
describing uncertainty.
Charge Question 4. ROE 2014 Web-based Product
The SAB noted in its 2009 report that "[i]n developing the ROE, EPA should emphasize producing a
powerful and interactive web-based platform with links to pertinent websites containing additional data
and information." The 2014 ROE is an exclusively online presence. Consistent with the SAB
recommendation, the goal of this transition to a web-based platform was to make the ROE indicators
and graphics easily accessible, while retaining the ROE theme/question structure. In the future this
format will facilitate providing users with the most current, real-time information possible, which is not
feasible with a print format.
EPA is seeking feedback on the enhancements made to the website, which include interactive mapping
and graphing, streamlined text, and links to related information outside the ROE. Specifically
enhancements to the site include:
• Interactive mapping and graphing. Tools allow users to customize graphical displays and view
certain indicators in zoomable map form.
• Streamlined text. The language has been edited to be more plain and direct (to the extent possible
in a document of a technical nature). Web pages are scannable; in many cases the web pages
display topics in an "accordion" format, allowing the user to click on that topic to reveal more
detail. Other pages provide main text and additional details available by clicking on "Learn
more" options. These changes are designed to maximize the usefulness of the website for a wide
range of audiences. The indicator text, which comprises most of the ROE text, was previously
peer-reviewed and therefore has not been changed.
• Numerous links have been added throughout the ROE.
A-4
-------
> EPA has created two new sections in the ROE, "Where You Live" and "What You Can
Do." that provide links to relevant EPA Web pages. These links were identified with
input from the ROE Indicators Work Group and others across the Agency.
> "Related Links" are provided at the bottom of all six theme areas and 24 question pages
that link to relevant EPA Web pages and, as appropriate, other federal websites.
> "For More Information" links, including links to data providers, are provided at the
bottom of all indicator pages.
4(a). Please comment on the scientific rigor and clarity of the ROE content with the transition
from a printed document to an online presence.
4(b). Please provide suggestions on other factors that could be considered concerning the overall
content, format, credibility, user friendliness and navigability of the site.
Charge Question 5. Communication
The audience for the ROE is very diverse. It includes EPA scientists and policy-makers, as well as
scientists and policy-makers in other Federal, State, Tribal and local governmental agencies. A broad
array of stakeholders, and the general public will be interested in the ROE indicators. The ROE may
also be useful to educators and students in a myriad of community settings.
Given the diverse nature of the ROE audience, EPA presented information in the ROE in multiple ways
and in increasing levels of detail, i.e., graphical presentations, explanatory narrative, corresponding links
to literature citations, and, where possible, information on statistical uncertainty for indicators. The
ROE also provides links and citations to underlying datasets so interested readers can get additional
information for specific indicators of interest.
5(a). Please provide feedback on the approaches used in the ROE to provide information such
that audiences with varying interests can efficiently and reasonably find information concerning
the status and trends of environmental conditions. Please provide any recommendations to
enhance the access to information.
5(b). Please comment on the accuracy of the ROE's presentations and their effectiveness in
communicating complex scientific information to a broad range of technical and non-technical
audiences. Please provide recommendations for specific components of the ROE as appropriate.
Charge Question 6. Additional Indicator Recommendations
Although the focus of this SAB review is on enhancements made to the ROE since the 2009 SAB
review, the SAB may have further advice on how well existing indicators address the specific questions
of interest to the Agency as outlined in the ROE. The SAB may also have suggestions concerning
additional indicators that might be considered to more fully address the questions outlined in the ROE.
In making recommendations, the SAB should consider information provided in the ROE concerning the
definition and selection of indicators, and the exclusion of purely administrative indicators.
6. Please provide suggestions concerning existing or potential future indicators so as to more
fully address the questions of interest to the Agency outlined in the ROE. Please provide
specific recommendations concerning approaches to an integrated understanding of the status
and trends for environmental and human health conditions related to the mission of the Agency.
A-5
-------
APPENDIX B. Draft ROE 2014 Questions and Indicators
Theme
Questions
Indicators
Air
• What are the trends in outdoor air quality and
their effects on human health and the
environment?
•Acid Deposition
•Acidity in Lakes and Streams
•Air Quality Index: Days Above 100
•Air Toxics Concentrations
•Air Toxics Emissions
•Carbon Monoxide Concentrations
•Carbon Monoxide Emissions
•Energy Use
•Lead Concentrations
•Lead Emissions
•Manganese Concentrations in
Region 5
•Mercury Emissions
•Nitrogen Dioxide Concentrations
•Nitrogen Oxides Emissions
•Ozone Concentrations
•Ozone Injury to Forest Plants
•Ozone and Particulate Matter
Concentrations Along
U.S./Mexico Border
•Ozone-Depleting Substances
Concentrations
•Particulate Matter Concentrations
•Particulate Matter Emissions
•Regional Haze
•Stratospheric Ozone Levels
•Sulfur Dioxide Concentrations
•Sulfur Dioxide Emissions
•Volatile Organic Compounds
Emissions
• What are the trends in greenhouse gas
emissions and concentrations and their impacts
on human health and the environment?
•Energy Use
•Greenhouse Gas Concentrations
•Sea Level
•Sea Surface Temperature
•Temperature and Precipitation
•U.S. Greenhouse Gas Emissions
• What are the trends in indoor air quality and their
effects on human health?
•Radon: Homes At or Above EPA's
Action Level
•Serum Cotinine
Water
• What are the trends in the extent and condition of
fresh surface waters and their effects on human
health and the environment?
•Acidity in Lakes and Streams
•Benthic Macroinvertebrates in
Wadeable Streams
•Freshwater Withdrawals
•Nitrogen and Phosphorus in
Agricultural Streams
•Nitrogen and Phosphorus in Large
Rivers
•Nitrogen and Phosphorus in
Wadeable Streams
•Pesticides in Agricultural Streams
•Stream Flows
•Streambed Stability
B-l
-------
Theme
Questions
Indicators
• What are the trends in the extent and condition of
ground water and their effects on human health
and the environment?
•Freshwater Withdrawals
•Nitrate and Pesticides in Ground
Water
• What are the trends in the extent and condition of
wetlands and their effects on human health and
the environment?
•Wetlands (Extent)
• What are the trends in the extent and condition of
coastal waters and their effects on human health
and the environment?
•Coastal Benthic Communities
•Coastal Fish Tissue
•Coastal Sediment Quality
•Hypoxia in Gulf of Mexico and Long
Island Sound
•Submerged Aquatic Vegetation in
Chesapeake Bay
•Trophic State of Coastal Waters
•Wetlands
• What are the trends in the quality of drinking
water and their effects on human health?
•Drinking Water (Quality)
• What are the trends in the condition of
recreational waters and their effects on human
health and the environment?
• What are the trends in the condition of
consumable fish and shellfish and their effects on
human health?
•Coastal Fish Tissue
•Lake Fish Tissue
Land
• What are the trends in land cover and their
effects on human health and the environment?
•Forest Extent and Type
•Land Cover
• What are the trends in land use and their effects
on human health and the environment?
•Land Use
•Urbanization and Population Change
• What are the trends in chemicals used on land
and their effects on human health and the
environment?
•Agricultural Fertilizer
•Pesticide Incidents
•Pesticide Residues in Food
•Toxic Chemicals in Wastes
• What are the trends in wastes and their effects
on human health and the environment?
•Hazardous Waste
•Municipal Solid Waste
• What are the trends in contaminated land and
their effects on human health and the
environment?
•Contaminated Ground Water
Migration at Cleanup Sites
•Human Exposures at Cleanup Sites
Human
Exposure
and Health
• What are the trends in human exposure to
environmental contaminants?
•Blood Cadmium
•Blood Lead
•Blood Mercury
•Serum Cotinine
•Serum Persistent Organic Pollutants
•Urinary Pesticides
•Urinary Phthalates
• What are the trends in health status in the United
States?
•General Mortality
•Infant Mortality
•Life Expectancy
B-2
-------
Theme
Questions
Indicators
• What are the trends in human disease and
conditions for which environmental contaminants
may be a risk factor?
•Asthma
•Birth Defects
•Cancer
•Cardiovascular Disease
•Childhood Cancer
•Chronic Obstructive Pulmonary
Disease
•Infectious Diseases
•Low Birthweight
•Preterm Delivery
Ecological
Condition
• What are the trends in the extent and distribution
of the nation's ecological systems?
•Ecological Connectivity
•Forest Extent and Type
•Forest Fragmentation
•Land Cover
•Land Use
•Urbanization and Population Change
•Wetlands
• What are the trends in the diversity and biological
balance of the nation's ecological systems?
•Benthic Macroinvertebrates in
Wadeable Streams
•Bird Populations
•Coastal Benthic Communities
•Fish Faunal Intactness
•Non-Indigenous Estuarine Species in
Pacific Northwest
•Submerged Aquatic Vegetation in
Chesapeake Bay
• What are the trends in the ecological processes
that sustain the nation's ecological systems?
•Carbon Storage in Forests
• What are the trends in the critical physical and
chemical attributes of the nation's ecological
systems?
•Acidity in Lakes and Streams
•Hypoxia in Gulf of Mexico and Long
Island Sound
•Nitrogen and Phosphorus in
Agricultural Streams
•Nitrogen and Phosphorus in Large
Rivers
•Nitrogen and Phosphorus in
Wadeable Streams
•Sea Level
•Sea Surface Temperature
•Stream Flows
•Streambed Stability
•Temperature and Precipitation
• What are the trends in biomarkers of exposure to
common environmental contaminants in plants
and animals?
•Coastal Fish Tissue
•Lake Fish Tissue
•Ozone Injury to Forest Plants
Sustainability
• What are the trends in consumption of natural
resources?
•Energy Use
•Freshwater Withdrawals
•Hazardous Waste
•Municipal Solid Waste
B-3
------- |