UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
May 12, 2000
EPA SAB-EC-LTR-00-004
Ms. Carol Browner
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: Review of the SAB Report "Toward Integrated Environmental Decision-
Making"
Dear Ms. Browner:
I am pleased to send you this letter, which represents the final peer review of the
document prepared by the Science Advisory Board's Integrated Risk Project, "Toward Integrated
Environmental Decision-Making". The Integrated Risk Project has been an enormous
undertaking, involving many participants who have worked long and hard to address a problem
that lies at the heart of EPA's mission: how to adopt a more comprehensive and integrated
approach to identifying, assessing and managing environmental risks. While it has not proved
feasible to offer detailed peer-reviewed procedural guidance to the Agency, the final report does
an excellent job of identifying the problem and proposing a broad philosophical approach which
should lead the Agency toward more integrated environmental decision making in the years to
come.
In May of last year, the Integrated Risk Project produced a two volume draft report. Our
Subcommittee voiced many concerns with these draft documents (Review of the "Integrated
Environmental Decision-Making in the Twenty- First Century", EPA SAB-EC-99-018). The
current, much simpler, revised report, "Toward Integrated Environmental Decision-Making," has
successfully addressed most of those concerns. In particular, our Subcommittee finds the ten
Recommendations to the Agency contained on pages 37-42 to be reasonable, appropriate, and
worthy of sustained Agency consideration and action.
The Integrated Risk Project took the SAB, and the Agency, into unfamiliar territory,
involving research literatures in behavioral decision science and decision theory with which they
have had limited past experience. The effort has emphasized the importance of expanding the
scope of expertise of both SAB membership and Agency staff into these important domains. It
has also emphasized the importance of adopting an interdisciplinary approach which combines
-------
deep understanding of environmental science with theory and empirical methods in behavioral
and decision science. Many of the difficulties with the more specific recommendations
contained in the two volume draft report of last May sprang from a lack of familiarity with the
state of research and the literature in these latter fields.
While "Toward Integrated Environmental Decision-Making" charts a valuable future
direction for the agency, it is important to note that a number of specific paths can lead to the
desired destination. There can be no substitute for a thoughtful strategy of experimentation
worked out in the specific settings of different environmental problems.
If the journey toward more integrated environmental decision making is to be successful,
the Agency will need to undertake a significantly expanded effort in developing improved tools
and guidance that have been vetted with real problems in environmental decision-making. As
the report clearly documents, specific, focused research is needed on problems that range from
improving methods for the informed synthesis and elicitation of public environmental values, to
tools and procedures that support: improved characterization and treatment of uncertainty;
reasoned science-based deliberative processes; and, the development of ordinal and cardinal
evaluations of multi-dimensional risks.
The challenges of improving and better integrating environmental decision-making are
considerable, but as this report clearly articulates, the end result should be worth the effort.
Recommendations
In the paragraphs that follow, we discuss several remaining issues which we believe
deserve modest further attention. These are followed by a list of more minor comments on
typographical and editorial matters.
Page 11, column 2, lines 1-5: Perhaps "reaching agreement" is a bit strong. How about
"seek consensus?" Sometimes in very controversial problems agreement among all stakeholder
is not possible even on problem formulation (in part because parties look ahead and reach
different conclusions about what alternative formulations imply in terms of outcomes). Clearly
as a default you want to leave decision-makers the freedom to listen and be informed by all the
parties and then make a decision (for a discussion of the limits of consensus see: Gary
Coglianese "The Limits of Consensus, Environment, 41, pp. 1-6, April 1999). You might also
want to rework text on page 12, bottom column 1 to place the decision-maker ultimately in
charge. If you make the suggested change, you might also wish to change page 34, lines 31-32
from "the frequency of reporting should be decided upon with all the participants..." to read "the
frequency of reporting should be chosen by the decision maker in consultation with all the
participants..." The last few lines in the box on page 15 adopt the "informed decision maker"
perspective we are suggesting.
On page 12, in the box at the bottom of column 2 you make the goals for the decision-
making exercise a fixed output of Stage 1. This is also the way Figure 6 on page 28 is drawn.
However, it is often the case that as one works through a problem and understands it better, goals
become better informed and change (sometimes even without a change in underlying values).
-------
This possibility needs to be acknowledged. Some feedback into the goals box in Figure 6 would
be a good addition. The change could be as simple as:
_ _ _ ., ,^ New insights from
Define Problem "^ 3
later stages of analysis.
Develop....
Page 15, column 1, lines 3-13 does acknowledge that goals may change. The language "...of the
initial risk rankings or other aspects..." might be changed to read "...of the initial goals, risk
rankings or other aspects..." On this general subject of fixed versus evolving goals, see the
March (1976) quote that we included on the last page of our previous review.
Page 16, column 2, lines 2-4: Because risk is a multi-attribute concept (i.e., depending
on things like equity and level of knowledge as well as expected mortality and morbidity), all
risk ranking is based on values as well as science. The problem of merging health and
environmental risks is just a more extreme instance. Perhaps the sentence might be reworked to
read: "...the members concluded that a merger would involve even larger value judgments than
those involved in the separate rankings of health and environmental impacts and was reluctant to
make such judgments absent some broader societal valuation process." Alternatively, since no
rankings are being reported (with the exception of the box on page 18) is it even necessary to say
this?
In its previous review, the subcommittee had some methodological concerns with the
ecological risk ranking procedure, and major concerns with the internet-based interview tool.
Since the details of these methods are no longer present, many of those concerns are less
pressing, but we would like to see the following language added at the end of the discussion on
page 20, just before Section 3.1.2:
"While the examples developed by the ERS and HEHS provide illustrations of the
type of new tools we believe need to be developed, these specific examples were
created with limited consideration of the literature in modern social and decision
science. As the EPA develops and refines such tools for future use, it will be
important to involve experts from these fields."
Then, on page 39, column 1, lines 30-31: "These approaches should be further explored..."
should read "These and similar approaches should be further developed and explored..."
The discussion of benefit-cost and of deliberative processes on pages 22-24 could use a
bit more discussion of equity (especially page 24). Benefit-cost is a tool to achieve efficiency.
While efficiency is very important, government spends much of its time dealing with issues of
equity. Thus, you might consider adding something on the need for better analytical methods for
assessing the equity implications of decisions. For example, on line 38, after the sentence that
-------
ends "...welfare goals." you might add something like the following: "Benefit-cost analysis is a
tool for assessing the efficiency of decisions. Government is also concerned with the equity
implications of decisions. More work needs to be done to develop assessment tools that address
issues of equity."
On page 22, column 1, line 36 "when benefit-cost information is considered..." might
better read "when benefit-cost and other information is considered..."
The discussion of page 25, column 25 and the box on page 26, implicitly assume that
people have well-articulated values and that the problem is one of measurement. Elsewhere the
report makes it clear that this is often not the case. These sections should be reworked slightly to
include that recognition here as well.
Table 1 on page 30 is still incomplete and the shading does not seem to help organize the
information. It should either be worked out more systematically or dropped. For example, if
tradable emissions permits are to be used it must be possible to:
a) Define and accurately measure the pollutant(s) of concern, their sources (both
natural and anthropogenic) and their atmospheric fate and transport (i.e.,
understand the science);
b) Define the quantity of emissions that the regulated sources will be allowed to emit
(the cap), and which will be available for trading in the market, which can vary
(typically decline) as a function of time;
c) Find an acceptable method to allocate permits to participating parties before
trading is initiated;
d) Create and operate a market with enforceable contracts and rules in which
specified classes of polluters must participate (or face penalties), and which
involve enough participants to assure competitive behavior; and
e) Demonstrate that all pollutants being traded cause similar damage (as in the case
of a uniform well mixed pollutant), or if they do not, devise a weighting system
acceptable to all participating parties that to normalize damages across emission
locations and times, and pollutant types.
Similar elaborations are possible for all the other cells. In addition, the layout (shading of
alternate rows) is somewhat confusing. Finally, the caption might better identify these as
strategy or approaches. "Options" implies the need to choose one or another row, while in fact,
many of these strategies can be combined.
Page 39, column 2, lines 5-11: It is unclear what the "prototype risk reduction
methodology" is. These lines might just be dropped.
-------
Page 22, Section 3.2.2 needs to introduce the notion of metrics that would be applied to
various risks. For instance, the following could be inserted:
"To integrate assessments of risk embedded in complex environmental problems, it is
essential that methodology be developed that presents the relevant science in a way that
the public can understand the nature of particular endpoints and make decisions across
diverse endpoints. Descriptors should be both qualitative and quantitative. In the health
effects area, the range of impairments produced by the disease should be described and it
should be made clear whether the data describes frank disease or a metric that is thought
to be predictive of the disease. Quantitatively, it should characterize the exposure-dose-
dose rate- response relationships that are involved. The valuation of different endpoints
in the environmental area must be done in public forum as these likely must be
collectively made if they are to be widely accepted. Entirely different values are
involved in environmental decisions compared to personal medical decisions."
The phrase in the second to last sentence in the first paragraph of section 3.2.2 suggesting that
the agency attempts to reduce risk to a specified level may not be a full characterization of the
Agency's policy. For example many regulations are not set at the 10"6 level. The Office of
Water sets a range from 10"6 to 10-4. Most of their standards are within this range. However,
there are standards whose calculated risks exceed this level (i.e., disinfectant by-products, radon
and arsenic).
In addition to our comments above, we have also attached an appendix with specific
editorial comments.
Conclusion
The Subcommittee congratulates the Integrated Risk Project Steering Committee and the
SAB staff on a dramatically improved report. It is well-organized and clearly written. The
"what we have"/" what we need" structure is a very useful organizing device. The set often
recommendations are reasonable and appropriate.
We thank the Agency for the opportunity to review the "Toward Integrated
Environmental Decision-Making" report by the Science Advisory Board. We think it offers
promised to advance the decision-making process at EPA.
Sincerely,
Prof. M. Granger Morgan, Chair
Integrated Risk Project Review Subcommittee
-------
APPENDIX A. SPECIFIC EDITORIAL COMMENTS
In the early part of the report, the format "benefit/cost" is used. Later "benefit-cost" is used.
While this is a matter of taste, there are good reasons to use "benefit-cost" (see for example EJ.
Mishan, Economics for Social Decisions: Elements of cost-benefit analysis, George Allen and
Unwin, 1972 for a discussion of when to use B-C and when to use B/C).
Page 2, column 2, line 20: "...as budgets continue to decline." Not all budgets are declining and
there is no reason to assume that all will in the future. You might consider saying "...in the face
of fiscal constraints." or "...in the face of multiple demands on limited budgets."
In the box on page 3, why include the example of a focus on "a particular industrial sector" if the
point of the new approach is to look in an integrated way at risks independent of their source?
p. 4 column 1 line 30ff. The sentence is awkward. ".. .endpoints (e.g ), and the elements
of exposure assessment, two essential components of risk assessment.
Page 5, column 1, lines 24-28: the English is a bit awkward.
Page 7, column 1, line 6: "...costs and benefits (either physical or monetary)..." might better read
"...costs and benefits (both tangible and intangible)..." or "...costs and benefits (both monetary
and non-monetary)..."
Page 11, column 1, lines 7-9: "...improved techniques for forming, eliciting and considering
public values, for communicating..." might better read "...improved techniques for helping
people develop considered values, for eliciting and using public values in decision making, for
communicating..." A similar problem exists on page 40, column 1, line 37. The issue is the
distinction between helping people construct their own values, as opposed to forming people's
values.
p. 12. Box in column 2. Item 2 should add ".. ..and the reasons for these decisions;
Page 13, column 2, lines 18-21: "Developments in the social sciences, for example, are
providing improved methods for multi-attribute decision making" might better read
"Developments in the social and decision sciences, for example, are providing improved
methods for value elicitation and multi-attribute decision making."
Page 22, column 2, line 23: "...can make themselves best off..." might better read "...can make
themselves better off..."
p. 23 column 1, line 9. what is meant by ".. .physical endowment available."
A- 1
-------
Page 23, column 2, line 42: "...cost across different groups..." might better read "...cost across
different individuals and groups..."
Page 23, column 2, line 17: "...derived from the vector of all..." should be less technical.
Page 26: There is missing text at the bottom of the boxed section.
Page 28, Figure 6: Do you really want to say "optimize options"? Might not "refine options" be
better? Real policy processes rarely are able to optimize anything (see writing by Simon,
Kingdon and others).
Page 29, Section 3.5.2: You might make some reference to the fact that some risk management
strategies are dictated by existing legislative mandates.
Page 29, column 2, line 26: The meaning of "...'root cause' or 'common sources/common
pathway'" is not clear.
Page 31, column 1, lines 40-41: "...have a common measure of risk or a common denominator
of all risks..." The report might place greater emphasis on the difficulty of achieving this. Risk
ranking typically produces an ordinal measure. The current language is calling for a cardinal
measure. You might at the end of the section (page 31, column 2, line 3) something like:
"While models can be developed to weight and combine all the different attributes in disparate
types of risk, getting wide-spread social buy-in for such weighting poses enormous problems."
Page 31, column 2, line 5: "Uncertainty of the analysis is likely to..." is awkward phrasing.
Page 31, column 2, lines 14-16: It may be easier to implement communication than regulation
when uncertainty is high but it is not clear that it is easier to get good results. Communication
methods work best when uncertainty is low. You might just drop the examples.
Page 7, Figure 7: The arrows might be dropped, at least the left one. Or you could use a graphic
such as:
to indicate a spectrum.
Page 34, column 2, lines 1-20: The language is awkward.
p. 34, column 2, line 9 of the second paragraph. Should it not be ".. .may not exist, may be
subtle and difficult to measure, or may be observable "
Page 35, column 1, line 4: "Figure 9" should read "Figure 8"
A-2
-------
Page 35, column 2, lines 28-36: A considerable amount of work of the sort proposed on
evaluating marketable permits has been done. If you need references, contact Dr. Alex Farrell
(afarrell® andrew.cmu.edu).
Page 36, Figure 8 is really not a Figure but a box like the earlier boxes in the report.
Box on p. 36. Should it be recognized that one needs to use several parameters to characterize
effectiveness of intervention. Not all remedial actions are equally effective and in certain areas
where failure is more highly feared (i.e. in drinking water treatment to prevent waterborne
infectious disease) multiple barriers are routinely used or at least advocated. It is important to
have measures of outcome, stressors, and process to judge what has been the most effective
when the overall effectiveness is evaluated. To assume that all the improvement that is seen is
all attributable to institution of a remedial process is analogous to ignoring in the placebo effect
in clinical trials.
Page 40, column 1, line 33: "...behavioral science and decision logic..." the more conventional
language in the field is "...behavioral and decision science..."
Page 40, column 2, line 4: "integrated environmental" should read "integrated environmental"
Page 40, column 2, line 30: "...manage and use that data..." should read "...manage and use those
data..."
Page 44, column 1, line 15: "c) Need to compare..." should read "c) The need to compare..."
A-3
-------
NOTICE
This report has been written as part of the activities of the Science Advisory Board, a
public advisory group providing extramural scientific information and advice to the
Administrator and other officials of the Environmental Protection Agency. The Board is
structured to provide balanced, expert assessment of scientific matters related to problems facing
the Agency. This report has not been reviewed for approval by the Agency and, hence, the
contents of this report do not necessarily represent the views and policies of the Environmental
Protection Agency, nor of other agencies in the Executive Branch of the Federal government, nor
does mention of trade names or commercial products constitute a recommendation for use.
Distribution and Availability: This Science Advisory Board report is provided to the EPA
Administrator, senior Agency management, appropriate program staff, interested members of the
public, and is posted on the SAB website (www.epa.gov/sab). Information on its availability is
also provided in the SAB's monthly newsletter {Happenings at the Science Advisory Board).
Additional copies and further information are available from the SAB Staff.
-------
US ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
INTEGRATED RISK PROJECT
PEER REVIEW SUBCOMMITTEE OF THE
EXECUTIVE COMMITTEE
CHAIR
Dr. M. Granger Morgan, Head, Department of Engineering & Public Policy, Carnegie Mellon
University, Pittsburgh, PA
MEMBERS AND CONSULTANTS
Dr. Henry A. Anderson, Chief Medical Officer, Wisconsin Bureau of Public Health, Madison,
WI
Dr. William E. Bishop, Associate Director, Global Technical Policy, The Procter & Gamble
Company, Cincinnati, OH
Dr. Donald F. Boesch, President, University of Maryland Center for Environmental Science,
Cambridge, MD
Dr. Richard J. Bull, Senior Staff Scientist, Battelle Pacific Northwest Division, Molecular
Biosciences, Richland, WA
Dr. Terry Davies, Director, Center for Risk Management, Resources for the Future,
Washington, DC
Dr. John D. Graham, Professor of Policy and Decision Sciences and Director of Harvard
Center for Risk Analysis, Harvard University, Boston, MA
Dr. Catherine Kling, Professor, Department of Economics, Iowa State University, Ames, IA
Dr. Debra Knopman, Director, Center for Innovation and the Environment, Progressive Policy
Institute, Washington, DC
Dr. Morton Lippmann, Professor, Nelson Institute of Environmental Medicine, New York
University School of Medicine, Tuxedo, NY
Dr. Warner North, President and Principal Scientist, North Works Inc., Belmont, CA
Dr. Richard Revesz, Professor of Law, New York University School of Law, New York, NY
Dr. Bruce Tonn, Senior Researcher, Oak Ridge National Laboratory, Oak Ridge, TN
11
-------
SCIENCE ADVISORY BOARD STAFF
Dr. John R. Fowle III, Deputy Staff Director/Designated Federal Officer, Science Advisory
Board (1400A), US EPA, 1200 Pennsylvania Avenue, NW, Washington, DC 20460
Ms. Wanda R. Fields, Management Assistant, Science Advisory Board, USEPA (1400A), US
EPA, 1200 Pennsylvania Avenue, NW, Washington, DC 20460
in
------- |