UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON D.C. 20460 OFFICE OF THE ADMINISTRATOR SCIENCE ADVISORY BOARD May 12, 2000 EPA SAB-EC-LTR-00-004 Ms. Carol Browner Administrator U.S. Environmental Protection Agency 1200 Pennsylvania Avenue, NW Washington, DC 20460 Subject: Review of the SAB Report "Toward Integrated Environmental Decision- Making" Dear Ms. Browner: I am pleased to send you this letter, which represents the final peer review of the document prepared by the Science Advisory Board's Integrated Risk Project, "Toward Integrated Environmental Decision-Making". The Integrated Risk Project has been an enormous undertaking, involving many participants who have worked long and hard to address a problem that lies at the heart of EPA's mission: how to adopt a more comprehensive and integrated approach to identifying, assessing and managing environmental risks. While it has not proved feasible to offer detailed peer-reviewed procedural guidance to the Agency, the final report does an excellent job of identifying the problem and proposing a broad philosophical approach which should lead the Agency toward more integrated environmental decision making in the years to come. In May of last year, the Integrated Risk Project produced a two volume draft report. Our Subcommittee voiced many concerns with these draft documents (Review of the "Integrated Environmental Decision-Making in the Twenty- First Century", EPA SAB-EC-99-018). The current, much simpler, revised report, "Toward Integrated Environmental Decision-Making," has successfully addressed most of those concerns. In particular, our Subcommittee finds the ten Recommendations to the Agency contained on pages 37-42 to be reasonable, appropriate, and worthy of sustained Agency consideration and action. The Integrated Risk Project took the SAB, and the Agency, into unfamiliar territory, involving research literatures in behavioral decision science and decision theory with which they have had limited past experience. The effort has emphasized the importance of expanding the scope of expertise of both SAB membership and Agency staff into these important domains. It has also emphasized the importance of adopting an interdisciplinary approach which combines ------- deep understanding of environmental science with theory and empirical methods in behavioral and decision science. Many of the difficulties with the more specific recommendations contained in the two volume draft report of last May sprang from a lack of familiarity with the state of research and the literature in these latter fields. While "Toward Integrated Environmental Decision-Making" charts a valuable future direction for the agency, it is important to note that a number of specific paths can lead to the desired destination. There can be no substitute for a thoughtful strategy of experimentation worked out in the specific settings of different environmental problems. If the journey toward more integrated environmental decision making is to be successful, the Agency will need to undertake a significantly expanded effort in developing improved tools and guidance that have been vetted with real problems in environmental decision-making. As the report clearly documents, specific, focused research is needed on problems that range from improving methods for the informed synthesis and elicitation of public environmental values, to tools and procedures that support: improved characterization and treatment of uncertainty; reasoned science-based deliberative processes; and, the development of ordinal and cardinal evaluations of multi-dimensional risks. The challenges of improving and better integrating environmental decision-making are considerable, but as this report clearly articulates, the end result should be worth the effort. Recommendations In the paragraphs that follow, we discuss several remaining issues which we believe deserve modest further attention. These are followed by a list of more minor comments on typographical and editorial matters. Page 11, column 2, lines 1-5: Perhaps "reaching agreement" is a bit strong. How about "seek consensus?" Sometimes in very controversial problems agreement among all stakeholder is not possible even on problem formulation (in part because parties look ahead and reach different conclusions about what alternative formulations imply in terms of outcomes). Clearly as a default you want to leave decision-makers the freedom to listen and be informed by all the parties and then make a decision (for a discussion of the limits of consensus see: Gary Coglianese "The Limits of Consensus, Environment, 41, pp. 1-6, April 1999). You might also want to rework text on page 12, bottom column 1 to place the decision-maker ultimately in charge. If you make the suggested change, you might also wish to change page 34, lines 31-32 from "the frequency of reporting should be decided upon with all the participants..." to read "the frequency of reporting should be chosen by the decision maker in consultation with all the participants..." The last few lines in the box on page 15 adopt the "informed decision maker" perspective we are suggesting. On page 12, in the box at the bottom of column 2 you make the goals for the decision- making exercise a fixed output of Stage 1. This is also the way Figure 6 on page 28 is drawn. However, it is often the case that as one works through a problem and understands it better, goals become better informed and change (sometimes even without a change in underlying values). ------- This possibility needs to be acknowledged. Some feedback into the goals box in Figure 6 would be a good addition. The change could be as simple as: _ _ _ ., ,^ New insights from Define Problem "^ 3 later stages of analysis. Develop.... Page 15, column 1, lines 3-13 does acknowledge that goals may change. The language "...of the initial risk rankings or other aspects..." might be changed to read "...of the initial goals, risk rankings or other aspects..." On this general subject of fixed versus evolving goals, see the March (1976) quote that we included on the last page of our previous review. Page 16, column 2, lines 2-4: Because risk is a multi-attribute concept (i.e., depending on things like equity and level of knowledge as well as expected mortality and morbidity), all risk ranking is based on values as well as science. The problem of merging health and environmental risks is just a more extreme instance. Perhaps the sentence might be reworked to read: "...the members concluded that a merger would involve even larger value judgments than those involved in the separate rankings of health and environmental impacts and was reluctant to make such judgments absent some broader societal valuation process." Alternatively, since no rankings are being reported (with the exception of the box on page 18) is it even necessary to say this? In its previous review, the subcommittee had some methodological concerns with the ecological risk ranking procedure, and major concerns with the internet-based interview tool. Since the details of these methods are no longer present, many of those concerns are less pressing, but we would like to see the following language added at the end of the discussion on page 20, just before Section 3.1.2: "While the examples developed by the ERS and HEHS provide illustrations of the type of new tools we believe need to be developed, these specific examples were created with limited consideration of the literature in modern social and decision science. As the EPA develops and refines such tools for future use, it will be important to involve experts from these fields." Then, on page 39, column 1, lines 30-31: "These approaches should be further explored..." should read "These and similar approaches should be further developed and explored..." The discussion of benefit-cost and of deliberative processes on pages 22-24 could use a bit more discussion of equity (especially page 24). Benefit-cost is a tool to achieve efficiency. While efficiency is very important, government spends much of its time dealing with issues of equity. Thus, you might consider adding something on the need for better analytical methods for assessing the equity implications of decisions. For example, on line 38, after the sentence that ------- ends "...welfare goals." you might add something like the following: "Benefit-cost analysis is a tool for assessing the efficiency of decisions. Government is also concerned with the equity implications of decisions. More work needs to be done to develop assessment tools that address issues of equity." On page 22, column 1, line 36 "when benefit-cost information is considered..." might better read "when benefit-cost and other information is considered..." The discussion of page 25, column 25 and the box on page 26, implicitly assume that people have well-articulated values and that the problem is one of measurement. Elsewhere the report makes it clear that this is often not the case. These sections should be reworked slightly to include that recognition here as well. Table 1 on page 30 is still incomplete and the shading does not seem to help organize the information. It should either be worked out more systematically or dropped. For example, if tradable emissions permits are to be used it must be possible to: a) Define and accurately measure the pollutant(s) of concern, their sources (both natural and anthropogenic) and their atmospheric fate and transport (i.e., understand the science); b) Define the quantity of emissions that the regulated sources will be allowed to emit (the cap), and which will be available for trading in the market, which can vary (typically decline) as a function of time; c) Find an acceptable method to allocate permits to participating parties before trading is initiated; d) Create and operate a market with enforceable contracts and rules in which specified classes of polluters must participate (or face penalties), and which involve enough participants to assure competitive behavior; and e) Demonstrate that all pollutants being traded cause similar damage (as in the case of a uniform well mixed pollutant), or if they do not, devise a weighting system acceptable to all participating parties that to normalize damages across emission locations and times, and pollutant types. Similar elaborations are possible for all the other cells. In addition, the layout (shading of alternate rows) is somewhat confusing. Finally, the caption might better identify these as strategy or approaches. "Options" implies the need to choose one or another row, while in fact, many of these strategies can be combined. Page 39, column 2, lines 5-11: It is unclear what the "prototype risk reduction methodology" is. These lines might just be dropped. ------- Page 22, Section 3.2.2 needs to introduce the notion of metrics that would be applied to various risks. For instance, the following could be inserted: "To integrate assessments of risk embedded in complex environmental problems, it is essential that methodology be developed that presents the relevant science in a way that the public can understand the nature of particular endpoints and make decisions across diverse endpoints. Descriptors should be both qualitative and quantitative. In the health effects area, the range of impairments produced by the disease should be described and it should be made clear whether the data describes frank disease or a metric that is thought to be predictive of the disease. Quantitatively, it should characterize the exposure-dose- dose rate- response relationships that are involved. The valuation of different endpoints in the environmental area must be done in public forum as these likely must be collectively made if they are to be widely accepted. Entirely different values are involved in environmental decisions compared to personal medical decisions." The phrase in the second to last sentence in the first paragraph of section 3.2.2 suggesting that the agency attempts to reduce risk to a specified level may not be a full characterization of the Agency's policy. For example many regulations are not set at the 10"6 level. The Office of Water sets a range from 10"6 to 10-4. Most of their standards are within this range. However, there are standards whose calculated risks exceed this level (i.e., disinfectant by-products, radon and arsenic). In addition to our comments above, we have also attached an appendix with specific editorial comments. Conclusion The Subcommittee congratulates the Integrated Risk Project Steering Committee and the SAB staff on a dramatically improved report. It is well-organized and clearly written. The "what we have"/" what we need" structure is a very useful organizing device. The set often recommendations are reasonable and appropriate. We thank the Agency for the opportunity to review the "Toward Integrated Environmental Decision-Making" report by the Science Advisory Board. We think it offers promised to advance the decision-making process at EPA. Sincerely, Prof. M. Granger Morgan, Chair Integrated Risk Project Review Subcommittee ------- APPENDIX A. SPECIFIC EDITORIAL COMMENTS In the early part of the report, the format "benefit/cost" is used. Later "benefit-cost" is used. While this is a matter of taste, there are good reasons to use "benefit-cost" (see for example EJ. Mishan, Economics for Social Decisions: Elements of cost-benefit analysis, George Allen and Unwin, 1972 for a discussion of when to use B-C and when to use B/C). Page 2, column 2, line 20: "...as budgets continue to decline." Not all budgets are declining and there is no reason to assume that all will in the future. You might consider saying "...in the face of fiscal constraints." or "...in the face of multiple demands on limited budgets." In the box on page 3, why include the example of a focus on "a particular industrial sector" if the point of the new approach is to look in an integrated way at risks independent of their source? p. 4 column 1 line 30ff. The sentence is awkward. ".. .endpoints (e.g ), and the elements of exposure assessment, two essential components of risk assessment. Page 5, column 1, lines 24-28: the English is a bit awkward. Page 7, column 1, line 6: "...costs and benefits (either physical or monetary)..." might better read "...costs and benefits (both tangible and intangible)..." or "...costs and benefits (both monetary and non-monetary)..." Page 11, column 1, lines 7-9: "...improved techniques for forming, eliciting and considering public values, for communicating..." might better read "...improved techniques for helping people develop considered values, for eliciting and using public values in decision making, for communicating..." A similar problem exists on page 40, column 1, line 37. The issue is the distinction between helping people construct their own values, as opposed to forming people's values. p. 12. Box in column 2. Item 2 should add ".. ..and the reasons for these decisions; Page 13, column 2, lines 18-21: "Developments in the social sciences, for example, are providing improved methods for multi-attribute decision making" might better read "Developments in the social and decision sciences, for example, are providing improved methods for value elicitation and multi-attribute decision making." Page 22, column 2, line 23: "...can make themselves best off..." might better read "...can make themselves better off..." p. 23 column 1, line 9. what is meant by ".. .physical endowment available." A- 1 ------- Page 23, column 2, line 42: "...cost across different groups..." might better read "...cost across different individuals and groups..." Page 23, column 2, line 17: "...derived from the vector of all..." should be less technical. Page 26: There is missing text at the bottom of the boxed section. Page 28, Figure 6: Do you really want to say "optimize options"? Might not "refine options" be better? Real policy processes rarely are able to optimize anything (see writing by Simon, Kingdon and others). Page 29, Section 3.5.2: You might make some reference to the fact that some risk management strategies are dictated by existing legislative mandates. Page 29, column 2, line 26: The meaning of "...'root cause' or 'common sources/common pathway'" is not clear. Page 31, column 1, lines 40-41: "...have a common measure of risk or a common denominator of all risks..." The report might place greater emphasis on the difficulty of achieving this. Risk ranking typically produces an ordinal measure. The current language is calling for a cardinal measure. You might at the end of the section (page 31, column 2, line 3) something like: "While models can be developed to weight and combine all the different attributes in disparate types of risk, getting wide-spread social buy-in for such weighting poses enormous problems." Page 31, column 2, line 5: "Uncertainty of the analysis is likely to..." is awkward phrasing. Page 31, column 2, lines 14-16: It may be easier to implement communication than regulation when uncertainty is high but it is not clear that it is easier to get good results. Communication methods work best when uncertainty is low. You might just drop the examples. Page 7, Figure 7: The arrows might be dropped, at least the left one. Or you could use a graphic such as: to indicate a spectrum. Page 34, column 2, lines 1-20: The language is awkward. p. 34, column 2, line 9 of the second paragraph. Should it not be ".. .may not exist, may be subtle and difficult to measure, or may be observable " Page 35, column 1, line 4: "Figure 9" should read "Figure 8" A-2 ------- Page 35, column 2, lines 28-36: A considerable amount of work of the sort proposed on evaluating marketable permits has been done. If you need references, contact Dr. Alex Farrell (afarrell® andrew.cmu.edu). Page 36, Figure 8 is really not a Figure but a box like the earlier boxes in the report. Box on p. 36. Should it be recognized that one needs to use several parameters to characterize effectiveness of intervention. Not all remedial actions are equally effective and in certain areas where failure is more highly feared (i.e. in drinking water treatment to prevent waterborne infectious disease) multiple barriers are routinely used or at least advocated. It is important to have measures of outcome, stressors, and process to judge what has been the most effective when the overall effectiveness is evaluated. To assume that all the improvement that is seen is all attributable to institution of a remedial process is analogous to ignoring in the placebo effect in clinical trials. Page 40, column 1, line 33: "...behavioral science and decision logic..." the more conventional language in the field is "...behavioral and decision science..." Page 40, column 2, line 4: "integrated environmental" should read "integrated environmental" Page 40, column 2, line 30: "...manage and use that data..." should read "...manage and use those data..." Page 44, column 1, line 15: "c) Need to compare..." should read "c) The need to compare..." A-3 ------- NOTICE This report has been written as part of the activities of the Science Advisory Board, a public advisory group providing extramural scientific information and advice to the Administrator and other officials of the Environmental Protection Agency. The Board is structured to provide balanced, expert assessment of scientific matters related to problems facing the Agency. This report has not been reviewed for approval by the Agency and, hence, the contents of this report do not necessarily represent the views and policies of the Environmental Protection Agency, nor of other agencies in the Executive Branch of the Federal government, nor does mention of trade names or commercial products constitute a recommendation for use. Distribution and Availability: This Science Advisory Board report is provided to the EPA Administrator, senior Agency management, appropriate program staff, interested members of the public, and is posted on the SAB website (www.epa.gov/sab). Information on its availability is also provided in the SAB's monthly newsletter {Happenings at the Science Advisory Board). Additional copies and further information are available from the SAB Staff. ------- US ENVIRONMENTAL PROTECTION AGENCY SCIENCE ADVISORY BOARD INTEGRATED RISK PROJECT PEER REVIEW SUBCOMMITTEE OF THE EXECUTIVE COMMITTEE CHAIR Dr. M. Granger Morgan, Head, Department of Engineering & Public Policy, Carnegie Mellon University, Pittsburgh, PA MEMBERS AND CONSULTANTS Dr. Henry A. Anderson, Chief Medical Officer, Wisconsin Bureau of Public Health, Madison, WI Dr. William E. Bishop, Associate Director, Global Technical Policy, The Procter & Gamble Company, Cincinnati, OH Dr. Donald F. Boesch, President, University of Maryland Center for Environmental Science, Cambridge, MD Dr. Richard J. Bull, Senior Staff Scientist, Battelle Pacific Northwest Division, Molecular Biosciences, Richland, WA Dr. Terry Davies, Director, Center for Risk Management, Resources for the Future, Washington, DC Dr. John D. Graham, Professor of Policy and Decision Sciences and Director of Harvard Center for Risk Analysis, Harvard University, Boston, MA Dr. Catherine Kling, Professor, Department of Economics, Iowa State University, Ames, IA Dr. Debra Knopman, Director, Center for Innovation and the Environment, Progressive Policy Institute, Washington, DC Dr. Morton Lippmann, Professor, Nelson Institute of Environmental Medicine, New York University School of Medicine, Tuxedo, NY Dr. Warner North, President and Principal Scientist, North Works Inc., Belmont, CA Dr. Richard Revesz, Professor of Law, New York University School of Law, New York, NY Dr. Bruce Tonn, Senior Researcher, Oak Ridge National Laboratory, Oak Ridge, TN 11 ------- SCIENCE ADVISORY BOARD STAFF Dr. John R. Fowle III, Deputy Staff Director/Designated Federal Officer, Science Advisory Board (1400A), US EPA, 1200 Pennsylvania Avenue, NW, Washington, DC 20460 Ms. Wanda R. Fields, Management Assistant, Science Advisory Board, USEPA (1400A), US EPA, 1200 Pennsylvania Avenue, NW, Washington, DC 20460 in ------- |