UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C 20460 OFFICE OF THE ADMINISTRATOR SCIENCE ADVISORY BOARD The Honorable Stephen L Johnson Administrator U S Environmental Protection Agency 1200 Pennsylvania Avenue, N W Washington, DC 20460 Subject SAB Advisory on Superfund Benefits Analysis Dear Administrator Johnson The U S EPA Science Advisory Board's (SAB) Superfund Benefits Analysis Advisoiy Panel met on February 24 -25, 2005 to review the Agency's draft "SuperfundBenefits Analysis " (SBA) EPA's Office of Solid Waste and Emergency Response (OSWER) requested a review of this draft assessment of the retrospective benefits (1980 - 2004) of the Superfund program The attached "Advisory on Superfund Benefits Analysis" constitutes the SAB Superfund Benefits Analysis Advisory Panel's detailed advice on OSWER's draft report In this draft SBA, the Agency presents an estimate of the economic benefits of Superfund, based on what is described as a meta-analysis and benefits transfer of selected studies of property values around Superfund sites The SBA also describes how the Agency proposes to quantify and monetize the benefits associated with three specific categories of effects (reductions in human health nsks, reductions in injuries to ecological systems, and protection of ground water) and seeks the SAB's advice concerning its proposals Estimating the full benefits of the Superfund program is an extremely important effort Thus, the Agency should be commended for initiating this study That said, the draft SBA, in our view, falls short of the kind of estimate of benefits that we believe is needed We detail in the executive summary, and the full advisory, steps we think EPA should consider in deciding whether or not to move forward with this particular study ------- As outlined in the attached report, the committee reached six major conclusions about the draft SBA 1) A more coherent framework for enumerating the many possible benefits of the Superfund program is needed 2) Better documentation of the study's approach and methodologies is needed 3) The benefits transfer and meta analysis in the hedonic property model study falls short of a national retrospective benefits estimate 4) The Agency's proposed approach to estimating the comprehensive health benefits of Superfund faces severe data constraints 5) The Agency's proposed approach to estimate ecological benefits faces severe data and methodological constraints 6) The Agency's proposed meta analysis and benefits transfer to estimate the benefits of the protection of ground water resources lacks an adequate empirical basis The attached Advisory discusses a range of options for the Agency to consider in moving forward from here In considering which, if any, parts of the SBA to continue, the Panel recommends the Agency separately evaluate the several components of the study since each will entail different schedules, expertise, and research aims The sections on human health, ecological and groundwater benefits are much less developed than the hedonics chapter Thus, one option for the Agency to consider is to complete the hedonics research following the suggestions offered in Section 3 2 of the Advisory and to put the other three analyses on hold Other options include forgoing an effort to develop a comprehensive estimate of monetary benefits and focusing on a set of illustrative case studies based on the available literature A third option is to terminate the current exercise and to focus attention on a long term strategy for developing methods and data for estimating future benefits Because of the importance of the Superfund Program and the questions raised in the Advisory, a multi-year research strategy for assessing the benefits of Superfund may be needed Finally, we want to point out that any assessment of the benefits of the Superfund Program will be of limited usefulness unless it is accompanied by a comparable assessment of the Program's full costs Sincerely, ------- Dr Granger Morgan, Chair Dr A Mynck Freeman, Chair US EPA Science Advisory Board Superfund Benefits Analysis Advisory Panel ------- NOTICE This report has been written as part of the activities of the EPA Science Advisory Board, a public advisory group providing extramural scientific information and advice to the Administrator and other officials of the Environmental Protection Agency The Board is structured to provide balanced, expert assessment of scientific matters related to the problems facing the Agency This report has not been reviewed for approval by the Agency and, hence, the contents of this report do not necessarily represent the views and policies of the Environmental Protection Agency, nor of other agencies in the Executive Branch of the Federal government, nor does mention of trade names or commercial products constitute a recommendation for use Reports of the EPA Science Advisory Board are posted on the EPA website at http //www epa gov/sab 4 ------- U.S. Environmental Protection Agency Science Advisory Board Superfund Benefits Analysis Advisory Panel CHAIR Dr. A. Myrick Freeman, Research Professor of Economics, Department of Economics, Bowdoin College, Brunswick, ME MEMBERS Dr. Anna Alberini, Associate Professor, Department of Agricultural and Resource Economics - AREC, University of Maryland, College Park, MD Dr. Robin L. Autenrieth, Professor, Department of Civil Engineering and Department of Environmental and Occupational Health, Texas A&M University, College Station, TX Dr. James Boyd, Senior Fellow, Director, Energy & Natural Resources Division, Resources for the Future, Washington, DC Dr. Ted Gayer, Associate Professor, Department of Public Policy, Georgetown University, Washington, DC, USA Dr. Mark Miller, Public Health Medical Officer, Office of Environmental Health Hazard Assessment (OEHHA), Oakland, CA Dr. Horace Moo-Young, Associate Dean and Professor, Civil and Environmental Engmeering, College of Engineering, Villanova University, Villanova, PA Ms. Katherine N. Probst, Senior Fellow and Director, Risk, Resource and Environmental Management Division, Resources for the Future, Washington, D C , Dr. Kathleen Segerson, Professor, Department of Economics, University of Connecticut, Storrs, CT Mr. Timothy Thompson, Senior Environmental Scientist, Science, Engineering, and the Environment, LLC, Seattle, WA SCIENCE ADVISORY BOARD STAFF Dr. Holly Stallworth, Designated Federal Officer, Science Advisory Board Staff Office, Washington, DC ------- Table of Contents 1.0 Executive Summary i 2.0 Background to the Advisory 1 3.0 Overview of the SAB's Response to Charge Questions 2 3.1. Benefits Framework 6 3.2 Hedonic Property Values 6 3.3. Health Benefits 10 3.4. Ecological Benefits 12 3.5. Ground Water Protection Benefits 13 4.0 Responses to Specific Charge Questions 15 4.1. Charge Question 1 15 4.2 Charge Question 2 19 4.3 Charge Question 3 22 4.4 Charge Question 4 26 4.5 Charge Question 5 29 4.6 Charge Question 6 30 4.7 Charge Question 7 35 4.8 Charge Question 8 36 5.0 References 37 ------- 1.0 Executive Summary This Advisory responds to a request from the EPA's Office of Solid Waste and Emergency Response (OSWER) to EPA's Science Advisory Board (SAB) for advice on its draft Superfund Benefits Analysis (SB A), a retrospective study of economic benefits associated with the Superfund Program from 1980 to 2004 In this draft SBA, the Agency presents an estimate of the economic benefits of Superfund, based on what is described as a meta-analysis and benefits transfer of selected studies of property values around Superfund sites The SBA also describes how the Agency proposes to quantify and monetize the benefits associated with three specific categories of effects (reductions in human health risks, reductions in injuries to ecological systems, and protection of ground water) and seeks the SAB's advice concerning its proposals The laudable intent of the draft SBA is to provide a counterpart to a number of studies on the costs of the Superfund program Indeed, no study has been published to date that attempts a systematic accounting of the benefits of the Superfund Program over the last 25 years, perhaps due to the difficulty of the undertaking While the Panel shares the desire for a credible assessment of the retrospective benefits of the Superfund Program, we believe the current draft SBA falls short of this goal This is in large part due to the difficulty of the task at hand, which faces severe methodological and data constraints In addition to our responses to detailed charge questions in section 4, we offer the following more general comments 1 The Agency should develop a more coherent framework for enumerating the many possible benefits of the Superfund program The development of a comprehensive and credible framework could be one of the most important "next steps" in encouraging the broader academic and research communities to engage in research on this topic The Panel believes that this task would require much more time and effort and would need outside experts to be involved in the development of a comprehensive benefits framework for the many components of the Superfund program 2 There needs to be a more thorough discussion of methodological issues and better documentation of the criteria guiding selection of studies for benefits transfer. The lack of discussion of important methodological decisions in itself raises questions about the quality of the report The promises and pitfalls of using an hedonic approach to capture an ex post stream of Superfund benefits over a 25-year period should be discussed Also, the Panel is concerned that the studies used in the meta-analysis may not be a representative sample of Superfund sites, and given the limited years of data each study contains, they are not designed to arrive at a comprehensive, 25-year retrospective estimate of the benefits of the Superfund program l ------- 3 The benefits transfer and meta analysis (from hedonic property model studies) falls short of a national retrospective benefits estimate. While the hedonic property value study is the most well developed of the four categories of benefits described in the study, there are major problems in the way the meta-analysis and benefits transfer have been conducted, given the goal of capturing the historical benefits of cleanup A hedonic property model can offer a sound means of assessing the benefits of reducing the perceived disamenities of a given Superfund site and of assessing how these benefits vary given the availability of different levels of information to the residents about the site However, it is a more difficult task for the hedonic approach to yield credible benefits estimates at the national level, that is, for all National Priorities List (NPL) sites (unless individual site-specific panel data were collected) The meta analysis contained in the draft SBA does not achieve this goal 4 The Agency's proposed approach to estimating the comprehensive health benefits of Superfund faces overwhelming data constraints. The feasibility of the Agency's proposed approach is limited due to the lack of epidemiological and exposure data and the difficulty in obtaining values for many health endpoints 5 The Agency's proposed approach to estimate ecological benefits faces severe data and methodological constraints. The Agency's approach relies too heavily on the value of natural resource damages as a proxy for "interim lost use values" Settlement figures from natural resource damage assessment cases reflect either negotiations between the Trustees and the responsible parties, or are the result of awards determined in federal court, they may not reflect the benefit that society places on the ecological effects of the Superfund program's activities The Panel recommends that rather than attempt to develop an aggregate monetary estimate of ecological benefits, the Agency more fully describe the various ecological consequences of remediation and then translate these into descriptions of beneficial effects that the public can understand The Panel notes that this would actually be part of the development of the comprehensive benefits framework suggested earlier 6 The Agency's proposed meta analysis and benefits transfer to estimate the benefits of the protection of ground water resources lacks an adequate empirical basis It is not clear what the Agency means by "protection" in this context It will be very difficult to quantify the amount of ground water protected under any of the possible interpretations of this word And over half of the ground water valuation studies cited as the basis for the proposed benefits transfer deal with nitrates from agricultural runoff or surface water contamination and thus are of questionable relevance for the kinds of ground water contamination associated with Superfund sites n ------- 7 There are several options for moving forward from here. One of the major questions discussed by the Panel was whether to recommend that the Agency continue with any or all of the four proposed approaches to capturing the benefits of Superfund Estimating the retrospective benefits of the Superfund program is an extremely difficult and controversial task It is laudable that the Agency is trying to fill this gap in existing research But the Panel has concerns about each of the components of the study In considering which, if any, parts of the SBA to continue, the Panel recommends the Agency separately evaluate the five individual projects (with the fifth effort being the development of a benefits framework), since each will entail different schedules, expertise, and research aims The sections on human health, ecological and groundwater benefits are much less developed than the hedonics chapter Thus, one option for the Agency to consider is to complete the hedonics research following the suggestions offered in Section 3 2 and to put the other three analyses on hold Other options include forgoing an effort to develop a comprehensive estimate of monetary benefits and focusing on a set of illustrative case studies based on the available literature A third option is to terminate the current exercise and to focus attention on a long term strategy for developing methods and data for estimating future benefits Because of the importance of the Superfund Program and the questions raised here, a multi-year research strategy for assessing the benefits of Superfund may be needed Such a multi-year research strategy could include the following components initiating studies that can be done now using existing data and methodologies, identifying and filling gaps in the data, and commissioning independent research to fill gaps in methodology The Panel also believes that estimating the full costs of the program is another critical component of a future research agenda in ------- 2.0 Background to the Advisory The purpose of this Advisory is to provide advice on the Superfund Benefits Analysis (SBA), a draft assessment of the retrospective benefits of the Superfund program produced by EPA's Office of Solid Waste and Emergency Response (OSWER) In response to OSWER's request for advice on this draft document, the Science Advisory Board Staff Office (SABSO) published a Federal Register notice on July 30, 2004 requesting nominations for expertise in (a) hazardous waste management, (b) valuation for cost-benefit analysis, specifically hedonic pricing models and methods, (c) ecological risk assessment, (d) public health and epidemiology, and (e) toxicology and human health risk assessment of toxic chemicals Subsequently, the SABSO received nominations for the Superfund Benefits Analysis Advisory Panel, solicited public comment on a "Short List" of candidates, and, on February 8, 2005 issued a final determination memo announcing the members of this Panel The Superfund Benefits Analysis Advisory Panel met on February 24 -25, 2005 in a face-to-face public meeting in Washington, D C and held 3 additional teleconferences to discuss draft responses to charge questions posed by OSWER This Panel produced an initial Draft Advisory, posted on April 25, 2005, and continued deliberations through two subsequent Draft Advisories This Advisory constitutes the final result of the Panel's deliberations 1 ------- 3.0 Overview of the SAB's Response to Charge Questions In the draft Superfund Benefits Analysis (SBA) the Agency presents an estimate of the economic benefits associated with the Superfund Program from 1980 to 2004 obtained from what it describes as a meta-analysis and benefits transfer based on studies of property values around Superfund sites The Agency also describes how it proposes to quantify and monetize the benefits associated with three specific categories of effects and seeks the SAB's advice concerning its proposals The three categories of effects are reductions in human health risks, reductions in injuries to ecological systems, and protection of ground water The Panel applauds the Agency for conducting much needed research to try to estimate the benefits of the Superfund program While there have been a number of studies of the economic costs caused by particular hazardous waste sites and of the costs of the Superfund and RCRA programs overall, there have been only a few studies of the benefits related to the Superfund program There have been no published studies that attempt a systematic accounting of the benefits of the Superfund Program over the last 25 years This lack of a comprehensive assessment of the benefits of Superfund may explain in part why the program is often viewed as controversial While the Panel shares the desire for a credible assessment of the retrospective benefits of the Superfund Program, we believe the SBA currently falls short of this goal This is in large part due to the difficulty of the task at hand, which faces severe methodological and data constraints Also, in some cases the SBA does not fully document its assumptions and procedures, leaving the Panel unable to judge the soundness of its conclusions and findings Because the hedonic property value study is the most well-developed of the four substantive chapters of the report, the bulk of our comments pertain to that effort Sections 3 1 - 3 5 below contain detailed comments on our concerns with the SBA's hedonic meta-analysis and with the approaches to estimating human health benefits, ecological benefits, and benefits of ground water protection In these sections, the Panel offers some recommendations for how these approaches can be improved Below is a general summary of the Panel's comments on the SBA 1 A more coherent framework for enumerating the many possible benefits of the Superfund program is needed This is not, in fact, an easy task As currently constructed, the report uses the organization of the current program as an organizing principle for identifying benefits In some cases important benefits have been omitted, for example, those related to removal actions In other cases the benefits identified would be better characterized as inputs 2 ------- rather than outcomes For example, deterrence refers to a change in incentives that leads to the prevention of harmful releases and thus leads to health and other benefits The development of a comprehensive and credible framework could be one of the most important "next steps" in encouraging the broader academic and research communities to engage in research on this topic The Panel believes that more time and effort needs to be devoted to this task, and that outside experts need to be involved in the development of a comprehensive benefits framework for the many components of the Superfund program 2 Better documentation is needed. The lack of discussion of important methodological decisions - most notably the selection of the nine existing hedonic studies used in the benefits transfer — in itself raises questions about the quality of the report The authors need to explain why they selected these nine studies The promises and pitfalls of using a hedonic approach to capture an ex post stream of Superfund benefits over a 25-year period should be discussed Because of the lack of explanation regarding the approach taken, the Panel is not able to assess whether the selection of these studies for the meta-analysis is appropriate A major concern about the analysis as currently conducted is that the studies used in the meta-analysis are not a representative sample of Superfund sites, and given the limited years of data each study contains, they are not designed to arrive at a comprehensive, 25-year retrospective estimate of the benefits of the Superfund program A better approach would be to use a larger, more representative sample of Superfund sites, and to obtain market data on housing prices at each stage of the Superfund process In thinking about alternative approaches to hedonic analysis, the Agency could examine the unpublished paper by Greenstone and Gallagher (2005), which addresses some of the Panel's concerns by using retrospective data covering the 25-year history of Superfund, using a more representative set of Superfund sites, and using an innovate research design 3 The benefits transfer and meta analysis (from hedonic property model studies) falls short of a national retrospective benefits estimate. At least in theory, many (but not all) of the benefits of Superfund can be captured by hedonic property models While the hedonic property value study is the most well developed of the four categories of benefits discussed in the study, there are major problems in the way the meta-analysis and benefits transfer have been conducted, given the goal of capturing the historical benefits of cleanup A hedonic property model can offer a sound means of assessing the benefits of reducing the perceived disamenities of a given Superfund site However, it is a more difficult task for the hedonic approach to yield credible benefit estimates at the national level, that is, for all National Priority List (NPL) sites, (unless individual site-specific panel data were collected ) In Section 3 2, we discuss why we believe the meta-analysis approach taken in this study is inadequate and offer some suggestions for an improved hedonic approach 3 ------- 4 The Agency's proposed approach to estimating the comprehensive health benefits of Superfund faces overwhelming data constraints. The feasibility of the Agency's proposed approach is limited due to the lack of epidemiological and exposure data and the difficulty in obtaining values for many health endpoints We instead offer some suggestions on how the Agency can present illustrative calculations of some components of the benefits to human health of Superfund, and how the Agency can analyze the benefits related to reduced lead exposure 5 The Agency's proposed approach to estimate ecological benefits faces severe data and methodological constraints. The Agency's approach relies too heavily on the value of natural resource damages as a proxy for "interim lost use values " As is discussed in more detail in Section 3 4, settlement figures from natural resource damage assessment cases reflect either negotiations between the Trustees and the responsible parties, or are the result of awards determined in federal court Under these circumstances, these NRD settlements do not reflect the benefit that society places on the ecological effects of the Superfund program's activities The Panel believes that estimating the monetary value of ecological benefits of the Superfund program is difficult at this time for the following reasons First, the beneficial effects of remediation and restoration on ecological system are not well understood and documented Second, despite much recent research in this area, it remains difficult to attach a monetary value to ecological benefits of environmental restoration programs Finally, connecting estimated interim lost use values with NRD settlements or awards, and then to the actual application of the settlement fund toward restoration activities is not well-established The Panel acknowledges that there is substantive body of developing NRD evaluations These include the Hudson River (NY), the Housatonic River (MA), the Grand Calumet River (ID), the Lower Duwamish River (WA), the Willamette River (OR), Coeur d'Alene Basin (ID) the Palos Verde Shelf (CA), and several other sites throughout the country Within the next 5 - 10 years, these sites, along with the Lower Fox River cited in the EPA's report, will have developed a sufficient body of data from which to calculate ecological economic benefits In the interim, the Panel recommends moving away from the notion of developing an aggregate monetary estimate of ecological benefits and instead suggests that EPA more fully describe the various ecological consequences of cleanup and then translate these into descriptions of beneficial effects into terms that the public can understand The Panel notes that this would actually be part of the development of the comprehensive benefits framework suggested earlier In addition, EPA could conduct quantitative but not monetary assessments of ecological benefits using ecological benefits indicators Finally, the Panel refers the Agency to the recently issued Millenium Ecosystem Assessment (2003) report, which addresses many of these issues, and to the SAB panel examining the valuation of the protection of ecological systems and services 4 ------- 6 The Agency's proposed meta analysis and benefits transfer to estimate the benefits of the protection of ground water resources lacks an adequate empirical basis It is not clear what the Agency means by "protection" in this context It will be very difficult to quantify the amount of ground water protected under any of the possible interpretations of this word And over half of the ground water valuation studies cited as the basis for the proposed benefits transfer are of questionable relevance for the kinds of ground water contamination associated with Superfund sites 7 There are several options for moving forward from here. One of the major questions discussed by the Panel is whether to recommend that the Agency continue with any or all of the four proposed approaches to capturing the benefits of Superfund Rather than offering a yes or no, the Panel offers a set of options and comments on the Agency's various approaches In addition, the Panel concludes that the data and methodologies do not support the development of comprehensive estimates of health, ecological, and ground water protection benefits Estimating the retrospective benefits of the Superfund program is an extremely difficult and controversial task It is laudable that the Agency is trying to fill this gap in existing research In considenng which, if any, parts of the SBA to continue, the Panel recommends the Agency separately evaluate the five individual projects (with the fifth effort being the development of a benefits framework), since each will entail different schedules, expertise, and research aims For example, it is clear that there is much more work to be done on all the sections, and that the sections on human health, ecological and groundwater benefits are much less developed than the hedonics chapter Thus, one option for the Agency to consider is to complete the hedonics research following the suggestions offered in Section 3 2 and to put the other three analyses on hold Other options include forgoing an effort to develop a comprehensive estimate of monetary benefits and focusing on a set of illustrative case studies based on the available literature as suggested in Sections 3 2-3 5 of this Advisory A third option is to terminate the current exercise and to focus attention on a long term strategy for developing methods and data for estimating future benefits Because of the importance of the Superfund Program and the questions raised here, a multi-year research strategy for assessing the benefits of Superfund may be needed Such a multi-year research strategy could include the following components initiating studies that can be done now using existing data and methodologies, identifying and filling gaps in the data, and commissioning independent research to fill gaps in methodology The Panel also believes that estimating the full costs of the program is another critical component of a future research agenda In the rest of this section, the Panel describes and critiques the approaches used or proposed for each of the four categories of benefits Section 4 provides the Panel's detailed 5 ------- responses to the charge questions and suggestions for ways in which the Agency might use the available literature and data on property values, health effects, ecological effects, and ground water protection to better charactenze the beneficial effects of the Program 3.1. Benefits Framework The Panel believes that the Agency could make a major contribution by providing a coherent framework for thinking about the benefits of the Superfund program, that is, by laying out an approach to benefits assessment in the specific context of Superfund Figure 1 1 in the SBA attempts to provide some structure for thinking about Superfund benefits, but as detailed in Section 4 1 below, it suffers from a number of shortcomings For this reason, the Panel recommends that the Agency lay out an alternative framework for Superfund benefits assessment A beginning point for benefits assessment would be to identify the elements of the Superfund program that have led to beneficial changes and the impacts of each of these elements, i e , what changes have occurred (in physical terms) because of each element One can then try to translate these "impacts" into measured "benefits" to the extent possible The report should be clear about how "benefits" are defined In particular, it should note that the concept of benefits used throughout the report is based on a broad definition that includes impacts on both human health and the environment With regard to the latter, the concept of benefits includes impacts on people that stem either from the flow of services provided by the environment or from the intrinsic value of the environment and its components Currently, the discussion of benefits in Chapter 1 appears to be focused on the different components of the Superfund program, not on a conceptual framework based on impacts Two recent studies on ecosystem benefits can provide some guidance on a conceptual framework for benefits assessment (National Research Council, 2004, and Millennium Ecosystem Assessment, 2003) In thinking about translating impacts into benefits, it is important to distinguish among the following (i) those impacts that can be valued in monetary terms using standard economic valuation methods and available data, (u) those that could be valued in this way (/"better data were available, and (in) those that cannot be captured through economic valuation either because of the limitations of tools or data For this latter category, it might still be possible to quantify the impacts in some way that provides information about the associated benefits, although in some cases even this may not be possible The report should discuss clearly what kinds of approaches and data are available to estimate benefits, what the challenges are, and, from the authors' perspective, what are the best approaches that can be taken given the current state of the art 3.2 Hedonic Property Values 6 ------- The Panel believes that the approach used in Chapter 4 provides some information on the benefit of living further away from a Superfund site, but it does not provide a credible estimate of the monetary value of the retrospective benefits of the Superfund program There are three major problems with the approach that lead us to this conclusion I Limitations of the conceptual model Chapter 4 relies on hedoruc property models, which attempt to estimate the marginal willingness to pay for a non-market housing amenity One difficulty with such studies is finding an appropnate measure of the environmental disamenity to be valued Most of the studies cited in chapter 4 use the distance to the Superfund site as a proxy for the environmental good to be valued Furthermore, some of the studies are cross-sectional analyses that, coupled with the distance measure, effectively estimate the marginal willingness to pay for moving further from the site at a point in time This measure is not easily adaptable to a measure of the benefits of eliminating the site altogether, let alone the benefits of the Superfund program (which, after all, does not lead to the equivalent results as complete removal of the disamenities associated with the site) Some of the other studies in the meta-analysis rely on panel data, which are better suited to estimating the marginal willingness to pay for changes in the status of the site (e g , as the site progresses through clean up stages) However, none of the studies estimate price changes between discovery and completed clean-up Further, the meta-analysis in chapter 4 still seems to rely on static estimates of the price-distance gradient even for the panel studies The Panel could not discern how the estimated price-distance gradients were mapped into the aggregate benefit estimates The Panel believes that the estimated gradients were used to compute the price change that would result from moving all the neighborhood houses to a point where there no longer is a price impact stemming from the site This assumes that the Superfund program leads to full recovery of housing prices However, the Panel believes that there is very limited evidence of a full pnce recovery For the most part, the literature shows a price-distance gradient, which is evidence of a willingness to pay for distance from a site But this does not necessarily imply that prices recover after EPA remediation There is some evidence in the literature that the price gradient changes over time (which could be due to perceived rather than real changes caused by EPA), but it is still not clear that prices fully recover due to EPA activities The claim that prices fully recover after EPA activities is difficult to support given that most studies in the meta-analysis are either cross-sectional or use a narrow range of years of housing sales, neither of which can provide strong evidence of a full price recovery from Superfund actions Moreover, these studies are limited to specific locales, and there is no reason to believe that the full pnce recovery-even if it was truly observed at these locales-would apply on a national scale 7 ------- A related concern is how to determine which price to use as the baseline Market prices should fluctuate through each of the many steps from discover}' of a site to full clean-up It is not clear which price to use as the starting price from which benefits are estimated as an increase in price or value Should this be the price at the time of discovery? At placement on the NPL? When the ROD is issued? To further elaborate on this, assume, for example, that a site is listed on the NPL, causing housing prices to drop Assume also that prices then increase after the remedial investigation, which includes the baseline risk assessment Should it then be assumed that after remediation, prices will return to pre-NPL listing, pre-remedial investigation or post-remedial investigation level? It could be that the initial decrease in prices was due to unfounded beliefs about contamination at the site, which were then corrected with the release of the risk assessment This suggests the possibility that Superfund has caused a decrease in benefits by spurring the initial beliefs in the first place It also suggests that the appropriate starting price for the benefits measure is post-remedial investigation, after people were informed about the risks they face Also, if pre-discovery is used as the starting price, doesn't that count any emergency removals as part of this benefits estimate? In sum, the report glosses over the loss in property values that may occur when a site is placed on CERCLIS or nominated to NPL This loss would presumably be a cost of the Superfund program The report misleadingly assumes that all price decreases are independent of Superfund and that Superfund actions then fully recover prices from their lowest levels It should be remembered, however, that if the Panel agreed on a starting price, it is even more difficult to estimate these pnces based on the studies used in the meta-analysis 2 What drives the price-distance relationship? Another limitation of this hedonic meta-analysis is that it implicitly assumes, without testing, that subjective risks are equal to objective risks Based on this assumption, if nsks were completely eliminated by remediation, property prices would rise to their no-contarrunation levels This assumption breaks down if people have a different assessment of the risk than does EPA, if they do not understand the possible health impacts of these risks, or if they do not believe that mitigation has, in fact, completely eliminated the risks More generally, most of the studies used in the meta-analysis in fact address a much different question than is appropriate for this report Those studies (primarily) either estimate the relationship between housing prices and distance (this is especially the case for the cross-sectional studies), or they estimate how the price-distance relationship changes as information is released or events occur Neither of these estimates is clearly transferable to an estimate of the benefits of Superfund actions Such an estimate would need to estimate price 8 ------- changes over each stage of the Superfund process, with the added assumption that those changes are due solely to Superfund activities 3 Weakness of the meta-analysis/benefits transfer The Panel is unconvinced that the sample studies used in the meta-analysis are representative of the full population of Superfund sites and that they can therefore be credibly used to estimate the full benefits of the 25 years of Superfund activities Chapter 4 states that a review of the literature produced 30 hedonic studies In the end, only 9 of them were used in the meta-analysis Chapter 4 does not discuss in detail the selection process that led to only 9 studies being used The Panel was unable to assess the studies that were not used for the meta-analysis, however, the Panel questions the appropriateness of some of the selected studies For example, the McClelland et al (1990) article seems like an odd choice for this meta-analysis First, the study only has 178 observations What's more, the variable of interest is a "neighborhood" measure, so identification comes from even fewer observations (and thus the standard errors are biased upwards) Aside from the econometric issues, the article's main claim is that housing prices respond to subjective risk and that subjective risk differs greatly from objective risk It does not seem appropriate to blend this study with others where the maintained assumption of the meta-analysis is that perceived and objective risks are equal Finally, McClelland et al attempt to estimate how housing values vary by subjective risk They do not estimate a distance gradient, nor do their findings suggest that benefits would accrue from Superfund remediation Also, the Gayer et al (2000) study estimates how the price-risk gradient changes when new information becomes available It is not clear how this is incorporated into the meta- analysis of the benefits of Superfund remediation Finally, the Mendelsohn et al (1992) study does not use distance to the site as the variable of interest, so it is not clear how these estimates were incorporated in the meta-analysis In summary, the meta-analysis is based on a limited number of applicable studies Given the small sample of housing price estimates from local markets, the Panel is not comfortable extrapolating benefits to the full population of sites There are many reasons why these studies may not be representative For example, the study sites tend to be old and early NPL sites, they tend to be in places with high population density, and they tend to be larger sites Chapter 4 does not contain enough information to assess whether the locations, chemicals and pathways, and other site characteristics are representative of the full population of NPL sites All of these differences lead to potential problems with benefits transfer Furthermore, the meta-analysis does not control for study characteristics or study site characteristics, which is necessary To obtain unbiased estimates 9 ------- Given these reservations about the estimation of national benefits from the meta-analysis and benefits transfer, the following options for inclusion of property value data in the report are offered • being explicit about the necessary qualifications and caveats, • providing a richer discussion of the conceptual issues involved (baseline price level, real vs. perceived risks, relevance of the price-distance gradient for examining the behavior of house prices over time, assumption of full recovery of prices), • revising the selection criteria and basing the benefits transfer on a different and possible larger set of studies, • providing a more complete discussion of the selection criteria for studies included in the meta-analysis, • de-emphasizing the final figure from the extrapolation, and placing this chapter after those on health, ecological and other effects The results of this benefits transfer would yield an estimate of the aggregate housing price effects The validity of interpreting the housing price effect as a national benefit estimate is contingent on such things as the validity of assuming a national housing market and the validity of assuming that non-marginal changes in environmental quality are captured by the hedonic price function (see, among others, Michaels and Smith (1990), Bartik (1987), Epple (1987), and Ekeland, Heckman, and Nesheim (2004)) Another approach is to replace the current hedonics exercise with a qualitative descnption of the existing empirical literature on housing prices near Superfund sites (with proper attention to the difficulties in interpretation) to demonstrate the significance that people apparently place on the presence of Superfund sites near their homes and changes in the information about Superfund sites 3.3. Health Benefits The Agency proposed to estimate the health benefits of reductions in 5 health endpoints acute accidents and injuries, birth defects, lead induced health effects (cognitive deficits and cardiovascular disease), other chronic non-carcinogenic effects, and adult cancer The proposed approach for the first four endpoints is based on Lybarger, et al (1998) This paper used existing epidemiological studies of the health effects of exposure to volatile organic compounds in drinking water to quantify health effects for seven endpoints associated with living in proximity to NPL sites and valued these effects using available direct cost-of-illness data For 10 ------- adult cancer, no specific method was proposed, but one of our specific charge questions suggests that extrapolation from a study by Hamilton and Viscusi (1999) was being considered In the Panel's judgment, a comprehensive and defensible estimate of health benefits from Superfund is not possible at this time for three reasons The first concerns the epidemiology data Superfund sites contain a vanety of substances of concern, exposure routes, and numerous potential adverse health outcomes The epidemiologic literature is too sparse to allow a complete assessment of health outcomes attributable to exposures in communities adjacent to sites Lybarger et al (1998) provides an estimate for one of the few classes of chemicals at Superfund sites for which relevant data exist The second reason is the limitations of the data on exposure Lybarger, et al (1998) used proximity to an NPL site as an indicator of exposure But there was no direct measure of the amount of exposure or dose Hence variation in degree of exposure across sites and within the population around any single site were not taken into account And it would not be feasible to attempt to develop direct measures of exposure for the large number of sites affected by Superfund over the past 25 years The third reason is difficulties in obtaining values for many of the health endpoints in question The Report correctly notes that the direct medical costs as estimated in the EPA Cost of Illness Handbook (2002) represent a lower bound on the true social cost of illness There are both revealed preference and stated preference methods for estimating the willingness to pay to avoid morbidity effects (See Freeman (2003), Dickie (2003), or EPA's Handbook for Non- cancer Health Effects Valuation (2000)) But the Panel is not aware of empirical estimates based on these methods for most of the health effects of interest here The Panel notes that the EPA Cost of Illness Handbook does not give cost of illness data for accidents and injuries or chronic non-cancer effects The Panel does not know whether there are cost of illness data for these effects from other sources The Panel is also skeptical of the use of the Hamilton and Viscusi (1999) study (H&V hereafter) to obtain estimates of the numbers of cancer cases avoided H&V studied a nonrandom sample of 150 sites on the NPL where RODs were signed during 1991-2 Rather than the upper-bound estimates used by EPA, H&V used mean values for ingestion rate, exposure duration, and chemical concentration to estimate individual and population risks of cancer for each site H&V combined the estimates of population risk with data on populations within 1 mile of each site to estimate the numbers of cancer cases over an assumed 30 year time horizon On the assumption that these excess cancer cases would be avoided with site remediation, this could be the basis for an estimate the benefits of Superfund remediation at these site There are two problems with using H&V data to estimate the benefits for all sites covered by the remediation program First, it is not clear that the estimate for the 150 sites in 11 ------- H&V can be the basis for extrapolation to the universe of sites covered by the Program H&V report that almost 90% of the predicted cancer cases in the sample came from one site, indicating a high degree of variability across sites Second, H&V used the Agency's estimates of cancer risk factors in their own calculation But at least for those chemicals where the risk factor is based on animal test data rather than epidemiology, the risk factors are 95% upper confidence levels rather than maximum likelihood values Thus an estimate of cancers avoided based on H&V would still be biased upward perhaps by as much as an order of magnitude Although the Panel believes estimates of the aggregate health benefits of Superfund are not possible due to data and methodological gaps, it is possible to present illustrative calculations of some important components of the benefits to human health For example, the Panel recommends that Lybarger, et al's (1998) estimates of reductions in several categories of health effects associated with exposures to VOCs be described Their estimates of the reductions in the costs of these illnesses should also be described along with the comment that cost-of-illness is an underestimate of the true social value of reduced adverse health effects Some authors have reported evidence that true social values for some health effects appear to be several times the direct cost-of-illness avoided (see Dickie, 2003, p 439, and Albenni and Krupnick, 2000) The Panel suggest that the Agency present a sensitivity analysis of the Lybarger, et al results based on this evidence The Panel also recommends that the Agency consider an analysis of the benefits related to reduced lead exposure In this regard, it would be helpful to use a model to quantify the full range of toxic effects that may result from exposure to lead, including cognitive changes, behavioral changes that may produce increased rates of criminality, drug abuse, and incarceration, and cardiovascular disease and stroke related to elevated blood pressure in adults An example of this approach is available in Landngan (2002). See also the lead benefits assessment model used by EPA in its Retrospective Benefit Cost Analysis of the Clean Air Act (EPA, 1997) These two approaches should be included as examples of estimates of benefits for a few of the many chemicals of concern The SBA might explain that due to lack of adequate data for many chemicals and for exposures to individuals residing near Superfund sites, a complete economic benefits analysis is not possible Also, support by Superfund of the planned "National Children's Study" and of oversampling of populations at strategic Superfund locations will aid in developing more complete future analysis 3.4. Ecological Benefits The Agency proposed to define the ecological benefits associated with restoration at Superfund sites as the decrease in the discounted present value of interim lost use value brought about by restoration compared to the counterfactual scenarios either natural recovery or no 12 ------- recovery (as appropriate) Under the law, interim lost use value includes passive use value (or what is sometimes called nonuse or existence value) This is a reasonable definition of ecological benefits To make use of this definition, the Agency needs to have an estimate of interim lost use value for each site at the time that the restoration action begins, as well as estimates of the time paths of interim lost use value under natural recovery and under active restoration To obtain estimates of interim lost use value the Agency proposes first to obtain the dollar values of the natural resource damage assessment (NRDA) settlements for those approximately 130 sites (including 70 NPL sites) where settlements have occurred For some of these sites, the Trustee's estimates of interim lost use value can be obtained from the NRDA documents For these cases, the Agency proposes to determine the relationship between the NRDA estimate of interim lost use value and the settlement amount For those sites for which interim lost use value is not available, the Agency would use the ratio of interim lost use value to the settlement amount to calculate an estimate of interim lost use value as a percentage of the known settlement amount The Panel applauds the Agency's effort to find a way to include ecological benefits in its accounting of the benefits of Superfund But the Panel doubts that what is proposed here will result in defensible estimates of the aggregate ecological benefits of the Superfund program At this time there are relatively few NRD sites for which settlement has been achieved Interim lost use value as a percentage of the settlement is probably not constant across sites NRD settlements - like any legal settlement - are products of negotiation, or court order, as much as they are the products of calculation or analysis Also, trustees are authorized by law to include restoration costs and replacement costs of lost resources in their claims Replacement cost can not be considered as a proxy for interim lost use values (for example, Freeman, 2003) NRD remedies often include dollar claims for other remedies besides restoration of damaged resources For instance, settlements may be used to construct trails, docks, or other facilities to compensate for losses Thus, the dollar value of an NRD settlement is not, and should not be construed as the "value" of ecological impacts Finally, the Agency has not identified a way to determine which of those sites for which settlements have not already occurred can be expected to have ecological benefits If the Agency opts out of pursuing a comprehensive estimate of Superfund's aggregate benefits, the Panel recommend the following as ways to provide illustrative and qualitative information on the potential ecological benefits of the Program The Panel recommends that the SBA more fully describe the various ecological consequences of cleanup and removal and then translate those into descriptions of beneficial effects that the public can understand This will be a qualitative exercise, but it is a way to convey the range of improvements to well-being that can result from site cleanups In addition, depending on the resources and time available, the Agency could conduct quantitative, but non-monetary assessments of ecological benefits This would involve the 13 ------- development of ecological benefit indicators This method was described and recommended in the report of the Panel to Examine Benefits, Costs, & Impacts to the Underground Storage Tanks (UST) and Resource Conservation Recovery Act (RCRA) Subtitle C Program in 2002 (EPA SAB, 2002) 3.5. Ground Water Protection Benefits The Agency proposes to quantify "the amount of ground water protected by Superfund (p 5-33)," and to use benefits transfer to estimate the monetary value of ground water protection The Superfund Program can affect ground water quality and yield benefits through three channels 1 Restoration of the quality of contaminated ground water through remediation, 2 Clean up of sites so as to prevent contaminants from migrating from the sites into ground water resources, and 3 Deterrence of poor disposal practices so as to prevent the future contamination of ground water Only the second and third channels can be characterized as "protecting" ground water The discussion of the areas of NPL sites with contaminated ground water (p 5-38) suggests that its focus is on the first two of these channels But a sentence on the next page of the SBA suggests that the concern is with the third channel1 The Agency needs to clarify which of these channels is being discussed The Panel believes that it would be possible to obtain a ball park estimate of the quantity of ground water that is affected by Superfund through the first channel However, this would require an examination of the conceptual models for each site individually to see how much cleanup has occurred in three dimensions For a given site, the cleanup criteria for that site may differ It is strongly suggested that the Agency use more carefully crafted case studies showing Superfund's impact on groundwater There are a number of case studies that can be used to estimate the quantity of ground water cleaned up, for example, the Fairchild Semiconductor Case, the Ft Devons Case, the Industri-Plex Case The Panel does not know of any reliable ways to estimate the quantities of ground water affected through the second and third channels above 'Quoting from the SBA, "Many of these areas have been controlled or reversed through Superfund response actions, and there may be some sites where removal actions or state actions may have prevented potential ground water contamination (p 5-38) " And, "It might be possible to estimate the amount of ground water that will not be contaminated because of Superfund, but would have been in the baseline case where no Superfund Program had ever come into being (p 5-39) 14 ------- Regarding the valuation of protected groundwaters, the Agency proposes to do " a meta-analysis of individual studies to get a range of willingness to pay for ground water quality, and possibly placing states into groups based on relevant metrics " Based on our examination of the 13 studies listed in Table 5 6 of the SBA, there is not at the present time an adequate basis for doing a meta-analysis or benefits transfer As the SBA points out (p 5-34), two of the studies cited in Table 5 6 cast doubt on the feasibility of benefits transfer And 7 of the 13 studies are about either surface water or nitrate contamination of ground water, making them of questionable relevance for the purpose of valuing Superfund 15 ------- 4.0 Responses to Specific Charge Questions Section 4 provides responses to the specific charge questions not addressed in Section 3 of this Advisory 4.1. Charge Question 1: Chapter 1 provides a framework for capturing the benefits of the Superfund program in the Superfund Benefits Analysis (SBA). Chapter 2 reviews the literature relevant to the SBA; and Chapter 3 describes the structure of the Superfund program. Please comment on the adequacy and appropriateness of these introductory chapters as a foundation for the SBA If the Agency chooses to go forward with this report, the Panel recommends that Chapter 1 receive major revisions The chapter does a nice job of introducing the reader to the Superfund program, but the language in the first few pages reads more like a public relations document than an objective description of the program In addition, the Panel recommends a revision of a few misstatements about the program, such as the statement (p 1-4) that many of the worst sites are now addressed by state programs and that Superfund addresses "abandoned" hazardous waste sites (p 1-7) Chapter 1 should have two primary goals (1) to lay out a framework for the report and (2) to provide a coherent framework for thinking about the benefits of the Superfund program These two goals are not independent, however, since the framework for thinking about benefits should in turn inform the framework for the report In terms of the first goal, the Panel recommends that the introduction provide a motivation for the report, a discussion of the analytical approach taken, any critical caveats, and a roadmap to the remainder of the report The introduction could provide a clearer statement of purpose of the report and could be more clearly organized to distinguish the following components roadmap for the report, overview of the Superfund program, definition of how the term "benefit" is used in the report, description of benefits, and methodology or approach for estimating benefits The Panel did not find the delineation of either "approaches" or "benefits" in Figure 1 1 very useful, and the mapping between the two was unclear Likewise, the Panel did not find the distinction between fundamental and embedded benefits, or their definitions, to be very helpful as an organizing principle For example, why is "community involvement" a separate approach (rather than part of "response") and why is "empowerment" per se a benefit category (rather than a means toward an end, namely, better outcomes)7 Why is reduced uncertainty about the nature and extent of the actual health risks associated with releases considered part of the amenities 16 ------- benefit, and why aren't deterrence and emergency preparedness simply means toward an end (reduced damages)7 Much of Chapter 1 appears to follow EPA's Guidelines for Preparing Economic Analyses (2000) However, the central focus on this guidance detracts rather than adds to this section, and hence the Panel recommends eliminating the discussion of this guidance This guidance was developed for prospective analyses, not retrospective ones, and is probably not even appropriate for this kind of study For example, starting a section on "Problem Definition" after already discussing Superfund and introducing a basic approach to be taken (through Figure 1 1) seems backwards Likewise, in the context of this retrospective analysis, which considers only benefits and not costs, the section on "Reasons for Market Failure and the Need for Federal Action" adds little The chapter then goes on to define Superfund approaches and Superfund benefits, but this is several sections after these concepts have already been introduced as a basic organizational framework for the report in Figure 1 1 The section on "Methodology" is a combination of a discussion of methods and a discussion of some benefit categories For example, the paragraphs on p 1-17 say very little, if anything, about methodologies that will be used in the assessment It is useful to have a mapping between benefits and methods, but the Panel generally found the attempt to do this through Figure 1 2 confusing The literature review in Chapter 2 is a thorough review of the literature that it covers However, much of the paper-by-paper description is tedious and might be better put in an appendix The Panel recommends that the text focus on general conclusions from the literature, rather than a detailed account of individual studies, with the purpose of identifying gaps in our understanding In addition, Chapter 2 omits some relevant literature For example, the chapter could include the literature on methods of benefit estimation as well as on previous studies of the Superfund program There is little in this chapter, or in the rest of the report, about the promises and pitfalls of the hedonics approach for captunng benefits, nor on some of the other approaches Another example is the recent literature on ecosystem valuation (e g, NRC report, Millenium Assessment) This literature provides both a logical framework for thinking about benefits assessment of ecosystem services, and reviews of the current state of knowledge in this area A third neglected literature relates to uncertainty The Panel recommends a greater elaboration of the uncertainty inherent in the estimation of the benefits of Superfund and discussion of how uncertainty can and should be treated The Panel recommends the case studies be more closely tied to text or eliminated They are well-written but not well integrated into the report or used as support for the text Chapter 3 of the SBA covers two distinct topics (1) a description of the Superfund program (pages 3-1 through 3-25) and (2) methods for characterizing NPL boundaries and 17 ------- potentially affected populations for use in subsequent chapters (pages 3-26 through 3-49) The latter topic is covered in the Panel's responses to Charge questions 2a - 2d The Panel's discussion of the first part of Chapter 3 centered on the three issues discussed below 1 the link between the problem definition in Chapter 1 and the description of Superfund responses in Chapter 3 With regard to the first point, Chapter 1 casts a broad net over CERCLA benefits that includes not only the response sections of the Superfund program, but also lists the following approaches community involvement, enforcement, research and development, training and natural resource damage assessments Yet, the Description of Superfund Responses in Chapter 3 deals principally with Response Actions, and does so without being clear as to its overall role in the benefits analysis In addition, while a description of the Superfund program is of course useful to this report, it is unclear if the reader needs, for example, to know the details of the site screening process and the remedy selection process in order to understand and estimate the benefits of the program As mentioned in the comments on Chapter 1, the core focus of the SB A is on "benefits," and the Panel recommends the rest of the report provide the information needed for that purpose The reader needs to understand CERCLA's basic authorities and goals, and something about how the program functions, but whether it is necessary to describe the remedial process in Chapter 3 in such detail is unclear It would be more useful to discuss what the law requires in terms of protection of public health and the environment, a very brief overview of the removal, remedial and enforcement program, summary data on the number of different kinds of actions, information on the evolution of the program over time, program accomplishments to date, and the heterogeneity among NPL sites, specifically noting how Federal Facilities are different from other NPL sites, and perhaps also discussing what kinds of sites are on the NPL 2 the representation of the Superfund process With regard to the second issue, the Panel believes that the overall description is a reasonable representation of the Superfund process However, correcting a number of statements throughout this chapter would improve the report Some examples are the discussion of EPA enforcement does not seem fully cognizant of the critical role that settlements play, the descnption of state capabilities is not accurate, there is no mention of the fact that Superfund liability is retroactive, and the text incorrectly states that sites must be on the NPL for the liability scheme to be invoked Also, citations are needed for many of the statements in this chapter For example, the Remedial Investigation and Feasibility Study process is described on pages 3-12 and 3-13 of the report A good reference document for this section is EPA's 1988 Guidance for Conducting Remedial Investigations and Feasibility Studies under CERCLA In addition, the discussion of the CERCLA Remedy Selection Criteria (EPA 1988) is given on page 18 ------- 3-13, and the SBA goes on to state under Selection of Remedy that the "FS identifies the best response options" More accurately, the Feasibility Study evaluates a range of alternatives and compares them to the first seven selection criteria The criteria of State and Community Acceptance are evaluated by the EPA, in consultation with stakeholder (states, Tribes, U S Fish and Wildlife Service), public input, and in consideration of environmental justice issues (1997a, 1999) More attention might be given to the role of risk assessment in the remedy-decision process Ultimately, all remedies are based to a large part upon protection of human health and the environment, so the ties between the Human Health and Ecological Risk Assessment processes (EPA 1989,1997b) need to be better explained and documented Substantively, the Panel's greatest concern related to the description of the roles of the states While it is true that CERCLA requires states to pay for 10% of fund-lead (that is, actions paid for by EPA, rather than by responsible parties) and 100% of operations and maintenance for these actions, it is not clear that most states would say that CERCLA provides a 'substantial role" for states When states do carry out actions, all decision-making powers still are with the EPA, and not the states At a more general level, the discussion of state funding and capabilities is not consistent with the research of others about state capabilities and capacity for NPL-level cleanups More weight is given in the SBA to the report from the Environmental Council of the States, which is not specifically about Superfund, than the Environmental Law Institute and Resources for the Future research that is more focused on cleanup activity The assumption that 25% of state cleanups are paid for with federal dollars seems highly speculative Also, the Panel would recommend against including a chart that conflates all Superfund actions Figure 3 4 aggregates state and EPA actions, and removal and remedial actions, in a single chart These are truly apples and oranges in terms of their costs and their accomplishments Also, the ELI report must be examined quite carefully as it includes cleanups under a variety of programs, not just Superfund The Panel strongly recommends taking proposed NPL sites out of the statistics (would note that the RFF 2001 Report includes a site by site description of the status of NPL "proposed" sites at that time), and separating federal facility sites from other NPL sites As an example, the SBA makes mention of the Lower Fox River site in Wisconsin as an NPL site, that site is nominated, but not listed The Lower Fox River remains a Wisconsin state-lead project It is true that the EPA is an active participant, and that some benefits may be construed from the Agency's participation, but the distinction needs to be clear 3 Whether the appropriate Superfund responses were used in subsequent sections to characterize Superfund benefits 19 ------- The Panel discussed whether the appropriate Superfund responses were used in subsequent sections to characterize and quantify Superfund benefits The majority of the SBA focuses on actions at NPL sites There are two concerns regarding this decision First, NPL sites are anything but homogeneous To the extent that the study relies on a subset of sites it is critical to assess how this subset represents the full NPL, and what types of sites should be analyzed in separate categories These categories, we would note, might be different for different types of benefits analyses (e g for the hedonic study vs the cost of illness study ) Still, it seems clear that, at a minimum, federal facilities should be kept separate from non-federal facility sites In addition, it is important to acknowledge that a hedonics approach to estimate the benefits to properties that are near sites that have simultaneously been the focus of redevelopment would capture not only the benefit of cleanup, but also the benefits of redevelopment investments and efforts at the site as well And these could be significant There are many sites where cleanup and redevelopment have been undertaken hand-in-hand in the Superfund program As an example, one of the most well-known of these sites that has been touted by EPA is the Industn-Plex site in Woburn, MA At this site, which was one of the most valuable undeveloped pieces of land in the area, EPA, along with local and state government officials, developed a reuse plan along side of the cleanup plan In fact, reuse in many ways drove the cleanup approach The site was the beneficiary of funding that came from many sources other than EPA and the PRPs to finance certain aspects of the plan that related to reuse Thus the final state of the site and its value are not only result of cleanup but also of the additional redevelopment And it is likely that this redevelopment has enhanced the values of nearby sites as well This is a critical issue that it would be helpful to address in the SBA And, to the extent that any "redevelopment" sites are included in the study sites in the hedonic approach, this should be noted Finally, a more troubling concern is the fact that while the authors note that removal actions may well result in much of the decrease in current risk, these actions appear to be ignored in the remainder of the report One small but important point is that some percentage of RODs are what are called "no action RODs" meaning that EPA determines that no remedy is needed at the site A second important point about RODs, in terms of assessing site progress, is that a larger percentage of NPL sites have more than one remedy, and more than one ROD While this is mentioned, it is unclear how this is addressed in terms of assessing site progress The Panel notes that it would be worth finding out what number or percentage of deleted and construction complete sites are "no action ROD" sites Defining the universe of sites as ROD sites (including no-action RODS) may be misleading in a retrospective study 20 ------- 4.2 Charge Question 2: The latter part of Chapter 3 discusses the data used for the SBA. With regard to this data discussion, please address the following. Charge Question 2a. The lack of NPL site boundary information makes it necessary to estimate the numbers of nearby residents and homes at various distances from NPL sites; these are needed for the analyses in Chapters 4 and 5. Is the use of circular areas based on site size, as illustrated in Figures 3.5-3.8, an adequate approach? Yes, this is an adequate approach Of course, this measure ignores the possibility that risk varies across sites, but there are no easy ways to account for differential risk At any rate, the authors do not have any information about contamination pathways, contaminant plumes, etc The report does sound GIS work on mapping the sites and the boundaries To our knowledge, the use of circular buffers is widespread in GIS-based type of studies in the absence of more detailed information about property boundaries This is probably sufficient for the purposes of the report, and it would seem reasonable if the site is small relative to the 2 5-mile or 5-mile buffer As a related thought, we note from Figure 3 10 that most of the exposed U S population lives near a small fraction of the sites Given that the benefit analysis is based on a limited number of sites, it isn't possible to calculate benefit estimates by population density However, Figure 3 10 can be improved (and some clarification can be added to the text of the report) descnbing how population is concentrated around a limited subset of sites Charge Question 2b. Is it appropriate to ignore proximity to more than one NPL site? What methods might be used to account for effects associated with proximity to two or more NPL sites? If one wishes to develop a measure of exposure to contaminants, it seems possible that a receptor could be exposed to pollution coming from more than one site Accounting for this, however, requires extensive information about pollution plumes and pathways, which is not possible within the scope of this study We conclude that it is acceptable to ignore proximity to more than one NPL site because there is no clear means to account for multiple sites An alternative measure was used in Gayer, Hamilton, and Viscusi (2000, 2002), which aggregated the lifetime excess cancer nsk estimates 21 ------- of the neighborhood sites This would be prohibitively difficult to replicate on a nation-wide scale, and would not be amenable to a benefits-transfer estimate using the distance gradients2 Charge Question 2c. Are the correct inferences about comparability between the NPL site groups in Table 3.3 correctly drawn? Are there other groups that it might be useful to define and analyze? The Panel isn't clear on the meaning of this question A retrospective study of benefits should include only the sites in which remediation is completed, especially given the mixed evidence that prices fully recover from Superfund actions Many of the ROD sites are not completed, so they should not be included in a benefits estimate that assumes full price recovery (or, at the very least, the benefits of unfinished sites should be discounted) What's more, some RODs recommend no-action, which suggests no benefit of remediation It may be useful to show benefits by different types of sites (by population density, by remediation strategy, by removal vs remediation, etc ), but this is not possible given the benefits-transfer method used in this study Turning to the attributes of the sites in the various groups of Table 3 3, it is reasonable to expect that the proportion of construction completed ROD sites is higher than the entire NPL universe because the construction of the remedies on site will be initiated and completed only after its ROD has been issued That a lower percentage of federal sites has CC complete is consistent with the idea that federal sites are large and complex H & V chose sites with well- developed ROD and contamination characterizations, as these were needed to develop the data The hedonic property method (HPM) sites probably looked at complex sites The authors of the SBA are correct in pointing out that population density in the HPM group of sites is higher than in the other group This is consistent with high population densities and numerous homes being sold at a continuum of distances from the site, as one would typically want when doing an HPM study The average size of NPL sites tend to be smaller in the HPM studies, but the median in this group is perfectly aligned with that of the NPL, ROD, MROD Federal sites are very large and HV tend to be smaller 2 Regarding proximity to the site in hedonic property models (HPMs), most of the previous studies have looked at the distance to the nearest Superfund site, ignoring the presence of others Ihlanfeldt and Taylor (2004) have also looked at the second-closest site, which became important when they calculated the TIF revenue afforded by the cleanup (which changed the value of a property) It should be borne in mind, however, almost all of the sites considered by these authors at their study locale (Fulton County, Georgia) are CERCLIS, not NPL, sites 22 ------- Other possible groups could be formed on the basis of the type of contaminant, the contaminated media (e g, groundwater), or of the possible involvement of the potentially responsible parties (PRPs) More information on the characteristics of the study sites is needed in the SBA, as well as the sites in each of the groups listed in Table 3 3 To the extent that the report relies on the estimates of the study sites, it is essential to get a sense of how representative those sites are of the universe of sites Charge Question 2d. Is it appropriate to assume a uniform distribution of populations and residences across census blocks? What other approaches could be taken? We agree that the approach is appropriate, especially given the relatively refined measure of census blocks Below are two additional points regarding Charge Question 2 1 The use of 2 5-mile rings seems reasonable, but it would be good to provide some more support for this decision Many studies estimate a price gradient, but don't assess the distance at which price effects go to zero As a result, many studies arbitrarily assume a distance in which the price effect goes to zero, or they choose a distance based on the data they have To the extent that the hedonic analysis moves forward, it would be useful to discuss this issue in more detail 2 There is a possible benefits-transfer problem because the hedonic property studies used in the meta-analysis are located in more populated areas than the typical NPL site Larger populations mean more and larger housing markets, which could affect the gradient estimate Riechert, Small, and Mohanty (1992) find some evidence property values in rural areas are not responsive to distance to a landfill Similarly, the studies used in the meta-analysis take longer to clean up, suggesting that they are more problematic (perhaps higher risk) More generally, a benefits transfer problem will exist to the extent that the study sites are not representative of the full population of sites This is one the Panel's reservations about using the meta-hedonic approach 4.3 Charge Question 3: Chapter 4 presents a benefits transfer analysis applied to all those NPL sites where the benefits are expected to occur from 1980 - 2004. With regard to this chapter, please address the following. Charge Question 3a. Are the challenges associated with the benefits methodology as applied to hedonic price studies in Chapter 4 satisfactorily met? 23 ------- In addition to the comments related to this question in Section 3 2, we offer the following - Using ROD sites This chapter should provide more information in which to assess whether the sites from the hedonic studies are representative of the universe of sites The SBA misleadingly defines the universe of sites as ROD sites (including no-action RODS), which seems inappropriate for a retrospective study The extent of these problems depends on whether the report moves forward with the current meta-hedonic analysis If the hedonic analysis is to be de-emphasized, then chapter 3 can be re-written to provide much more information about each of the types of sites listed in Table 3 3, as a means of explaining the different type of remedial work that Superfund does But with the hedonic analysis de-emphasized, it would no longer be necessary to show that the hedonic sites are representative of the universe of sites (however defined) Also, the Panel has a concern with using all ROD sites as the basis for the benefits estimate Many of the ROD sites are not yet cleaned up, so the uncertainty of whether they will happen and the discounted benefits of the delay would reduce the estimate of retrospective benefits of Superfund - Omitted-variable bias Omitted-vanable bias is potentially a big problem with hedonic property studies and needs to be mentioned The NPL sites are likely located in unattractive areas, so the price-distance gradient could be picking up un-measurable characteristics of the neighborhood Atkinson and Crocker (1987) and Graves et al (1988) both find evidence of a problem with omitted variables This problem would be especially acute for the cross-sectional studies used in the meta-analysis The Greenstone and Gallagher paper provides a research design that could potentially address this problem - Inferences about infra-marginal changes The report should be more explicit about the difficulty of using estimates of the hedonic price function to make inferences about mfra- marginal changes - Market size and market segmentation Considerable space is dedicated to discussing the issue of the size of the market, and of whether there are separate housing markets Unfortunately, the criteria used to identify the size of the market (40% of the homes in Middlesex Co are within 2 5 miles of the NPL site, commuting times) are unconvincing The discussion on page 4-4 does not clearly address concerns about appropriate market size and segmentation - Also, it is unclear why using a single market would underestimate benefits This might be the case with Michaels and Smith (1990), but this claim may not apply to all hedonic property studies To elaborate on this, consider the claim made by the authors that if the disamemty 24 ------- impacts are stronger on the high-end homes, then single-pooled hedonic regressions will likely give underestimates of the effects It seems to us that the effect estimated from single-pooled data would be a weighted average of the two effects, which may or may not be an underestimate, depending on the specific market Charge Question 3b. Both Circular A-4 (Office of Management and Budget 2003 pp. 24-26) and the recent comments on the Underground Storage Tanks (UST) Cleanup & Resource Conservation & Recovery Act (RCRA) Subtitle C Program Benefits, Costs, & Impacts (BCI) Assessments: An SAB Advisory (Science Advisory Board 2002 pp. 20-22) contain specific comments associated with the methodology used in Chapter 4. Have these comments been addressed adequately? In light of these comments, has the benefits transfer methodology been applied correctly? This is difficult to answer, since Circular A-4 is meant to guide prospective studies of proposed regulations rather than retrospective studies As such, Circular A-4 places much emphasis on the importance of justifying the need for the proposed regulation and showing that it is the best alternative None of this applies when considenng retrospective benefits Circular A-4 also stresses that both benefits and costs should be evaluated where possible, whereas the SBA is concerned only with benefits While Circular A-4 offers useful guidance to agencies in their development of proposed regulations, it is less useful for the task at hand in the current retrospective study Nonetheless, Circular A-4 does provide some guidance on benefit-cost analysis that might offer some insight for this study It stresses the importance of identifying a baseline in order to assess projected benefits, of providing transparent and reproducible results, and it list potential pitfalls of benefit-transfer studies In Section 3 of this report, as well as in the remainder of charge question 3, we discuss our thoughts and concerns with how this study addresses these issues This study does follow some of the guidelines of Circular A-4, such as the need to examine the market failure that justifies the regulation, the potential advantage of using a revealed preference method for assessing benefits, and the consideration of ancillary benefits Regarding the comments on the hedonic methodology in the SAB Advisory (Science Advisory Board, 2002), this question is addressed in our response to charge question 3g Charge Question 3c. Are the estimates of the price effect (Figures 4.3 and 4.4 and Tables 4.4 and 4.5) based on the best available data and a sound methodology? As discussed earlier, the selection of studies needs better documentation and justification More specifically, the literature review produced a total of 30 studies, including book chapters, reports, and journal articles In the end, only 9 of them are used for the benefit transfer The 25 ------- authors must explicitly discuss the reasons why the others were dropped and these were selected Additional considerations include - Was a subjective assessment of the quality of the study ever a consideration in the selection of these nine studies'? None of these studies control, for example, for whether the homes in an area that are sold are representative of those that did not sell In other words, none ask the question whether proximity to the site altered the frequency at which homes are sold, in addition to the sale price Moreover, only two of the studies in the sample of nine use panel data, in spite of the accepted notion that cross-sectional studies are inadequate in this type of analysis - Nine is a very small sample, perhaps too small for extrapolation Perhaps the results based on this sample could be compared with those from a broader sample, where the studies selected by the authors are supplemented with others, even if the latter do not focus on NPL sites, as long as a comparable announcement event is identified - Table 4 2 is unclear What is the "absolute effect" in column 2"? What distance does this price effect refer to7 - Do the studies' sites differ for type of contaminated site, type of contaminant, PRP participation and state government involvement1? More information is needed here - Most of the studies used in this chapter are old and refer to early NPL sites Would they be still suitable for sites more recently nominated to the NPL, which have been described as being larger and more complex than earlier sites'? This raises an issue of temporal stability of the benefits/benefit transfer - Table 4 4 is unclear - Once the selected studies are used to estimate an appreciation effect associated with the issue of the ROD, such appreciation is aggregated over the housing units within the specified distance of each Superfund site Doing so, however, in some cases assumes that the size and structural characteristics of the dwellings are similar across locales Is it possible to obtain information about the typical housing size at the various locales, perhaps using the Census or the American Housing Survey, express prices and appreciation on a per square foot basis, and then multiply the latter by the size of the typical dwelling in the vicinity of each Superfund site to get a better sense of the total benefits'? 26 ------- In sum, more information is needed on both the selection criteria and the studies (See 3a above for concerns about inferences drawn from the studies) It is important that the assumed causal link between the issuance of the ROD and a rebound effect be clearly detailed Charge Question 3d. Does the application of the price effect to estimate the value of Remedial Actions at NPL sites match the relevant theoretical principles? In particular, does the analysis of reversals of the negative price effect found in the literature (and associated with Figure 4.3) provide a sound basis for assuming that reversals always occur? The Panel has some problems with the inferences, and questions using full reversals of negative price effects as a basis for the benefits estimate See all the points listed under 3a and under 3c Charge Question 3e. Are the four model specifications (Equations 4.2 through 4.5) appropriate? In particular, these models calibrate the basic benefits transfer model (Equation 4.2) for non-linear effects and/or differences in home values. Are these calibrations appropriate? Are there other calibrations that could be made to improve the estimate? Yes, they seem to be correct Charge Question 3f. Are the study sites reasonably representative of the policy sites? The Panel's answer to this charge question sounds common themes with that of charge question 3c The report needs to provide a more informative discussion on how the 9 studies were chosen and why the other 21 were dropped out of the initial sample of 30 In addition, Gayer et al (2002) seems more appropriate for this report than does Gayer et al (2000) Finally, as to whether the study sites are reasonably representative of the NPL sites, the SBA does not contain enough information about locations, chemicals and pathways, PRP involvement, community involvement, role of the State and the EPA, etc to allow us to conclude whether the study sites are or are not representative of the universe If housing price depreciation/appreciation is affected by these factors (which we also do not know), it is unclear how the benefit transfer can be applied to the universe Charge Question 3g. Although there are no federal facilities among the study sites, federal facilities (e.g., the Camp Pendleton and Savannah River sites) are included among the policy sites. The analysis of NPL site groups in Chapter 3 suggests there 27 ------- may be no significant differences in relevant characteristics (e.g., nearby population, cost of nearby homes, and the price effect). Should federal sites be included in this analysis or not? If so, how? Can estimates both with and without federal sites be included, and which one does the Panel think would be more reliable? Earlier evidence suggests that federal sites take longer than non-federal sites to transition through the different phases of the Superfund program In many cases, we would expect them to be large sites with complex pollution problems and heavy "dread" effects (radioactive waste, nuclear plants, etc) Absent studies that specifically looked at the price effects of the proximity to federal NPL sites, or that at least controlled more carefully for the type of contamination, the assumption that the depreciation/appreciation mechanism associated with the ROD applies to federal sites goes a bit too far In addition, two other points are offered Regarding the question of "Market Size" (pp 4-4 to 4-6), the concern of the earlier SAB Panel was whether or not the hedonic price function (HPF) would shift as a result of a change in the vector of characteristics If it does, then using the sum of the predicted changes in prices would lead to a biased estimate of the welfare change The SBA is not clear on this point Also, there are no guidelines for determining in advance whether to expect the HPF to shift So the discussion in the SBA that this will not be a problem isn't fully persuasive Second, regarding "Ex Ante Data" pp 4-6 to 4-7), the SBA misstates the Bartik conclusion What Bartik wrote was, " the WTP for the improvements of households originally at improved sites will underestimate benefits (Bartik, 1988, p 176, emphasis added) " 4.4 Charge Question 4: Chapter 5 presents a proposed analysis for capture health effects of Superfund. Should the health efTect-by-effect analysis proposed in Chapter 5 proceed? Charge Question 4a. Is the epidemiology-based approach adapted from Lybarger et al. (1998) appropriate? If so, does the Panel have any specific recommendations for implementing it? See Section 3 3 Charge Question 4b. Are there other feasible methods for monetizing the value of avoided morbidity besides Cost of Illness? Of these other methods, how can they be prioritized (i.e. which ones should EPA investigate first)? See Section 3 3 28 ------- Charge Question 4c. Have the content and limitations of the literature on the epidemiology of hazardous substances in the environment and related material been described adequately? The review does an adequatejob of delineating the limitations of the epidemiologic literature In view of the limited exposure data, the lack of the ability to address additive or synergistic exposures, and limitations of sample size related to the sometimes small populations involved, it should be pointed out that it is remarkable that some studies have found elevated (and sometimes statistically significant) risks Charge Question 4d. What recommendations does the Panel have for using the Integrated Exposure Uptake Biokinetic model for lead? The EPA Integrated Exposure Uptake Biokinetic (IEUBK) model is proposed for use in this analysis A modeling approach to evaluate lead blood levels was initiated in 1985 and has been developed to its present state as the IEUBK model The current model provides many advantages over the previous explicit mathematical methods used for estimating the potential for adverse health risks as a result of exposures to lead However, the model has definite limitations that if violated will potentially jeopardize the accuracy of predictions provided by simulations The IEUBK model for children exposed to lead was developed for ages 0 to 84 months that may potentially be exposed through a range of possible pathways It is a probability based model rather than a deterministic model which has the advantage of providing estimates of outcomes within a range of conditions Long term exposures are used to estimate a geometric mean blood level for the exposed children This model was developed for individual estimates, but can be used to evaluate neighborhood exposures to predict blood lead levels The model can be used at several scales including a single location or a neighborhood Single locations, such as a single dwelling, are used to estimate exposures for a single child Multiple locations within a single neighborhood that has homogeneous media or a heterogeneous media are two other scales This can be extended to more than one neighborhood with heterogeneous media The multiple locations are appropriate for exposure of a population of children Advantages of the IEUBK model include a predictive capability to estimate blood lead levels and evaluate effects of efforts to reduce exposures The deterministic models rely on slope factors that are not universal constants Slope factors change due to differences in uptake, site characteristics, among other conditions The IEUBK model allows for multiple media exposure and multiple pathways Model simulations can be run to evaluate the effect of mitigation strategies to reduce risks Isolation of key pathways of exposure can be used to guide more effective remediation strategies and set clean up targets Individual or neighborhood blood 29 ------- levels can be predicted as a consequence of remediation alternatives Such an approach can reduce removal and remediation costs associated with lead contamination There are significant limitations to the model, notably that the model was developed for children and most childhood exposures to lead are the result of household exposures Any model is only as good as the data available and assumptions made in its execution and the IEUBK model is no different There are few pathways through which such young children would be exposed to NPL sites However, the model has been adapted for adult lead exposures and this development has potential, especially related to fetal exposures to lead which are known to potentially have significant adverse health effects If the model is applied within its limitations, the error of the analysis can be determined which provides an analysis of associated uncertainty The IEUBK model would be useful in the SBA since it offers probabilistic based estimates which provide a more realistic prediction of potential outcomes due to exposures Charge Question 4e. What recommendations does the Panel have for using the results from "Calculating Risks?" for estimating the benefit of avoided adult cancers? See the Panel's discussion in Section 3 3 of this Advisory In addition, the following comments are offered Hamilton and Viscusi develop the thesis that the current risk assessment practice is overly conservative, providing quantitative evidence to support this assertion They focus on the effect that parameters have on the estimated risks according to the following equation ED, x EF. x IR LECR"--i bw;at ^cc^tox' where ED is exposure duration, EF is exposure frequency, IR is the ingestion rate, 1 is the contaminant, j is the pathway, AT is the averaging time, BW is the body weight, CC is the contaminant concentration, and TOX is the toxicity By varying the values for the 'constants' the estimated lifetime excess cancer risk (LECR) can vary several orders of magnitude They argue that using the reasonable maximum exposure (RME) is overly conservative They recommend using mean or median values for CC instead of the RME They further argue that EPA recommended ED, EF, and IR default values exceed those observed at several sites They evaluate the use of probabilistic approaches to estimates of risk 30 ------- They do not address the TOX values and how these can vary The uncertainty of this parameter can be significant This value can be adjusted by uncertainty factors (UF) or modifying factors (MF) that reflect uncertainties in extrapolating toxicity values determined for different species, between organisms within the same species, among several other adjustments These adjustments could surpass those of the uncertainty in the constants factors The equation must be considered in its entirety The magnitude of variability of all parameters must be evaluated The value of this approach is to adopt more widely a probability based risk assessment approach This approach will result in estimates that reflect the range imposed by the uncertainties in the approximation Such results can help identify those parameters most responsible for creating the uncertainty 4.5 Charge Question 5: Chapter 5 also presents a proposed method for capturing the ecological benefits of Superfund. Should the ecological benefit analyses proposed in Chapter 5 proceed? First the Panel would like to comment on a central premise that runs through the report - that the ecological benefits of CERCLA are equated with improved "ecological services" (see Table 1 2) Services and the economic definition of "benefits" are anthropocentric - that is they define and calculate benefits in relation to changes in human wellbeing There are those who feel this is too restrictive a definition of benefits Because of this we note that ecological resources and risks under CERCLA may be valued intrinsically m-and-of themselves, and not solely in relation to their potential to bring benefits to humans (NRC 2001, EPA 1992,1997b, 1999) With this said, several members of the panel wish to make two additional points First, if benefits are not defined in economic terms (e g , via changes in social well-being) it is not clear what alternative methods would be used to assess the magnitude of "benefits " Second, it is likely that any ecological improvement resulting from Superfund would have at least some social (and thus economic) benefit, even if those benefits are intangible, spiritual, or purely philosophical Nevertheless, the panel acknowledges the sensitivity of alternative definitions of "ecological value" and urges the agency to include a corresponding acknowledgment in its report The Panel's general comments on the SBA's proposed ecological benefits analysis may be found in Section 3 4 of this Advisory 31 ------- Charge Question 5a. Is the method of using data from detailed Natural Resource Damage Assessments to estimate benefits on a site-specific basis appropriate and in accordance with accepted theory? See Section 3 4 Charge Question 5b. Will the method of investigating NRDAs proposed on pages 5- 31 through 5-33 provide insight into the value of ecological benefits created by CERCLA and SARA? In particular, will an investigation of specific NRDA examples, be helpful? Will the proposed comparison of settlement amounts and estimated benefits be helpful? If not, what better approaches might be used to understand these benefits? See Section 3 4 Charge Question 5c. In cases where natural recovery would otherwise take place over finite but lengthy periods (decades to centuries), the benefits of active restoration accrue over similar periods. It is not clear whether these should be considered //ilra-generational or i/t/ir-generational. Is there a way to decide, or perhaps to avoid making this decision? The approach taken by the report is appropnate namely, the use of 3 alternative discount reference points - a zero, three, and seven percent discount rate This allows for "sensitivity analysis" of results and is consistent with accepted federal and economic practice 4.6 Charge Question 6: Are each of the non-quantified benefits discussed in Chapter 6 presented appropriately and sufficiently? This chapter of the report presents these benefits in a very cursory manner The Panel recommends either presenting a more thorough discussion of these benefits and how they might be captured, or noting that they are really just being "mentioned" here and are not truly addressed in this benefits analysis Also, depending on how the description of Superfund benefits is revised (see comments on Charge question #1) it is not clear if these particular "benefits" will still be included in this section of the report Amenities This section correctly points out that a benefit of the Superfund program is the "removal of unsightly, often abandoned facilities " This is likely to be a significant component of the benefits of Superfund It's less clear that "psychological benefits associated 32 ------- with reducing the uncertainty and fear of unknown risks" constitute an amenity component of the benefits If anything, this constitutes a health or information component of benefits But one must also consider the possibility of negative "psychological" effects of the program The SBA lists amenities as "non-quantifiable " It is not clear that amenities are indeed non-quantifiable, since the hedonic property models used in Chapter 4 could be capturing amenity effects Indeed, hedonics may over-estimate amenities Chapter 4 uses the estimated pnce gradients from the HPM studies and assumes that remediation leads to a full recovery of the housing prices For the cross-sectional studies, this means that the estimate is based on the assumption that remediating a site is the same as moving a house to a distance in which there is no price drop-off This implies that the remediation eliminates all health nsks and removes the entire disamenity of living near the site If anything, this over-states the amenity benefits of the Superfund program, because remediation does not necessanly rid the site of all its visual disamenities Materials In environmental economics textbooks, the classic examples of materials damages include damage to buildings and soiled clothes caused by noxious emissions and soot The SBA does not provide a definition of the materials benefits of the program it focuses on In addition, this section confuses two types of possible impacts of contamination on commercial or industrial (C/I) sites (1) the impact that contamination of a site has on the pnce of a neighboring C/I site, and (2) the impact that contamination of a given C/I site has on the price of that same C/I site In the first case the pnce impact of cleanup reflects benefits of reductions in contamination, while in the second case it reflects costs of cleanup (reflected in the liability) rather than benefits The discussion in this sections flips back and forth between these two different impacts without a clear indication of the distinction between them In addition, it incorrectly suggests that price effects due to liability (case 2) provide a measure of a benefit of Superfund The SBA begins this section with a claim that "In terms of avoiding material damages, the Superfund program often helps convert unusable commercial properties back into productive real estate In many cases, the avoided damage is associated with removal of both uncertainty about the presence of hazardous substances and with uncertainty about the cost of restoring the site to a usable condition " This is a statement about case 2 impacts However, the discussion of hedonic valuation that follows switches back and forth between arguments relevant to case 1 and those relevant to case 2 For example, the reference to "proximity to a hazardous waste" in the section on Structural Economic Changes is relevant for case 1 impacts, while the discussion of financing and lender liability in Financing Practice is only relevant for case 2 impacts In addition, the discussion of Other Literature does not distinguish between studies that focus on case 1 impacts (e g , Ihlanfeldt and Taylor, 2004) and those that focus on case 2 impacts (e g , Howland, 2000, and Schoenbaum, 2002) As noted above, this distinction is crucial for the SBA 33 ------- since estimates of case 1 impacts are a measure of benefits while estimates of case 2 impacts are not In addition, although Superfund has in some cases led to the conversion of unusable property into productive real estate, there is a widely held view that it has also had the opposite effect, 1 e , that the Superfund program actually created abandoned and underused properties because of fear of possible liability associated with the cost of cleaning up the site (Simons, 1998) Some observers have even claimed that listing in CERCLIS alone creates stigma, and it is often felt that the purpose of many state and local legislation and programs passed in the 1990s (e g , voluntary cleanup programs, brownfield programs) and offenng relief from liability and various incentives to parties that voluntarily clean up sites was to offset the perverse incentives over real estate created by the liability features of the Superfund program The SBA needs, therefore, to be careful about this kind of claim, and to justify carefully any statements made in this regard In particular, the SBA needs to discuss how exactly the Superfund program removes the uncertainty associated with the presence of hazardous substances (through the investigation activities7) and about the cost of restoring the site to a usable condition In general, the SBA does a good job of emphasizing that the market for commercial and industrial real estate has a completely different nature from the residential property market Specifically, the size of the market and the number of players are much smaller, there are far fewer transactions, and there are different rates of property turnover It would be useful to see some statistics, perhaps drawn from national statistics, to support these claims Similarly, it would be useful to cite formal studies to support the report's claims that commercial and industrial developers and end users are interested in a different set of amenities than homeowners The committee also found the subsection on Financing Practices to be rather underdeveloped and its implications were not clear Among other things, this section needs to lay out clearly the implications of CERCLA for property owners, third-party defenses, etc In addition, the literature review in the subsection entitled "Other Literature" missed the paper by McGrath (2000) In sum, the SBA needs to clarify (i) what "materials" benefits are, (n) what exactly the contribution of the Superfund program is to "materials" benefits, (in) the distinction between price effects from own contamination and from contamination of neighboring sites, (iv) whether migration or potential migration of contaminants is important here, and (v) how dis-incentives and effects have changed over time, as the US Environmental Agency became more efficient at recognizing and addressing contaminated sites, and potentially responsible parties have become better acquainted with the expectations imposed upon them by the agency (The type of sites and the effects on neighboring properties may have changed, too ) 34 ------- Finally, we feel that there insufficient evidence is provided for the following assertion in the SBA "It is important to consider if and how the materials benefit would appear in the policy case (1 e, no Superfund program) Similar to other benefit categories, the fact that without Superfund fewer responses would occur and uncertainty associated with toxic contamination of real property would be greater suggests that a large fraction of the materials benefits should be assigned to Superfund " In addition, the meaning of "a large fraction of the materials benefits" is unclear Empowerment The Panel has some concerns about the use of the word "empowerment" to describe this particular set of "benefits" of the Superfund program (This refers as well to the use of "empower" throughout the text) Public education and involvement may well be more appropriate terms to referring to the set of activities described here This raises a second point, which is that much of this section describes activities and programs of EPA and ATSDR, and while these are important "inputs" to the full range of Superfund benefits, they are not, in and of themselves "benefits " This relates to the Panel's comments on Chapter 1 suggesting ways in which the overall description of Superfund benefits might be improved As noted just above, SBA's section on empowerment appears to be more a description of program activities than of the benefits that result, without an explanation of how these activities are benefits of the Superfund program This section also appears to be wntten from the perspective of an agency advocate, in that it describes all the public activities in glowing terms, rather than in a more objective and analytical fashion For example, on page 6-9 the text reads "EPA maintains a substantial outreach and information effort " and in the paragraph that follows "The Superfund program also uses it community outreach mechanisms to create partnerships " While these statements are likely true for some sites, it is well documented (as shown in the number of sites with and without Technical Assistance Grants (TAGs)) that at some sites there are very active communities, and at others, none In addition, most likely the intensity and quality of community involvement activities vary according to EPA region, and to the individual conducting these activities None of this variation is alluded to, much less documented, in this section The Superfund program has several programs intended to provide information to communities In addition to the (TAG) program, which provides up to $50,000 for communities to procure technical advice on site documents related to final and proposed NPL sites, there is a similar program, the Technical Outreach Services to Communities (TOSC) program, which is similar to TAGs but is for communities with non-NPL sites Other mechanisms for providing information to the community and the general public include the Superfund website and various outreach activities, as well as the Superfund Job Training Initiative (Super JTI) which provides information on all sites, investigates sites, and makes recommendations Other efforts in community outreach are required as part of the Superfund Basic Research Program The 35 ------- outreach efforts in these Superfund Centers take many forms from education to children, museum displays, community programs, among many others These efforts provide information to people If risks do exist, people can learn how they can protect themselves If the public is educated on the issues, they are better able to participate in the decision making processes Perhaps more importantly, it is unclear exactly what the "benefits" to be discussed are While community education is a good thing, should it really be considered a "benefit" of the Superfund program"? How effective these programs are is difficult to determine This section would benefit from a more nuanced discussion about community education and involvement benefits, and which of these should be considered benefits of the program, in comparison to important components of, for example, the remedy selection process Once the section is revised to focus on benefits, the next challenge would be to discuss what metrics can be used to evaluate their impact For example, is trust being built between EPA and the communities7 Have the various outreach efforts improved citizens' knowledge about current risks at a site, steps they can take to protect themselves, and the pros and cons of alternative remedies7 This section is plagued by many assertions about various benefits that could well be true, but are not backed up by any kind of independent research and analysis For example, the discussion suggesting that community empowerment might reduce declines in property values does not appear grounded in any research, but is an untested hypothesis For example, the authors could have documented the number of NPL sites with active vs inactive community groups, as well as examined some of the internal and external reports that have tried to evaluate the quality of EPA's community involvement efforts Similarly, there is a lengthy description of the role of ATSDR, that describes its activities, but just what the benefits are is not clearly defined nor measured in any fashion Deterrence As noted in the response to charge question 1, the Panel feels that deterrence is a means to an end, namely, reduced contamination, which in turn leads to reduced negative environmental or health impacts The SBA notes that the deterrence benefit is "indirect" However, the reduction in impacts that results from deterrence or avoidance of contamination in the first place is no less important potentially than the reduction that results from cleaning up existing contamination Thus, there is no apparentjustification or logic for including this as an indirect or "embedded" benefit In revising Chapter 1 to develop a benefits assessment framework, the authors should consider including deterrence not as a separate "indirect" benefit category but rather as part of the health, amenities, ecological and materials benefits of Superfund Of course, in doing so, it will be important to distinguish between impacts or benefits stemming from past contamination (where the opportunities for deterrence are limited to 36 ------- containment) and those that would result from future contamination (which can still be prevented or reduced) The report's treatment of deterrence focuses almost exclusively on TRI, which is not really considered to be part of the Superfund program since it is funded through a separate appropriation It does not address the primary deterrence effect created by CERCLA's liability provision, which is a key component of the legislation The Panel recognizes the challenges associated with measuring the deterrence effects of Superfund, 1 e, estimating the benefits resulting from prevention or containment of contamination However, there is a growing body of empirical evidence, as well as a substantial theoretical literature, regarding the impact of liability on firm behavior (References for some of the empirical studies are given below ) While limited, this literature does suggest some conclusions regarding the impact of CERCLA liability The Panel advises the Agency to draw on the work that has been done to date to examine the deterrence benefits of Superfund in an expanded discussion in the report Emergency Preparedness This appears to be more of a description of the EPA removal and emergency response program and capabilities than an objective discussion of the benefits of these activities The question is not what resources are put into this activity, but what are the accomplishments If this section is supposed to describe benefits, then the SBA could be improved with a focus on accomplishments and how these accomplishments could be captured in economic terms In addition, while participating in the response to the World Trade Center, the anthrax attacks and picking up the debris from the Columbia shuttle disaster are important, and worth mentioning, they are by no means the center of the emergency response program And, in shifting resources to these new areas, the SBA fails to mention that resources were most likely drawn from other Superfund activities Information and Innovation The Panel believes that one of the "shining stars" of the CERCLA program has been the innovative methods and technologies developed from the various programs funded by Superfund, and in particular the Office of Research and Development This section of the SBA Report covers those topics well A recent SAB Advisory on the Office of Research and Development's Contaminated Sites and RCRA Multi-Year Plans (U S EPA-SAB, 2005) concluded that these programs have played, and will continue to play, a vital role in developing the science and technology for evaluating and cleaning-up the nation's hazardous waste sites The Panel recommends that the SBA incorporate some of those findings and conclusions herein International benefits As the SBA notes, although some Superfund activities generate benefits in other countries, most of these activities are funded through other sources and are not strictly speaking part of the Superfund program However it should be noted that the Superfund Program is responsible for developing capabilities and generating knowledge and data that could 37 ------- be interpreted as lowering the cost or increasing the opportunities for these international activities, a kind of public good benefit Examples include - Risk assessment approaches, - Databases for health effects and ecological impacts from exposures, and - Remediation technologies 4.7 Charge Question 7: Please comment on the overall organization and flow the report. The Panel has addressed this issue in multiple areas of this Advisory 4.8 Charge Question 8: Chapter 5 presents a proposed analysis for assessing the ground water effects of Superfund. Please comment on the proposed ground water efTect- by-effect analysis with a consideration of the following questions. Charge Question 8a. Will the approach described on pages 5-38 and 5-39 for quantifying the fraction of aquifers in the United States protected by Superfund provide useful information? Is the proposed approach to monetizing this benefit feasible? Is there a better approach to addressing the question of ground water protection? See Section 3 5 Charge Question 8b. The proposed methodology will probably not adequately capture the amount of ground water that will not become contaminated because of the Superfund program. Does the Panel have any suggestions about how to do so. The Panel agrees with the SBA's assessment of the difficulties in developing the counterfactual of ground water contamination without Superfund The Panel is without suggestions about how to do this in a credible fashion 38 ------- 5.0 References Albenni, Anna, and Alan Krupnick 2000 "Cost-of-Illness and Willingness-to-Pay Estimates of the Benefits of Improved Air Quality Evidence from Taiwan," Land Economics, 76(1) 337-53 Atkinson, Scott E and Thomas D Crocker 1987 "A bayesian approach to assessing the robustness of hedonic property value studies" Journal of Applied Econometrics, 2 27-45 Bartik, T J 1987 "The Estimation of Demand Parameters with Single Market Data The Problems Caused by Unobserved Tastes," Journal of Political Economy, 95 81-88 Bartik, T J 1988 "Measuring the Benefits of Amenity Movements in Hedonic Price Models," Land Economics, 64(2) 172-183 Dickie, Mark 2003 "Defensive Behavior and Damage Cost Methods," in Patricia A Champ, Kevin J Boyle, and Thomas C Brown, eds , A Primer on Nonmarkel Valuation, Dordrecht Kluwer Academic Press Ekeland, Ivar, James J Heckman, and Lars Nesheim 2004 "Identification and Estimation of Hedonic Models " Journal of Political Economy, 112 S60-S109 EPA, 1988 Guidance for Conducting Remedial Investigations and Feasibility Studies Under CERCLA EPA-540-G-89-004 EPA, 1989 Risk Assessment Guidance for Superjund, Vol 1 Interim Final (Part A) of Human Health Evaluation Manual EPA 540/1 89 002 EPA, 1997 Final Report to Congress on Benefits and Costs of the Clean Air Act, 1970 to 1990 EPA 410-R-97-002 Office of Air and Radiation Available at http //www epa.gov/airprogm/oar/sect8I2/index html EPA 1997a Rules of Thumb for Remedy Selection Office of Solid Waste and Emergency Response EPA 540-R-97-013 EPA, 1997b Ecological Risk Assessment Guidance for Superfund Process for Designing and Conducting Ecological Risk Assessments Interim Final EPA 540-R-97-006 United States Environmental Protection Agency, Office of Solid Waste and Emergency Response, Edison, New Jersey June 5 39 ------- EPA, 1999 A Guide for Preparing Superfund Proposed Plans, Records of Decision, and Other Remedy Selection Decision Documents EPA/540/R-98/031, OSWER Directive 92001 1- 23P, July 1999 EPA 2002 Cost of Illness Handbook Office of Pollution Prevention and Toxics Available at www epa gov/oppt7coi/loc html EPA, 2000 Guidelines for Preparing Economic Analyses Office of the Administrator, EPA 240-R-00-003 Available at http //yoserrute epagov/ee/epa/eed nsf/webpages/Guidelines html EPA, 2000 Handbook for Non-Cancer Health Effects Valuation, EPA Science Policy Council Available at www epa gov/osa/spc/htm/Homeqs htm EPA-SAB 2002 Underground Storage Tanks Cleanup and and Recovery Act (RCRA) Subtitle C Program Benefits, Costs and Impacts An SAB Advisory (EPA-SAB-EC-ADV-03-001) EPA-SAB, 2005 Advisory on the Office of Research and Development's Contaminated Sites and RCRA Multi-Year Plans (EPA-SAB-05-009) Epple, D 1987 "Hedonic Prices and Implicit Markets Estimating Demand and Supply Functions for Differentiated Products," Journal of Political Economy, 95 59-80 Freeman, A Mynck III 2003 The Measurement of Environmental and Resource Values, 2nd ed , Washington, DC Resources for the Future Gayer, Ted, James T Hamilton, and W KipViscusi 2000 "Private Values of Risk Tradeoffs at Superfund Sites Housing Market Evidence on Learning about Risk " Review of Economics and Statistics, 82(3) 439-451 Gayer, Ted, James T Hamilton, and W KipViscusi 2002 "The Market Value of Reducing Cancer Risk Hedonic Housing Prices with Changing Information" Southern Economic Journal, 69(2) 266-289 Graves, Phil, James C Murdoch, Mark A Thayer, and Don Waldman 1988 "The Robustness of Hedonic Price Estimation Urban Air Quality " Land Economics 64(3) 220-233 Greenstone, Michael and Gallagher, Justin 2005 "Does Hazardous Waste Matter'? Evidence from the Housing Market and the Superfund Program," unpublished mimeograph 40 ------- Hamilton, James T , and W KjpViscusi 1999 "How Costly is 'Clean"? An Analysis of the Benefits and Costs of Superfund Site Remediations " Journal of Policy Analysis and Management, 18(1) 2-27 Howland, M 2000 "The Impact of Contamination on the Canton/Southeast Baltimore Land Market," Journal of the American Planning Association, 66(4), 411-420 Ihlanfeldt, K R , and L O Taylor 2004 "Externality Effects of Small-Scale Hazardous Waste Sites Evidence from Urban Commercial Property Markets," Journal of Environmental Economics and Management, 47(1), 117-139 Landngan, P J , 2002 "Environmental Pollutants and Disease in American Children Estimates of Morbidity, Mortality, and Costs for Lead Poisoning, Asthma, Cancer, and Developmental Disabilities et al , Environmental Health Perspectives, 110 721-728 Lybarger, Jeffrey A, et al 1998 "Medical Costs and Lost Productivity from Health Conditions at Volatile Organic Compound-Contaminated Superfund Sites," Environmental Research, 79 (1), 9-19 McClelland, G H,W D Schulze, et al "The Effect of Risk Beliefs on Property Values A Case Study of a Hazardous Waste Site" Risk Analysis, 10(4) 485-97 McGrath, D T 2000 "Urban Industrial Land Redevelopment and Contamination Risk," Journal of Urban Economics, 47(3) 414-442 Mendelsohn, Robert, et al, 1992 "Measuring Hazardous Waste Damages with Panel Models," Journal of Environmental Economics and Management, 22 (3) 259-271 Michaels, R Gregory and V Kerry Smith 1990 "Market Segmentation and Valuing Amenities with Hedonic Models The Case of Hazardous Waste Sites " Journal of Urban Economics, 28 223-242 Millennium Ecosystem Assessment 2003 Ecosystems and Human Well-being, Washington, DC Island Press National Research Council 2004 Valuing Ecosystem Services Toward Better Environmental Decision-Making, Washington, DC National Academy Press Office of Management and Budget 1992 OMB Circular A-94 Guidelines and discount rates for benefit-cost analysis of federal programs Washington, DC Executive Office offihesident 41 ------- Reichert, A , M Small, and S Mohanty 1992 "The impact of landfills on residential property values," The Journal of Real Estate Research, 7 297-314 Schoenbaum, M 2002 Environmental contamination, brownfields policy and economic redevelopment in an Industrial area of Baltimore, Maryland Land Economics, 78(1) 60- 42 ------- |