IPA-SAB-EC-LTR- SABECLTR940
4-002
wEPA
THE SCIENCE ADVISORY BOARD
Overview of SAB Comments and
Recommendations on the Proposed RIA
for the RCRA Corrective Action Rule
EPA-SAB-EEAC-94-001 "Review of the Contingent
Valuation Method for the Proposed RIA for RCRA
Corrective Action Rule" by the Environmental Economics
Advisory Committee
EPA-SAB-EEAC-LTR-94-001 "Review of Economic
Aspects of the Proposed RIA for the RCRA Corrective
Action Rule" by the Environmental.Economics Advisory
Committee
EPA-SAB-EEC-94-002 "Review of MMSOILS component of
the Proposed RIA for the RCRA Corrective Action Rule"
by the Environmental Engineering Committee
EPA-SAB-EPEC-COM-94-001 "Commentary on the
Ecological Risk Assessment for the Proposed RIA for the
RCRA Corrective Action Rule" by the Ecological
Processes and Effects Committee
EPA-SAB-EHC-LTR-94-003 "Review of the Health Benefits
for the Proposed RIA for the RCRA Corrective Action
Rule" by the Environmental Health Committee
EPA-SAB-EC-LTR-94-002 "Overview of SAB Comments on
the Proposed RIA for RCRA Corrective Action Rule" by
the RCRA/RIA Steering Committee
November 1993
Recycled/Recyclable
Printed with Soy/Canda Ink on paper that
contains at least 50% recycled fiber
-------
EPA-SAB-EC-LTR-94-002 'Overview of SAB
Comments on the Proposed RIA for RCRA
Corrective Action Rule" by the RCRA/RIA Steering
Committee
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF THE ADMINISTRATOR
November 19, 1993 SCENCE ADVISORY BOARD
EPA-SAB-EC-LTR-94-002
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, S.W.
Washington, D.C. 20460
Subject: Overview of SAB Comments and Recommendations on the Proposed
RIA for the RCRA Corrective Action Rule.
Dear Ms. Browner
This report is one of a series of six reports (listed in Appendix A) generated by
the SAB in response to the subject request from the USEPA's Office of Solid Waste
and Emergency Response (OSWER). It contains a brief summary and overview of
the salient conclusions of the other five reports as well as some observations,
comments and recommendations of the RCRA/RIA Steering Committee (RRSC). A
roster of RRSC is in Appendix B.
At the October, 1992 meeting of the Executive Committee (EC), the Science
Advisory Board (SAB) was asked by OSWER to review its then-nearly-complete RIA
methodology which was being applied to the cost/benefit analysis required prior to
promulgation of the Agency's final RCRA Corrective Action Rule. The EC, recognizing
the importance, complexity, creativity, and novelty of OSWER's work and its
multi-disciplinary character, established the RRSC to assure that certain significant
aspects of the RIA-both methodology and application-received appropriate attention
from the relevant SAB standing committees.
At a public meeting on January 29,1993, the RRSC concluded, on the basis of
presentations by and discussions with OSWER personnel, that four individual SAB
standing committees should undertake reviews of the major segments of the
RCRA-RIA with appropriate inter-committee liaison. The Environmental Economics
Advisory Committee (EEAC) reviewed the contingent valuation (CV) methodology
-------
(hereafter called CV-1) and the application of CV in the specific case of the RCRA-RIA
(hereafter called CV-2). The Environmental Engineering Committee (EEC) reviewed
the principal fate and transport model. MMSOILS. used in the RCRA-RIA. The
Ecological Process and Effects Committee (EPEC) reviewed the ecological risk
assessment portion of the RCRA-RIA and the Environmental Health Committee (EHC)
reviewed the human health risk assessment portion of the RCRA-RIA. The RRSC
provided coordination and its own insights with respect to the RCRA-RIA methodology
and its application.
Interpretation of the Charge
The charge for these reviews is contained in two separate requests for reviews
of RCRA-RIA components: the Groundwater Contingent Valuation (CV) Study
(October 21,1992) and the MMSOILS model for fate and transport (March 26, 1993).
The latter request also asked the SAB to comment on "the implications that the model
has on the human and ecological risk assessment" and to "consider during their
review several practical factors including the baseline risk assessment and the fact
that the RIA is a predictive analysis". Thus the RRSC concluded that how these risks
were determined needed to be reviewed as well. Each of the standing committees of
the SAB has addressed its portion of the charge in their separate reports. The RRSC
has taken as its charge the task assigned it by the EC of ensuring that the significant
aspects of the RIA - both methodology and application - received appropriate
attention from the relevant SAB standing committees, of ensuring that coordination
exists where needed and, ultimately, providing its own comments and overview. As
noted by the EHC, this draft methodology is actually a screening analysis which
provides preliminary estimates rather than definitive analysis reflecting site specific
details.
^**^ -—
The SAB was not asked to review the costs of corrective action or the
procedures for estimating them, nor was it supplied with the detailed background
information needed to da so. The SAB has thus reviewed only the estimation of the
benefits of the corrective action and the methodologies used in deriving the
expressions of the benefits. A review of the costs and their estimation methods might
produce comments. Thus, the absence of comments in this area does not constitute
any SAB position as to the costs of remediation or their methods of estimation.
-------
General Comments
This RIA is one of the most complete and complex that OSWER has
undertaken and is the first to undergo detailed review by the SAB. In spite of the
scientific criticisms which follow, we commend the Agency for this major effort, for its
openness of discussion with the SAB, and for its innovative attempts to apply a wide
variety of types of information and procedures to assess the risks associated with solid
waste management units and the benefits of abating those risks.
The OSW/ORD working group is also to be commended for a well-coordinated
and focused effort to develop an RIA that will help the Agency and the Nation better
understand the risks associated with RCRA sites and the costs and benefits of
remediation - and the size, complexity and difficulty of the analysis. The intra-agency
coordination represented by this RIA is itself a model approach that the Agency should
apply to other programs to promote effective and efficient interactions of Program
Office and ORD to ensure that Program Office activities represent state-of-the-art
science and technology.
Finally, despite the large amount of good work that has gone into this RIA, it
only accounts for part of the benefits that may accrue from reducing health and
ecological risks from RCRA sites. The comments which follow include
recommendations for both short term changes to improve this RIA and long-term
investments in research and analysis to improve RIAs within the Agency.
Overview of the Major Comments and Recommendations of the SAB Standing
Committees
To place into context the specific comments and recommendations of the
RRSC, some of the major findings of the SAB's standing committees are summarized
here. The reader is referred to the individual reports for full and detailed descriptions
of these and other findings. In each case, the standing committees offer
recommendations for both short term and long term improvements.
The Agency's CV-1 document represents a substantive contribution, extending
understanding of the issues associated with contingent valuation as a method of
estimation of non-market values. Even so, concerns were raised about the method
which need answering: whether the pretesting and design are such as to truly assure-
-------
that a well-defined groundwater commodity was understood properly by the
respondents; whether use of the Box-Cox econometric estimates alone is acceptable
and defensible; whether embedding was adequately treated; whether the "per
household" non-use values from CV-1 can be regarded as either upper or lower bound
values; and others. Among the concerns raised about CV-2, given also the concerns
about CV-1, itself, are: whether the application of values obtained in CV-1 to the
different set of circumstances of CV-2 is possible; that EEAC could not endorse the
McClelland et al study as a basis for EPA to determine the non-use values of
groundwater; and that the hedonic analysis was not actually used in the RIA.
CV methods in general are still controversial and the EEAC concluded that
more information was needed to apply CV in this RIA. Nonetheless, the EEAC
showed sufficient confidence in the approach to recommend that further research be
undertaken, particularly to resolve whether the CV approach can produce information
useful in RIAs.
Fate and transport information is fundamental in assessing exposures and,
therefore, human health and ecological risks. While noting that the methods and
formulations used in the MMSOILS model are well known, documented and accepted
and that underlying assumptions are dearly stated in the RIA, the EEC concluded that
here, too, there are difficulties (though if MMSOILS is applied to simplified case
studies it might be a valid screening tool for assessing the relative risks and costs
associated with alternative regulatory options). The primary difficulties are: sparse or
inaccurate information, poor parameter estimation especially relative to source terms,
suspected over-reliance upon default parameters, and that the Model is applied to
cases outside the range of its validity. Given these shortcomings, many of which were
already realized by the Agency, the most basic and pressing concern is whether the
use of a generic model such as MMSOILS is appropriate as a basis for the
assessment of regulatory costs and benefits at the national level since the fate and
transport estimates that comprise the model output may be wrong by orders of
magnitude for many complex sites. In its report the EEC recommends ways in which
the Agency can augment exposure and cost/benefit estimates using alternative
approaches.
The human health risk assessment methods used in the RIA are well known
and often used in the regulatory arena. There is a question as to how well they can
be used to estimate risks - or exposure levels of concern for risk - in this case of
-------
multiple exposures, for analytical rather than regulatory purposes. Also there are
fundamental differences between the usual methods used for carcinogens and non-
carcinogens and the results that can be obtained in each case. Much of the criticism
of the human health risk analysis in CV-2 has to do with a confusing use of risk and
exposure terminology and is easily rectified. We urge that it be rectified. The EHC
also urges that quantities be calculated to describe the population exposed at levels of
concern for cancer and non-cancer adverse effects, and the attendant risks so they
are comparable and offers suggestions for presenting non-monetized benefits. Some
of the proposed calculations can be carried out in the short term (calculating the
number of people exposed to carcinogens at levels of concern for cancer risk, as is
already done for noncarcinogens for non-cancer adverse effects risk) whereas_the
method proposed for calculating population risks for noncancer adverse effects will
take more time and development. Of all of the sections of RIA, this section is the
most easily improved through the application of existing scientific knowledge.
The EPEC recognizes the formidable task undertaken by the Agency in the
ecological risk assessment but raises several questions about it. Among the concerns
raised are these: the major pathway considered is not necessarily the most likely to
cause adverse ecological effects, the range of ecological endpoints considered is
limited, there is no consideration of the ecological risks and benefits of site
remediation, there is insufficient discussion of data sources and assumptions, and
there are a number of application and interpretive errors. The EPEC also suggests
that the ecological risk assessment be recast in a form consistent with the Agency's
"Framework for Ecological Risk Assessment" (EPA/630/R-92/001, February, 1992).
Specific Comments and Recommendations of the RRSC on RCRA-RIAs
_ a). The RRSC noted with pleasure that each major chapter in the RIA
contained a final section on limitations which served to enhance the
understanding of the reader/user of what meaning can be ascribed to the
contents of the chapters. It is recommended that the sections be
enlarged where necessary to address the criticisms from the SAB
relevant to the final text of the RIA.
b) It is important to improve the assessment of fate and transport, and
therefore of estimated exposures, as much as possible since the large
error band seen by the EEC can only seriously compound and make
-------
more uncertain the assessment of both human health and ecological
risks and the benefits to be derived by their abatement. Greater reliance
should be placed on measurements of exposure to supplement and
validate model predictions, as called for in the Exposure Assessment
Guidelines.
c) The risks created by the remediation process should be addressed.
EPEC identified additional ecological risks in their report such as
impacts on terrestrial wildlife, habitat, and biodiversity which were not
adequately addressed. Additional risks include: loss of contaminants to
other media during pumping, treatment, excavation, and hauling, e.g.,
transferring groundwater eventually to surface water, stripping of volatile
compounds, air entrapment of soil particles; accidents to workers during
remediation and transportation; and puncturing a confining structure and
contaminating a deeper aquifer during installation of wells. These risks
are relevant because several Superfund Records of Decisions have been
amended/overturned due in part to risks to workers and off-site
communities during remediation.
d) While the cost estimates in the RIA for corrective action were not
evaluated in a consensus manner by the SAB, several issues of possible
concern were identified by the EEC: Should additional sites be included.
e.g., pre-HSWA land treatment units, very large DoD sites, more spill
sites? Was the cost of a given cleanup underestimated, e.g., quantity of
soil to be remediated, labor costs under hazardous conditions, insurance,
inflation? What is the comparison of the cost in the draft RIA report with
costs developed in 1992 by the University of Tennessee's Waste
Management Research and Education institute? Should other cost
categories be included, e.g., transaction costs and government
administrative costs?
We recommend that these questions be addressed by EPA during
the public comment period.
e) The RRSC has considered sampling strategy since, as in all
assessments, results and interpretations depend on the samples used in
the RCRA-RIA. The current RIA gives much consideration to this subject-
-------
and the sample used is large relative to that used in other cases. RIAs
operate on a national scale and have multiple components (economics,
engineering, human health, and ecological effects). The choice of
sample sites has to be representative of the population of sites in the
country and it also has to consider the attributes of these multiple
components. The RCRA RIA sampling design was based primarily on
the size and type of waste sites but did not consider the nature of the
ecological risks (based on exposure or effects) or, necessarily, the
different nature of health risks (and exposures), in choosing the sites.
Thus it is not evident that either type of risk assessment is representative
because ecological and health criteria were not part of the stratification
process.
We therefore recommend that in future RIAs the sample designs
incorporate criteria appropriate for all aspects of the RIA. The categories
of samples should include estimates of central tendency and dispersion
and a discussion of the sources of variability within each category.
Estimates of uncertainty should be included in the National Assessment.
In some cases, it may be necessary to use different designs to address
particular types of risks. For the long-term the Agency should consider
whether calculation of a National Assessment is a useful decision-making
tool for rulemaking, particularly where site-specific conditions are highly
variable.
f) The RRSC supports the recommendations for further research into and
development of the methodologies needed to perform RIAs as set forth
in the other five reports. Some further questions which research may be
able to answer are: (a) Can the CV methods be sensitive enough to
distinguish between the values of clean, cleaner and cleanest ground
water? This question is of importance in examining different corrective
action options which may yield significantly different costs and different
levels of dean-up. If this distinction cannot be made, the distribution of
CV values would stand as an invariant and might have little to say about
which option to choose, if any. And (b) what are the CV values of sound
ecosystems or of good health? These questions, if they can be answered
without overlap with other CV values, could greatly assist in valuing
benefits.
-------
g) Chapter 13 of the RIA should be renamed to indicate that costs and
benefits are characterized, since the non-monetized and the monetized
benefits cannot be compared to costs in the same manner. Monetization
is not always possible so other types of characterizations should be
presented in ways that make comparisons possible and that facilitate
judgments about costs versus benefits; also terms should be dearly
defined and used consistently. Indeed, even if complete monetization is
achieved, presenting additional characterizations of costs and benefits is
highly desirable. All characterizations should include a description of
their uncertainties. Specific suggestions for improvements are provided
in Appendix C to this report Similar clarifications of meaning and,
definition should be made consistently throughout the entire report.
In closing, we commend the Office of Solid Waste for its pioneering efforts in
the development of this RCRA RIA. Regulatory impact assessments by their very
nature are not site specific and operate at the national scale or even international
and/or global scale. Based on our review of this RIA, we recommend that the Agency
build on the experience gained here to develop a technical support document (TSD)
providing guidance on the development of an RIA. The TSD should include a variety
of approaches for assessing the economic, human health, and ecological benefits and
costs associated with proposed regulation. We suggest that the TSD incorporate as
building blocks the Human Health Risk Assessment Guidelines and the Framework for
Ecological Risk Assessment
Finally, we make one additional recommendation. In the first stages of
approaching and defining a major project such as the RIA, the Agency might consider
availing itself of the consultation role of the SAB. In this role the individual members
and consultants of SAB committees offer advice and comments as individuals in
public meetings on points of interest raised by the staff members about their nascent
project Although the occurrence of such a consultation is recorded and reported to
the Administrator, the details of the advice are not. Such advice at an early stage,
can serve to raise questions that are better addressed early rather than dose to the
end of a project The SAB may later conduct a peer review of the final agency
document
8
-------
The SAB is pleased to have had the opportunity to review this important
project, and we look forward to your response to these comments, as well as
reviewing other RIAs in the future.
Sincerely yours,
Dr. Raymond C. Loehr, Chair Dr. Paul Deisler, Chair
Executive Committee RCRA-RIA Steering Committee
Science Advisory Board Science Advisory Board
9
-------
APPENDIX A. LIST OF SAB REPORTS REVIEWING DIFFERENT ASPECTS OF
OSWER'S RCRA/RIA CORRECTIVE ACTION COST/BENEFIT ANALYSIS
METHODOLOGY AND ITS APPLICATION.
The reports are, in brief:
1. EPA-SAB-EEAC-94-001 "Review of the Contingent Valuation Method for the
proposed RIA for RCRA Corrective Action Rule" by the Environmental
Economics Advisory Committee (Also referred to as CV-1)
2. EPA-SAB-EEAC-LTR-94-001 "Review of Economic Aspects of the p'roposed
RIA for the RCRA Corrective Action Rule" by the Environmental Economics
Advisory Committee (Also referred to as CV-2)
3. EPA-SAB-EEC-94-002 "Review of MMSOILS component of the Proposed RIA
for the RCRA Corrective Action Rule" by the Environmental Engineering
Committee
4. EPA-SAB-EPEC-COM-94-001 "Commentary on the Ecological Risk
Assessment for the proposed RIA for the RCRA Corrective Action Rule" by the
Ecological Processes and Effects Committee
5. EPA-SAB-EHC-LTR-94-003 "Review of the Health Benefits for the proposed
RIA for the RCRA Corrective Action Rule" by the Environmental Health
Committee
6. _ .EPA-SAB-EC-LTR-94-002 "Overview of SAB Comments on the proposed RIA
for RCRA Corrective Action Rule" by the RCRA/RIA Steering Committee
A-1
-------
APPENDIX B.
US ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
RCRA/RIA STEERING COMMITTEE
ROSTER
CHAIRMAN
- Dr. Paul F. Delsler, Jr., 11215 Wilding Lane, Houston, Texas 77024
MEMBERS
Dr. Richard A. Conway, Union Carbide Corporation, South Charleston, West
Virginia 25303-0361
Dr. Kenneth L. Dickson, University of North Texas, Denton, Texas 76203-3078
Dr. Allen V. Kneese, Resources for the Future, Washington, D.C. 20036
Dr. Verne A. Ray, Pfizer Inc., Groton, Connecticut 06340
Dr. Arthur C. Upton, New York Medical Center (Retired), Sante Fe, New Mexico
87501
SCIENCE ADVISORY BOARD STAFF
Dr. Edward S. Bender, Designated Federal Officer, US EPA/Science Advisory
"Board, 401 M Street, S.W. (A-101F) Washington, D.C. 20460
Mrs. Marcia K. Jolly (Marcy)
Secretary to the Designated Federal Official
B-1
-------
APPENDIX C. Recommendations from the Steering Committee for Clarifying the Cost
Benefit Presentation
The characterizations of costs and benefits in Chapter 13 can be improved,
particularly in the way the information is presented in Section 13.3 and in Exhibits 13-1
and 13-2. These exhibits, in particular, are very important since they summarize the
output of the entire cost/benefit study; great care should therefore be taken to be sure
they are not easily misunderstood or misused.
In Exhibit 13-1, for example, the values given might be designated as
"Preferred Value" instead of "Effect of Corrective Action" and a second line'added
entitled "Range of Estimates" with the corresponding figures to give the decision
maker some immediate understanding of the uncertainties. In the draft as written, the
non-use value of ground water would then be shown as the preferred value of $2.3
billion (as now, unless changed in response to the SAB review) and the ranges of the
estimate would be given as $0.17-18.0 billion (using different numbers if the numbers
should change as a result of the SAB review). Exhibit 13-2 also offers opportunities
for improvement ranging from changing captions to more suitable ones (including
correcting risk terminology) to adding further information to make useful comparisons
possible. Here are some examples of desirable changes: the figure of $18.7 billion
should be shown as the preferred value with its ranges of estimate as suggested for
Exhibit 13-1; the caption "Avoided Non-Cancer Effects" should be changed to
"Avoided Non-cancer Exposures of Concern", which is what they are; the 100 to 12
million "cases" under the Non-Cancer column should become "exposures of concern"
since these are not cases of actual effect as in the case of cancer, and at least one
additional column needs to be added to the exhibit. It would shed further light on the
benefits to be obtained is one giving the number of "Avoided Cancer Exposures of
Concern"; it would be based on the numbers of people, exposed at levels of exposure
yielding a risk of cancer of 10 or higher using much the same information already
used to estimate cancer risks (as suggested in the EHC report). As the exhibit now
stands, the cancer and non-cancer effects - the totality of health effects - cannot be
compared or placed in context with each other. Although the definitions of what is "of
concern" differ for cancer and non-cancer effects, they do nonetheless exist and are
accepted as meaningful and so this additional column, compared to the one retitled
"Avoided Noncancer Exposures of Concern", would provide useful additional
information to the decision maker. If, in the future, columns for non-cancer effects
C-1
-------
comparable to the column entitled "Averted Population Cancer Cases" can be
provided, so much the better. Such information might assist in providing a basis for
monetizing non-cancer effects avoidance as do the figures for cancer cases now in
hand in appropriate cases (medical costs and productivity losses avoided and the
like). Finally, expanding the footnote to offer some idea of the meaning of current
ecological risk and whether corrective action will have any effect on it will add further
to the value of this Exhibit in shedding light on benefits to be expected from the
corrective action.
The two exhibits should also be changed to reflect the fact that there are two
baseline cases: one with the entire population either treating or substituting its_water
supply (which now forms the basis of the small monetary benefit ascribed to averting
treatment and substitution in Exhibit 13-1) and a second in which no one in the
population treats or substitutes their water supply (the one for which the relatively
large decreases in cancer incidents shown in Exhibit 13-2 are estimated). As they
stand now, Exhibits 13-1 and 13-2 appear to be anomalous in this regard and the full
information that could be displayed in the characterization is not displayed.
These few examples illustrate ways in which the characterization of the costs
and benefits can be greatly enriched; they and others like them apply not only to the
two exhibits cited but to Chapter 13 as a whole.
C-2
-------
APPENDIX D. List of Acronyms
CV Contingent Valuation Methodology
EEAC Environmental Economics Advisory Committee (SAB/EEAC)
EEC Environmental Engineering Committee (SAB/EEC, also referred to as
"The Committee")
EHC Environmental Health Committee (SAB/EHC)
EPEC Ecological Processes and Effects Committee (SAB/EPEC)
MMSOILS A Mathematical Model for Soils (Includes other media transfer from
soils.)
OSW Office of Solid Waste (U.S. EPA)
OSWER Office of Solid Waste and Emergency Response (U.S. €PA) '
RCRA Resource Conservation and Recovery Act
RRSC RCRA RIA Steering Committee
RIA Regulatory Impact Analysis
SAB Science Advisory Board (U.S. EPA)
SWMUs Solid Waste Management Units
D-1
-------
EPA-SAB-EEAC-94-001 'Review of the Contingent
Valuation Method for the Proposed RIA for RCRA
Corrective Action Rule" by the Environmental
Economics Advisory Committee
-------
United States Sdtnct Advteoiy ' EPA^AB-EEAC^44)01
EnvfconiMntel Board (1400F) Novwnbtr 1993
Protection Agency W««hhgtoo DC
©EPA AN SAB REPORT:
CONTINGENT VALUATION
METHODOLOGY (CV 1)
REVIEW OF THE CONTINGENT
VALUATION METHOD FOR THE
PROPOSED RIA FOR RCRA
CORRECTIVE ACTION RULE BY THE
ENVIRONMENTAL ECONOMICS
ADVISORY COMMITTEE
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
November 19, 1993
SCENCE ADVISORY BOARD
EPA-SAB-EEAC-94-001
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, S.W.
Washington, DC 20460
Subject: Science Advisory Board's Review of the Contingent Valuation
Method for the proposed RIA for the RCRA Corrective Rule
Dear Ms. Browner.
At the October, 1992 meeting of the SAB's Executive Committee (EC) the
Board was asked by the Office of Solid Waste and Emergency Response (OSWER) to
review the methodology for the draft Regulatory Impact Analysis (RIA). This
cost/benefit analysis is required prior to promulgation of the Agency's final Resource
Conservation and Recovery Act Corrective Action (RCRA) Rule. The EC, recognizing
the importance, complexity, and novelty of OSWER's work and its multi-disciplinary
character, established an ad hoc RCRA-RIA Steering Committee (RRSC) to assure
that certain aspects of the RIA - in both methodology and application - received
appropriate attention from the relevant SAB committees.
At a public .meeting on January 29, 1993, the RRSC concluded, on the basis of
presentations by, and discussions with, OSWER personnel, that four SAB individual
committees should review the major segments of the RCRA-RIA. Specifically, the
RRSC agreed to review: a) the contingent valuation (CV) study used in the RCRA RIA
analysis (CV-1, by the Environmental Economics Advisory Committee (EEAC)); b) the
application of CV in the RCRA-RIA (CV-2, by the EEAC); c) the principal fate and
transport model (MMSOILS). used in the RCRA-RIA (by the Environmental Engineer-
ing Committee (EEC)); d) the ecological risk assessment portion of the RCRA-RIA (by
the Ecological Processes and Effects Committee (EPEC)); and f) the human health
benefits assessment portion of the RCRA-RIA (by the Environmental Health Commit-
tee (EHC)).
Rtcyctodfltocyc
Pi««d M piper Hut ooMInt
-------
Addressing other aspects of the Charge:
a) The Committee does not believe that the pre-testing and survey design
techniques offer convincing evidence that a well-defined groundwater
commodity was understood properly by all the respondents.
b) The Committee does not believe that any of the three possible methods
for separating the non-use or passive use values from total values can
be established as preferred at this time. Each should be regarded as
exploratory rather than a tried-and-true method for estimating non-use
values from survey responses.
c) The Committee deems it impossible to judge whether the Box-Cox
econometric estimates alone provide an acceptable and defensible
method for dealing with the scenarios and the large bids associated with
them.
d) The Committee does not believe that the approaches for treating embed-
ding, scenario rejection, and the potential effects of non-bidding respons-
es can be assessed for their reliability on the basis of the information
provided in the report.
In many respects, the Committee feels that the problems in using the study
results to meet the needs of the RIA effort arise from requirements imposed on the
research. These include the need for both a separate estimate of nonuse value and
for a method that abstracted from the specific features of the local conditions associat-
ed with each specific case of groundwater contamination. After reviewing the full
record, the Committee feels that this strategic decision on EPA's part is the cause of
many of the problems with the McClelland study as well as the difficulties of imple-
menting the research findings in the RIA context. The approaches taken to deal with
these requirements have no basis in the theory of non-market valuation, nor precedent
in practice, and were never subjected to peer review.
The Committee's report offers specific suggestions for further research to help
resolve the questions raised by this study. Five general areas seem especially
relevant for the corrective action RIA:
a) For the most part, economic research has focused on developing esti-
mates of the values of the typical or representative household
-------
We appreciate the opportunity to review these issues, and look forward to
receiving your response to our comments.
Sincerely,
Dr. Raymond C. Loehr, Chair
Science Advisory Board
Dr. Allen Kneese, Co-Chair Dr. V.TCerry Smith, Co-Chair
Environmental Economics Environmental Economics
Advisory Committee Advisory Committee
ENCLOSURE
-------
ABSTRACT
The EEAC addressed the design, conduct, and results of the contingent
. valuation study (undertaken for the EPA Office of Solid Waste by Drs. McClelland,
Schulze, et a/.), focusing on a Charge organized around five major questions: a) the
survey respondents' understanding of groundwater resources; b) selection of the best
method for estimating non-use values from the survey responses; c) use of the Box-
Cox econometric procedure to address large bids; d) the problems of embedding, non-
bids, and scenario rejection; and e) the applicability of the valuations obtained in this
study as a basis for EPA to determine the non-use values of groundwater. The
Committee commends EPA staff for supporting exploratory research of this nature.
There is little doubt that this study represents a substantive contribution, extending our
understanding of the issues associated with contingent valuation estimation of non-
market values. Addressing the last, but most encompassing element of the Charge
first, the Committee can not endorse the McClelland et a/, study as a means'of
generating valid and reliable estimates of the nonuse values associated with cleaning
up contaminated groundwater Specifically, the Committee has no confidence that the
respondents were dear about what it is they were being asked to value. Although the
study was innovative in a number of respects, this most basic failing gives the
Committee no choice but to question the validity of the findings. Addressing other
aspects of the Charge: a) The Committee does not believe that the pre-testing and
survey design techniques offer convincing evidence that a well-defined groundwater
commodity was understood properly by all the respondents; b) The Committee does
not believe that any of the three possible methods for separating the non-use or
passive use values from total values can be established as preferred at this time; c)
the Committee deems it impossible to judge whether the Box-Cox econometric esti-
mates alone provide an acceptable and defensible method for dealing with the
scenarios and the large bids associated with them; and d) the Committee does not
believe that the approaches for treating embedding, scenario rejection, and the
potential effects of non-bidding responses can be assessed for their reliability on the
basis of the information provided in the report. The EEAC feels that the problems in
using the study results to meet the needs of the RIA effort arise from requirements
imposed on the research by the EPA, including the need for separate estimate of
nonuse value and for a method that abstracted from the specific features of the local
conditions associated with each specific case of groundwater contamination. The
approaches taken to deal with these requirements have no basis in the theory of non-
market valuation, nor precedent in practice, and were never subjected to peer review.
The Committee's report offers specific suggestions for further research to help resolve
the questions raised by this study, including the criteria for deciding which households
would be among the groups demanding increases in the amount available of specific
commodities or values and study of the sensitivity of CV outcomes to the survey
methods used
KEYWORDS: contingent valuation; groundwater; hedonic models; nonuse values;
RCRA RIA
ii
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
ENVIRONMENTAL ECONOMICS ADVISORY COMMITTEE
RCRA RIA REVIEW
CO-CHAIRS
Dr. Allen V. Kneese, Resources for the Future, Washington DC
Dr. V. Kerry Smith, Department of Economics, North Carolina State University, Raleigh NC
MEMBERS
Dr. Nancy E. Bockstael, Department of Agricultural and Resource Economics, University of
Maryland, College Park, MD
Dr. A. Myrick Freeman, Department of Economics, Bowdoin College, Brunswick, ME *
Dr. Charles D. Kolstad, Department of Economics, University of Illinois, Urbana, IL
Dr. William Nordhaus, Department, of Economics, Yale University, New Haven CT
Dr. Bryan Norton, School of Public Policy, Georgia Institute of Technology, Atlanta GA
Dr. Wallace E. Dates, Department of Economics, University of Maryland,College Park, MD
Dr. Paul R. Portney, Resources for the Future, Washington DC
Dr. Robert Repetto, World Resources Institute, Washington, DC
Dr. Richard Schmalensee, Massachusetts Institute of Technology, Cambridge MA
Dr. Robert N. Stavins, Kennedy School of Government, Harvard University, Cambridge, MA
Dr. Thomas H. Tietenberg, Department of Economics, Colby College, Waterville, ME
Dr. W. Kip Viscusi, Department of Economics, Duke University, Durham, NC
SAB COMMITTEE LIAISONS
Dr. William Cooper (EPEC), University of Michigan
Mr. Richard Conway (EEC), Union Carbide Corporation
Dr. Morton Lippmann (IAQC), Nelson Environmental Institute, New York University
Dr. Roger McClellan (CASAC), Chemical Industry Institute of Toxicology
DESIGNATED FEDERAL OFFICER
Mr. Samuel Rondberg, Environmental Health Committee, Science Advisory Board (1400F),
U.S. Environmental Protection Agency, Washington, DC 20460
iii
-------
TABLE OF CONTENTS
1 EXECUTIVE SUMMARY 1
2 INTRODUCTION 6
2.1 Background 6
2.2 Charge 6
3 DETAILED DISCUSSION 9
3.1 Organization of the Report 9
• 3.2 Outline of Committee's Review Procedures for the Preparation of
this Report .^. 9
3.3 Committee Evaluation and Response to the EPA Charge ".'... 11
3.3.1 Comparison with the NOAA Panel Guidelines 11
3.3.2 Specific Issues of Concern 14
3.3.2.1 Definition of the Commodity ..: 14
3.3.2.2 Embedding 17
3.3.2.3 Partitioning of Total Values 17
3.3.2.4 Consistency with Mitchell-Carson Existence Value
Study 19
3.3.2.5 Box-Cox Estimation and Treatment of Zero Bids ... 20
3.3.2.6 Statistical Analysis of Pre-test and Final Samples ... 21
3.3.3 Responses to Specific Charges 23
4 CONCLUSIONS AND RECOMMENDATIONS 26
4.1 Summary 26
4.2 Recommended Research 27
5 REFERENCES R-1
-------
1 EXECUTIVE SUMMARY
. The EPA Office of Solid Waste (OSW) has been developing methodologies to
assess the benefits of cleaning up (corrective action) contamination at active hazard-
ous waste facilities operating under the authority of the Resource Conservation and
Recovery Act. OSWs definition of potential benefits includes non-use values of dean
groundwater. The OSW (along with the Office of Policy, Planning, and Evaluation)
has supported a Cooperative Agreement with the University of Colorado to conduct a
contingent valuation study (McClelland et a/. ,1992) to estimate these non-use values.
The OSW requested that the SAB Environmental Economics Advisory Commit-
tee (EEAC) address the design, conduct and results of the contingent valuation study
with particular attention to methodological limitations that have been identified in the
literature on contingent valuation. The Committee was asked to review the study in
the context of the state of the art for contingent valuation methodology, but to be
aware of the practical constraints under which the effort had been carried out (e.g.,
budget limitations). The Committee devoted three meetings (in whole or in part) to
this review - December 18, 1992, April 30, 1993, and July 13, 1993, completing a
detailed review of the research. The Charge for the EEAC review was organized
around five major questions posed by the OSW, and summarized below:
a) Did survey respondents understand groundwater sufficiently to give a
meaningful value to the commodity in question?
b) Which of three similar methods should be used for estimating non-use
values from the survey responses?
c) Is the Box-Cox econometric procedure an acceptable and defensible
method for addressing large bids?
d) Embedding has been identified as a problem in contingent valuation
studies. Were the methods used to address the issue of non-bids,
scenario rejection and embedding reasonable?
e) Are the valuations obtained in this study sufficiently accurate and repro-
ducible so as to be used, in part, as a basis for EPA to determine the
non-use values of groundwater? Can the SAB advise the Agency on
-------
analyses of the survey results to improve the estimation of non-use
values for groundwater?
Addressing the last, but most encompassing, element of the charge first, the
Committee can not endorse the McClelland et a/, study as a means of generating valid
and reliable estimates of the non-use values associated with cleaning up contaminated
groundwater. Moreover, the Committee questions the study's applicability within the
context of the forthcoming RCRA Corrective Action Rule Regulatory Impact Analysis
(RIA). This CV study design flows from the premise that the values people place on
cleaning up groundwater that they are told will have no effect on their own water
supplies can be measured independently of the specific circumstances associated with
each contaminated groundwater resource. The benefits (or value) provided to people
from cleaning up contaminated groundwater are measured by what people'would be
willing to give up to obtain that improvement. With most commodities, these "sacrific-
es" can be measured from market transactions. For many environmental commodities
this is not possible. Indeed, in situations where people experience satisfaction from
the knowledge that a resource is restored or preserved irrespective of any direct use
they might make of it, there is a particular need for the type of innovative research
strategies attempted in this McClelland et a/, study. Their approach used survey
methods to ask a representative sample of adult decision makers what they would be
willing to pay to dean up a contaminated groundwater resource. While the survey
approach is not accepted by all economists, it has been accepted in principle for
natural resource damage assessment by the recent (Federal Register, January 15,
1993) report of the National Oceanographic and Atmospheric Administration (NOAA)
panel of distinguished social scientists (including two Nobel Laureates) composed of
five economists and a survey researcher.
The Committee's concerns about the McClelland et a/, findings relate primarily
to specific details of this study's application of the CV technique. It was apparent to
Committee members that people answering the study's survey instrument could have
interpreted the services provided by cleaning up contaminated groundwater in a
numberof different, conflicting ways. There is no way to know which of these multiple
meanings these respondents adopted in answering the valuation questions.
Addressing other aspects of the Charge:
a) The Committee does not believe that the pre-testing and survey design
techniques offer sufficient evidence to ensure that a well-defined ground-
water commodity was understood property by all the respondents. The
sample sizes for the pre-tests were not large enough to permit
-------
multivariate analysis that would allow evaluation of some of the judg-
ments made by the principal investigators.
b) The Committee does not believe that any of the three possible methods
for separating the non-use or passive use values from total values can
be established as preferred at this time. Each should be regarded as an
innovative research technique and not one of proven reliability for esti-
mating non-use values from survey responses.
c) It is not possible to judge whether the Box-Cox econometric estimates
alone provide an acceptable and defensible method for dealing with the
scenarios and the large bids associated with them. Comparative evalua-
tions of these results, and the results of other approaches for dealing
with the skewness in the valuation responses across a number of sur-
veys, should be undertaken before accepting the Box-Cox results.
Because there are several different hypotheses about the source of large
bids, the literature does not provide one set of criteria for discriminating
among these approaches. Comparative evaluations provide one means
of evaluating whether the specific approaches used to address skewed
bid distribution influences the summary statistics used for policy analysis
or the conclusions drawn or based on survey findings.
d) We do not believe that the approaches for treating embedding, dealing
with scenario rejection, and reflecting the potential effects of non-biding
responses can be assessed for their reliability on the basis of the infor-
mation provided in the report.
The above problems notwithstanding, the Committee wishes to commend EPA
staff for supporting exploratory research of this nature. There is little doubt that this
report represents a substantive contribution, extending our understanding of the issues
associated with contingent valuation estimation of non-market values. It provides new
research insights to the evolving literature on contingent valuation. However, such
innovation is not the criterion that the Committee was asked to use in evaluating the
report, nor the foundation for use of the study by EPA. In many respects, the Commit-
tee feels that the problems in using the findings of the study to meet the needs of the
RIA effort arise from requirements imposed on the research - the need for a separate
estimate of non-use value and for a method that abstracted from the specific features
of the local conditions associated with each specific case of groundwater contamina-
tion. After reviewing the full record, the Committee feels that this strategic decision on
-------
EPA's part is the cause of many of the study's problems as well as the difficulties of
implementing the research findings in the RIA context. The approaches taken to deal
with these requirements have no basis in the theory of non-market valuation, nor
precedent in practice, and were never subjected to peer review.
The Committee's report offers specific suggestions for further research to help
resolve the questions raised by this study. Five general areas seem especially
relevant for EPA's activities associated with the corrective action RIA:
a) For the most part, economic research has focused on developing esti-
mates of the values of the typical or representative household demand-
ing increases in the amount available of specific commodities (or im-
provements in them) and not the criteria for deciding which households
would be among the groups having such demands or values. Address-
ing this question is fundamental to the task of measuring aggregate
values for the cleanup of groundwater resources and may well have a
much greater quantitative impact on these aggregate estimates than do
refinements in estimates of the representative household's values for
changes in some environmental commodities.1
b) Detailed study of the sensitivity of CV outcomes to the survey methods
used - whether telephone, in-person, or mail surveys is needed. The
evidence provided to the Committee suggests that there remains some
questions in the profession about the results provided by telephone or
mail surveys in comparison with in-person interviews. More research on
this topic is dearly warranted.
c) Research addressing issues associated with defining changes in envi-
ronmental commodities is central to understanding whether the contin-
gent valuation responses are internally consistent. The recommended
' research would involve developing methods and practical protocols that
could be used to understand how environmental commodities are best
measured and how those measures relate to the descriptions offered in
the framing of contingent valuation questions.
'Given the extremely fmKed nature of the available reeeareh on the value of groundwater cleanup, the Committee feete that
In thfe caee both typee of research are exceptional/ important
-------
d) Research on the need for, and the ability of, respondents separating their
total value of increases in environmental commodities (such as the
cleanup of contaminated groundwater) into use and non-use compo-
nents.
e) Research on the development of methods for gauging the potential
implications of using a hypothetical setting to elicit valuations in relation-
ship to real choices is important to the ongoing development of the
contingent valuation method. Research evaluating whether or not
contingent valuation results can be transferred from one setting to others
is also critical. The Committee believes this should be a central compo-
nent of the future research that EPA undertakes in support of its pojicy
evaluation.
The Committee assumes that there will be a continuing need to evaluate
analyses intended to appraise household's willingness to pay for improvements in
environmental commodities. There is simply not sufficient information on the proverbial
"research shelf' to address in a meaningful way many of the questions that have been
posed to the EEAC about specific decisions in the design of contingent valuation
surveys and in the transfer of results from such surveys. EPA staff have attempted to
do their best to meet policy needs in the presence of this limited information. Their
Charge to the EEAC reflected a desire to have the Committee's judgement, based on
its collective research experience, substitute for a documented research record on
these issues. Our conclusions reflect the fact that the problem is not simply that EPA
has been unable to sustain a research program to address these questions. Overall,
there does not exist a set of research available for the Committee to use in forming its
own judgements, EPA should begin to sponsor the research required to evaluate the
methods used in its policy evaluations and to develop more experience in valuing
changes to important environmental resources.
-------
2 INTRODUCTION
• 2.1 Background
The EPA Office of Solid Waste (OSW) is working to complete a final rule
requiring the dean-up of contamination of active hazardous waste facilities operating
under the authority of the Resource Conservation and Recovery Act (RCRA). A
significant area of benefits from the rule is expected to be the dean-up of groundwater
contamination, as well as other environmental media. To provide dedsion makers
with a complete assessment of the benefits of corrective action, OSW has adopted a
broatJ definition of potential benefits that indudes non-use values of dean groundwa-
ter.
The OSW has provided support to a Cooperative Agreement with the University
of Colorado to conduct a contingent valuation study to estimate these non-use values:
The contingent valuation study is moving toward completion, under the direction of
Drs. McClelland, Schulze, Lazo, Waldman, Doyle, Elliot, and Irwin.
The OSW requested that the SAB address the design, conduct and results of
the contingent valuation study. The researchers were aware of the need to cope with
four potential sources of error that have been identified in the literature on contingent
valuation. These sources indude: (1) large bids not indicative of willingness-to-pay;
(2) scenario rejection or refusals to bid; (3) embedding; and (4) effects of context on
bids.
Although the study was conducted to place emphasis on pretesting, instrument
design, sampling, and econometric analysis of CV survey data, there were also a
number of practical constraints. The survey instrument length was limited to insure a
reasonable response rate from a mail survey. A mail survey was used because
estimates on a national scale were needed; the sample size had to be large enough to
allow comparison among survey variants and tests of several key questions. Lastly,
the budget was limited. The Committee was asked to review the study in the context
of the state-of-the-art for contingent valuation methodology, but to be aware of the
constraints noted above.
2.2 Charge
The specific charge was organized around five questions. They are summa-
rized below:
6
-------
a) The survey instrument design was developed through focus groups using
a cognitive survey design approach and different survey variants. Two
levels of context were included among the survey variants to test the
effect of context. This was particularly important because groundwater is
an exotic commodity. There was concern during pretesting that ground-
water would not be understood well enough by respondents to properly
give a value. Were the pretesting and subsequent survey design tech-
niques reasonable methods to use? Does OSWER have sufficient
evidence and support, after using these methods, to insure that ground-
water was understood enough to property give a value?
b) The survey was designed to allow estimation of non-use values for,
groundwater by three possible methods. The three methods produced
similar results. Does the SAB have a preference regarding which
method should be used for estimating non-use values from the survey
responses?
c) The study used the econometric technique of Box-Cox transformations to
address large bids. OSWER believes that the information content of
large bids needs to be considered in the results. Is the Box-Cox econo-
metric procedure an acceptable and defensible method for addressing
large bids?
d) The survey had to address the issue of non-bids or scenario rejection by
using careful survey design techniques. Scenarios that people would
accept were pretested extensively. The cognitive survey design ap-
proach was instrumental in facilitating the pretesting. Embedding has
also been identified as a problem in contingent valuation studies.
Embedding was addressed by posing explicit questions that ask the
respondent to rethink a previous answer and reduce their bid based on
how much the values previously given for groundwater cleanup were
actually for other environmental problems. Were the methods used to
address the issue of non-bids, scenario rejection and embedding reason-
able?
e) In developing regulations under its several legislative mandates, the
Agency is required to produce analyses that determine the benefits of
regulations intended to protect groundwater. Are the valuations obtained
in this study sufficiently accurate and reproducible so as to be used, in
-------
part, as a basis for EPA to determine the non-use values of groundwa-
ter? Can the SAB advise the Agency on analyses of the survey results
to improve the estimation of non-use values for groundwater?
8
-------
3 DETAILED DISCUSSION
3.1 Organization of the Report
The Committee's report on the review of this contingent valuation study is
organized into three sections. In the first section we describe our evaluation process.
We include this section because the steps we took in our review are quite relevant to
our judgment on the report
In the second section, we discuss the implications of the NOAA Contingent
Valuation Panel's Report for the evaluation of McClelland, et a/., as well as several
issues that were not addressed in the charge. We summarize our conclusions-with
respect to each of the issues identified in the charge. And we present our overall
reactions to this report.
In the third section, we identify three major areas of substantive research
concerning the use of contingent valuation methods for the valuation of nonmarket
resources such as groundwater. We recommend that EPA initiate a program of
research in each of these areas.
3.2 Outline of Committee's Review Procedures for the Preparation of this Report
The McClelland et at. contingent valuation study of groundwater reviewed was
prepared as part of the research necessary for EPA's Regulatory Analysis for its
Corrective Action Rule for RCRA sites. Our review was undertaken in several stages.
A detailed review was required because the Office of Solid Waste did not provide any
information indicating that the McClelland et a/, contingent valuation report had been
subjected to a peer review process prior to submitting it to the Committee.2 Indeed, it
is our understanding that the research design for the benefit transfers planned as part
of the Regulatory impact Analysis was not subjected to a comprehensive external
review. We believe this was a mistake. It had a significant effect on the design of the
McClelland et a/, research, and therefore we will return to it below. In the absence of
a documented peer review of the draft final report, the Committee believed it was
essential to conduct a detailed peer review of the research as well as to evaluate the
issues associated with the Charge given to the Committee.
^Subsequent to the preparation of this report, peer review material* were provided to one of the Committee Chair*. These
material* addre»*ed an interim report, not the draft final report. Moreover, there. wa« no indication of how the research design
was modified to respond to the comments.
9
-------
The first stage of our review consisted of a briefing on December 18,1992 by
EPA staff on the goals specified for the contingent valuation study. In this presenta-
tion EPA staff outlined its needs in relationship to the Regulatory Impact Analysis.
The staff identified a pre-defined set of research goals that were imposed on the
principal investigators and that conditioned their decisions concerning research
strategies. One of the goals was to obtain an estimate of the non-use values (or
"passive use" value, using the terminology of the 1989 Court of Appeals decision and
the NOAA Contingent Valuation Panel Report) for the cleanup of groundwater
resources that could be applied as a separate component to any use values estimated
separately on a site specific basis. It is our understanding, based on that briefing, that
thi*goal was specified by EPA staff. It was not based on a recommendation of the
principal investigators as a component of their research design. Nor was it a goal that
was developed as a result of existing research findings on the measurement of use
and non-use values (i.e., the earlier Mitchell-Carson (1989) study using focus groups
to evaluate how people conceived of groundwater resources and whether they might
have non-use values for them).
Following the overview of EPA goals, the Committee Co-Chair (Smith) present-
ed an overview of contingent valuation methods and an initial summary of the key
elements in the McClelland et a/, contingent valuation survey. Because the Commit-
tee was aware of the ongoing NOAA Contingent Valuation Panel's assessment of the
CV method, the Committee decided to wait until the report of that panel was available
before undertaking a detailed review of the McClelland et a/, study (The report was
subsequently published in the Federal Register for January 15,1993).
The second stage of our review process involved assembling a detailed set of
questions for the principal investigators of the CV study. These questions summarized
the issues that arose in the Committee's initial review of the McClelland et a/, study.3
The questions posed to the authors also outlined the concerns that the Committee felt
needed to be addressed by the principal investigators in a subsequent meeting.
As part of the preparation of the equivalent of a peer review, the Committee
requested three sets of consultative activities. First, the Committee asked Dr. Robert
C. Mitchell to evaluate the background information forming the basis for the design of
the contingent valuation survey. Mitchell, along with Dr. Richard T. Carson, had
conducted for EPA qualitative research on the issues associated with attempting to
3Th« qu»*ttont were trantmtted to the author* In a memorandum from Kerry Smith and Aten Km«M to WBfam Schutzt
January 11,1993.
10
-------
measure non-use or existence values for groundwater prior to the McClelland et al.
report cited above). The McClelland et a/, report indicates that it intended to build
upon the findings of the Mitchell-Carson study. The charge given to Mitchell (Smith
and Kneese memorandum, March 23,1993) proposed that his review focus on the
interpretation given to this earlier Mitchell-Carson work by McClelland et al.
The Committee asked Drs. Richard Bishop and Gregory Poe to evaluate the
implications of their recent study of the influence of information on individual's valua-
tion of groundwater cleanup. These researchers were asked to consider their findings
in relation to the information that was provided to respondents in the McClelland et al.
contingent valuation study. Finally, Dr. Kevin Boyle was asked to prepare a detailed
review of the literature including the McClelland et al. study. This review was to-
include a comparative evaluation of all aspects of the earlier literature on the contin-
gent valuation method to value cleanup of groundwater resources (K. J. Boyle, 1993).
The third stage in the Committee's review process involved a second meeting
of the Committee (a) to: discuss the report of the NOAA Panel on contingent valuation
methods; (b) to allow the principal investigators to describe their research design in
more detail and to provide detailed responses to the January comments and ques-
tions; and (c) to hear the reports of the three groups of commissioned consultants to
the Committee. Following this meeting, a draft report of the Committee was prepared
and circulated to the Committee members. The Committee's third and final meeting
evaluated and revised this report and finalized the Committee's recommendations
about the study. During the final meeting, Dr. William Schulze presented some
clarifying information to the Committee. A summary of the authors' written responses
to Committee questions (McClelland, Schulze, and Lazo, June 23, 1993) was circulat-
ed shortly before the final meeting.
3.3 Committee Evaluation and Response to the EPA Charge
3.3.1 Comparison with the NOAA Panel Guidelines.
The Committee decided to delay its review of the McClelland et al. report until
after the NOAA Panel's report on the contingent valuation method was available. This
was done because the Committee did not have the resources or time to undertake its
own review of the CV method. The NOAA Panel concluded that "contingent valuation
studies can produce estimates reliable enough to be the starting point for a judicial or
administrative determination of natural resource damages - including lost passive use
value." After reviewing the objectives of the NOAA Panel and its recommendations,
11
-------
the EEAC acknowledges that the mandate requested of the NOAA panel differs from
that associated with using the results of a CV analysis for general policy purposes.
Moreover, the Committee does not intend to endorse or agree with all aspects of the
NOAA report Rather, we have simply used the NOAA Panel's guidelines as one of
several starting points for an evaluation of the McClelland et al study.
The NOAA Panel's guidelines for the design and implementation of CV survey
methods included4: use of in-person interviews; use of a referendum (or discrete
choice) format in asking contingent valuation questions; obtaining a high response rate
to the questions; investigating the responsiveness of the "willingness to pay respons-
es" to changes in the amount of the environmental commodity offered to respondents;5
attempting to determine whether respondents understood the tasks, considered their
budgets in the process of answering contingent valuation questions, and believed the
scenario presented to them. With discrete choice questions, the NOAA Panel
recommended a follow-up to any yes/no questions with specific attempts to identify
responses that indicated concern over available resources to make the stated pay-
ments. Scenarios should demonstrate that respondents considered seriously the
private and public substitutes for the commodities offered as well as these budget
constraints.
Because the McClelland et al. study was conducted long before the NOAA
report was available, it is unrealistic to expect a direct correspondence between the
procedures used in the McClelland et al. report and the NOAA Panel's recommenda-
tion. It also should be noted that not all of the members of this Committee fully
These comments paraphrase the specific requirements identified In the NOAA Pane's summary of conditions required to
satisfy their burden-of-proof requirement They also include the recommendations made about specific implementation proce-
dures. The specific cftabon tor text of the report is: federa/ ffegister. January 15,1983. VoL 58, f 10, pp. 4601-4614.
5 The NOAA Panel report uses the term 'scope* to refer to changes In the environmental commodities that are intended to
represent the- injuries to one or more specific environmental resources. K tt identified ae 'scope* In their burden of proof
requirements and that specific patterns of responsiveness In wBngness to pay acknowledges that the predictions torn economic
theory about the properties of the wfflngness to pay are lotted. This is especialy true when there may be more than one
environmental resource affected and the reductions hi Injuries (Le., Increases In the environmental commodities Involved) that are
being valued may be different across resources. The specific text of (he NOAA Paners report that seems to be identified as
being associated win the scope requirement describe* the condUon as blows: fiaffonalry in fe weakest form requires certain
Und* of consistency umong cnofce* made by MMduafe. for instance, t tn MMdutl cnooss* some purcnases af a gtvon aef of
price* and Income, tout I some prices M and (her* are no other changes, th» good* 1h»t th* MMdiMl woutt now buy m»k»
Mm or ner Better off. Slmfany, we would expect an MMdutf* preference* over pubic good* fie. btfdgo*. nfenwsys, a* ouaify)
to reflect me same Und e/consistency.
Common notion* ofnOoniKy Impose other rvowre/nenb which tn relevant In dMarsnf contexts, t/susfy, ttough not
a/ways, ft is /MsenaMe to suppose fief more of something i^fMt* good *b»lt»r to king** in MMdu»l I* not a»1*ML
Thh *i gene/i/^/issued1 silos wmngnou to pey somewhat mo/e fermcye of a good t* Judgod by fte MMdutL Also, JT
me/gine/orJncramento/wff*vnesstopeytoredkflfone/a/noestfeclrw wMdemountalrMoVaivisMe. l(Isusuajy/K>(
rseaoneMe to assume thef* decfnes very ateuptV (emphasis added) pp 4604.
12
-------
endorse all of the NOAA Panel's guidelines. Nonetheless, the EEAC started its
evaluation by considering the McClelland et al. study's design in relationship to the
• NOAA Panel recommendations. Several notable differences were identified.
McClelland et al. used mail surveys and not in-person interviews, a fact that was a
concern of some members of the Committee. The interview format did not allow
evaluation of whether the questionnaires were understandable to respondents of the
final survey and did not incorporate a test for the effects of changes in the scope of
the commodity offered respondents, as recommended by the NOAA panel. While
considerable attention was paid to a specific set of substitutes, the set of substitutes
for contaminated groundwater was narrowly defined and the set of substitutes
available with scenario rejection was not made dear. The NOAA Panel did not
discuss the procedures employed in the McClelland et al. report for dealing wjtb-
disembedding and for partitioning the total willingness to pay into use and passive use
components. Additionally the study did not utilize the close-ended or referendum
format for the contingent valuation questions, using instead a payment card approach:
While the discussion of the pre-test results suggests that a closed-ended contingent
valuation was evaluated, the questions used were not in the same format as recom-
mended by the NOAA Panel (based on information given in the report, the principal
investigator's comments, and the Boyle (1993) review). The authors' pre-test did not
evaluate the discrete choice approach recommended by the NOAA Panel. Thus, the
questioning format did not correspond to that recommended by the NOAA Panel.
Overall, the McClelland et al. study does not satisfy the burden-of-proof
standards described by the NOAA Panel for reliable CV assessments of the passive
use values associated with natural resource damages. It is important to acknowledge,
however, that the NOAA Panel report did not describe these features of CV studies as
absolute criteria for reliability. The report acknowledges that reliable estimates could
be realized without full adherence to all the recommendations. As the NOAA Panel
report dearly indicates, it is possible for CV studies to achieve comparable levels of
reliability without adhering to their recommendations, but the burden of proof must be
satisfied" by other means. Accordingly, given the Committee's position that it does not
necessarily endorse the NOAA Panel's recommendations, our evaluation of reliability
and validity was based on the general definitions of these concepts as they have been
used in the CV literature (R. C. Mitchell and R. T. Carson, 1989). As such, this
evaluation is a professional judgment that took the comparison with the NOAA criteria
as an approximate template in the Committee's first stage review of the McClelland et
al. study.
13
-------
The McClelland et al. report exhibited several commendable features. Although
it used mail questionnaires, it did achieve a relatively high response rate (63.4 percent
.of total questionnaires were returned; 43.7 percent of the total were available for
regression analysis). The report included an extensive amount of qualitative analysis
associated with the design of their survey instrument and several pre-tests to evaluate
its performance. At the early design stage of questionnaire development, there was
extensive attention to verbal protocol analysis, focus groups, and other qualitative
approaches to evaluate the extent to which individuals were comprehending the
commodity that was being offered to them. Equally important, a series of pre-tests
were used to evaluate the sensitivity of responses to the structuring of the valuation
questions. While concerns were raised about the relative sizes of the samples
underlying these pre-tests, the Boyle (1993) appraisal and other reviewers suggested
that this amount of work was among the most extensive ever conducted in a contin-
gent valuation study for groundwater cleanup. Unfortunately, there does not appear to
have been evaluation of the final survey instrument to determine whether the judge-
ments from the pretests were consistent with an independent set of respondent's
understanding of the CV questions.
3.3.2 Specific Issues of Concern
The Committee found difficulties with several elements of the design and
analysis presented in the McClelland et al. report. Some of these are highlighted in
brief terms in our summary judgment. We discuss a few of them in more detail below.
3.3.2.1 Definition of the Commodity
The first of these concerns the definition of the commodity to be valued by
respondents. The Committee found that there were multiple interpretations of what
was offered to respondents based on the information conveyed by the questions
comprising the valuation task. The most direct explanation of the Committee's
reasons for concern stems from the reactions of Committee members to the question-
naires. Careful readings of the survey instrument by several members of the Commit-
tee resulted in different interpretations of the commodity that was being valued. These
differences (along with similar questions about the survey instruments associated with
the experimental variations included as different versions) raised questions about the
ability of lay persons to understand the nature of the commodity they were being
asked to value.
14
-------
The Committee found that the sample sizes and evaluation of the information
materials and pre-testing and design techniques were not sufficient to statistically test
whether respondents had enough information and understood the commodities offered
to them. Equally important, it was not possible to discriminate between the various
interpretations of the commodity offered to respondents. In order to present a
comprehensible choice to respondents, the CV survey instrument must dearly specify
the nature of the environmental commodity they will receive if they choose not to
purchase what is offered and the modifications to that commodity, the access condi-
tions and the nature of payment(s) if they choose to purchase what is offered. The
no-purchase condition will be referred to as the default condition and all aspects of the
change in the commodity as the commodity change.
There is a basic problem in interpreting the valuation question (Q11), p'rimarily
associated with an ambiguity in the problems that can arise from the contaminated
water. In the base scenario, forty percent of the water used by the community was
described as coming from groundwater contaminated as a result of a leaking public
landfill, in the discussion of the context for the commodity, contamination increased
the risk of dying from cancer by about ten additional deaths per million per year
among people who drink the water. An event with comparable risk was described for
respondents.6 However, it is subsequently stated that the water must not be used for
drinking and cooking. Thus, individuals may not perceive health risk to be a feature of
the default conditions of water supply and risk perception may not enter into the
individual's decision process. In this case the only implication of contamination is the
possibility of water shortage.
In fact, it is not dear what respondents view as the default scenario for their
valuation. That is, in order to place a value on the primary scenario of complete
deanup, respondents must dearly understand the reference point for their valuation,
that is, the condition of their water supply if complete deanup did not occur. This is
important because acceptance/rejection of a proposed dean-up plan relative to a
default option characterizes the commodity purchased. The scenario could be
interpreted as describing an increased risk and the prospect that a substantial fraction
of their water supply would be contaminated. But since the scenario also specified
that it could not be used for drinking or cooking, it is not dear that those answering
this question would perceive any risk as being present Furthermore no information
was provided on alternative supplies or response patterns. Other options are de-
scribed before the contingent valuation scenario. But there is no statement about
HTie term risk to uMd here to mean the Metime probability of death.
15
-------
whether these alternatives would be able to be used for the particular groundwater
contamination being valued. Under the most optimistic interpretation of questions Q6
through Q10, individuals might have selected one of four alternative base cases to
define the default scenario in considering their valuation of complete cleanup. The
default scenario constitutes the reference point for their valuation estimate. The
difficulty is that we do not know which of these alternatives (if any) would function as
the method of choice for any particular respondent would choose in the event that
they did not select complete cleanup.
In the base survey instrument, the respondent must somehow judge whether
the uncontaminated 60% of the water supply will be adequate. There seems to be an
implicit assumption that the community must find an additional water source. .This
implication appears to arise from Q10 and is not stated elsewhere. If the respondent
perceives a water shortage, all that is being asked is the demand for water. Of
course, this interpretation makes the further assumption that respondents have
accepted the conclusion offered in the instrument that risk has been precluded by the
water policy in effect.
Respondents' satisfaction ratings for each of the alternative approaches (i.e.,
containment, public treatment, home treatment, or water rationing) do not help resolve
the ambiguities because they do not provide a complete description of the adjustments
to their values associated with each of these reference points. Versions C and D of
the questionnaire allow comparison of public treatment with complete treatment, and
Version D elicits valuation adjustments in response to varying percentages of domestic
water supply contamination. Nonetheless, this would not allow us to completely
decompose (by scenario comparisons) the values that were offered for anyone who
did not receive one of these questionnaires.
The method of eliciting valuation estimates from some versions of the question-
naire as a percentage of the complete cleanup valuation caused some concern.
When individuals are offered changes in the scenario, they are offered adjustments to
the base case (i.e. a complete cleanup scenario) with specific percentages of value
offered as the responses. Where we have a priori expectations that the values given
should be larger, the percentages are scaled at a larger rate; where there are a priori
expectations that the values should be smaller, the percentages are scaled over a
smaller range. This formatting of the questions preconditions responses that individu-
als can give. The literature provides no basis for judging the reliability of these
percentage adjustments. Moreover, there does not appear to have been a test of
16
-------
using alternatives to determine whether the formatting influenced the plausible
response.
3.3.2.2 Embedding
Another source of concern is the question associated with disem bed ding.
Embedding is a term that has had a variety of interpretations in the economics
literature, with widespread use of the concept generally associated with research
reported by Kahneman and Knetsch (1992). In the context of this report, we have
interpreted embedding as a situation where survey respondents report values for
commodities that are more inclusive than what is desired by those analysts or policy
makers intending to use their responses. This can arise because respondents...
interpret the commodity differently from the analyst or because they assume that other
commodities would be provided along with the one that has been offered (even though
the survey instrument does not imply this to be the case). The McClelland et al.
disembedding question (Q12) asks individuals to reconsider the dollar amount they
stated (in response to the preceding question, Q 11) they would be willing to pay for
complete groundwater cleanup and asks them to characterize this amount as: (a) "Just
for the stated groundwater program;" (b) "somewhat for the groundwater program and
somewhat a general contribution to all environmental causes;" (c) "basically a contri-
bution to all environmental or other worthwhile causes;" or (d) "other." In question 13,
a percent of a dollar amount is requested. If individuals are asked in question 11 to
describe the most they would be willing to pay each month on top of their current
water bill for each of the next ten years for complete groundwater cleanup program,
the Committee wondered why they would immediately thereafter be willing to admit
that this response was actually for other things. It would be difficult to interpret how
any household would respond to this question if they did not know it in advance, i.e. if
the questions were conducted at an in-person interview or a telephone interview where
respondents do not know what is coming next in the question sequence. When these
questions are presented in a mailed questionnaire (where it is assumed that respon-
dents can read ahead), the response to this type of question is even more difficult to
interpret and there is no basis in the literature for assessing this issue. Thus the
methods used in this study to adjust for embedding have not been evaluated in the
literature.
3.3.2.3 Partitioning of Total Values
One of the objectives of the study was to estimate the non-use values of
cleaning up groundwater contamination. These estimates were to be used in the
17
-------
benefit assessment required for the Regulatory Impact Analysis of RCRA corrective
action rule. The Committee concludes that the report does not provide evidence that
respondents adequately distinguished the use and non-use values for cleaning up
contaminated groundwater. Three methods were used to estimate non-use values:
direct questions requiring allocation of total bids over composite categories; compari-
son of "willingness to pay" (WTP) responses for different commodity scenarios; and
extrapolation by fitting an assumed functional form to WTP bids for different levels of
water shortage.
The first method was implemented through Question 14 which asked respon-
dents to decompose their complete groundwater cleanup bid into motivational catego-
ries. A limited number of investigators have used this approach to isolate use .and
non-use values, but there is no evidence in the literature about the method's perfor-
mance.7 The maintained hypothesis that respondents to CV questions can decom-
pose their bids into constituent parts is difficult to accept and there is little evidence to
support its use. For example, it seems reasonable to suggest that people would have
difficulty decomposing their willingness to pay for something as familiar as an ice
cream sundae into percentages they would associate with texture, temperature, looks,
taste, etc. At the minimum we would expect that this type of detailed information
would need to be elicited very carefully - in effect "coaxed" from them in a very
detailed but logical way. The reliability of the second method is questionable due to
confusion over the definition of the commodity and default scenarios, as discussed
above. With regard to the third method, we have no past experience with the relation-
ship between this method and the tasks requested of respondents, nor any reason to
believe that the intercept of a fitted function can be interpreted as a non-use value.
Finally, there is little basis provided in the study for gauging the transferability of such
results from other situations requiring the estimation of non-use values for cleaning up
contaminated groundwater resources.
By criticizing the techniques used in this study to separate use from non-use
values7~the Committee does not intend to address the broader issue of the validity of
that distinction. Some members of the Committee questioned whether it is appropriate
to ask respondents to partition their willingness to pay into discrete elements that were
associated with components of those respondents' total values. This should be a
matter for further research.
7S«« Boyto (1893] for dtecuMfcn and tvakjatton.
18
-------
3.3.2.4 Consistency with Mitchell-Carson Existence Value Study
The Committee found some inconsistency between the lessons learned from
the earlier Mitchell-Carson evaluation of existence values as reported by Mitchell and
the McClelland et a/, interpretation of these results. For example, McClelland et al.
summarize the objectives of their study:
The aim of our current study is to estimate non-use values for groundwa-
ter cleanup. This commodity, of great interest to the U.S. EPA, also
appeared to be ideal fora methodology study since in early development
work undertaken for U.S. EPA by Mitchell and Carson, it was apparent
that (1) people were generally poorly informed about groundwater. con-
tamination, and (2) people resisted non-use scenarios used for evalua-
tion in which groundwater was preserved but never used. In other
words, the scenario was rejected by respondents. Delighted with our
exotic commodity, groundwater cleanup, our strategy was to apply two
new methods in designing this survey instrument (page 22).
By contrast, the Mitchell-Carson report concluded somewhat differently from
what the authors indicated. It notes:
We believe it is possible to use the contingent valuation method to obtain
a credible dollar measure of the existence values of groundwater by
using a scenario of the type described in this chapter. The device of
having people evaluate hypothetical GNN (groundwater not needed for
human use) groundwater (aquifer) that can be plausibly isolated from
other aquifers and which lies at a great distance from those who are
being interviewed should effectively eliminate any use values. The
concrete barrier plan promises to provide a credible protective option for
the GNN aquifers so that those who choose it will be stating how much
they would be willing to pay to preserve the aquifer defined as GNN from
contamination. These containment features should minimize any influ-
ence on people's willingness to pay that might result if they continue to
believe that contamination in groundwater travels at a much higher
velocity than is actually the case.... Given the difficulty of convincing
people that the aquifer will never be needed for human use, a portion of
the values will include the utility people get from vicarious protection.
The vicarious protection values will be minimized by the scenario fea-
tures that are intended to protect others from inadvertently using ground-
19
-------
water in the basin.... There is no valid way to obtain separate mea-
sures of the several types of existence value to this study. First, it is too
difficult to overcome people's belief about future use by others to design
a scenario that would only capture stewardship values. Likewise, we see
no way to design a scenario that would only capture bequest or inherent
values for groundwater. Second, it is cognitively unrealistic to ask
respondents to state what proportion of the total value they ascribe to
each of these three types of existence values. However, it will be
possible to assess in a qualitative fashion the degree to which respon-
dents are influenced by these several types of value by using the follow-
up motivational questions (pp 83-85, emphasis added).
Both studies identify difficulties in separating use and non-use values. In part
because of the Mitchell-Carson work, and in part because of their own pre-test and
verbal protocol analysis, it was dear to the principal investigators that it would be
extraordinarily difficult to explain the nature of groundwater resources that would not
have any foreseeable uses to them or other people, to respondents. Mitchell and
Carson recommended a separation by distance as well as an identification of other
aquifers to try to deal with this problem. McClelland et a/, adopted a different frame-
work. Concern about substitutes in the form of alternative aquifers and about the
importance of describing plans for protecting groundwater that would be believable to
respondents did not appear to influence the way in which McClelland et a/, structured
their scenarios. The allocation format used in the McClelland et al. report does not
conform to the follow-up motivational questions as described by Mitchell and Carson.
3.3.2.5 Box-Cox Estimation and Treatment of Zero Bids
The Box-Cox regression methods analyzed what is referred to as the "reduced
willingness to pay," the willingness to pay bid for complete groundwater cleanup
adjusted by the reported disembedding percentage. The results reported by
McClefland-SchuIze in the second of the Committee meetings suggest that the Box-
Cox modelling approach used to deal with a skewed bid distribution was quite
sensitive to the authors' treatment of zero bids. The magnitude of the Box-Cox
parameter exhibited substantial variation with the substitutions imposed for the zero
values of the willingness to pay bids. Despite the sensitivity to zero values, there was
no discussion of the treatment of zero bids or scenario rejection in the present report.
Moreover, the Box-Cox transformation is only one of a number of ways of dealing with
skewed error distributions and no comparative evaluation of the methods was under-
taken. Thus, the Committee concludes (in response to the third question of the
20
-------
Charge) that the report does not provide clear-cut evidence.that the Box-Cox proce-
dure adequately addresses the issues posed by large bids, zero bids, and scenario
• rejection. In addition, an evaluation of the zero responses, non-responses, large bids,
and non-responses to the valuation scenarios in total are essential for judging the
reliability of the analysis. The existing contingent valuation literature offers an array of
possibilities for dealing with each of these issues, and no single method has been
identified as preferred. In view of this work, it seems reasonable to suggest that
comparative evaluations of the different methods for treating outlying observations,
zero responses, and non-responses would be important for gauging the comparative
performance of the contingent valuation scenarios offered in this study.
The Committee recognizes that there are a variety of statistical methods that
can be used to address large bids, zero bids, and other anomalies. For example, one
might employ some robust regression approach. Using a statistical procedure to
address extreme observations does not, however, solve the more fundamental
problem of why such extreme responses arose. Do these responses reflect legitimate
heterogeneity in valuations, or do they reflect a failure by respondents to understand
the survey and give meaningful responses? Choice of the appropriate statistical
solution depends largely on the factors generating these extreme responses. As a
result, the Committee views these extreme bids as a matter of continuing concern that
would warrant further research in future studies.
3.3.2.6 Statistical Analysis of Pre-test and Final Samples
The design of the McClelland et al. survey instrument was intended to include a
set of activities that would evaluate the amount of information that respondents
needed to value groundwater cleanup. These activities included the verbal protocol
analyses and pretests conducted in the initial stage of the research to permit judg-
ments about the final success of the form of the survey instruments in communicating
the information people required to decide about a groundwater cleanup policy offered
to them. Because the pretests played a key role in the survey instrument's design,
the Committee considered the nature of the empirical analyses undertaken to develop
the conclusions on the instrument used in the final survey.
The empirical analysis included summary statistics, primarily means and
standard deviations, and some frequency plots of responses to the willingness to pay
questions against a few of the attitudinal variables for the pre-test and full samples
together. There were also comparable statistics for a variety of subsets of the full
21
-------
sample. There was little multivariate analysis of the other willingness to pay ques-
tions. While there is a much smaller sample associated with each of these variations
. in the questions, a comparative evaluation of multivariate functions estimated for each
of the willingness to pay responses using the final sample would contribute information
to help assess the reliability of the valuation estimates, it would provide indirect
evidence on the effects of changes in the scope of the commodity; for example, larger
percentage shortfalls would be expected to be associated with greater use-related
motives, where scenario changes from complete cleanup to containment or public
treatment would be regarded as smaller amounts of the "cleanup" commodity.
* Table 7.5 in the McClelland et a/, report provides suggestive evidence, both in
the full sample and the regression sample, of a response to variations in the com,-
modity design . However, it is not dear whether these differences are associated with
the actual characteristics of the commodities or with particular features of the sub-
sample (i.e. lower income level, socio-economic status, or other factors), so that these
factors would need to be taken into account in appraising whether these were
consistent differences in the valuation responses across scenarios. The McClelland et
a/, evaluation of the effects of changes in the commodity (across the versions of the
survey questionnaire) did not independently evaluate the change in respondents'
willingness to pay to the features of the commodities. It was conditioned on accepting
the Box-Cox model (and its predictions) as a maintained hypothesis. These predic-
tions were then used, together with the percentage responses to commodity changes
elicited from each respondent based on which version of the questionnaire they
received. No attempt was made to model the adjustments and to incorporate this in a
broader description of the adjusted valuations. The Committee believes that such
models, both for the base case valuation and for other valuation responses that relate
them to the characteristics of the respondent, would provide indirect evidence of the
reliability of the survey methods.
Concerns were also raised with respect to the responsibility variable (Q 15).
While trie" briefing from the principal investigators suggested that the results are not
markedly changed by removing this variable from the willingness to pay models, the
performance of the variable as a gauge of acceptance of the scenario and its relation-
ship to other characteristics of respondents could be quite important
More complete treatment of the sub-samples that include a) individuals living in
areas with sites on the National Priority List; b) individuals who do not receive water
bills - i.e. individuals with private wells; and c) individuals with varying experience with
groundwater contamination, would offer opportunities for additional insights into the
22
-------
reliability of the responses. Because this work was not undertaken as part of the
analysis of the survey and remains to be done, it is difficult to reach an overall
•conclusion on the reliability of the willingness to pay responses interpreted as total
values for groundwater cleanup. Preliminary analysis undertaken by Committee
members suggests that there may be further insights from this type of empirical
evaluation.
3.3.3 Responses to Specific Charges
The Committee's response to each of the specific questions associated with the
charge is as follows:
a) We do not believe that the pre-testing and survey design techniques
used in this study offer sufficient evidence to ensure that a well-defined
groundwater commodity was understood properly by all the respondents.
The results of these analyses support the conclusion that groundwater
issues were important to respondents. However, the sample sizes for
the pre-tests were not large enough to permit multivariate analysis that
would allow evaluation of some of the judgments made by the principal
investigators. In particular, the process of reducing the information set
that was provided to individuals, the decisions on eliminating the risk
information, the selection of a valuation payment card over alternative
methods, and the definition of the commodity itself were not sufficiently
tested in the pre-test and verbal protocol analysis in a way that would
enable us to judge whether these were the most understandable ap-
proaches for dealing with these issues.
b) The Committee does not believe that any of the three possible methods
for estimating the total willingness to pay for non-use or passive use
values can be established as most preferred on the basis of the evi-
dence that is currently available in the.literature. This is not to suggest
that the methods are incorrect or that they might not ultimately provide a
basis for estimating non-use values in some cases. The evidence to
date however provides no basis for judging them. They should be
regarded as innovative research techniques and not verified methods for
estimating non-use values from survey responses.
The Committee remains skeptical of the method of asking respondents to
decompose reported total values into use versus non-use or various
23
-------
motivational categories. These procedures have not been verified in the
literature. There fore we recommend the principal focus of the analysis
be on the total willingness to pay estimates for the complete containment
scenario only.
c) The Box-Cox econometric transformation as applied in this study is one
of several methods for dealing with large bids and outliers. The esti-
mates appear sensitive to the treatment of zero bids. Comparative
evaluations of other approaches for dealing with the skewness in the
valuation responses should be undertaken before accepting the Box-Cox
results. In the absence of such comparative evaluations and detailed
analysis of all the possible outcomes of the valuation questions, it is.not
possible to judge whether the Box-Cox econometric estimates alone
provide an acceptable and defensible method for dealing with the sce-
narios and the large bids associated with them.
d) We do not believe that the approaches for treating embedding, dealing
with scenario rejection, and reflecting the potential effects of non-bids
can be assessed for their reliability on the basis of the information
provided in the report The procedures described in the McClelland et al.
report are research innovations that have not been evaluated in the
literature and were not evaluated as part of their study. Further,
multivariate modelling of determinants of the disembedding responses
and of the factors influencing whether individuals provided no bids, zero
bids, or large bids would be essential to answer the fourth question in
the EPA charge. Because this information was not provided as part of
the report, it is impossible for the Committee to make the judgment
required to address question four of the charge.
e) The valuation responses in the survey for non-use value have not been
' demonstrated to be accurate and reliable enough for the EPA to estimate
the non-use values for groundwater contamination. It is not dear that
any of the three decomposition methods provides a basis for estimating
non-use values accurately and in a reproducible manner. The literature
does not provide specific documentation indicating that these methods
are capable of separating the total values, especially for a situation
where the commodity definition involves water rationing, risk, and
groundwater protection as part of the complete containment scenario.
Disaggregating the components of the total value on the basis of motives
24
-------
alone would not necessarily assure that the values are exclusively
associated with non-use values or that they could be transferred to
another hypothetical situation with distinctly different features.
Improving the estimates of non-use values will require a composite
scenario that includes the recommendations earlier provided by Mitchell-
Carson along with the insights provided in this study. That is, it will
require an evaluation that attempts to control the respondents' percep-
tions of uses for the groundwater by identifying other sources for their
water and localizing the effects of contamination to deposits that were
not now (or not thought to be) likely resources for future use. Because
these alternatives were part of what was recommended from the Mitch-
ell-Carson report and were identified as possible alternatives in the Boyle
(1993) summary of existing literature, it seems they should have been
part of the experimental design.
25
-------
4 CONCLUSIONS AND RECOMMENDATIONS
• 4.1 Summary
it is possible to evaluate the report from a number of different perspectives.
From the perspective of contributions to methodology and the practice of contingent
valuation analysis, there is little doubt that this report represents a substantive
contribution, extending our understanding of the issues associated with contingent
valuation estimation of non-market values. Its progressive development of information
treatments, innovative experimental design, and attention to comparative analysis of
the values provided by alternative methods of separating use from non-use or passive
use values should be treated as new research insights to evolving literature on
contingent valuation. However, this is not the criterion that the Committee was asked
to use in evaluating the report In many respects, the Committee feels that the
problems in using the study results to meet the needs of the RIA charge arise from
requirements imposed on the research - the need for a separate estimate of non-use
value and for a method that abstracted from the specific features of local conditions.
After reviewing the full record, the Committee feels that this strategic decision on
EPA's part is the cause of many of the study's problems as well as the difficulties of
implementing the research findings in the RIA context. The approaches taken to deal
with these requirements have no basis in the theory of non-market valuation, nor
precedent in practice.
Finally, because of our criticisms of the definition of the commodity being
valued and our concerns about whether respondents fully understood what was being
valued, we are unable to make a judgement as to whether the estimates of total value
for groundwater cleanup are too high or too low. Thus, the Committee concludes the
study does not offer evidence that the survey has in fact developed estimates of non-
use (or passive use) values. Moreover, there is no basis for believing that estimates
of the representative household's non-use values (or the total values) could be
developed for groundwater cleanup independent of the particular circumstances
associated with the groundwater that has been contaminated.
As noted below, this conclusion does not imply that further research with the
survey would not yield valuable insights into how households evaluate groundwater
cleanup.
26
-------
4.2 Recommended Research
The Committee would like to commend EPA for supporting innovative policy
research of this type. Environmental commodities have distinctive characteristics in
that they are often not the object of explicit or implicit market transactions. In this
instance, there are no readily available economic data to enable economists to
precisely assess the value of removing groundwater contamination to those who will
not be using the potentially contaminated water. To obtain such values, original
economic research is needed. Research frequently will not completely resolve all of
the uncertainties with respect to benefit assessment in these controversial areas.
However, beginning the process of exploring the appropriate value to attach to these
problematic benefit components should be a high priority for future EPA funding..
As the above comments indicated, the Committee believes that there are a
number of ways in which this specific research effort could be enhanced. For
example, providing respondents with a better understanding of the commodity and
ensuring that the responses isolate the non-use value of environmental damage are
clearly of paramount concern. Indeed, had such a program existed, prior to the
McClelland et a/, report, it would have been possible to provide more specific answers
to the methodological questions posed in the Charge to this Committee.
The research issues associated with developing aggregate estimates of the
total value households place on groundwater cleanup are much too detailed to discuss
completely in this report Five general areas that are identified from the McClelland et
al. report and seem especially relevant for EPA's activities associated with the
corrective action RIA will be discussed. It is, of course, also true that further
multivariate analysis with the survey data from the McClelland et a/, survey along with
research investigating the survey mode, role of substitute groundwater resources, and
other features of the commodity described in their survey instrument would be very
desirable and seems likely to enhance our ability to interpret the findings from the
study.
The first of these has received very limited research attention. It concerns the
extent of the market for environmental commodities like groundwater. For the most
part, economic research has focused on developing estimates of the typical or
representative household values for specific commodities and not on the criteria for
determining how many households can be expected to hold these values. The
McClelland, Schulze, Lazo report (1993) includes some very preliminary and therefore
limited discussion of these issues. Addressing this question is fundamental to the task
27
-------
of measuring aggregate values for the cleanup of groundwater resources and may well
have a much greater quantitative impact on these estimates than would refinements in
estimates of the representative household's values for changes in some environmental
commodities.8
The second area of research would involve a detailed study of the sensitivity of
CV outcomes to the survey methods used • whether telephone, in-person, or mail
surveys. The evidence provided to the Committee by Professor Dillman (letter, March
31,1993) when compared with the recommendations of the NOAA Panel suggested
that there is not a consensus in the profession about the superiority of in-person
interviews over mail surveys. More research on this topic is dearly warranted. Such
research would consider not only the implications of the method used to implement
the survey but also the elicitation procedure and the role of information in the design
of the survey questions. It would require parallel efforts conducting the same valua-
tion exercise with at least two, and preferably all three, of the methods available. It
would be important not to compromise the evaluation by requiring that it provide
"definitive" answers for an ongoing policy issue. Rather it would be important to have
the commodity offered be a "real one" that is of some significance to the households
that are interviewed, while the goal of the evaluation be dearly identified as generic
research, not estimates to be used in a specific subsequent RIA.
A third area of research would address issues assodated with defining changes
in environmental commodities. This research is central to understanding whether the
contingent valuation responses are internally consistent. The recommended research
would involve developing methods and protocols for practice that could be used to
understand how environmental commodities are best measured and how those
measures relate to the descriptions offered in the framing of contingent valuation
questions. Consideration of the differences that might arise in circumstances when
total values are dominated by use-related components in comparison with those when
non-use related services dominate should be an important component of this design.
Equally'important, to the extent that consideration of risk is an important element in
the ongoing activity of EPA policy evaluations, some component of perceived risk
should be incorporated as an additional factor in understanding commodity definition
for CV work.
*Given the extremely Imtted nature of the avatabte research on the value of groundwater cleanup, the Committee feeh that
in thte ca»e both type* of research are excepttonaty important
28
-------
The fourth area of research arises from an implicit strategy for benefit transfer
that influenced the design of the CV study. Based on the introduction to the study (as
well as earlier interim study reports and EPA staff briefings), it is dear that the study
was intended to be used as a basis for estimating national non-use values that would
be combined with separately computed estimates of components of the use values
resulting from groundwater cleanup. This strategy implicitly accepts the validity of
separating the use from non-use value. Before this approach can be adopted, there is
a dear need for research on whether respondents can understand this task in a
meaningful way.
• 'The final high priority area for research concerns the desirability of evaluating
the potential of these methods developed in marketing research for calibration At CV
estimates and the development of methods for gauging the potential implications of
using a hypothetical setting to elidt valuations in relationship to real choices. Is
calibration a meaningful concept in the relationship to contingent valuation responses?
What would be the mechanism for developing calibration adjustments? These are
issues that are important to the ongoing development of the contingent valuation
method. Research evaluating whether or not contingent valuation results can be
transferred from one setting to others is also critical. The Committee believes this
should be a central component of the future research that EPA undertakes in support
of its policy evaluation.
This research should not be regarded as peripheral to policy evaluations, but
instead central to the ability of the EEAC to respond to the spedfic types of questions
posed by EPA staff about this study and the assodated RIA analysis. The Committee
assumes that there will be a continuing need to evaluate analyses intended to
appraise household's willingness to pay for improvements in environmental commodi-
ties. There is simply not sufficient information on the research shelf to address in a
meaningful way many of the questions that have been posed to the EEAC about
contingent valuation and its use. EPA should begin to sponsor research required to
evaluate" "the methods used in its policy evaluations and to develop a greater set of
experience in valuing changes to important economic resources.
29
-------
Distribution List *
Administrator
Deputy Administrator
Assistant Administrators
Deputy Assistant Administrator for Research and Development
Deputy Assistant Administrator for Water
EPA Regional Administrators
EPA Laboratory Directors
EPA Headquarters Library
EPA Regional Libraries
EPA Laboratory Libraries
-------
5 REFERENCES
Boyle, K. J. A Review of Contingent Valuation Studies of the Benefits of Groundwater
Protection (report to U.S. Environmental Protection Agency), Research Triangle
Institute, April 1993).
Dillman. D.A. Letter to Alan Cariin, EPA, March 31,1993
Kahneman, D. and Knetsch, J. K. Valuing Public Goods: The Purchase of Moral
Satisfaction," Journal of Environmental Economics and Management, Vol 22,
January 1992, 57-70.
Kneese, A. and Smith, V.K. Memorandum to R.C. Mitchell, March 23,1993. _ —
McClelland, G.H., Schulze, W.D. Lazo, J.K., Waldman, D.M., Doyle, J.K., Elliott, S.R.
and Irwin, J.R. Methods for Measuring Non-Use Values: A Contingent Valua-
tion Study of Groundwater Cleanup (Draft report to the U.S. EPA), University of
Colorado, Center for Economic Analysis, October, 1992.
McClelland, G.H., Schulze, W.D. and Lazo, J.K. Additional Explication of Methods for
Measuring Non-Use Values: A Contingent Valuation Study of Groundwater
Cleanup, Memorandum to the SAB/Environmental Economics Advisory Commit-
tee. Center for Economic Analysis, University of Colorado, June 29,1993.
Mitchell, R.C. and Carson, R.T. Existence Values for Groundwater Protection. Draft
Final Report to U.S. Environmental Protection Agency (Washington, D.C.:
Resources for the Future, May 1989).
National Oceanographic and Atmospheric Administration (NOAA), Report of the NOAA
Panel on Contingent Valuation, Federal Register, January 15,1993, Vol. 58,
#10, pp. 4601-4614.-
R-1
-------
EPA-SAB-EPEC-COM-94-001 'Commentary on the
Ecological Risk Assessment for the Proposed RIA for
the RCRA Corrective Action Rule" by the Ecological
Processes and Effects Committee
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
EPA-SAB-EPEC-COM-94-001
November 19, 1993
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, S.W.
Washington, DC 20460
Subject: Commentary on the Ecological Risk Assessment for the
Proposed RIA for the RCRA Corrective Action Rule
Dear Ms. Browner:
In response to a request from the Office of Solid Waste and Emergency
Response (OSWER), the Science Advisory Board (SAB) has reviewed several aspects
of the draft Regulatory Impact Analysis (RIA) prepared in support of the RCRA
Corrective Action Rule. At the October 1992 meeting of the SAB Executive
Committee, the Board was asked to review several components of the draft RIA. The
Executive Committee, recognizing the importance, complexity and creativity of
OSWER's work and its multi-disciplinary nature, established an ad hoc Steering
Committee to assure that certain significant aspects of the RIA-both methodology and
application-received appropriate attention from the relevant SAB Committees.
At a public meeting on January 29, 1993, the Steering Committee concluded,
based on presentations by and discussions with OSWER personnel, that four SAB
committees, with appropriate inter-committee liaison participation, should review
major segments of the RCRA Corrective Action RIA as follows: the Environmental
Economics Advisory Committee (EEAQ would review the contingent valuation
methodology and its application in the RIA; the Environmental Engineering
Committee (EEC) would review the MMSOILS fate and transport model; the
Ecological Processes and Effects Committee (EPEQ would review the ecological
risk analysis; and the Environmental Health Committee (EHC) would review the
human health risk assessment. In addition, the Steering Committee agreed to
Mffed on p«ttr«MI evUM
* toot 7S% ncyctod »•
-------
prepare an overview report, if one is deemed necessary, to accompany the
individual committee reports.
This report is the result of EPEC's review of the ecological risk analysis in
the draft RIA. We hope our comments on the draft RIA will assist the Agency in
further developing and integrating contemporary concepts of ecological risk
assessment into the proposed rule on corrective action for releases of hazardous
substances from solid waste management units.
1. Statement of Charge and Review Process
In the charge to the SAB, dated March 26, 1993, OSWER requested that
the Board assess "the implications of the fate and transport modelling .assumptions
on the ecological and human health risk assessment." To accomplish this, EPEC
designated two consultants to participate in the EEC's review of the MMSOILS
model, and comments on this topic are largely included in the report of that
committee. However, we felt that it was important to evaluate the adequacy of
the ecological risk chapter itself with the hope of improving this and future
Agency risk assessments and cost/benefit analyses. In consultation with OSWER,
we developed the following additional questions:
a) Given the constraints on available data and modeling assumptions, is
the ecological risk assessment contained in the RCRA RIA consistent
with the Ecorisk Framework developed by the EPA Risk Assessment
Forum?
b) Are the ecorisk methodologies used in the RIA appropriate for
assessment of risk at this broad scale (i.e., national vs. site-specific
assessment)? What additional analyses could be added to strengthen
the assessment?
Our comments on the draft RIA are the result of a conference call on June
7, 1993, by the Ecorisk Subcommittee of EPEC, and subsequent discussion of the
document at a public meeting of EPEC on June 21, 1993.
2. Summary of Findings
We are pleased that the Agency is beginning to incorporate ecological risk
assessment into the day-to-day policy and regulatory decision making process. We
fully recognize the immensity of the task faced by the Agency in developing an
evaluation program sufficiently general to have application to the estimated 5,800
facilities and 100,000 solid waste management units (SWMUs) across the country
-------
which will be subject to the RCRA Corrective Action Rule. Any such exercise
necessarily involves numerous trade-offs of site-specificity and accuracy vs. general
applicability to many diverse facilities and locations. Nevertheless, we feel that
the present document is incomplete in its consideration of ecological risks and
benefits of site remediation, and is not fully consistent with the Agency's Ecorisk
Framework document. Substantial additional effort will be required to complete a
defensible ecological risk analysis on a national scale, even one which covers only
baseline conditions.
There are at least three approaches open to the Agency: a) the final RIA
could present the full range of environmental endpoints that should be considered,
then state clearly that only a subset have been evaluated in the document; b) the
current ecological risk analysis could be refined and used to answer a-less"
sweeping set of questions (e.g., how widespread is the risk of exceeding water
quality standards and other ecological benchmarks in nearby surface waters); or, c)
the Agency could choose to allocate additional resources to complete the full
ecological risk assessment on a national scale. If resources are not available for a
full risk assessment, the Agency should, at a minimum, adopt the first alternative.
3. Prioritization of Ecological Risks
The assessment of benefits in the document is incomplete in that no
attempt is made to assess the ecological benefits (or risks) which would result
from site remediation. Although the table of contents describes Chapter 8 of the
draft RIA as addressing "ecological benefits," the chapter itself is titled "ecological
threats." The overall impression left by the document, and exemplified by Exhibit
13-2, is that ecological considerations, even the extent of ecological risk from the
"no action" scenario, are not being factored into the proposed cost/benefit analysis.
The authors appear to have relegated both human and ecological risks to
secondary issues in deference to non-use benefits calculated from the developing
contingent valuation method.
— As noted previously, the Environmental Economics Advisory Committee of
the SAB is conducting a detailed assessment of the contingent valuation (CV)
methodology. However, we feel strongly that the CV approach cannot yield a
realistic valuation of a resource unless the respondents have a firm knowledge of
and appreciation for the human health and ecological risks involved.
A much better definition of the problem is needed at the very outset of the
document in order to avoid misleading or false expectations as to what can really
be done to quantify ecological risks in the absence of corrective action, as well as
benefits (or risks) of site remediation. A clear discussion of the realistic state-of-
-------
the-science limitations, broad-based assumptions and generalized nature of results
is needed to properly qualify and caveat any conclusions.
4. Risk Assessment Framework
The Framework for Ecological Risk Assessment (Ecorisk Framework,
EPA/630/R-92/001) developed by the Agency's Risk Assessment Forum provides a
conceptual outline for the conduct of ecological risk assessments. The authors of
the draft RIA do refer to the framework in the case study discussion; however,
Chapter 8 of the draft RIA could be substantially improved by reorganizing it to
more closely follow the ecological risk paradigm. For example, the proximity
analysis is essentially equivalent to the conceptual step referred to in the Ecorisk
Framework as problem formulation. Discussing the elements of the proximity
analysis in terms of the problem formulation phase will ensure that the first step
of the ecorisk assessment is properly structured, that key questions will be
addressed and that meaningful results will be produced. Similarly restructuring
the concentration-based screening analysis and case studies to follow the
framework document steps of stressor-response assessment and risk
characterization phases will help identify weaknesses in the current document and
provide guidance to ensure that ecologically sound results are provided.
The problem formulation step appeared to be driven more by conveniently
accessible data than by development of a conceptual model, probably due to time
and resource constraints. As a result, the major exposure pathway which was
considered (chronically contaminated surface water) is not necessarily the most
likely to cause adverse effects from RCRA sites, as was shown in the case studies
in Appendix F of the RIA. The results of the case studies should now be used to
reformulate the conceptual model. The proximity analysis should also include a
component which identifies particularly valuable habitats, such as those containing
rare and endangered species. One source of information on the location of rare
and endangered species is The Nature Conservancy's Heritage System database.
— Clearly, the Ecorisk Framework approach has not been used to its full
benefit. It is the process of this approach that is useful. By taking a systems
perspective on risks, the choice of endpoints, models, and stressors all make sense
(e.g., see pp. 13-14 of the Ecorisk Framework document). This approach is
missing in the draft RIA, Rigorous application of the risk assessment approach
would have clarified numerous questions left unresolved in the current draft,
including:
a) Why was 1 mile selected as the distance to be used in the proximity
analysis? The distance should be selected based on ecological or
-------
physical considerations. Also, relative rather than absolute distances
should be considered. For example, sites down-slope or along steeper
topographic gradients may be more susceptible than sites up-slope or
on less steep areas. The choice of area should also depend on the
stress being considered (e.g., some chemicals are more mobile than
others).
b) What was the ecological basis for the choice of the time scales (e.g.,
why were exposures evaluated over a 128-year period)?
c) Why were particular receptors selected (e.g., shrew, hawk, owl)? An
ecosystem endpoint (e.g., increasing biodiversity, improving habitat
quality in terms of food availability and cover) would have-been
useful. We recommend that the authors of the RIA consult with the
Agency's Risk Assessment Forum for assistance in identifying
appropriate ecological endpoints.
Another important component of the Ecorisk Framework approach is the
analysis of uncertainty in the overall risk assessment. The draft RIA lacks
adequate treatment of uncertainties in the risk estimates. For example, it would
be useful to know the variability/uncertainty surrounding outputs from the
MMSOILS model, expected uncertainties of risk, and how valuation methods deal
with uncertainties.
5. Stratification of the Sample Frame
Appendix A to the draft RIA provides details of how the universe of sites
was stratified and how the subsample of facility sites was selected. Facilities
potentially subject to the Corrective Action Rule were stratified according to the
magnitude of potential costs of corrective action (i.e., very large, large, and other)
and the availability of data for an assessment. This stratification does not
consider the distribution of ecologically-relevant site characteristics, and there are
no data'to indicate that the subsample selected is indeed representative of the
range of ecological risks at all sites. We were unable to assess what other
extremes may be represented by those sites excluded. For example, does the
subsample accurately represent the myriad of soil types, groundwater and surface
water flow regimes that could be encountered? Thus, it is unclear how the Agency
can relate results obtained using the subsample of sites to the entire population of
sites.
-------
6. Emphasis on Modeling - MMSOILS
The results of the ecological risk assessment are highly dependent on the
results of the MMSOILS model. Since MMSOILS has been the subject of a
separate review by the Environmental Engineering Committee of the SAB, we
include only overview comments here.
We question the use of this simplistic model for the RCRA evaluation. The
model should be subject to formal, comprehensive sensitivity and uncertainty
analyses. Results of these analyses can be used to: a) determine the range of
model outputs in relation to uncertainty on input data; b) determine what the
critical data are for improving model predictions; and c) simplify the model
structure without sacrificing accuracy or precision of model results. At the very
least, sensitivity analysis should be done to compare outputs from MMSOILS with
other chemical fate models such as EXAMS. It may be that using a variety of
different existing fate and transport models, rather than any single model, is the
most appropriate way to evaluate risks.
The MMSOILS model is emphasized as a screening tool. However, it is
clear that the model is used beyond screening in estimating quantitatively the fate
and transport of contaminants. Care must be taken in implementing and
evaluating the model for these two different purposes.
One major problem that must be confronted in the development of
multimedia models, such as MMSOILS, is the forcing of differently scaled
environmental transport processes into a single model construct. Forcing
disparate temporal and spatial scales into a single model can produce inaccuracies
in model results. These scale considerations should be used to examine the basic
model constructs of MMSOILS.
7. Adverse Impacts of Site Remediation
— As mentioned earlier, the draft RIA does not address the potentially adverse
ecological impacts of corrective actions. These actions could include mobilization
and release of large quantities of contaminants (e.g., during dredging of
contaminated sediment or due to accidental ruptures of tanks, dams, etc.);
destruction of terrestrial habitat due to road-building, soil removal, etc.; and
greatly increased loadings of silt in streams due to erosion of exposed soil
Methodologies for addressing these effects should be developed and the impacts
should be considered in the overall analysis.
-------
8. Use of Case Studies
In the draft RIA, the three ecorisk case studies are used simply to document
the fact that ecologically significant contamination can be observed at some sites,
and that pathways for potential exposure do exist. A much more valuable use of
the case studies would be to evaluate the conceptual model used for the ecological
risk assessment and the fate and transport predictions of the MMSOILS model.
Actual contaminant concentrations and hazard indices could be compared to
predicted values; land-use designations obtained from on-site surveys could be used
to verify land use designations obtained from maps, etc.
9. Assumptions and Incomplete Information
The information provided in Appendix F of the RIA on the concentration-
based screening methodology is quite sketchy, and lacks sufficient discussion of
data sources and assumptions. However, from the information given, it seems
possible that some inappropriate assumptions have been made. Areas of
incomplete information include:
a) The time frame used in the model for predicted concentrations of
contaminants in surface waters was an annual tune step, while the
time frame used for deriving chronic ambient water quality criteria is
4 days. Mixing these different time scales is likely a problem since
the annual average value could mask short-term extreme events. The
document should clearly state that this is a problem.
b) Many of the extrapolation factors described in Exhibit F-l are
consistent with general usage by regulatory agencies, although better
methods are now available in the scientific literature. They are,
however, derived principally from studies involving fish. It appears
from the documentation that data from invertebrate toxicity tests may
also have been used in the assessment. The values in Exhibit F-l
should not be applied to toxicity test data for organisms other than
fish.
c) The Hazard Index results (Exhibit F-3) are poorly documented.
Contaminants responsible for the values are identified, but the
pathways and receptors are not. In addition, some of the index
values are extremely high (1(T or more). If environmental
contaminants were really present at the levels indicated and the
exposed species were as sensitive as assumed in the analysis, overt
ecological effects should already have occurred and should be easily
-------
detectable. The concern here is that most regulatory applications of
the hazard index approach are deliberately conservative and designed
to overstate the magnitudes of ecological risks. In this case,
conservatism can result in poor decision making because contaminant-
related risks may be overstated relative to corrective action-related
risks, which are not even being considered.
d) The document describes methods for estimating exposures and effects
on benthic invertebrates and terrestrial wildlife, but is vague about
data sources and assumptions.
e) In the equation used to calculate aqueous benchmarks for piscivorous
•wildlife (p. F-7), what is the source of the values for "species-
sensitivity factor " (SSF)? This may be the most important value in
the equation.
In addition to the many limitations of the methodology acknowledged in the
draft RIA, there are a number of specific application and interpretive errors in
Appendix F, including:
a) Proximity Analysis
(1) p. F-l, paragraph 2: Land use categories were ranked in terms
of habitat value (i.e., surface water and terrestrial ranked
higher than residential, agricultural and industrial land uses).
However, assessing land uses in terms of vulnerability
("ecosystems at risk") would result in the opposite ranking.
(2) p. F-l, paragraph 3: "Smaller areas" of surface water are not
necessarily at lower risk than "larger areas": for example,
greater volume and surface area tend to reduce risk in larger
bodies of water due to dilution and air stripping.
b) Ecological Benchmark levels
(1) p. F-2, paragraph 4: There is no scientific rationale provided
for the extrapolation factors used to account for variation in
species sensitivity, extrapolation from acute to chronic value,
and high bioaccumulation potential.
(2) p. F-4, paragraph 2: One cannot estimate BCF values when
log KQW is greater than 4 or 5, since the correlation is not
8
-------
linear in that range: many of the chemicals of concern have
log K^ values greater than 4.
(3) p. F-5, paragraph 2: "No observed effect levels" (NOELs),
which are protective of individual organisms, are not the
appropriate population threshold for wildlife.
(4) p. F-8, paragraph 1: The document states that, "BCF values in
the Superfund Chemical Data Matrix are the highest measured
for any aquatic species." What is the impact of all these worst-
case estimates on the final values for the screening-level surface
water criteria?
c) Case Studies
(1) p. F-19, paragraph 2: If, as stated, "the data do not allow for
separation of the effects of SWMU releases from permitted
releases under the facility's NPDES outfalls," the methodology
will not he useful for diagnostic purposes. Is this a generic
situation?
(2) p. F-21, paragraph 2: The finding that parasites and fin rot
were most prevalent among fish collected closest to the facility
may not be significant, since most fish are not territorial in
lakes and move around a lot.
10. Conclusion
In conclusion, we support the inclusion of an ecological risk assessment in
the RCRA Corrective Action RIA. However, we are concerned that the current
draft does not incorporate the approach contained in the Ecorisk Framework, and
the assessment of ecological risks is incomplete. We recognize that resource
limitations may preclude a complete ecological risk assessment in this document,
yet the approach taken may serve as a model for other risk assessments and RIAs.
Consequently, we recommend that the RIA be modified to: a) more explicitly
follow the Ecorisk Framework; b) discuss which ecorisk factors were considered in
the RIA and why, c) discuss which ecorisk factors were not considered and why;
and, d) discuss uncertainties associated with insufficient knowledge, inadequate
data, natural variability, etc.
-------
We sincerely appreciate the opportunity to review the progress of this
important activity. We hope these comments will be helpful to the Agency in
revising the RIA, and we look forward to your response on the issues we have
raised.
Sincerely,
Dr. Raymond C. Loehr, Chair
Science Advisory Board
Dr. Kenneth L. Dickson, Chair
Ecological Processes and
Effects Committee
Dr. Alan W. Maki, Chair
Ecorisk Subcommittee
10
-------
US. ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
ECOLOGICAL PROCESSES AND EFFECTS COMMITTEE
ROSTER
June 21-23, 1993
CHAIR
Dr. Kenneth L. Dickson, Institute of Applied Sciences, University of North
Texas, Denton, Texas
MEMBERS/CONSULTANTS
Dr. Steven M. Bartell, SENES Oak Ridge, Inc., Center for Risk Analysis, Oak
Ridge, Tennessee
Dr. Edwin L. Cooper, Department of Anatomy and Cell Biology, School of
Medicine, UCLA, Los Angeles, California
Dr. William E. Cooper, Zoology Department, Michigan State University, East
Lansing, Michigan
Dr. Virginia Dale, Environmental Sciences Division, Oak Ridge National
Laboratory, Oak Ridge, Tennessee
Dr. Mark A. Harwell, Rosenstiel School of Marine and Atmospheric Science,
University of Miami, Miami, Florida
Dr. Robert J. Huggett, Virginia Institute of Marine Science, College of William and
Mary, Gloucester Point, Virginia
Dr.-Alan W. Maki, Exxon Company, USA, Houston, Texas
Dr. Frederic K Pfaender, Institute for Environmental Studies, University of North
Carolina, Chapel Hill, North Carolina
Dr. Anne McElroy, SUNY at Stoney Brook, Stoney Brook, New York
Dr. William H. Smith, Professor of Forest Biology, School of Forestry and
Environmental Studies, Yale University, New Haven, Connecticut
Dr. Terry F. Young, Environmental Defense Fund, Oakland, California
-------
SCIENCE ADVISORY BOARD STAFF
Ms. Stephanie Sanzone, Designated Federal Officer, Science Advisory Board
(1400F), U.S. EPA, 401 M Street, S.W., Washington, DC 20460
Mrs. Marcia K Jolly, Staff Secretary, Science Advisory Board (1400F), U.S. EPA,
401 M Street, S.W., Washington, DC 20460
-------
NOTICE
This report has been written as part of the activities of the Science
Advisory Board, a public advisory group providing extramural scientific
information and advice to the Administrator and other officials of the
Environmental Protection Agency. The Board is structured to provide balanced,
expert assessment of scientific matters related to problems facing the Agency.
This report has not been reviewed for approval by the Agency and, hence, the
contents of this report do not necessarily represent the views and policies of the
Environmental Protection Agency, nor of other agencies in the Executive Branch
of the Federal government, nor does mention of trade names or commercial
products constitute a recommendation for use.
-------
DISTRIBUTION LIST
Administrator
Deputy Administrator
Assistant Administrators
Deputy Assistant Administrator for Research and Development
Deputy Assistant Administrator for Solid Waste and Emergency Response
EPA Regional Administrators
EPA Laboratory Directors
EPA Regional Libraries
EPA Laboratory Libraries
-------
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
November 19, 1993 C*FK*C*THEADMINISTRATOR
SCIENCE ADVISORY BOARD
EPA-SAB-EHC-LTR-94-003
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, S.W.
Washington, DC 20460
Subject: Review of the Health Benefits for the proposed RIA for the RCRA
Corrective Action Plan Rule by the Environmental Health Commit-
tee
Dear Ms. Browner:
At the October, 1992 meeting of the SAB's Executive Committee (EC) the
Board was asked by the Office of Solid Waste and Emergency Response (OSWER) to
review the methodology for the draft Regulatory Impact Analysis (RIA). This
cost/benefit analysis is required prior to promulgation of the Agency's final Resource
Conservation and Recovery Act Corrective Action Rule. The EC, recognizing the
importance, complexity, and novelty of OSWER's work and its multi-disciplinary
character, established an ad hoc RCRA-RIA Steering Committee (RRSC) to assure
that certain aspects of the RIA - in both methodology and application -received
appropriate attention from the relevant SAB committees.
At--a-public meeting on January 29,1993, the RRSC concluded, on the basis of
presentations by and discussions with OSWER personnel, that four SAB individual
committees should review the major segments of the RCRA-RIA. Specifically, the
RRSC agreed to review: a) the contingent valuation (C\fl methodology used in the
RCRA RIA analysis (CV-1, by the Environmental Economics Advisory Committee
(EEAC)); b) the application of CV in the RCRA-RIA (CV-2, by the EEAC); c) the princi-
pal fate and transport model (MMSOILS). used in the RCRA-RIA (by the Environmen-
tal Engineering Committee (EEC)); d) the ecological risk assessment portion of the
RCRA-RIA (by the Ecological Processes and Effects Committee (EPEC)); and 0 the
human health risk assessment portion of the RCRA-RIA (by the Environmental Health
Committee (EHC)).
-------
This letter was prepared by the SAB's Environmental Health Committee
following the circulation (by mail) of initial comments prepared by a Committee
-Member, and a public teleconference held on September 24, 1993. The report
focuses on the risk assessment methodology used to generate the estimated impacts
on human health resulting from proposed corrective action at RCRA facilities. The
March 23, 1993 memorandum from Richard Guimond to Dr. Donald Barnes described
the Charge for the SAB's review expresses an interest (page 4) in the implications of
the fate and transport model assumptions on the ecological and human health risk
assessments. Before such an interest can be addressed, the risk assessments must
be reviewed to determine whether they are sound, and to suggest improvements
where they are not; otherwise comments on the implications of fate and transport
assumptions may not be meaningful.
Our report is organized into two sections - some overall strategic comments
which follow in the body of this letter, and detailed technical points, keyed to specific .
sections of the draft RIA, which are incorporated in an Appendix enclosed with this
letter.
Based on our understanding of OSWER's goals, the Committee views the draft
methodology as a screening analysis, as opposed to a more detailed and definitive
analysis. We have therefore reviewed the material with the following question in mind
as our Charge: "Is this the best that can be done to provide a method for conducting a
screening analysis?"
The screening analysis methodology produced by the Agency was a very
ambitious undertaking. It is also of great potential importance since implementing the
proposed RCRA Corrective Action regulation could cost many billions of dollars and, in
the future, these techniques for estimating and comparing costs and benefits may well
find application to other important cases as stated in the document.
Irf general, the methods used are well known and correspond to "much-used"
guidelines, methods and practices (GMP). These GMP have, for the most part, been
developed for use in setting prudently protective standards of exposure to individual
substances in specific regulatory situations. In the proposed RCRA/RIA methodology
the GMP are being applied to a large set of complex cases involving multiple expo-
sures to make calculations of total impact not contemplated when the GMP were first
derived. Determining whether the GMP can be so applied, whether they are ade-
quately applied, whether their application to determine overall health impacts is
reasonable, whether the important assumptions and limitations have been clearly
-------
identified, and what reasonable and feasible improvements can be suggested has
been an objective of our review.
\
The methods developed in Chapter 7 and Appendix E are obviously the result
of much careful and thoughtful work. Care has been taken to lay out and define the
risk assessment process used and its components. The basic assumptions used in
each of the steps are clearly stated, (including the way in which numerical values were
chosen to bring in as much realism as possible at each step through the use of site
specificity where possible). Alternatives considered and different ways used to
calculate the end results are set forth so as to give an idea of sensitivities or possible
alternative results (e.g., the decision to analyze a "less than 100 percent effective-
ness" corrective action scenario). Lastly, possible biases and uncertainties inherent in
the whole procedure are identified.
The net effect of the effort was to produce a construct formed of many carefully
selected parts with which a formal estimate of cancer population risk can be made,
both before and after corrective action, and with which a formal estimate can be made
of the numbers of persons exposed to contaminant levels which have some probability
(extent unknown) of producing adverse, non-carcinogenic effects , also before and
after corrective action. Thus, at least formally, the effect of the regulation can be
measured in two different ways, one for cancer and one for non-cancer effects.
Although those responsible for developing this construct can be congratulated
on their achievement, another group of workers might have made different specific
choices at various junctures (for example, some other length of time than the 9 year
exposure period, or the 128 year timeframe). However, it is doubtful if, given the state
of science, a more "rigorous" method with greater certainty of giving "right" answers is
possible at the present time.
As a construct, given both the state of science and the impossibility of validat-
ing the=£alculations through any realistically achievable, actual measurements, the
results reported must be regarded as coming from an enormously complex, logically
consistent, but mainly hypothetical calculation - as must the results of most low
exposure-level risk assessments. We do not use the term "hypothetical" in a negative
sense; rather, as carefully devised as it is, the construct cannot give, at best, more
than a rough estimate of the actual situation. Using these results in any kind of
cost-benefit balancing must be done with this ambiguity firmly in mind. We suggest
that in the final RIA, this point be made abundantly dear to avoid even the possibility
that the results wfll be treated as definitive by anyone.
-------
There are some areas where we suggest possible improvements, i It would be
very useful to include a way to estimate, even crudely, the fraction of the population
presumably exposed to significant levels of contaminants (HI > 1) who actually
manifest adverse, non-cancer effects. Without some attempt at providing such
estimates for the most important cases, the cost/benefit calculation remains seriously
incomplete. By including such estimates, the monetization of both cancer and non-
cancer effects avoidance benefits can be done in a formal sense, and that portion of
the cost/benefit calculation would at least be present in the overall screening analysis.
The Committee suggests that there is a methodology available to estimate the
size of the population affected at a specified level of effects. We suggest (and this is
probably the most significant revision we propose) a change in the basic approach
used for relating toxicant exposure and effects. The proposed methodology can be
expanded to include use of the benchmark dose concept for determining adverse
effects, including reproductive and developmental toxicity, as described by Crump
(1984) and Kimmel and Gaylor (1988), and offered as a suggestion in the EHC's
report (EPA, 1990) reviewing proposed revisions to the EPA Guidelines for Develop-
mental Toxicity. From this benchmark dose, a straight line can then be drawn to the
dose level that represents a 100-fold margin-of-safety, plus any relevant uncertainty
factors, i.e., the reference dose for developmental toxicity. The reference dose is
considered to be an exposure level unlikely to cause human developmental effects.
At the beginning of the low-dose extrapolation below the benchmark dose, the actual
probabilities of adverse effects may be close approximations of the linear extrapola-
tion. As the extrapolation progresses toward the reference dose, the calculated
probability of adverse effects may exceed reality to some unknown extent due to the
presumed threshold phenomena operative for most developmental toxicants.
Use of this model would allow both pre- and post-remediation quantification of
reduction of risk in a manner markedly consistent with that for low-dose extrapolations
for carcinqgenests. This calculation could be used to monetize remediation efficacy.
The example given here has been for developmental toxicity, but it is suggested that
the report also attempt similar type calculations for reproductive toxicity and other
non-carcinogenic toxicologic endpoint assays. (It should be noted that this method
does not lend itself to quantification of effect severity, e.g., den lip verses auricular
tags, but it does permit quantification of classes of adverse effects.)
Addressing other areas, we note that the term "population risk" (and related
terms in connection with both cancer and non-cancer adverse effects) is employed
correctly vis-a-vis cancer, but not with non-cancer effects. As discussed in more detail
-------
in the Appendix, the population risk for cancer is correctly given as an estimate of the
number of people affected by cancer within the exposed populations. In contrast, for
. non-cancer effects, estimates of the number of people who are exposed at levels
exceeding the Hazard Index (HI > 1) do not necessarily coincide with the numbers
presumed affected; indeed, the fraction of those actually expected to be affected by
non-cancer effects among the numbers cited is likely to be very small, possibly zero.
Also, within the cancer/non-cancer dichotomy, consideration might be given in some
way to those autoimmune diseases such as lupus, and certain genetic diseases
whose causes may be related to environmental factors. The Agency should be more
explicit in distinguishing cancer and other disease conditions with respect to risk and
"population at risk." The SAB's Environmental Engineering Committee, in its review of
the RCRA MMSoils model (EPA-SAB-EEC-94-002) had similar concerns about the
EPA practice of using different approaches to cancer and non-cancer risk assessment
(as well as on some misidentification of critical endpoints for some common landfill
constituents).
The benefits of abating disease are not monetized in the RIA document. At
some point, the decrease in cancer cases and the decrease in numbers exposed to
possible risks of non-cancer adverse effects may need to be balanced, along with
other benefits (either monetized or not) against the dollar costs of corrective action.
This is a difficult if not impossible aim to achieve in any objective way. Alternatively,
using existing methodologies, the results of the calculations could be used to estimate
direct monetary benefits of cancer avoidance (medical costs avoided, lost productivity,
etc...) as well as indirect costs (pain and suffering, damage to family relations,
damage to quality of life, etc...). However, a similar calculation cannot be made for
non-cancer effects based on the results of the proposed screening method. (The
SAB's Environmental Economics Advisory Committee, as part of its review of the
RCRA/RIA, is considering the monetization of health benefits.) Another consideration
which might be taken into account in estimating impact or monetizing effects is age of
onset of a fatal and/or disabling disease; i.e., cancer in an 80-year old will have
differerlf societal and personal impacts than cancer in a 24-year old person.
It would be useful to estimate for cancer, the number of individuals "at risk"
(already done for non-cancer effects) so as to have comparable numbers of people at
risk for cancer and non-cancer effects. An estimate of the population exposed at
levels of exposure of concern for cancer (i.e., levels leading to a lifetime individual risk
of 10* or greater) would yield such estimates recognizing the fact that what is of
concern is not identically defined in the two cases.
-------
Lastly, we urge increased emphasis on the collection and management of good
exposure data as a foundation for this, and other efforts by the Agency. The impor-
tance of good exposure data can not be underestimated. From what we now under-
stand, concentrations predicted via the MMSOILS model (as reviewed by the Environ-
mental Engineering Committee) are subject to large uncertainties, affecting exposure
estimates significantly, and thereby affecting the results of the risk assessment. More-
over, the screening methodology will produce results which are completely useless
and inaccurate if chemicals released from the subject facilities are not included in the
assessment. We note that several chemicals which have caused problems around
municipal landfills are not included in Table E-1 - particularly methane and hydrogen
sulfide. These chemicals may well also be generated at solid waste management
units at RCRA facilities. Although not generally considered a toxic problem, methane
has accumulated In houses and caused explosions; this is indeed a public health
problem. Hydrogen sulfide is produced when sulfur containing compounds such as
gypsum are buried and become wet. Hydrogen sulfide releases from several landfills
have produced documented health effects in people living in nearby communities.
Although the importance of exposure data is discussed in the Appendix, we believe
that it is such a basic and important consideration that it needs to be highlighted in our
comments especially since some problematic chemicals have not been Included in the
assessment.
We look forward to receiving your response to our comments.
Sincerely,
Dr. Raymond C. Loehr, Chair
Science Advisory Board
iy£
Or. Arthur C. Upton, Chair
Environmental Health Committee
ENCLOSURE
-------
APPENDIX
SPECIFIC COMMENTS: CHAPTER 7
- Page 7-7. last para.: The SAB/EEC, in its review of the MMSOILS transport model
as part of the RCRA/RIA review, apparently has reservations about the use of such a
general model to predict concentrations for a wide variety of different, specific cases,
along with other reservations about it. It would be desirable to contact the SAB/EEC
(and the liaison members) to determine what bias or uncertainty this might introduce
into the risk assessment, what might be done instead, and under what criteria
MMSOILS might not be adequate, with suitable precautionary considerations, for a
screening analysis.
- Page 7-15. references 17 and 18: It would be useful to review the SAB's reviews
of these documents, which took place subsequent to the dates on them, to ensure that
the comments therein are adequately taken into account in this proposed methodolo-
gy; perhaps the SAB's review should be added as a reference.
- Page 7-22. top Of the page: The selection of the nine-year period needs more
justification or explanation.
- Page 7-23. last sentence of 1st para.: This is an inadequate risk descriptor (See
comments re Page 7-41. 2nd para, and further related comments as to why).
- page 7-29.1st paragraph: the word "may" appears twice; delete the second occur-
rence.
- Page 7-41. 2nd para.: The statement,"... and 25,000,000 persons experiencing
non-cancer health effects over the 128 year modeling period..." is an example of an
inaccurate interpretation of the extent of the population whose exposure exceeds an
HI of 1.0. (The next sentence is another example as is the wording in Exhibit 7-17).
The 25,000,000 people correspond to the population exposed at levels yielding an HI
of 1.0 or higher, not those "...experiencing non-cancer health effects...". Exposure at
or above' effect levels does not equal effect; only the probability of effect (Suppose a
news story were to appear asserting that EPA estimates that 25,000,000 people suffer
from etc...!) The number experiencing non-cancer health effects will actually only be a
fraction of the 25,000,000 — 25, 250 or maybe as many as 2,500. The number of
people exposed at an HI of 1.0 or greater is therefore not a measure (or a descriptor)
of population risk: the number of such people who do actually experience non-cancer
health effects is (as in the case of cancer.population risk is the number of people
adversely affected). The numbers of people so exposed and the numbers actually
experiencing non-cancer health effects can, in fact, easily be in reverse order for
different cases so that the figure of 25,000,000 is, again, not a measure of risk. This
wording should be changed (as should similar wording elsewhere in the text) to reflect
A-l
-------
this fact and to emphasize it so that the user/reader will not be misled. As mentioned
above, one of the matters the SAB's Environmental Economics Advisory Committee is
, addressing in their portion of the RCRA/RIA review is the question of monetizing the
' health risks so as to calculate the benefits obtained directly from reducing the risks
through regulation. Use of the 25,000,000 figure in such monetization under the
impression that it measures the number of people experiencing non-cancer health
effects would lead to grossly high dollar values.
- Page 7-42.1st para.: This paragraph is a most important observation. It could
imply that most of the cost and most of the benefit of Corrective Action could be
attributed to this one site: It would be highly desirable to quote the site population
figures in this paragraph, compared to the relevant totals, and to underline or highlight
the paragraph to help ensure that its importance for the rest of the analysis is not
missed by the reader.
~ Page 7-42. 2nd para., last two sentences: The last of the two sentences corrects
the statement of the first one; I suggest eliminating the first one and using the sense
of the second one, instead, to give a correct impression in the first place.
~ Pace 7-42. section entitled " Number of Facllffies..."Exhlbit 7-18 and all other
sections of the material under review where this comment applies; The cancer and
non-cancer population "risk" figures should not be combined in any way and indicated
to be somehow of the same type since they are intrinsically differently defined; they
should preferably be presented separately. The cancer figures represent an estimate
(however uncertain) of the number of cancer cases whereas the non-cancer figures
represent the number of individuals who merely might become non-cancer health
effects cases. Any kind of "sum" or statement of "jointness" is meaningless unless
very carefully labelled. Stating the number or percentage of sites where the number
of cancer cases is expected to be insignificant (less than one in a million, say) and in
which the HI is less than one (and therefore the number of people in which non-
cancer health effects might not occur in significant numbers if at all) is a
not-misleading statistic that might be helpful to the reader/user. The text and Exhibits
need to be modified to not mix the two types of estimates in a misleading way. For
exampte/jri Exhibit 7-18, the wording No Risk might be changed to W Risk and the
wording' % Risk might be changed to % Risk and Possibly At Risk. Other
examples abound; for example, in Exhibit 7-24 (a very useful Exhibit), even though the
non-cancer ordinate has a parenthetical statement defining what is really meant, the
title of the chart should be changed to CUMULATIVE NON-CANCER POPULATION
POSSIBLY AT RISK and the title of the ordinate should be changed accordingly.
Throughout the entire text the distinction between the two types of population
estimates should be carefully maintained: population risk, in the case of cancer, and
population at riskJn the case of non-cancer effects. There are many such instances
and I suggest the text be carefully edited to find and change each and every one of
A-2
-------
them (for example, where the term "risk/effects' is used one might use, instead "risk/at
risk," or "risk/concern'' or "effects/possible effects," etc.
— Sections 7.4.1. 7.4.2 and 7.4.3: These two sections are well done and very wel-
come.
SPECIFIC COMMENTS: APPENDIX E
- page 5. first paragraph: Too flat and sharp a distinction is made between carcino-
gens and non-carcinogens on the basis of the existence or nonexistence of thresh-
olds. I would suggest the following words: "It has been the custom to suppose that for
non-carcinogenic or systemic effects protective physiological mechanisms exist that
must be overcome before the adverse effect is manifested. This may not, in fact, be
universally true and for the non-carcinogenic effects of lead, for example, K appears
not to be true. Similarly thresholds are thought to be absent in the case of cancer,
Le., any level of exposure, however small, could result in cancer although there are a
very few instances now known in which this assumption may not hold. Nonetheless,
in this method, the existence of thresholds will be assumed as usual in the case of
non-carcinogens (except for lead, as discussed below) and the lack of thresholds will
be assumed for carcinogens, also as usual"
These words describe just what is being done, and the real assumptions being
made, for the user of the method.
- page 5. second paragraph: The use of "benchmark" to describe RfDs could be
confused with the term "benchmark dose," which quite different from and RfD. We
suggest changing the subject sentence to read "For many chemicals the RfD ap-
proach has been used as a basis for regulatory decisions in relation to potential
impacts on human health."
— Page 27. first paragraph: This is an entirely sensible way to handle the aggregation
of risks from exposures to multiple carcinogens, it should be mentioned here, howev-
er, that synergism (and/or antagonism) is possible (referring to section 7.4.3 of
Chapter^ -Where it is already mentioned).
— Page 27. second paragraph: The second sentence, beginning "Ratios of contami-
nant level... " should be added to as follows:"... non-carcinogenic health affect for
exposure to a particular contaminant. The hazard quotients for different contaminants,
even If they have equal RfDs, do not necessarily indicate which substance poses the
greater risk.' (the material in italics is the added material) - Also, regarding the HI, the
SAB/EHC commented in considerable depth somewhat over a year ago on the
limitations of the use of the Hi in its review of the "Risk Assessment Guidelines for
Superfund Sites". We suggest that the Agency needs to refer to the SAB/EHC review,
especially its Appendix, and to include mention of the principal limitations of the use of
A-3
-------
the HI, here, in Appendix E, since the HI is incorrectly used in identifying populations
as being significantly exposed to contaminants having non-cancer adverse health
»effects.
- paoe 27.last paragraph fE.3.31: Here, again, the comments made on Chapter 7
need to be taken into account on the subject of the difference between population risk
and population at risk. The first sentence of this paragraph, as well as further
statements within it, need modification.
The problem is that the estimates of population risks associated with carcino-
gens, despite their well known weaknesses, at least purport to make some kind of
estimate, using a model which may or may not apply in a given case, of the number of
people affected (true population risk) as a result of exposure whereas counting the
number of people with HI > 1 for non-carcinogens estimates the number of people at
exposure levels such that they are potentially affected but not necessarily affected
(this is not population risk). Whereas the estimates for carcinogens at least attempt to
get at the number of people affected, the method for non-carcinogens does not
attempt to do so since it includes those exposed and affected, plus those exposed but
not affected in its count. Unless some effort is made to reconcile the two methods,
the result of the non-carcinogen procedure is inconsistent with that for carcinogens
and the two results are not comparable. Moreover, as mentioned above, the
"population-at-risk" result for non-carcinogens does not measure risk and should not
be said or inferred to do so here or elsewhere in the report it estimates only one
factor in characterizing risk, namely, an estimate of whether an exposed population is
exposed to possibly meaningful levels of the agents involved, but it takes no account
of the probability of such a population actually exhibiting adverse effects. This point
needs to be made and maintained dearly in the text to avoid any misunderstanding by
users if the method.
A-4
-------
REFERENCES
•Crump, K.S. 1984. A new method for determining allowable daily intakes. Fund.
•Appl. Toxicol. (4):854-871.
EPA. 1990. Review of proposed developmental guidelines. EPA-SAB-EHC-90-013.
Kimmel, C.A. and Gaylor, D. W. 1988. Issues in qualitative and quantitative risk
analysis for developmental toxicology. Risk Analysis (8): 15-20.
R-l
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
ENVIRONMENTAL HEALTH COMMITTEE
RCRA RIA HEALTH BENEFITS REVIEW
CHAIRMAN
Dr. Arthur Upton, Santa Fe, NM
MEMBERS AND CONSULTANTS
Dr. William B. Bunn, Mobile Administrative Services, Company, Inc., Princeton, NJ
Dr. David Gaylor, Department of Health & Human Services, National Center for lexicological
Research, Jefferson, AR
Dr. Rolf Hartung, School of Public Health, University of Michigan, Ann Arbor, Ml
Dr. Rogene F. Henderson, Inhalation Toxicology Research Institute, Albuquerque, NM
Dr. Marshall Johnson, Jefferson Medical College, Philadelphia, PA
Dr. Nancy K. Kim, New York Department of Health, Albany, NY
Dr. David Howe Wegman, University of Lowell, Lowell, MA
Dr. Bernard Weiss, University of Rochester School of Medicine, Rochester, NY
DESIGNATED FEDERAL OFFICER
Mr. Samuel Rondberg, Environmental Health Committee, Science Advisory Board (1400F),
U.S. Environmental Protection Agency, Washington, D.C. 20460
STAFF SECRETARY
Ms. Mary L. Winston, Environmental Protection Agency, Science Advisory Board (1400F),
Washington, D.C. 20460
-------
NOTICE
This report has been written as a part of the activities of the Science Advisory
Board, a public advisory group providing extramural scientific information and advice
to the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters
related to problems facing the Agency. This report has not been reviewed for
approval by the Agency and, hence, the contents of this report do not necessarily
represent the views and policies of the Environmental Protection Agency, nor of other
agencies in the Executive Branch of the Federal government, nor does mention of
trade names or commercial products constitute a recommendation for use.
-------
Distribution List
Administrator
Deputy Administrator
Assistant Administrators
Deputy Assistant Administrator for Research and Development
Deputy Assistant Administrator for Water
EPA Regional Administrators
EPA Laboratory Directors
EPA Headquarters Library
EPA Regional Libraries
EPA* Laboratory Libraries
-------
EPA-SAB-EEAC-LTR-94-001 'Review of Economic
Aspects of the Proposed RIA for the RCRA
Corrective Action Rule" by the Environmental
Economics Advisory Committee
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
November 19, 1993
1 OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
EPA-SAB-EEAC-LTR-94-001
Honorable Carol M. Browner
Administrator
U.S.JEnvironmental Protection Agency
401 M Street, S.W.
Washington, DC 20460
Subject: Review of Economic Aspects of the proposed RIA for the RCRA
Corrective Action Rule by the Environmental Economics Advisory
Committee (also referred to as CV-2)
Dear Ms. Browner:
At the October, 1992 meeting of the SAB's Executive Committee (EC) the
Board was asked by the Office of Solid Waste and Emergency Response (OSWER) to
review the methodology for the draft Regulatory Impact Analysis (RIA). This
cost/benefit analysis is required prior to promulgation of the Agency's final Resource
Conservation and Recovery Act Corrective Action Rule. The EC, recognizing the
importance, complexity, and novelty of OSWER's work and its multi-disciplinary
character, established an ad hoc RCRA-RIA Steering Committee (RRSC) to assure
that certain aspects of the RIA - in both methodology and application - received
appropriate attention from the relevant SAB committees.
At a public meeting on January 29, 1993, the RRSC concluded, on the basis of
presentations by and discussions with OSWER personnel, that four SAB individual
committees should review the major segments of the RCRA-RIA. Specifically, the
RRSC agreed to review, a) the contingent valuation (CV) methodology used in the
RCRA RIA analysis (CV-1, by the Environmental Economics Advisory Committee
(EEAC)); b) the application of CV in the RCRA-RIA (CV-2, by the EEAC); c) the princi-
pal fate and transport model (MMSOILS). used in the RCRA-RIA (by the Environmen-
tal Engineering Committee (EEC)); d) the ecological risk assessment portion of the
RCRA-RIA (by the Ecological Processes and Effects Committee (EPEC)); and f) the
human health risk assessment portion of the RCRA-RIA (by the Environmental Health
Committee (EHC)).
-------
This letter comprises the report of the EEAC, which completed its review of the
analytical methodology for the draft RIA at its meeting of September 23, 1993. The
.Committee evaluated four aspects of the proposed economic methodologies, includ-
ing:
a) the desirability of disaggregate information as part of the development
and presentation of benefit-cost information in the RIA for a large,
complex national rule
b) the exploration and presentation of "human health benefits," including
both the cancer and non-cancer health effects associated with the
proposed rule
c) the possibility of using the McClelland et a/. (1992) CV analysis as the
basis for estimating the non-use values for groundwater cleanup on a
national scale
d) the application of hedonic methods to evaluate the effects of contaminat-
ed sites on residential property values (Hedonic models recognize that
many commodities within the same broad categories, like houses and
automobiles, for example, have different features. The models imply that
the prices of products within each category should be related to the mix
of features that each type of commodity has. The models are sometimes
used to estimate people's willingness to pay for dimensions of environ-
mental quality that are location specific, such as air or water quality.)
Overall, EPA staff are to be commended for a number of innovations in the
methodologies outlined for use in the full RIA and illustrated with the examples in the
draft report Especially notable from the perspective of the economic analysis were:
a) the recognition, as part of the human health benefit analysis (and associ-
ated risk computations), that policies are not always effective
b) careful descriptions of both baseline and "with-policy" effects in all
aspects of the evaluation
c) attempting to separate the effects of the sub-part S rule from those of
other initiatives
d) recognition of the importance of the extent of the market for benefits
transfer
-------
e) full documentation of the econometric analysis associated with the
hedonic property value models
With respect to the development and presentation of national benefit-cost esti-
mates, the Committee recognizes that the RIA must, to the extent possible, represent
the aggregate benefits estimated to be associated with the proposed regulatory
alternative. Nonetheless, this goal does not preclude EPA staff from presenting
disaggregate information for the benefits and costs associated with different compo-
nents of the aggregate. For example, it should be possible to classify Solid Waste
Management Units (SWMU) by size, location, and other characteristics and to develop
the estimates according to these categories. This practice would facilitate evaluation
of the methods used and improve the transferability of disaggregate results to the
aggregate level.
Classification of SWMU would also permit evaluation of the likely distributional
impacts of policy alternatives across different types of facilities and types of communi-
ties impacted. The Committee recognizes that the limits implied by confidentiality
requires such disaggregation be conducted at levels that assure sites cannot be
identified. However, this requirement does not preclude substantive effort to develop
informative decompositions for the policy alternatives.
The Committee also recommends that in developing the benefit-cost analysis,
EPA staff should provide a more detailed description of the role of each type of
analysis, as well as more complete perspectives on the judgements made in develop-
ing estimates of benefits and costs. For example, the Executive Summary of the draft
RIA notes that benefits estimates would likely be larger if the health effects were
monetized. In an apparent contradiction to the Summary, the Health Benefits chapter
(Chapter 7) indicates that monetization was not undertaken because it was judged to
overlap the monetary measures developed using the avoiding cost model; no discus-
sion or explanation of these differing views are offered. Similarly, the hedonic esti-
mates are provided without developing their role in the comprehensive benefit-cost
analysis~nor their implications for further RIA analyses.
These descriptions would permit a second type of evaluation associated with
the aggregations that must accompany an RIA evaluation at a national level. Moneti-
zation of multiple components of benefits can lead to double counting if the concepts
being measured and the rationale underlying each method are not well-understood
and well-documented. Discussion of these concepts and rationale can provide a
means of avoiding this. Together with the presentation of the estimates, such a
discussion often allows plausibility checks, particularly if one approach produces
-------
estimates that can be expected to provide an upper*or lower bound on the estimates
of an alternative approach.
• The second major element of our review addressed the RIA's evaluation of the
benefits from avoiding adverse human health effects (the Committee prefers the term
"effects" to the term "benefits" as noted in the Technical Appendix). The RIA offers an
innovative analysis of exceptionally difficult issues. The analysis is to be commended
for recognizing and developing methods to reflect the fact that regulatory policies are
not always completely effective. While there are also notable features in this compo-
nent of the RIA, the Committee believes that there are also three important limitations.
First, the Committee concludes that efforts to monetize the mortality risks reduction
(e.g., reduction in statistical lives) should be included in the RIA. The Agency's
Guidelines for Preparing Regulatory Impact Analyses discusses these procedures and
recommends monetization, but the RIA does not do so. Monetizing of health effects is
also an integral part of the benefit-cost analyses being undertaken for the Clean Air
Act evaluation currently under way.
The argument that averting cost and monetized health effects would imply
double counting is correct, but it ignores the role these estimates could serve as
bounds on estimates of benefits. The inability to monetize all health effects should not
preclude this effort. It may be possible to offer better resolution about how benefits
might vary across different classes of facilities, regions, or other characteristics of the
SWMU through quantification of the health effects than by using information on
averting expenditures (because the exposed population and response estimates seem
more firmly grounded in scientific evidence than the averting expenditure information),
Aggregation of non-cancer health effects based on exceedances of the chronic
reference dose provides an index that is no more than informative. The Committee
has been advised that there are a number of judgements used in developing the
hazard indexes from hazard quotients (i.e., estimates of individual exposures to the
chronic reference dose) for specific substances. The Committee recommends that
efforts To" aggregate across substances be avoided and disaggregated information for
classes of substances be reported instead.
Finally, the cost effectiveness analysis and presentation of overall impacts
aggregates effects over a 128 year time horizon. The Committee recommends that
alternative strategies be investigated for dealing with the effects of this long time
horizon, including: discounting the measures of physical effects before gauging cost
effectiveness; calculating cost effectiveness based on a year-by-year cost effective-
ness and these ratios then discounted to a base period or an annualization of all
factors relevant to a comparison.
-------
(The Committee also identified some further technical issues on the health
benefits topic, as well as on the next two aspects of our review; a detailed summary is
. enclosed as a Technical Appendix to this letter.)
The EEAC devoted considerable effort to the issue of using the McClelland et
al. CV estimates as a basis for making national estimates of non-use values for
ground water cleanup. Because the focus of the McClelland et al. effort was to
develop oer-household estimates of these non-use values, a number of issues would
need to be addressed to use the results for national estimates relevant to the RIA.
The Committee's detailed findings on these issues are provided in the Technical
Appendix.
Based on our earlier review of the McClelland et al. study itself, and two further
issues raised by the methodology proposed for adopting the McClelland et al. results
for use in the RIA, the Committee believes that the McClelland et al. estimates cannot
be used for the intended purposes. The RIA's evaluation primarily deals with benefits
from cleaning up over a long period of time (and in some cases partially) groundwater
contaminated by industrial facilities. The McClelland et al. study focuses on municipal
sources and describes a situation with instantaneous cleanup. Both issues are very
important features of the problem and past literature suggests that they are likely to be
important to people's willingness to pay for cleanup. Unfortunately, simple adjust-
ments do not appear to be possible.
Last, a key element in the development of aggregate estimates is the determi-
nation of the number of households which would be willing to pay (a non-use value)
for cleanup of the groundwater. This factor is the key determinant of the wide range
of estimates of aggregate non-use values. No specific evidence has been developed
on how to determine the number of these households. The original McClelland et al.
study focused on per-household values. Subsequent work appears very preliminary
based on the information made available to the Committee.
"EPA is to be commended for recognizing the extent of the market question.
Nonetheless, the McClelland et al. methodology does not offer an approach to deal
with the problem.
Overall, the Committee recommends against using the specific approach
proposed in the draft RIA for developing the aggregate estimates of non-use values.
Unfortunately, there is no other information in the literature to be used to meet the
needs of the RIA effort.
-------
The Committee found the hedonic analysis to be a careful and systematic
evaluation of the effect of proximity of a contaminated site on the prices of nearby
residential properties. The Committee has some technical suggestions described in
the Appendix, and recommends that if the results of the hedonic analysis are included
in the RIA, two additions be made to the discussion. First, the relevant chapter should
describe dearly the analysis as providing a record of studies undertaken and summa-
rize in greater detail the reasons for not using the results in the benefit measurement.
Lastly, a brief discussion of the potential (at a conceptual level) for using
hedonic estimates to bound valuation estimates would be desirable.
In summary, from the perspective of its economic methodology, the draft RIA
reflects considerable creativity by EPA staff. Although the Committee had a number
of detailed suggestions, these can, with the exception of the estimates of aggregate
non-use values, be addressed with revisions and extensions to existing methods.
We look forward to receiving your response to our comments.
Sincerely,
Dr. Raymond C. Loehr, Chair
Science Advisory Board
«= •-
Dr/Allen Kneese, Co-Chair Dr/vTWetry^Smith, Co-Chair
Environmental Economics Environmental Economics
Advisory Committee Advisory Committee
ENCLOSURES
6
-------
TECHNICAL APPENDIX
The purpose of this Appendix is to summarize some detailed further sugges-
tions elaborating on the Committee's comments regarding human health benefits
estimates, the use of the McClelland et a/, findings as a basis for national non-use
value estimates, and the hedonic analyses.
aj Human Health "Benefits"
The Agency constructed two alternative baseline scenarios as descriptions of
conditions in the absence of the Proposed Rule:
1) human exposures would NOT be capped through existing MCLs and
taste/odor limits
2) exposures would be capped through existing MCLs and taste/odor limits.
The agency then estimated the reductions in the numbers of total cancer cases and
non-cancer health effects associated with the Proposed Rule assuming that the rule
would be less than 100 per cent effective in eliminating risks. The reductions in health
effects are listed as the human health benefits.
A positive feature of this analysis is the recognition that policies are not always
100 per cent effective and that realistic estimates of benefits should be based on the
expected level of effectiveness rather than the theoretical ideal. However, the
reported numbers are not particularly informative for policy purposes for several
reasons:
1) there is no economic valuation of the reductions in either cancer cases
or non-cancer health effects. This is puzzling, since the Agency's
Guidelines for Performing Regulatory Impact Analysis discusses ap-
proaches for estimating economic values for reductions in both morbidity
and mortality and other Agency RIAs have utilized these approaches to
monetize health effects.
~2) the measure of non-cancer health effects involves a difficult-to-interpret
aggregate index that consists of the sum of the numbers of individuals
with daily intakes in excess of the chronic reference doses (RfD) for the
substances analyzed plus numbers of individuals whose exposure to
lead results in blood lead levels above the threshold. Two important
pieces of information are obscured by this form of aggregations:
i) Different chemicals cause different types of health effects; and not
all health effects are of the same severity. Thus mild and severe
A-1
-------
adverse effects are lumped together and given the same weight in
this analysis.
ii) This form of aggregation treats a small exceedance of the RfD as
equivalent to a large exceedance. Also, using the number of
exceedances as an indicator of adverse health effects involves the
implicit assumption that all exceedances cause an adverse effect.
But in general, one would expect that as the average dose in-
creases above the RfD, the percentage of the exposed population
experiencing adverse health effects would rise, or the severity of
the effects would increase, or both.
3) there is no systematic effort to describe and quantify the uncertainties in
the analysis. In Chapter 13 a range of effects is reported. But this
range reflects only the two alternative baseline exposure scenarios.
In addition to these limitations, the Committee also has reservations about
several features of the analysis:
1) The uncapped scenario is not plausible as a matter of public policy. It
assumes, in effect, that existing regulations affecting human exposures
through a variety of pathways are ignored. This scenario undoubtedly
seriously overstates the number of cases of cancer and non-cancer
health effects in the absence of the rule and therefore overstates the
health benefits of the rule.
2) The report estimates baseline risks for people who move onto sites after
they are dosed (p. 7-39 to 40). Apparently these risks were not used in
the calculations for Table 7-23 (see the first line on p. 7-41). But this
raises the question of why these calculations were done and reported in
the first place. In any case, it seems unlikely that in the absence of the
rule, sites would be used in this way, given the high calculated risks.
Jl) - The report states that one big facility dominates the risk estimates. It
would be useful to see the results with this facility omitted. It also
suggests a policy design issue: should the regulations have two tiers
with stricter regulations for facilities like the one big one included in the
sample?
4) The report uses IRIS cancer slope factors which are mostly 95 percent
UCLs, not maximum likelihood estimates. The report is also based on
other standard "conservative" practices in risk assessment Thus, the
estimated health benefits cannot be considered to be expected values or
most likely values. Given the level of effort devoted to other parts of the
A-2
-------
analysis, and given the potential economic impact of the Proposed Rule,
more effort should be devoted to both expected values and upper and
lower bounds for the health benefits.
This decision to measure effects with count variables leads to the failure to take
account of the temporal distribution of health effects. One hundred twenty-eight years
of health effects are aggregated as if they were equally important and measured with
equal precision. This is especially troublesome when it is recognized that these
aggregates (over 128 years) provide the basis for the cost effectiveness analysis -
comparing discounted costs to undiscounted physical effects. At a minimum, the
comparison should be made year by year and then discounted.
b) Feasibility of Using McClelland et al Estimates for National Estimates
of Non-use Values for Groundwater Cleanup
Our earlier report (CV-1) raised a number of questions with the results of the
McClelland et. al contingent valuation study. The focus of that review was on the per
household estimates of nonuse values for complete cleanup of contaminated ground-
water. The overall judgment of the Committee was that we had no confidence that the
respondents to the McClelland et. al survey were clear about the commodity they were
being asked to value. This basic failing called into question the usefulness of the
willingness to pay (WTP) estimates as indications of a typical household value for
complete cleanup of contaminated groundwater.
There are also a number of problems with adopting this perspective, including
serious discrepancies between the context of the commodities to be evaluated as part
of the effects of the planned RIA and what has been described to the respondents in
the McClelland et. al survey. Equally important, there are significant unresolved
issues in the benefit transfer procedures used. These problems are independent of
our review of the McClelland et. al report and relate exclusively to the use of their
estimate in the draft methodology document for the RIA. The most important of our
concerns are:
JU- Technical Questions
0 At the dose of our review of the survey procedures, questionnaire and esti-
mates, it was suggested that the Committee might consider the feasibility of
using the estimates as an approximate upper bound on the per household WTP
to meet the goal of an upper bound evaluation of this source of aggregate
benefits generated by the Rule. There is no basis in the McClelland et. al
study, related literature (see Boyle [1993]) or the analysis presented in the RIA
to assume the estimates are upper bounds for per household WTP. The RIA
does not use the full range of estimates developed by McClelland et. al but
focuses instead on a comparison of two of the five scenarios, arguing incorrect-
A-3
-------
ty, that such comparisons were judged to be the most reliable in the NOAA
panel report on contingent valuation (see pp 10-14 to 10-15). Nonuse values
were measured by the difference between the complete cleanup and the public
treatment scenarios. No recommendation of this type was part of the NOAA
panel's report.
ii) The analysis did not rely on the primary data from the survey. The analysis
uses predictions from the Box-Cox model for WTP as if they were primary data,
performs a subsequent linear regression treating these predictions as depen-
dent variables and income plus regional dummy variables as independent
variables. Based on the McClelland et. al report, these variables were argu-
ments of the original Box-Cox models. This exercise has no basis in statistical
methods. Predictions from this model were adjusted by a single (for all esti-
mates) adjustment factor of .503 to compute the WTP for public treatment. The
difference was then the basis for the nonuse values. It is hard to understand
why this strategy was adopted when primary data on the difference that is
sought are available in version C of the McClelland et. al survey questionnaires.
These are not the only technical problems associated with the economic
methodology used in the benefit transfer. They are examples and reinforce the
need for appropriate peer review of methods prior to the presentation of materi-
als to committees of the SAB.
2) Benefit Transfer
i) The most fundamental problem arises with the source of the contamination.
All of the McClelland et. al questionnaire variations identify the source of
contamination as a public landfill The majority of the sites affected by the
RCRA rule would not fit this description. The pretest and design work from the
McClelland et. al study suggest this distinction is very important. Virtually all of
the literature on people's responses to contamination of groundwater finds
similar results. The source matters to people's evaluation of the problem and to
the character of the response.
_«•" *
ii) The extent and timing of cleanup activities presented in the McClelland et. al
survey are completely different from all of the potential RCRA actions and there
is no basis for gauging the temporal properties of the nonuse estimates in
response to changes identified as critical in the McClelland et. a/ pretest as well
as in the earlier Mitchell-Carson focus group analyses.
iii) The most fundamental issue giving rise to the range in estimates of the
aggregate nonuse values is the extent of the market Estimates range from
$170 million to $18 billion for aggregate nonuse values depending on the
assumption about extent of the market This variation does not relate to the
A-4
-------
variability in contingent valuation estimates or "to the Committee's concerns
about the commodity understood by survey respondents. It results exclusively
from assumptions about which households are actually concerned about each
site.
The RIA presents alternative estimates, identifies the fact that McClelland et. al
use the phrase "in your community" to describe how households were intended
to infer responsibility. Subsequent research is sketched in four pages of a later
McClelland et. al [1993] report where it is suggested that the concept of
community was examined in March 1993 post-testing of the survey. The
authors describe a question added to the survey to elicit information about how
respondents would evaluate their community. No information was provided in
that report about how the results used to propose a definition for the community
(as a proxy measure for the extent of the market) were derived - sample size,
implementation, and correspondence of other variables for the new survey with
the original survey were not discussed.
These comments are intended to illustrate the substantive questions in the
development of the nonuse benefit estimates. Taken together with the ques-
tions about interpreting the per household estimates of nonuse values, we are
forced to conclude that there is no basis for accepting any of the EPA estimates
of nonuse values.
c) Hedonic Analyses
The hedonic chapter and "sanitized" companion report provide an analysis of
residential property values using a hedonic framework where housing prices are
related to the home and site characteristics including the distance from hazardous
waste treatment, storage and disposal facilities (TSDF) at three distinct sites.
In each case study, a notable "event" took place at the TSDF during the
observation period. For two of the studies, distance from the TSDF became a
significant explanatory variable in the hedonic price equation only after the publicizing
of the Invent" In the third case study, distance from the site was strongly significant
throughout the entire period. The results from this study are qualitatively reasonable
and promising. They are based on large samples of housing prices. But, the authors
and the Committee caution against adoption of the quantitative estimates of benefits
of remediation based on these studies at this time. We reiterate that list adding
additional reasons to those reported and introducing some new concerns:
1) Functional form. The report relies primarily on a linear hedonic model,
although it employs a semi-log and a piece-wise (in distance from site)
linear form. The "benefit" estimates (see point 5 below) obtained seem
quite sensitive to functional form and the authors admit the need to
A-5
-------
investigate. Additionally, the implication of linear in characteristics
models, i.e. constant marginal prices of characteristics, has not been
found especially satisfactory in other studies. The fact that housing
prices are being underestimated in some house size ranges and overes-
timated in others suggests to them and to us the need to experiment with
more flexible functional forms with respect to housing characteristics as
well.
2) Explanatory power. Little of the variation in housing prices is explained
by the model probably because data are not available on important .
housing characteristics (especially neighborhood characteristics). Both
the report and the Committee agree on this point This is a particularly
difficult problem in one case study where the effects of one TSDF are
being estimated, when in fact several such facilities exist in the surround-
ing area.
While it should be possible to incorporate some additional neighborhood
characteristics, others may pose more complicated problems than the
authors anticipate. Some neighborhood characteristics might themselves
be endogenous - brought about by the location of the facility. The
report suggests using socio-demographic characteristics of the neighbor-
hood to explain price differences but these, too, may pose an
endogeneity problem.
3) Outliers. The report suggests that outliers (i.e. houses that sell for less
than $50,000 and over $500,000) are quite influential in their estimates.
Clearly the model is not explaining these very well. If, in addressing
problems 1 and 2 above, this outlier problem is not resolved, the Com-
mittee suggests the authors adopt some of the numerous available
statistical techniques for dealing with outliers rather than relying on ad
hoc procedures.
4) Distance as a proxy variable. The authors use distance from a TSDF as
^ ' a proxy for the relative damage due to the site. But, all effects from a
site are not radially symmetric. Prevailing winds and topographical
features can increase/decrease the impact of noxious fumes and other
aesthetic effects. The significance of these will be case specific.
5) Benefit measurement Without any discussion, the report takes the
change in the hedonic price function (as if each affected house were
moved farther from the site) as a measure of benefits. Welfare measure-
ment in hedonic models is extremely complicated and depends on such
factors as the extent of the market, whether the market could be consid-
ered closed or open, the heterogeneity of participants, etc. Even if one
A-6
-------
considers only a short run, when indivfduals (and therefore the hedonic
price functions) do not adjust, the hedonic price function is a locus of
equilibrium points and not a demand function.
Under certain circumstances, McConnell (in Kopp and Smith, Valuing
Natural Assets [1993]) has suggested that the change in the hedonic
price as predicted by the change in policy could serve as an approximate
welfare measure. However, the report shows no appreciation for this
complex problem and needs to consider the factors that would make this
a good or bad approximation, or an upper or lower bound. From our
understanding of the problem, this predicted change is likely to be an
upper bound on the welfare measure sought, but this needs to be
examined. Other literature of interest on the subject includes Palmquist
(in Braden and Kolstad, eds., Measuring Demand for Environmental
Commodities. [1991]; Kanemoto Econometrics [1988]; Cropper, Deck
and McConnell REStat [1988]; Bartik and Smith in Mills, eds., Handbook
of Urban Economics. [1987].
6) Repeat sales/time series. In addition to the hedonic study described
above, the report analyzed some time series data in a "repeat sales"
analysis for a small sample of houses that were sold more than once
during the time period. Little came of that analysis and, for a number of
reasons, we do not recommend that they pursue this part of the study.
A-7
-------
REFERENCES
Boyle, K. J. A Review of Contingent Valuation Studies of the Benefits of Groundwater
Protection (report to U.S. Environmental Protection Agency), Research Triangle
Institute, April 1993).
McClelland, G.H., Schulze, W.D. Lazo, J.K., Waldman, D.M., Doyle, J.K., Elliott, S.R.
and Irwin, J.R. Methods for Measuring Non-Use Values: A Contingent Valua-
tion Study of Groundwater Cleanup (Draft report to the U.S. EPA), University of
Colorado, Center for Economic Analysis, October, 1992.
R-1
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
ENVIRONMENTAL ECONOMICS ADVISORY COMMITTEE
RCRA RIA REVIEW
CO-CHAIRS
Dr. Allen V. Kneese, Resources for the Future, Washington DC
Dr. V. Kerry Smith, Department of Economics, North Carolina State University, Raleigh NC
MEMBERS
Dr. Nancy E. Bockstael, Department of Agricultural and Resource Economics, University of
Maryland, College Park, MD
Dr. A. Myrick Freeman, Department of Economics, Bowdoin College, Brunswick, ME
Dr. Charles D. Kolstad, Department of Economics, University of Illinois, Urbana, IL
Dr. William Nordhaus, Department of Economics, Yale University, New Haven CT
Dr. Bryan Norton, School of Public Policy, Georgia Institute of Technology, Atlanta GA
Dr. Wallace E. Oates, Department of Economics, University of Maryland,College Park, MD
Dr. Paul R. Portney, Resources for the Future, Washington DC
Dr. Robert Repetto, World Resources Institute, Washington, DC
Dr. Richard Schmalensee, Massachusetts Institute of Technology, Cambridge MA
Dr. Robert N. Stavins, Kennedy School of Government, Harvard University, Cambridge, MA
Dr. Thomas H. Tietenberg, Department of Economics, Colby College, Waterville, ME
Dr. W. Kip Viscusi, Department of Economics, Duke University, Durham, NC
SAB COMMITTEE LIAISONS
Dr. William Cooper (EPEC), University of Michigan
Mr. Richard Conway (EEC), Union Carbide Corporation
Dr. Morton LJppmann (IAQC), Nelson Environmental Institute, New York University
Dr. Roger McCleUan (CASAC), Chemical Industry Institute of Toxicology
DESIGNATED FEDERAL OFFICER
Mr. Samuel Rondberg, Environmental Health Committee, Science Advisory Board (1400F),
U.S. Environmental Protection Agency, Washington, DC 20460
STAFF SECRETARY
Ms. Mary L Winston, Environmental Protection Agency, Science Advisory Board (1400F),
Washington, DC 20460
-------
NOTICE
This report has been written as a part of the activities of the Science Advisory
Board, a public advisory group providing extramural scientific information and advice
to the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters
related to problems facing the Agency. This report has not been reviewed for
approval by the Agency and, hence, the contents of this report do not necessarily
represent the views and policies of the Environmental Protection Agency, nor of other
agencies in the Executive Branch of the Federal government, nor does mention of
trade names or commercial products constitute a recommendation for use.
-------
EPA-SAB-EEC-94-002 'Review of MMSOILS
component of the Proposed RIA for the RCRA
Corrective Action Rule" by the Environmental
Engineering Committee
-------
United States Science Advisory EPA-SAB-EEC-94-002
Environmental Board (1400F) November 1993
Protection Agency
xvEPA AN SAB REPORT: REVIEW
OF MMSOILS COMPONENT
OF THE PROPOSED RIA FOR
THE RCRA CORRECTIVE
ACTION RULE
REVIEW OF THE OSWER & ORD
DRAFT DOCUMENTATION AND
USER'S MANUAL AND RIA OF
THE MMSOILS MULTIMEDIA
CONTAMINANT, FATE, TRANSPORT,
AND EXPOSURE MODEL
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON. D.C. 20460
November 19, 1993 OFFICE OF THE ADMINISTRATOR
' SCIENCE ADVISORY BOARD
EPA-SAB-EEC-94-002
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, S.W.
Washington, D.C. 20460
Re: Review of MMSOILS Component of the Proposed Regulatory
Impact Analysis (RIA) for the RCRA Corrective Action Rule
Dear Ms. Browner:
The Science Advisory Board (SAB) is pleased to submit its report on review
of the Agency's draft document entitled "MMSOILS: Multimedia Contaminant
Fate, Transport and Exposure Model Documentation and User's Manual," dated
September 1992. The MMSOILS (Multi-Media Contaminant, Fate, Transport and
Exposure Model) document was developed jointly by the Office of Research and
Development's (ORD's) Office of Health and Environmental Assessment (OHEA),
Exposure Assessment Group (EAG) and the Office of Environmental Processes and
Effects Research (OEPER). This report by the MMSOILS Model Review
Subcommittee (MMRS) was prepared as part of the SAB's review of the "Draft
Regulatory Impact Analysis for the Final Rulemaking on Corrective Action for
Solid Waste Management Units: Proposed Methodology for Analysis." Our report
resulted from the MMRS public reviews on April 22 and 23 and June 29, 1993.
The Agency, through the Office of Solid Waste and Emergency Response
(OSWER) asked the SAB to review specific elements of the multi-media
contaminant fate, transport and exposure model, MMSOILS, with regard to the
methodology used to predict contaminant concentrations in the environment and
the resultant implications on human health and ecological risk assessments.
Specifically, the review dealt with:
a) the adequacy of methods for using a screening level model where
there is substantial subsurface heterogeneity and/or where non-
aqueous phase liquids (NAPLs) are present,
b) the appropriateness of the Agency's approach for aggregating releases
from solid waste management units (SWMUs) in order to estimate
concentrations at exposure points as a function of time, and
Piw»do«upwindo«»m«
-------
c) the adequacy of the Agency's approach for developing long-term
effectiveness and failure scenarios for site remedies.
The OSW/ORD working group is to be commended for a well-coordinated
and focused effort to develop a regulatory impact analysis (RIA) that will help the
Agency and the Nation better understand the costs and benefits of the proposed
rule. The Subcommittee wishes to recognize the responsiveness and progress made
by the OSW/ORD working group to many of the recommendations made at the
April 22-23, 1993 meeting and candidly displayed in detail at the June 29, 1993
meeting. The Subcommittee appreciates and recognizes the significant effort
expended to date, the positive attitude and open candor displayed by the
P^W/ORD working group in their presentations and interactions during the two
review meetings. The Subcommittee considers the intraagency coordination
represented by this RIA to be a "model approach" that the Agency would do well
to adopt in other programs.
On the positive side, the Subcommittee observes that MMSOILS uses
simple, conservative, and computationally efficient equations for estimating
chemical transport via ground water, surface water, soil erosion, atmospheric, and
food chain pathways. Pathway documentation is well organized with appropriate
references. Applied mathematical formulae are widely used and accepted by the
scientific community for use in simple situations. Underlying assumptions have
been identified, clearly stated, and appear to be reasonable yet not overly
restrictive. Given these strengths, MMSOILS, when applied to simplified case
studies, might certainly be a valid screening tool for assessing the relative risks
and costs associated with alternative regulatory options.
However, the Subcommittee notes that two problems create unquantifiable
uncertainties that seriously diminish the utility of MMSOILS relative to its use in
the draft Corrective Action Regulatory Impact Analysis (RIA), namely:
a) inaccurate input parameters, and
b) application of the model to cases outside its range of validity.
Inadequate input data are a consequence of sparse or inaccurate
information, poor parameter estimation especially relative to source terms, and
suspected over-reliance upon default parameters. The Subcommittee recommends
a documented and thorough peer review of all aspects of the data base, focusing
particularly on those parameters to which the results are most sensitive.
Equally serious is the inappropriate application of the MMSOILS model to
scenarios for which it was not intended, such as sites with complex hydrogeological
conditions or sites where NAPLs ar<, ^resent. To some extent, as discussed in the
Subcommittee's full report, the latter could be addressed by means of appropriate
-------
revisions to model formulations. However, for a significant number of sites, the
Subcommittee suspects that no generic model is likely to provide answers of
acceptable quality. OSWER is generally aware of the limited usefulness of generic
models for the analysis of complex environmental settings.
The Subcommittee also observes that uncertainly analysis for the RIA is in
its infancy and will require much greater effort to meet the needs of the
assessment process. Given the high stakes involved in terms of potential
commitment of national resources, defensible estimates of the uncertainties
associated with risk" and benefits are critical, and the protocol followed to obtain
such estimates deserves as much forethought and careful peer review as that
required to obtain the central estimate. As a related issue, the Subcommittee is
concerned that the simple protocol followed to obtain high-end risk estimates may
be inadequate in that these estimates in some cases apparently gave rise to lower
exposures than did the central tendency estimate.
Given these serious shortcomings, many of which were already recognized
by the Agency, the most basic and pressing concern of the Subcommittee is
whether the use of a generic model such as MMSOILS is appropriate as a basis
for the assessment of regulatory costs and benefits at the national level, given the
fate and transport estimates that comprise the model output may be wrong by
orders of magnitude for many complex sites. We recommend that the Agency:
a) augment its RIA with cost/benefit estimates derived by alternative
approaches, such as:
1) utilizing assessment data generated for Superfund sites,
2) using more sophisticated models with better-defined data to
develop estimates for representative sets of waste sites, or
3) applying site-specific models to analyze that relatively small
number of facilities which MMSOILS results indicate dominate
the total costs or risks, and
b) at a minimum, expert review of the latter cases should be undertaken
to judge the reasonableness of model outputs.
This augmentation should help validate the present reliance on the screening
studies that use MMSOILS model output as a starting point.
-------
The SAB appreciates the opportunity to comment on the EPA's MMSOILS
model. We are gratified that the Agency has brought this issue before us and look
forward to receiving a summary of the EPA's response, particularly to the points
raised in this letter to you.
Sincerely,
Dr. Raymond C. Loehr, Chair Mr. Richard A. Conway, Chair
Executive Committee Environmental Engineering Committee
Science Advisory Board Science Advisory Board
Dr. C. Herb Ward, Chair
MMSOILS Model Review Subcommittee
Environmental Engineering Committee
Science Advisory Board
-------
NOTICE
This report has been written as a part of the activities of the Science
Advisory Board, a public advisory group providing extramural scientific
information and advice to the Administrator and other officials of the
Environmental Protection Agency. The Board is structured to provide a balanced,
expert assessment of scientific matters related to problems facing the Agency. The
report has not been reviewed for approval by the Agency; hence, the comments of
this report do not necessarily represent the views and policies of the
Environmental Protection Agency or of other federal agencies. Any mention of
trade names or commercial products does not constitute endorsement or
recommendation for use.
-------
ABSTRACT
The MMSOILS Model Review Subcommittee (MMRS) of the Environmental
Engineering Committee (EEC) of the EPA Science Advisory Board (SAB) has
prepared a report on the Agency's Office of Solid Waste (OSW) MMSOILS
Multimedia Contaminant Fate, Transport, and Exposure Model. This model and
guidance document was developed as a technical resource for estimating potential
health risks at sites contaminated by toxic wastes or spills of toxic chemicals.
The review by the SAB's MMRS dealt with the adequacy of methods for
using a screening level model where there is substantial subsurface heterogeneity
or where non-aqueous phase contaminants are present, the appropriateness of the
Agency's approach for aggregating releases from solid waste management units
(SWMUs) to estimate concentration at exposure points over time, and the
adequacy of the Agency's approach for developing long-term effectiveness and
failure scenarios for site remedies.
The general consensus of the MMRS was that the use of a multimedia
pathway model for screening purposes could be an appropriate approach for
developing risk and cost estimates for a national-level Regulatory Impact Analysis
(RIA), as long- as the input parameters are accurate and the model is not applied
outside its range of validity. Furthermore, the Agency's use of a single model, to
the extent defensible for each facility considered, was viewed by the MMRS as
necessary in order to ensure consistency among model results. The major
overriding concerns of the MMRS were: a) application of MMSOILS outside its
range of validity; b) large uncertainties in input parameters; c) consequent large
uncertainties in MMSOILS results; d) clear communication of this uncertainty to
decision-makers; and e) presentation of the results in the draft RIA document in a
scientifically defensible manner that communicates the uncertainties of the
calculations and their implications for the cost/benefit analysis.
The MMRS recommended that the Agency augment the MMSOILS results
with cost/benefit estimates derived by alternative approaches, such as utilizing
assessment data generated for Superfund sites, using more sophisticated models
with better-defined data to develop estimates for representative sets of waste sites,
applying site-specific models to analyze that relatively small number of facilities
which MMSOILS results indicate dominate the total costs or risks, and submission
of selected case studies to expert panel review.
Key Words: Mathematical Models, Cleanup, Corrective Action, Regulatory Impact
Analysis, RCRA Models
11
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee
MMSOILS Model Review Subcommittee
CHAIR
Dr. C. Herb Ward, Professor and Director, Energy and Environmental Systems
Institute, Rice University, Houston, TX
VICE CHAIR
Dr. George F. Carpenter, Environmental Quality Manager, Environmental
Response Division, Michigan Department of Natural Resources, Lansing MI
AND CONSULTANTS
Dr. Steven M. Bartell, Vice President, Center for Risk Analysis, SENES Oak
Ridge, Inc., Oak Ridge, TN
Dr. Randall J. Charbeneau, Professor and Director, Center for Research in Water
Resources, Department of Civil Engineering, University of Texas, Austin, TX
Dr. Calvin C. Chien, Principal Consultant, duPont Corporate Remediation Group,
E.I. duPont de Nemours & Company, Wilmington, DE
Dr. Rolf Hartung, Professor of Environmental Toxicology, Department of
Environmental and Industrial Health, School of Public Health, University of
Michigan, Ann Arbor, MI
Dr. Wayne M. Kachel, Corporate Environmental Management, Martin Marietta
Corporation, Oak Ridge, TN
Dr. June Fabryka-Martin, Hydrogeologist, Isotope and Nuclear Chemistry Division,
Los Alamos National Laboratory, Los Alamos, NM
Dr. Ishwar P. Murarka, Senior Program Manager, Land & Water Quality Studies,
Environmental Division, Electric Power Research Institute, Palo Alto, CA
Dr. Bernard Weiss, Professor, Department of Environmental Medicine, University
of Rochester Medical Center, Rochester, NY
Science Advisory Board Staff
Dr. K. Jack Kooyoomjian, Designated Federal Official, U.S. EPA, Science Advisory
Board, (1400F), 401 M Street, SW, Washington, DC 20460
Mrs. Diana L. Pozun, Staff Secretary
Dr. Donald G. Barnes, Staff Director, Science Advisory Board
in
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
1.1 Overall Comments 1
1.2 Response to Charge 2
1.3 Additional Observations and Recommendations 4
1.3.1 Model Selection, Development, Formulation and
Documentation 4
1.3.2 Possible Improvements to Model Formulations for
Specific Processes 5
1.3.3 Issues of Parameter Estimation 6
1.3.4 Issues of Range of Model Validity 7
1.3.5 Issues Relating to Pathway Model Verification and
Validation 7
1.3.6 Comments on Remediation Effectiveness 8
1.3.7 Issues Relating to Assessment of Uncertainty 9
1.3.8 Comments on Results for Health Risk Analysis 9
1.3.9 Comments on Use of MMSOILS in Corrective Action RIA . 10
1.3.10 Other User Groups for MMSOILS 11
2. INTRODUCTION 12
2.1 Charge for SAB Review 12
2.2 SAB Review Procedure 12
3. COMMENTS ON MODEL SELECTION, FORMULATION,
DOCUMENTATION AND APPLICATION 14
3.1 MMSOILS Selection, Development, Formulation and
Documentation 14
3.1.1 Model Selection and Development 14
3.1.2 Use of Standard Formulations 15
3.1.3 Documentation of Modeled Pathways 15
3.1.4 Documentation of Assumptions Underlying Multimedia
Treatment 15
3.2 Possible Improvements to Model Formulations for Specific
Processes 16
3.2.1 Additional Types of Solid Waste Management Units
(SWMUs) 16
3.2.2 Recognition of Natural Biodegradation Processes in
Ground Water Pathway 16
3.2.3 Modeling Transport through the Vadose Zone 17
3.2.4 Plume Aggregation in Groundwater Pathway 17
3.2.5 Food-Chain Module 17
3.2.6 Mass Balance 18
3.2.7 Disparity in Relevant Time and Space Scales for
Transport Mechanisms 18
3.2.8 Relevant Time Scales for Ecological Risk Assessment .... 18
3.3 Issues of Parameter Estimation 19
iv
-------
TABLE OF CONTENTS: CONTINUED:
3.3.1 Source Term Parameters 19
3.3.2 Waste Release and Solubility 20
3.3.3 Use of Default Values 21
3.3.4 Peer Review of Data Base 21
3.3.5 Data Base for Future Related Modeling Efforts 21
3.4 Issues of Range of Model Validity 21
3.4.1 Extreme Events 22
3.4.2 Complex Sites 23
3.4.3 NAPLs 23
3.4.4 Development of Guidelines for Assessing Model
Applicability to Specific Cases 23
3.5 Issues Relating to Pathway Model Calibration, Verification and
Validation 24
3.5.1 Ground Water Model 24
3.5.2 Other Pathway Models 24
3.5.3 Guidelines for Calibration, Verification and Validation ... 24
3.6 Comments on Remediation Effectiveness 24
3.6.1 Remediation Times 24
3.6.2 Effect of Unknown Presence of DNAPLs on Remediation
Times 25
3.6.3 Remediation Effectiveness 25
3.6.4 Inclusion of Biologically-Based Remediation Technologies . 26
3.6.5 Risks of Remediation 26
3.7 Issues Relating to Assessment of Uncertainty 26
3.7.1 Uncertainty Estimation Protocol 26
3.7.2 Development of High-End Risk Estimates 27
3.8 Interpretation of Results for Health Risk Analysis 27
3.8.1 Health Risk as the Assessment Endpoint 27
3.8.2 Empirical Validation of Exposure Estimates 28
3.8.3 Inconsistent Treatment of Cancer and Noncancer Health
Risks 28
3.8.4 Inaccurate Identification of Critical Health Effects 28
3.8.5 Questionable Treatment of Different Waste Classes 29
3.8.6 Other Sources of Hazardous Wastes 29
3.9 Comments on Use of MMSOILS in Corrective Action RIA 29
3.9.1 Facility Selection Process 29
3.9.2 Use for National-Level Screening 30
3.9.3 Presentation of Results in Corrective Action RIA 30
3.9.4 Presentation of Uncertainty Analysis in RIA 31
3.10 Other User Groups for MMSOILS 31
3.10.1 Applicability to Other EPA Program Activities 31
3.10.2 Use for State-Level Screening 31
3.10.3 Other User Groups 31
-------
TABLE OF CONTENTS: CONTINUED:
APPENDIX A - BRIEFING AND REVIEW MATERIALS A-l
APPENDK B - REFERENCES CITED B-l
APPENDK C - GLOSSARY OF TERMS AND ACRONYMS C-l
VI
-------
1. EXECUTIVE SUMMARY
In response to a request from the Office of Solid Waste and Emergency
Response (OSWER), the Science Advisory Board (SAB) has reviewed several
aspects of the draft Regulatory Impact Analysis (RIA) prepared in support of the
Resource Conservation and Recovery Act (RCRA) Corrective Action Rule. At the
October 1992 meeting, the SAB Executive Committee, recognizing the importance,
complexity, and creativity of OSWER's work and its multi-disciplinary nature,
established an ad hoc Steering Committee to assure that certain significant aspects
of the RIA - both methodology and application - received appropriate attention
from the relevant SAB standing Committees.
At a public meeting on January 29, 1993, the Steering Committee concluded,
based on presentations by and discussions with OSWER staff, that four SAB
committees, with appropriate inter-committee liaison participation, should review
major segments of the RCRA Corrective Action RIA as follows: the Environmental
Economics Advisory Committee (EEAC) would review the Contingent Valuation
(CV) methodology and its application in the RIA; the Environmental Engineering
Committee (EEC) would review the MMSOILS multi-media contaminant fate,
transport and exposure model; the Ecological Processes and Effects Committee
(EPEC) would review the ecological risk analysis; and the Environmental Health
Committee (EHC) would review the human health risk assessment. In addition,
the Steering Committee agreed to prepare an overview report to accompany the
individual committee reports.
The MMSOILS Model Review Subcommittee (MMRS) of the EEC reviewed
the Agency's draft document entitled "MMSOILS: Multimedia Contaminant Fate,
Transport and Exposure Model Documentation and User's Manual," dated
September 1992 (See Appendix B, Reference 6), as well as the supporting RIA and
Appendices (See Appendix B, References 7 & 8). The draft documentation and
user's manual was developed jointly by the Office of Research and Development's
(ORD's) Office of Health and Environmental Assessment (OHEA) and Office of
Environmental Processes and Effects Research (OEPER). The MMRS report
resulted from a review of the above draft documents and briefing materials at
meetings on April 22-23, 1993 and June 29, 1993 (See Appendix A, and Appendix
B; References 6 through 8).
1.1 Overall Comments
The OSW/ORD working group is to be commended for a well-coordinated
and focused effort to develop a regulatory impact analysis (RIA) that will help the
Agency and the Nation better understand the costs and benefits of the proposed
rule. The Subcommittee wishes to recognize the responsiveness and progress made
by the OSW/ORD working group to many of the recommendations made at the
April 22-23, 1993 meeting and candidly displayed in detail at the June 29, 1993
meeting. The Subcommittee appreciates and recognizes the significant effort
-------
expended to date, the positive attitude and open candor displayed by the
OSW/ORD working group in their presentations and interactions during the two
review meetings. The Subcommittee considers the intraagency coordination
represented by this RIA to be a "model approach" that the Agency would do well
to adopt in other programs.
The consensus of the MMSOILS Model Review Subcommittee (MMRS) is
that the use of a multimedia pathway model for screening purposes could be an
appropriate approach for developing risk and cost estimates for a national-level
RIA as long as the input parameters are accurate and the model is not applied
outside its range of validity. The Agency's use of a single model, to the extent
defensible, ensures consistency among model results.
The major overriding concerns of the MMRS are the application of
MMSOILS outside its range of validity; large uncertainties in input parameters; .
consequent large uncertainties in MMSOILS results; clear communication of this
uncertainty to decision-makers; and the generation of credible guidance on
exposure, risk, costs, and benefits. Consequently, the recommendations contained
in this report are focussed at efforts to decrease the level of uncertainty, to
validate the MMSOILS results by comparison with alternative estimation methods,
and to ensure that the results of the modeling exercise are expressed in the RIA
background documents in a scientifically defensible manner that communicates the
uncertainties of the calculations and their implications for the cost/benefit analysis.
1.2 Response to Charge
The following issues were presented in the charge to the Subcommittee.
(Please note that numbers following specific observations and recommendations
refer the reader to more detailed discussion in Section 3 of this review report.):
Issue 1. The adequacy of methods for using a screening level model to
characterize situations where there is a substantial subsurface
heterogeneity or where non-aqueous phase contaminants are present.
While the Subcommittee (the MMRS) agrees that a screening-level model
may be appropriate for developing risk and cost estimates for a national-level RIA,
the MMRS recommends that the current version of MMSOILS not be applied to
the characterization of contaminant distributions in ground water in complex
hydrogeological settings or where Non-Aqueous Phase Liquids (NAPLs) may be
present. For these facilities, the MMRS recommends that alternative approaches
to characterization should be used. Such approaches include modification of the
MMSOILS ground water module to more accurately model contaminant movement
in complex hydrogeologic settings; utilization of assessment data generated at
Superfund sites; application of more sophisticated models with better-defined data
to develop estimates for representative sets of waste sites; application of site-
specific models to analyze that relatively small number of facilities which
MMSOILS results indicate dominate the total costs or risks; and submission of
-------
these case studies to expert panel review to develop estimates of contaminant
migration.
Issue 2. Appropriateness of the Agency's approach for aggregating releases
from solid waste management units (SWMUs, the source terms for
the contaminant modeling) to estimate concentration at exposure
points over time.
The Subcommittee is concerned that the method of aggregation used to
obtain the concentration distributions for application to individual wells may not
conserve mass. Rather, it appears that the resulting apparent mass and the
average concentrations for each concentric ring downgradient from the SWMUs
will always exceed that from the untransformed plumes. Such an approach may
nonetheless be defensible for the purposes of the RIA because it is conservative;
however, the MMRS recommends that the degree of conservatism be evaluated
through comparison with a number of simulations which do not use this method
of aggregation. Unduly conservative estimates can cause the inappropriate
prioritization of risks. In addition, the MMRS recommends that the Agency
evaluate whether movement of the contaminant plumes could result in a decreased
concentration for population wells. The required transformations from cartesian
to cylindrical coordinates should not require much computational effort compared
with that required for the model to begin with.
Issue 3. Adequacy of the Agency's approach for developing long-term
effectiveness and failure scenarios for site remedies.
The Subcommittee observes that the annual time scale for exposure
estimates produced by MMSOILS may be inappropriate for many ecological
applications. Typical organisms of concern exhibit short life spans, or critical
stages in their complex life histories that occur at time scales substantially shorter
than one year. Thus, the Subcommittee recommends the modification of
MMSOILS to produce more realistically-scaled exposures for meaningful inputs to
ecological risk analysis (Recommendation #13; also Section 3.2.8). The
Subcommittee recommends that the ecological risk assessment component be
constructed using the principles for ecological risk assessment as suggested by the
Risk Assessment Forum. (Recommendation #9; also Section 3.2.5; See also
Appendix B, Reference 23). The Subcommittee further recommends that
ecologically relevant exposure scenarios be modified so as to be capable of
simulating acute impacts from waste sites on aquatic environments due to surface
run-off after major rain events. (Recommendation #10; also Section 3.2.5).
Issue 4. The implications of the fate and transport modeling assumptions on
the ecological and human risk assessment.
The Subcommittee observes that certain hazardous agents are not easily
controlled and may pose health risks beyond the substances discussed in the RIA.
(Observation #43; Section 3.8.6). The Subcommittee recommends that the Agency
-------
revise its practices for assessing cancer and noncancer health risks so as to make
them more consistent with one-another (Recommendation #40; Section 3.8.3). The
Subcommittee also recommends that the Agency review its assumption of
additivity of Hazard Indices, and additivity of risks across Class A (known) and
Class C (suspected) carcinogens (Recommendation # 42; Section 3.8.5). The
Subcommittee further recommends that the Agency review its discussion of critical
health effects and correct any inaccurate information (Recommendation #41;
Section 3.8.4).
The Subcommittee recommends that the Agency consider how the general
validity of its exposure estimates might be tested by comparison with empirical
field data. This is being recommended as a result of the observation by the
Subcommittee that the translation of contaminant concentrations to estimates of
exposure necessarily involves a long chain of assumptions and requires the
adoption of parameter values of variable uncertainty (Recommendation #39;
Section 3.8.2).
1.3 Additional Observations and Recommendations
1.3.1 Model Selection, Development, Formulation and Documentation
Recommendation 1. The MMRS recommends that the criteria and rationale
for the selection of MMSOILS be more fully documented in the RIA so that the
scientific and strategic bases for the selection will be clear to all concerned -
regulator, regulated, and scientific/risk assessment/economic communities at large.
(3.1.1)
Observation 2. The model uses simple, conservative, and computationally
efficient equations for estimating chemical transport via ground water, surface
water, soil erosion, the atmosphere, and foodchains. Mathematical formulae used
to estimate transport rates for each pathway are widely used and accepted by the
scientific community for application to simple situations. Underlying assumptions
for each pathway model have been identified, are clearly stated, are reasonable and
are not overly restrictive. However, for a significant number of sites, the MMRS
suspects that no generic model is likely to provide answers of acceptable quality.
OSWER is generally aware of the limited usefulness of generic models for the
analysis of complex environmental systems, including aquifers. (3.1.2)
Recommendation 3. While documentation of the formulations for individual
pathway models is well organized with appropriate references, the manual would
benefit from another round of editing. (3.1.3)
Recommendation 4. Documentation for MMSOILS would benefit from a
concise and explicit presentation «r the model's basis, assumptions and limitations
in a central location at the beginning of the user's manual. (3.1.4)
-------
1.3.2 Possible Improvements to Model Formulations for Specific Processes
Recommendation 5. MMSOILS should be modified to make it capable of
handling other, potentially more costly, types of SWMUs such as leaky sewer
systems which are currently excluded from the RIA. Eventually these problematic
SWMUs will be impacted by the proposed corrective action rule, so an estimate of
their cost contribution to the implementation of the proposed rule should be
developed. (3.2.1)
Recommendation 6. Because of the long time periods involved, the MMRS
believes that it is critical that the role of natural biodegradation processes be
explicitly incorporated into the ground water fate and transport pathway by the
use of an appropriate biodegradation coefficient value. This function is essential
and provides realism for actual mechanisms taking place. Even a small
biodegradation coefficient would have a big impact.(3.2.2)
Recommendation 7. The MMRS recommends that the unsaturated-zone
transport module be replaced with a simple kinematic model in order to make its
treatment consistent with the other transport process models. (3.2.3)
Recommendation 8. The MMRS recommends that the Agency quantitatively
assess the degree of conservation introduced by its method of plume aggregation
through a comparison with simulations which do not use this method. As a part
of this exercise, the Agency should quantitatively evaluate whether movement of
the plumes could result in a decreased concentration for population wells. (3.2.4)
Recommendation 9. The MMRS recommends that the ecological risk
assessment component be constructed using the principles for ecological risk
assessment as suggested by the Risk Assessment Forum. (3.2.5)
Recommendation 10. The MMRS recommends that the ecologically relevant
exposure scenarios be modified so as to be capable of simulating acute impacts
from waste sites on aquatic environments due to surface run-off after major rain
events. (3.2.5)
Observation 11. The MMRS is concerned that the MMSOILS model may
not effectively estimate long-term consequences of remediation alternatives due to
a suspected breakdown of mass balance as a result of model output post-
processing. (3.2.6)
Observation 12. The MMRS notes that a major problem that must be
confronted in the development of any multimedia model, such as MMSOILS, is the
forcing of differently scaled environmental transport processes into a single model
construct. Attempts to force disparate scales into a single model by selecting a
compromise in time step will necessarily result in a loss of accuracy in model
predictions. (3.2.7)
-------
Recommendation 13. The Subcommittee recommends the modification of
MMSOILS to produce more realistically-scaled exposures for meaningful inputs to
ecological risk analysis. This recommendation results from the observation that
the annual time scale for exposure estimates produced by MMSOILS may be
inappropriate for many ecological applications. Typical organisms of concern
exhibit short life spans, or critical stages in their complex life histories that occur
at time scales substantially shorter than one year. (3.2.8)
1.3.3 Issues of Parameter Estimation
Recommendation 14. The Subcommittee recommends that the Agency
ensure that the uncertainty estimates in the RIA fairly reflect the uncertainties in
quantification of the source term of the model input. The MMRS believes that the
largest single source of uncertainly in the risk analysis is probably that related to
quantification of the source term. Problems include sparse or inaccurate
information on identification of types of wastes present (e.g., presence of NAPLa),
on quantification of waste quantities, and on estimation of waste distribution.
(3.3.1)
Recommendation 15. The MMRS recommends that the Agency consider
the quantity and quality of waste information as a reasonable criterion or
requirement for the inclusion of a particular facility in the facility selection
process. The Subcommittee believes that the expected improvement of the
confidence in the modeling results is obvious. (3.3.1)
Observation 16. The MMRS observes that the uncertainty of the waste
transport calculations may be increased by the fact that the existing data that
have been developed for SWMUs were generally not constructed or collected for
the purpose of estimating risks to humans or to ecosystems, but rather for the
purpose of defining the extent of contamination at a site rather than defining the
exposures at or near the site. (3.3.1)
Recommendation 17. The MMRS recommends that the solubility models
used for metals and organics be submitted to peer review to assess their scientific
basis and limitations. (3.3.2)
Recommendation 18. The MMRS recommends that the input data for the
case studies undergo peer review in order to evaluate a suspected over-reliance on
the use of default parameter values. (3.3.3)
Recommendation 19. The MMRS recommends a documented and thorough
peer review of all aspects of the data base, focusing particularly on those
parameters to which the results are most sensitive. (3.3.4)
-------
Recommendation 20. The MMRS recommends that the Agency build upon
the extensive data base it has accumulated for the Corrective Action RIA, to begin
the development of an extensive data base that could be tapped for other EPA
programmatic efforts, such as for a comparable assessment of the risks associated
with NORM wastes and radiologically contaminated sites. The intraagency
modeling task force, the Ad Hoc Agency Task Force on Environmental Regulatory
Modeling (AFTERM), may be an appropriate vehicle for organizing and
coordinating such an effort in a manner that would be most beneficial to the
potential users. (3.3.5)
1.3.4 Issues of Range of Model Validity
Recommendation 21. The MMRS recommends that the Agency evaluate the
validity of each pathway model to assess the extent to which extreme events might
be expected to contribute to the bulk of contaminant releases, and the extent to
which the model may under- or over-estimate transport. (3.4.1)
Recommendation 22. For facilities in complex hydrogeological settings
outside the range of validity of the MMSOILS model, the MMRS recommends that
alternative approaches to characterization be used. Examples include the
following: modification of the ground water module in MMSOILS to more
accurately model contaminant movement under these conditions; utilization of
assessment data generated for Superfund sites; application of more sophisticated
models with better-defined data to develop estimates for representative sets of
waste sites; application of site-specific models to analyze that relatively small
number of facilities which MMSOILS results indicate dominate the total costs or
risks; and submission of these case studies to expert panel review to develop
estimates of contaminant migration. (3.4.2)
Recommendation 23. The MMRS strongly endorses ORD's recommendation
that the Agency develop an improved screening-level model for non-aqueous phase
liquid (NAIL) transport, either by modification of the existing MMSOILS model or
by conducting independent modeling exercises. (3.4.3)
Recommendation 24. The MMRS recommends that the Agency develop
guidelines - perhaps including a requirement for peer review for key case studies -
in order to assess the applicability of MMSOILS to specific cases. (3.4.4)
1.3.5 Issues Relating to Pathway Model Verification and Validation
Recommendation 25. The MMRS recommends that the Agency prepare a
documented comparison of model predictions of chemical transport to field data
that would strengthen the scientific credibility of the results and provide a basis
for readers to evaluate the model validity and magnitude of uncertainty. For
similar reasons, the MMRS recommends that, for a subset of SWMUs where
ground water plume predictions are made by using MMSOILS, NATIVEPACMS
models also be exercised so as to permit comparison of plume predictions. (3.5.1)
-------
Recommendation 26. The MMRS recommends that documented validation
exercises be undertaken for the remaining environmental transport pathways, e.g.t
aerosolization, volatilization, surface water runoff and bioaccumulation, in order to
assess the ability of these pathways models to provide meaningful input to the
RIA, (3.5.2)
Recommendation 27. The MMRS recommends that the Agency, perhaps
through AFTERM, develop generic guidelines for model calibration, verification
and validation, including criteria for judging whether or not discrepancies among
alternative modeling results or between calculated and measured field data are
significant In the case of MMSOILS, it recommends that the Agency undertake a
root-cause analysis for discrepancies, where significant, in order to evaluate the
potential for systematic bias in the modeling approach. (3.5.3)
1.3.6 Comments on Remediation Effectiveness
Recommendation 28. The MMRS recommends that the sensitivity of the
RIA conclusions to these estimated remediation clean-up times be evaluated. This
recommendation is made from the observation that experience gained from the
Superfund program with respect to remediation effectiveness and time has shown
that time estimates are commonly overly optimistic for ground water extraction
systems. Because of unidentified sources, vadose zone contamination,
heterogeneities, and the unknown presence of NAPLs, remediation has gone on at
a number of sites for periods well in excess of initial estimates. (3.6.1)
Recommendation 29. The MMRS recommends that the Agency discuss the
implications of unknown presence of NAPLs in the Corrective Action RIA. The
MMRS observes that NAPLs are not always recognized during site
characterization, and that this oversight may result in selection of a remediation
system that is not appropriate for NAPLs, resulting in excessive remediation times
and associated costs, and possibly in remediation goals not being achieved. (3.6.2)
Recommendation 30. The Subcommittee recommends that the Agency
evaluate the sensitivity of the RIA analysis to assumptions about remediation
effectiveness. The MMRS believes that, for some cases, especially cases in which
NAPLs are present or those sites located in fine-grained soils and fractured or
karst rock, the assumed extent of remediation effectiveness may be too high.
(3.6.3)
Recommendation 31. The MMRS recommends that a closer review be made
of the derivation and scientific basis of the soil-water partition coefficient (K
value) used in the post-processing of model results to calculate the change in
concentrations at the exposure location. (3.6.3)
Recommendation 32. The MMRS recommends that the suite of remediation
technologies used in the analysis be expanded to include biologically-based
treatment technologies. The Subcommittee observes that a significant advantage
8
-------
of these treatment technologies is that, where applicable, they may provide a more
cost-effective treatment approach than other currently available remedial
technologies. (3.6.4)
Recommendation 33. The MMRS recommends that the risk analysis be
modified to recognize risks that may be incurred through the remediation process.
(3.6.5)
1.3.7 Issues Relating to Assessment of Uncertainty
""Recommendation 34. The MMRS recommends that guidance be provided in
the MMSOILS user m*n"«l concerning why and how the user should obtain
qualitative or quantitative estimates of the uncertainties associated with each
pathway. (3.7.1)
Recommendation 35. The MMRS recommends that any numerical results
emanating from the RIA analysis be presented as a range. The MMRS stresses
that presenting results as "a number" tends to give the reader a false sense of
accuracy which, in this instance, is particularly dangerous given the
incompleteness of the input data set and our incomplete comprehension of the fate
of hazardous constituents in the environment. (3.7.1)
Recommendation 36. The MMRS recommends that the MMSOILS model
and results be subjected to more thorough, formal and comprehensive sensitivity
and uncertainly analyses in order to identify the critical parameters associated
with predictions of contaminant concentrations along various pathways. This
information can then be used to determine what the critical data are for
improving model predictions, and possibly to simplify the model structure without
sacrificing accuracy or precision of model results. (3.7.1)
Recommendation 37. The MMRS recommends that the Agency review its
risk estimation protocol. The MMRS is concerned that the simple protocol
followed to obtain high-end risk estimates may be inadequate, in that these
estimates in some cases apparently give rise to lower exposures than those
generated using the central tendency estimate. (3.7.2)
1.3.8 Comments on Results for Health Risk Analysis
Observation 38. The MMRS observes that since health risks are the
predominant focus of current environmental protection initiatives, the adequacy of
risk estimates has to serve as the ultimate criterion of model relevance and
accuracy. (3.8.1)
Recommendation 39. The MMRS recommends that the Agency consider how
the general validity of its exposure estimates might be tested by comparison with
empirical field data. This is being recommended as a result of the observation by
the MMRS that the translation of contaminant concentrations to estimates of
-------
exposure necessarily involves a long chain of assumptions and requires the
adoption of parameter values of variable uncertainly. (3.8.2)
Recommendation 40. The MMRS recommends that the Agency revise its
practices for assessing cancer and noncancer health risks so as to make them more
-consistent with one another. (3.8.3)
Recommendation 41. The MMRS recommends that the Agency review its
discussion of critical health effects and correct any inaccurate information. (3.8.4)
Recommendation 42. The MMRS recommends that the Agency review its
assumption of additivity of Hazard Indices, and additivity of risks across class A
(known) and Class C (suspected) carcinogens. (3.8.5)
Observation 43. The MMRS observes that certain hazardous agents are not
easily controlled and may pose health risks beyond the substances discussed in the
RIA. (3.8.6)
1.3.9 Comments on Use of MMSOILS in Corrective Action RIA
Recommendation 44. The MMRS recommends that the word "random" be
deleted from any reference to the sample; the fact that various facilities were
eliminated from the analysis for various reasons, some of which are quite valid,
belies the concept of the sample being "random."
Observation 45. The MMRS agrees that MMSOILS may be appropriate to
use as a screening-level model at the national level, but observes that the model is
actually used beyond screening in estimating the fate and transport of
contaminants. The MMRS observes that the acceptability of uncertainties
associated with model predictions must be evaluated in the context of model use.
The model use determines the objective of any validation effort. If the model is
used in a screening mode, then greater uncertainties on the model outputs can be
tolerated in making a coherent decision and validation efforts should focus on how
well the model screens. If the model is to be used in estimating spatial-temporal
values of contaminant concentrations, for example, to feed into a site-specific risk
assessment, then validation requires comparisons with these kinds of data which
are highly likely to need greater accuracy and precision, if the model is to
effectively contribute to these estimations. Care must be taken in not confusing
the two different uses of the model and that such a distinction be made to the
model users. (3.9.2)
Observation 46. The MMRS observes that, as an alternative to attempting
to estimate a national average by aggregating the 38 site-specific applications of
MMSOILS, it might be just as valid to use as much data as possible from the
5,800 sites to construct an "average" national waste site and apply the model to
this single hypothetical site. This approach might be particularly effective given
that the validity of each site-specific simulation is not held to be very accurate.
10
-------
Analyzing the hypothetical site with the model might be more in-line conceptually
with the notion of screening. (3.9.2)
Observation 47. The MMRS commends the Agency for having drafted such
a well-organized and well-written report for such a highly complex issue as that
for the Corrective Action RIA. However, the MMRS observes that the major goal
of the RIA is to provide a quantitative estimate of the cost with incremental
benefit for corrective actions, and it is commonly recognized - and accepted as a
necessary reality - that the MMSOILS model application could derive exposure
estimates no better than "order(s) of magnitude". Although the results may still
be valuable for the purposes of screening, e.g., for assessing relative clean-up costs
or cost versus incremental benefits between various sites, their utility is brought
into question when the results are intended to be used for evaluating remediation
costs, i.e., how meaningful it is when a cost estimate is given with a built-in
uncertainty of one or more orders of magnitude, considering the total cost at the
national level would probably involve hundreds of billions of undiscounted dollars?
(3.9.3)
Recommendation 48. The MMRS recommends that the Agency give
priority to highlighting the uncertainties in the MMSOILS model screening effort
and the propagation and perhaps magnification of that uncertainty in the
subsequent estimates of exposure, risk, costs, and benefits, because of the critical
importance of this aspect of the RIA. The MMRS observes that a major deficiency
with the draft RIA relates to an inadequate representation of the magnitude of the
uncertainties associated with the cost and benefit estimates. The MMRS further
recognizes that communication of the relevance and implications of uncertainty
analysis to decision-makers is a difficult and challenging problem. (3.9.4)
1.3.10 Other User Groups for MMSOILS
Recommendation 49. Because of the potential utility of MMSOILS for
estimating ecological risks in relation to other EPA programmatic efforts, the
MMRS recommends that this modeling construct continue to receive attention,
both in terms of review and in resources, to ensure that it has utility beyond
RCRA. (3.10.1)
Recommendation 50. On a longer-term perspective, the MMRS recommends
that the Agency consider what might be its role in providing guidance to states as
to the appropriate types of models to use for state-level screening calculations.
(3.10.2)
Recommendation 51. The MMRS observes that the model documentation
makes clear that MMSOILS is meant to be used by non-specialists. Consequently,
the MMRS recommends that the manual be revised to contain stronger statements
that emphasize the model limitations to such users, to recommend alternative
models, and to emphasize the inapplicability of the model to site-specific
evaluations. (3.10.3)
11
-------
2. INTRODUCTION
2.1 Charge for SAB Review
In accordance with the charge to the MMSOILS Model Review
Subcommittee (MMRS), [See Appendix A, Reference 1 (memorandum dated March
26, 1993 from Richard Guimond to Donald Barnes regarding Charge for SAB
review of Regulatory Impact Analysis Supporting the Corrective Action
Regulation), as well as Appendix A, Reference 2 (a jointly-signed memorandum
dated June 26, 1992 from Richard Guimond and Peter Preuss requesting a SAB
review of the RCRA Corrective Action RIA), and Appendix A, Reference 3 (Federal
figflgter, Vol. 58 (April 9, 1993), pg. 18395, which states the charge to the SAB)]
the MMRS focussed on the technical aspects of the MMSOILS model. The specific
issues that the Subcommittee was asked to address include:
Issue 1. The adequacy of methods for using a screening level model to
characterize situations where there is a substantial subsurface
heterogeneity or where non-aqueous phase contaminants are present.
Issue 2. Appropriateness of the Agency's approach for aggregating releases
from solid waste management units (SWMUs, the source terms for
the contaminant modeling) to estimate concentration at exposure
points over time.
Issue 3. Adequacy of the Agency's approach for developing long term
effectiveness and failure scenarios for site remedies, and
Issue 4. The implications of the fate and transport modeling assumptions on
the ecological and human risk assessment.
2.2 SAB Review Procedure
The primary review document is the EPA/ORD report, MMSOILS:
Multimedia Contaminant Fate, Transport, and Exposure Model - Documentation
and User's Manual (September 1992 draft; See Appendix A, Reference 4, as well as
Appendix B, Reference 6). The Subcommittee also relied heavily upon the
EPA/OSW Draft Regulatory Impact Analysis for the Final Rulemaking on
Corrective Action for Solid Waste Management Units: Proposed Methodology for
Analysis (including its Appendices) (March 1993 draft; See Appendix A, References
5 and 6, as well as Appendix B, References 7 and 8). These documents were
addressed at a meeting of the MMSOILS Review Subcommittee (MMRS) of the
Environmental Engineering Committee (EEC) in Arlington, VA on April 22-23,
1993, at which time the MMRS was also briefed by Agency staff on the selection,
development and application of the MMSOILS models in the RIA (See Appendix A,
References 7 through 12).
12
-------
A second meeting was held on June 29, 1993 with some of the MMRS and
Environmental Engineering Committee (EEC) members and consultants who
wished to focus of the MMSOILS model and data [See Appendix A, Reference 13
(Federal Register. Vol. 58, No. 108, June 8, 1993, pg. 32122). Also, see Appendix
A, References 13 through 19, for a listing of the briefing materials for the June
29, 1993 meeting]. The specific purpose of this meeting was to further discuss
with the Agency staff the selection of the corrective action sample facilities, data
sets, data acquisition and facility conceptualization process, as well as progress by
the OSW staff on verification and validation of the MMSOILS model.
13
-------
3. COMMENTS ON MODEL SELECTION, FORMULATION,
DOCUMENTATION AND APPLICATION
3.1 MMSOILS Selection, Development, Formulation and Documentation
The Subcommittee would like to note that, while a number of
recommendations are made in this report to improve the MMSOILS model and
overall approach, such an approach lends itself well toward the context of better
understanding and dealing with reducing risk concepts (See Appendix B,
References 9, 10, 14 through 17 and 19). This approach also is a systematic way
of characterizing and assessing ground water oriented and multi-media oriented
approaches for grappling with the admittedly very complex, difficult, demanding
and challenging risk assessment concepts that are being applied in a national
context to deal with the Corrective Action RIA (See, for instance, Appendix B,
References 1 through 5, 10 through 13 and 19 through 22).
3.1.1 Model Selection and Development
The OSW/ORD working group is to be commended for a well-coordinated
and focused effort to develop a regulatory impact analysis (RIA) that will help the
Agency and the Nation better understand the costs and benefits of the proposed
rule for the final corrective action for solid waste management units. The
Subcommittee considers the intraagency coordination represented by this RIA to
be a "model approach" that the Agency would do well to adopt in other programs.
The consensus of the MMSOILS Model Review Subcommittee (MMRS) is
that the use of a multimedia pathway model for screening purposes could be an
appropriate approach for developing risk and cost estimates for a national-level
RIA, as long as the input parameters are sufficiently accurate and the model is not
applied outside its range of validity. The Agency's use of a single model, to the
extent defensible ensures consistency among model results. The rationale for the
selection of MMSOILS for the corrective action RIA was explained during a
briefing to the MMRS by OSW. The MMRS recommends that the criteria and
rationale for the selection of MMSOILS as expressed in that briefing be more fully
documented in the RIA so that the scientific and strategic bases for the selection
will be clear to all concerned - regulator, regulated, and scientific/risk
assessment/economic communities.
The major overriding concerns of the MMRS are: a) the application of
MMSOILS outside its range of validity; b) large uncertainties in input parameters;
c) consequent large uncertainties in MMSOILS results; d) the lack of clear
communication of this uncertainty to decision-makers; and e) the generation of
credible guidance on exposure, risk, costs, and benefits. Consequently, the
recommendations contained in this report are focused at efforts to decrease tbe
level of uncertainty, to validate the MMSOILS results by comparison with
alternative estimation methods, and to ensure that the results of the modeling
exercise are expressed in the RIA in a scientifically defensible manner that
14
-------
communicates the uncertainties of the calculations and their implications for the
cost/benefit analysis.
3.1.2 Use of Standard Formulations
MMSOILS was selected by EPA for use as a screening model for simulation
of contaminant transport from waste management units through multiple
environmental pathways, and for evaluation of potential exposures and associated
risks. The model uses simple, conservative, and computationally efficient
equations for estimating chemical transport via ground water, surface water, soil
erosionj the atmosphere, and foodchains. Mathematical formulae used to estimate
transport rates for each pathway are widely used and accepted by the scientific
community for application to simple situations. Underlying assumptions for each
pathway model have been identified, are clearly stated, are reasonable and are not
overly restrictive. For the mc^L part, the representations used in MMSOILS for
the various exposure pathways are consistent in their level of treatment.
However, for a significant number of sites, the MMRS suspects that no generic
model is likely to provide answers of acceptable quality. OSWER is generally
aware of the limited usefulness of generic models for the analysis of complex
environmental systems, including aquifers.
3.1.3 Documentation of Modeled Pathways
Documentation of the formulations for individual pathway models is well
organized with appropriate references. However, the manual would benefit from
another round of editing; a distracting aspect of the review draft is that several
terms in the equations and figures have not been defined in the text (e.g., Cwl,
Cdwl in equation 3-11, page 3-14 of the Users Manual). The MMRS recommends
that each term be defined in three places in the manual: (a) in the beginning or
end of each chapter in which it appears, (b) at the time each is first used in a
given chapter, and (c) in the compilation of terms in Chapter 12. The terms
should also be reviewed for internal consistency. In at least one case, two
different symbols have been used for the same parameter (Q^ qm). In another
case, the same symbol (DF) has been used for two different parameters.
3.1.4 Documentation of Assumptions Underlying Multimedia Treatment
Basic underlying assumptions, such as the assumption in the ground water
transport model of an "idealized homogeneous, uniform porous media," are
dispersed throughout the manual. Documentation for MMSOILS would benefit
if om a concise presentation of the model's basis, assumptions and limitations in a
central location at the beginning of the user's manual. The presentation should
include descriptions of key aspects of the overall model structure, such as:
modeling time frames for each pathway, whether the model assumes finite or
infinite sources, whether the model is steady-state or dynamic, and how the model
deals with competing mechanisms or pathways.
15
-------
This introductory section should be sufficiently comprehensive that a
knowledgeable reader could use it to quickly assess the level of sophistication of
each pathway component in the MMSOILS model and thus develop a level of
comfort with the predictions provided by the model. Chapter 2.0 of the user
manual, "Applications and Limitations of the Methodology," does not fulfill this
requirement, although it could be revised to do so. The cursory discussion
presented in the section of questions that should be asked may be interesting but
does not inform the user of the assumptions that were made in the model
construction. Assumptions and their resulting model simplifications are crucial to
the evaluation of the model. The brief section on model limitations also fails to
discuss what the true limitations are.
Possible Improvements to Model Formulations for Specific Processes
3.2.1 Additional Types of Solid Waste Management Units (SWMUs)
As currently constructed, MMSOILS is only capable of handling the more
traditional SWMUS such as landfills and surface impoundments. Other,
potentially more costly, types of SWMUs such as leaky sewer systems have been
excluded from the analysis. Although they have been identified as SWMUs, few of
these less traditional SMWUs have been remediated largely because no one really
knows how to deal with them. Their remediation could be quite costly and result
in disruption of industrial operations. Eventually these problematic SWMUs will
be impacted by the proposed corrective action rule and will have to be addressed.
Thus, an estimate of their cost contribution to the implementation of the proposed
rule should be developed.
3.2.2 Recognition of Natural Biodegradation Processes in Ground Water Pathway
Approximately 130 years of simulations are made using the MMSOILS
model to predict the existence, development or dissipation of ground water plumes.
Because of the long time periods involved, it is critical that the role of natural
biodegradation processes be explicitly incorporated into the ground water fate and
transport pathway by the use of an appropriate biodegradation coefficient value.
This function is essential and provides realism for actual mechanisms taking place.
Even a small biodegradation coefficient would have a big impact. For instance, a
value of only 0.0001/yr for the degradation coefficient of organic constituents in
ground water will have a very large effect on the distribution of contaminants
when taken over simulation periods much less than 130 years.
Neglecting the role of biodegradation processes in the transport model could
result in overestimation of exposure concentrations. This omission might not be
critical in screening applications where bias towards overestimation may provide
appropriately conservative results. However, the literature continues to increase in
terms of estimates of biodegradation of many organic contaminants and these data
should be examined for possible use in MMSOILS. (See also comment 3.6.4 on
biologically-based remediation technologies.)
16
-------
3.2.3 Modeling Transport through the Vadose Zone
The finite element numerical model used for the partially saturated zone is
adequate for the job, but it may not necessarily be consistent with the precision of
other model components. This is especially true in light of other assumptions
made in the RIA. For example, only a single layer (or small number of layers
with only roughly estimated properties) was used to represent the unsaturated
zone and the net infiltration rate was taken as piece-wise constant. Given these
assumptions, simple kinematic models are appropriate and consistent with the rest
of the transport process models. Kinematic models for flow and transport will
conserve mass, can address step-wise constant infiltration rates, and will provide
simple algebraic equations for use in the model.
32.4 Plume Aggregation in Groundwater Pathway
The method of aggregation used to obtain the concentration distributions for
application to individual wells does not conserve mass. Rather, it appears that the
resulting apparent mass will always exceed that from the untransformed plumes.
While this approach is conservative, the degree of conservation should be evaluated
through comparison with a number of simulations which do not use the method of
aggregation. In addition, the question of whether movement of the plumes could
result in a decreased concentration for population wells should be evaluated. The
required transformations from cartesian to cylindrical coordinates should not
require much computational effort compared with that required to operate the
model.
32.5 Food-Chain Module
The food-chain module in MMSOILS is very synthetic and unrealistic. Food
chains are highly site-specific and depend upon the gathering of the contaminant
into the receptor environment, the structure of the ecological community, and the
ultimate receptor of interest (humans or eagles or others). It makes considerable
difference in the risk estimate whether the ecological community is terrestrial or
aquatic and to what extent the contaminated food contributes to the total food in
each trophic level. Furthermore, the efficiency of contaminant transfer from one
trophic level to the next varies, and is dependent in part upon the octanol-water
partition coefficient (KQW) and molecular size of specific organic compounds.
The ecological risk assessment component should be constructed using
recommendations structured in the SAB's report of the Ecology and Welfare
Subcommittee of the Relative Risk reduction Strategies Committee of the SAB [See
Appendix B, Reference 10 in particular, as well as Appendix B, References 9, 11
and 12, and the principles for ecological risk assessment as suggested by the Risk
Assessment Forum, Appendix B, Reference 23.]
17
-------
3.2.6 Mass Balance
The effectiveness of the model for estimating long-term consequences of
remediation alternatives appears to be weakened by the breakdown of mass
balance as a result of post-processing of model outputs (See also Section 3.2.4
regarding issues related to the method of aggregation and conservation of mass).
The exponential decay of ground water contaminants as described represents one
particular example of this apparent breakdown. Similar concerns for overall mass
balance in MMSOILS remain. For example, if the combined degradation or
transport along all pathways accounts for more contaminant mass than is
available, the flux along each pathway is merely normalized according to the
"demand." This procedure may provide results of questionable accuracy given the
different time scales applied to contaminant flux along different pathways.
3.2.7 Disparity in Relevant Time and Space Scales for Transport Mechanisms
One major problem that must be confronted in the development of any
multimedia model, such as MMSOILS, is the forcing of differently scaled
environmental transport processes into a single model construct. For example, the
temporal dynamics of volatilization of organics may vary on the order of hours or
less, given changes in microclimatic conditions that drive this process (e.g.,
temperature, wind velocity). Surface water runoff leading to transport of soils and
dissolved contaminants is strongly event-driven, that is, local precipitation patterns
and strong storms can move significant amounts of chemicals in time scales of
hours. These dynamics contrast markedly with the slow movement of
contaminants in ground water, where years to decades may be the relevant time
scale.
Attempts to force the above disparate scales into a single model by selecting
a compromise in tune step will necessarily result in a loss of accuracy in model
predictions. Parallel considerations apply in the spatial domain. Representing the
spatial distribution of ground water plumes of contaminants may allow a
comparatively coarse spatial description of the waste site and surrounding region.
To achieve comparable accuracy in estimating atmospheric transport, a more
detailed spatial representation of the system may be necessary. Again, attempting
to force disparate spatial scales into a single model can produce inaccuracies in
model results. Finally, the combination of time and space scale selections required
in establishing a single model can compound problems outlined above.
3.2.8 Relevant Time Scales for Ecological Risk Assessment
At present the ecologically relevant exposure scenarios are inadequate. The
known major source for acute impacts from waste sites on aquatic environments is
surface run-off after major rain events. The MMSOILS model cannot simulate
this in its present version. On a long-term basis, major effects can be due to
biomagnification, changes in biodiversity, etc.
18
-------
Methods for estimating ecological risk continue to evolve, e.g., the EPA
Framework for Ecological Risk Analysis. Nonetheless, it is quite apparent that
the methods will surpass simple multiplicative models of food chain accumulation
and comparison of exposure concentrations to acute or chronic toxicity
benchmarks. The annual time scale for exposure estimates produced by MMSOILS
seems inappropriate for many ecological applications. Typical organisms of
concern exhibit short life spans, or critical stages in their complex life history that
occur at time scales substantially shorter than one year. Thus, an average annual
concentration is not particularly useful as an input to many ecological risk
assessments. Strong seasonal constraints are typically important, especially for
temperate surface waters, e.g., streams and lakes. Thus, MMSOILS may have to
be modified to produce more realistically-scaled exposures for meaningful inputs to
ecological risk analysis.
3.3 Issues of Parameter Estimation
The MMRS notes that the use of inaccurate input parameters is a suspected
major source of unacceptable errors and unreasonable magnitudes of uncertainties
in MMSOILS results relative to their use in the draft Corrective Action RIA.
Inappropriate input data are a consequence of sparse or inaccurate information,
poor judgement in parameter estimation, and suspected overreliance upon default
parameters. Regardless of the high quality of the model formulation, the quality
of the outputs will retain the deficiencies of the inputs, and may well amplify
them further. Agency personnel are clearly aware of problems in this area. Below
are highlighted some of the parameters of greatest concern.
3.3.1 Source Term Parameters
Most members of the MMRS believe that the largest single source of
uncertainty in the risk analysis was probably that related to quantification of the
source term. Problems include sparse or inaccurate information on identification
of types of wastes present (e.g., presence of NAPLs), on quantification of waste
quantities, and on estimation of waste distribution. Given the time and budget
constraints under which it is operating, it is highly questionable whether the
Agency could significantly improve upon the extensive and thorough job it has
already done in compiling the source-term data. Nonetheless, at a minimum, the
Agency should ensure that the uncertainty estimates in the RIA fairly reflect the
uncertainties in this aspect of the model input.
The MMRS believes that consideration of the quantity and quality of waste
information should be a criterion for the inclusion of a particular facility in the
facility selection process. As explained by the Agency in its presentations, the 79
sites were selected randomly from approximately 5,800 sites nation-wide; for many
of these sites, information on the wastes is often sketchy or non-existent. Because
a computer model cannot provide results that are any more precise or accurate
than the input data used, there may be no issue more important than ensuring
that the model input has the most accurate information possible on waste
19
-------
characteristics and history. An appropriate criterion for facility selection might be
a requirement for "waste information with both reasonable quality and quantity."
Evaluation of the waste information against such a criterion should not take much
additional effort or affect the purpose of the statistical selection process,
particularly when less than 2% of the sample sites are chosen from 5,800 sites.
The expected improvement of the confidence in the results from this study is
obvious.
Another problem that may increase the uncertainty of the waste transport
calculations is that the existing data that have been developed for SWMUs were
generally not collected for the purpose of estimating risks to humans or to
ecosystems. The data were collected to define the extent of contamination at a
site, rather than defining the exposures at or near the site. The data are often
deficient in providing information on environmental properties that influence the
dynamics of releases to air, si^ace water, or ground water. In addition, the
structure (dimensional distribution of materials, physical characteristics) and
processes occurring at a site are only partially understood. These short-comings
significantly affect the utility of input data.
3.3.2 Waste Release and Solubility
Predictions of ground water contamination and future growth in the plume
are directly proportional to the mass of leachate assumed to be released to the
subsurface from a SWMU. The release models used for metals and organics,
while being good starting points, could be improved.
With regard to metals release, the model's use of the maximum observed
concentrations near known source terms might over- or underestimate metal
solubility depending on the environmental context of the waste site, e.g., pH,
redox, aerobic/anaerobic, soil type, organic content of soils and ground water, etc.
Perhaps some of the chemical speciation models e.g., MINEQL, might be examined
to see if they can provide more meaningful estimates of the solubility, complexed,
adsorbed, etc. fractions of metals for use in MMSOILS calculations.
With regard to organics release, it appears that using a multiplier of 100 to
estimate organic leachate concentration is arbitrary at best. The choice of the
100 value incidently corresponds to the generic Dilution Attenuation Factor (DAF)
of 100 used in the Toxicity Characteristic (TC) rule promulgated by the Agency in
1991. The ideal or the most reliable method for choosing leachate concentrations
might involve choosing the most important solid phases for metals and perhaps
Aqueous solubilities based on Raoult's Law for most of the organic compounds. It
is also recommended that only a fraction of the total mass of a chemical in the
SWMUs be allowed/available for leaching instead of the entire 100%. [NOTE: It is
recognized that there should bt ~*gnificant attention paid to what the fraction of
total mass that would be allowed to leach might be.]
20
-------
3.3.3 Use of Default Values
The MMRS suspects that the modelers relied more heavily than warranted
on the use of default values, although this aspect is difficult to judge from the
information provided by the Agency. Clearly, the OSW/ORD working group needs
to address the issue of use of default values, making it very clear when they are
used and why the default value makes sense for the particular application, in the
absence of better data.
3.3.4 Peer Review of Data Base
•«.
The MMRS recommends a documented and thorough peer review of all
aspects of the data base, focusing particularly on those parameters to which the
results are most sensitive. Such a peer review does not need to involve the SAB,
but occasional consultation with the SAB or the interagency modeling task force,
ATFERM, on the approach and issues to be grappled with might be a useful
exercise.
3.3.5 Data Base for Future Related Modeling Efforts
The MMRS recommends that the Agency build upon the extensive data base
it has accumulated for the Corrective Action RIA, to begin the development of an
extensive data base that could be tapped for other EPA programmatic efforts, such
as for a comparable assessment of the risks associated with NORM wastes and
radiologically contaminated sites. The intraagency modeling task force, AFTERM,
may be an appropriate vehicle for organizing and coordinating such an effort in a
manner that would be most beneficial to the potential users.
3.4 Issues of Range of Model Validity
MMSOILS was not designed to estimate contaminant transport and fate for
chemicals in sites with complex hydrology, nor to assess the environmental
behavior of non-aqueous phase contaminants in ground water. However, these
limitations should not necessarily be considered as weaknesses of the model.
Complex sites and Non-Aqueous Phase Liquids (NAPLs) need to be addressed for
the RIA, but remain outside the domain of applicability of the current MMSOILS
construct. The solution lies either in developing separate models to examine these
issues or modifying the current MMSOILS so as to extend its applicability. Given
the current state-of-the-art in our understanding and ability to model either
complex hydrogeology or NAPLs, it may be some time before these aspects can be
realistically introduced into MMSOILS. For example, stochastic modeling of
ground water may contribute toward addressing the complex hydrogeology issue.
The literature on this subject continues to grow; however, the complexity of these
kinds of models may preclude their easy incorporation into a scheme such as
MMSOILS. This is not to undermine the importance of these issues, but merely
to emphasize that we are at the cutting edge of science in the development and
applications of MMSOILS (albeit that MMSOILS deals with simple, conservative
21
-------
and computationally efficient equations for estimating chemical transport) and
progress may be slow. Some additional questions about model validity and some
suggestions for how the Agency might deal with this issue are provided below.
The Subcommittee offers the following comment on use of stochastic models
and Monte Carlo models. Stochastic models are usually for simple hydrogeology,
but with complex parameterization, and it is doubtful that they are or will be
considered useful for a screening analysis. Further, these (the stochastic models)
should be distinguished from Monte Carlo models which generally are simple
deterministic models with random input data. The later are appropriate for
uncertainty analysis with a screening model.]
3.4.1 Extreme Events
How well does the model deal with processes that are event-driven? These .
applications may be outside the range of validity of the model. Two examples are
offered. First, the water balance approach is not expected to work well for sites in
arid regions in which average precipitation and evapotranspiration are
approximately equal. Use of the balance approach would lead one to expect no
recharge. While some infiltration and recharge in arid regions does occur as a
result of extreme rainfall events, extreme rainfall events are typically associated
with flash flooding. Recharge is more likely due to confluence of flows along
arroyos, and can occur with "normal" rainfall events. Such recharge is localized,
and probably should not be considered in the screening model calculations, unless
a facility is placed along the ephemeral stream.]
The model appears to under-estimate waste transport via surface runoff.
Net infiltration is calculated from precipitation less runoff and evapotranspiration.
Runoff is calculated using the curve number method, which is based on daily
rainfall amounts. Precipitation is provided through monthly rainfall amounts.
Daily rainfall amounts are calculated by dividing the monthly rainfall by the
average number of days with rainfall greater than 0.01" per month. The "wet"
days are then distributed uniformly though the month to calculate infiltration and
runoff. This approach increases the likelihood of infiltration and decreases the
potential for runoff. Runoff tends to occur when storms of greater than normal
precipitation occur, or when storms occur on consecutive days. The approach may
be conservative because the ground water pathway appears to dominate the
exposures and risks. However, it is not clear that this was the intent. Actual
daily rainfall data should be used for specific locales and regions where available.
The MMRS recommends that the Agency evaluate the validity of each
pathway model to assess the extent to which extreme events might be expected to
contribute to the bulk of contaminant releases, and the extent to which the model
may under- or over-estimate transport.
22
-------
3.4.2 Complex Sites
The Subcommittee observes that applicability of MMSOILS is limited by its
inability to deal with complex sites. It is likely that critical investigations of the
applicability of MMSOILS will turn up other complex sites beyond karst (e.g., a
limestone region marked by sinks, abrupt ridges, irregular protuberant rocks,
caverns and underground streams) and fractured rock. Sites in glacial till, such as
former gravel pits, may also exhibit more complexity than MMSOILS could
reasonably address.
For complex sites, the MMRS recommends that the Agency pursue
alternative approaches and should develop a strategy that combines MMSOILS
with an appropriate monitoring strategy. The outputs from such an effort can
also be used to modify and improve the MMSOILS model.
3.4.3 NAPLe
The MMRS strongly endorses ORD's recommendation that the Agency
conduct a separate modeling exercise and obtain expert opinion to develop an
improved screening-level modeling of NAPLs. For dense non-aqueous phase
liquids (DNAPLs), a possible modification might be to locate the source term in
the saturated zone. For light non-aqueous phase liquids (LNAPLs), a possible
modification might be to use an alternative volatilization model. Four
volatilization models are available, representing two conceptualizations of the
process. One approach is based on Fick's first and second law, and gives classical
solutions to the diffusion equation. The second approach combines Fick's first law
with a moving boundary model for continuity. This is the "Landfarming Equation"
of Thibodeaux and Hwang. A comparison between these approaches shows that
the moving boundary model predicts fluxes that are smaller than the classical
diffusion model by a factor of about 0.8 (square root of 2 divided by pi).
Given the level of accuracy expected from a screening model, these
approaches provide essentially the same result. By choosing the moving boundary
model, one can represent cases with a finite region of contamination, both with
and without a cover, using simple algebraic equations. Use of an effective
diffusion coefficient in soil along with appropriate partitioning relationships will
also allow one to account for the presence of air, NAPLs, and water within the
pore space, and partitioning of the constituent within the air, water, soil, and
NAIL system.
3.4.4 Development of Guidelines for Assessing Model Applicability to Specific
Cases
The MMRS recommends that the Agency develop guidelines in order to
assess the applicability of MMSOILS to specific cases.
23
-------
3.5 Issues Relating to Pathway Model Calibration, Verification and Validation
3.5.1 Ground Water Model
The ground water flow module has been verified by comparison of results to
those of numerical models. However, documented comparison of model predictions
of chemical transport to field data would strengthen the scientific credibility of the
results and provide a basis for readers to evaluate the model validity and
magnitude of uncertainty.
Another possible means to provide a limited validation of the ground water
pathway component of MMSOILS is to compare its output to that of other peer-
reviewed EPA models. The EPA Office of the Solid Waste (OSW) has developed
and used EPA Composite Model for Landfills (NATD/EPA Composite Model for
Surface Impoundments (EPACMS) model for ground water transport and fate of
contamination for the purpose of regulating RCRA wastes on the national level.
MMSOILS is similar in many respects to the NATL/EPACMS models. Therefore,
the MMRS recommends that, for a subset of SWMUs where ground water plume
predictions are made by using MMSOILS, NATL/EPACMS models also be
exercised so as to permit comparison of plume predictions.
3.5.2 Other Pathway Models
The MMRS strongly recommends that documented validation exercises be
undertaken for the remaining environmental transport pathways, e.g.,
aerosolization, volatilization, surface water runoff, and bioaccumulation, in order to
assess the ability of these pathway models to provide meaningful input to the RIA.
3.5.3 Guidelines for Calibration, Verification and Validation
The MMRS recommends that the Agency, perhaps through AFTERM,
develop generic guidelines for model calibration, verification and validation,
including criteria for judging whether or not discrepancies among alternative
modeling results or between calculated and measured field data are significant. In
the case of MMSOILS, it recommends that the Agency undertake a root-cause
analysis for discrepancies, where significant, in order to evaluate the potential for
systematic bias in the modeling approach.
3.6 Comments on Remediation Effectiveness
3.6.1 Remediation Tunes
As part of the RIA process, experts have estimated the time for clean-up of
contaminated sites. Experience cited in EPA's "19 Sites" (a 1989 publication; See
Appendix B, References 24 and 25) and "24 Sites" (a 1992 publication; See
Appendix B, References 26 and 27) documents has shown that these time
estimates may be overly optimistic for ground water extraction systems. Even less
24
-------
is known about the operation and efficiency of other remedial systems. Because of
unidentified sources, vadose zone contamination, heterogeneities, and the unknown
presence of NAPLs, remediation has gone on at a number of sites for periods well
in excess of initial estimates. In addition, for a number of these sites, the
remediation goal has changed from site clean-up to plume containment. The
impact of underestimating the clean-up time is that the clean-up costs will be
underestimated, and the benefits associated with clean-up will be overestimated.
The sensitivity of the RIA conclusions to these estimated remediation clean-up
times should be evaluated.
3.6.2 Effect of Unknown Presence of DNAPLfi on Remediation Times
MMSOILS and the RIA do not adequately address the presence of NAPLs.
It is well recognized that NAPLs are not always recognized during site
characterization. This may result in selection of a remediation system that is not
appropriate for NAPLs, resulting in excessive remediation times and associated
costs, and possibly in remediation goals not being achieved. NAPLs strongly
influence the source term. Effective solubility and partition coefficients are far
different when NAPLs are present, compared to their absence. The degree and
timing of contamination events are different when NAPLs are mobile, as compared
to cases with only dissolved phase contamination. These issues should be
addressed in the Corrective Action RIA.
3.6.3 Remediation Effectiveness
According to the RIA documentation, most of the risk is associated with
the ground water pathway. To calculate the benefits of corrective action, the
MMSOILS model is run for each site to develop the extent and concentration of
the plume over the time period of interest. The plumes are then aggregated to
obtain the baseline or pre-remediation conditions. For sites requiring clean-up, a
remedial action plan is identified, and the effectiveness of each remedial scheme is
assumed. The extent of remediation is applied to each plume, and the plumes are
aggregated to provide the post-remediation concentration distributions. Exposures
and risks are then calculated for the two conditions to evaluate the benefits of
corrective action.
At least two questions arise with regard to this procedure. The first
concerns the assumed level of remediation achieved with each applied technology.
For some cases, especially NAPLs, the assumed extent of remediation may be too
high. In fine-grained soils and fractured or karst rock, it is doubtful that complete
remediation of LNAPLs is achievable. For DNAPLs the assumed levels of
containment and remediation are also in question. The sensitivity of the RIA
analysis to these assumed levels of remediation should be evaluated.
The remediation effectiveness predictions are, at times, made by using the
MMSOILS model directly. But at other times the Agency has used post-processing
through the use of a decay or a percent removed value. For simplicity reasons,
25
-------
several assumptions regarding failure of caps, liners and barriers etc., have been
used. A simplified decay rate coefficient (K) has been employed to calculate the
change in concentrations at the exposure location. In the case of ground water
pump and removal remedy, it is very unclear as to how the K value is derived or
how realistic this approach is in representing the appropriator of the response. It
is recommended, therefore, that a closer review of the K value for choosing the
quantitative number for sites be made.
3.6.4 Inclusion of Biologically-Based Remediation Technologies
••The suite of remediation technologies used in the analysis should be
expanded to reflect the realities of emerging technologies. The principal defect is
the parsimonious use of biologically-based treatment technologies. Although these
may not at this time be considered as proven technologies, the scientific principles
upon which the technologies are based are sound. It is therefore entirely
reasonable to assume that over the 128-year tune frame spanned by the analysis
that these technologies will become widely used. The magnitude of uncertainty
associated with these technologies is certainly no greater than that associated with
the data base of source terms and transport parameters upon which the EPA has
based its analysis.
A significant advantage of the biologically-based treatment technologies is
that they will likely provide a more cost-effective treatment approach than other
currently available remedial technologies.
3.6.5 Risks of Remediation
The MMRS recommends that the risk analysis be modified to recognize
risks that may be incurred through the remediation process. It is conceivable that
the very act of remediation could incur a higher level of risk than what would be
reduced through remediation. This potential trap should be avoided by estimating
the risks of remediation and including them in the analysis.
3.7 Issues Relating to Assessment of Uncertainty
3.7.1 Uncertainty Estimation Protocol
No guidance is provided to the user as to how one can obtain a qualitative
or quantitative estimate of the uncertainties associated with each pathway. Given
the high stakes involved in terms of potential commitment of national resources,
such an estimate is as important as, or possibly even more important than, the
final result.
The MMRS recommends that any numerical results emanating from the RIA
analysis must be presented as a range. Presenting results as "a number" tends to
give the reader a false sense of accuracy which in this instance is particularly
dangerous given the incomplete level of the input data set and our incomplete
26
-------
comprehension of the fate of hazardous constituents in the environment. Adding
on the -uncertainties associated with risk analysis even further expands the error
bands.
The model should be subject to formal, comprehensive sensitivity and
uncertainty analysis. Some initial efforts have been completed by the Agency,
emphasizing the ground water pathway. Similar analyses should be completed for
the other pathways in the model, and finally for the entire model.
Sensitivity analyses can be used to identify the critical parameters
associated with predictions of contaminant concentrations along various pathways.
This information can be used to determine what the critical data are for improving
model predictions, and to simplify the model structure without sacrificing accuracy
or precision of model results. Both applications of sensitivity results are
important in increasing the capabilities of the model for site-specific use and in
improving its utility in the RIA process.
3.7.2 Development of High-End Risk Estimates
The MMRS is concerned that the simple protocol followed to obtain high-
end risk estimates may be inadequate in that this estimate in some cases
apparently gave rise to lower risks than did the central tendency estimate. The
Subcommittee recommends that the Agency review its protocol.
3.8 Interpretation of Results for Health Risk Analysis
3.8.1 Health Risk as the Assessment Endpoint
The model produces outputs which estimate concentrations in various
environmental media over time. These concentrations in various media are then
applied to various exposures gathered as an initial phase of risk assessment for
the protection of humans and ecosystems. The result of the risk assessments
comparing no-action and remedial action scenarios are then used as major inputs
into a cost-benefit analysis. The use of various assumptions in exposure
assessments as part of the risk assessment process has been highlighted previously
as a major source for uncertainties in risk assessments.
Three documents refer to health risk assessment implications: MMSOILS
(Documentation and User's Manual, the Corrective Action RIA draft, and
appendices for the latter. See Appendix B; References 1 through 3). These
implications are drawn from exposure analyses based on applying the models
described in MMSOILS. Because health risks are the predominant focus of
current environmental protection initiatives, the adequacy of risk estimates has to
serve as the ultimate criterion of model relevance and accuracy.
27
-------
3.8.2 Empirical Validation of Exposure
Section 9.0 of the MMSOILS document (See Appendix B; Reference 6)
provides equations for determining exposure due to inhalation, ingestion, and
dermal contact. (Section E of the RIA document addresses some of these as well.)
The relevant variables come from the list on page 9-4, and includes 21 parameters,
some of which are given default values in tables. These values, however, are
simply guides to translating air, water, and soil concentrations into exposure
estimates. These presumed concentrations are connected to human exposures
through a long chain of assumptions. Because of this lack of direct coupling,
questions arise about how the chain might be shortened. Are there any empirical
data by which the models might be tested? Is there any way in which they might
be acquired? Measures as simple as uptake by plants might be useful, for
example. Or, analyses of animal carcasses at particular sites to which the
modeling has been applied. Is empirical validation, the process by which theories
and models are tested in science, out of the question?
3.8.3 Inconsistent Treatment of Cancer and Noncancer Health Risks
The other aspect of health risk assessment treated in these documents is
described in the draft report on Regulatory Impact Analysis (See Appendix B;
Reference 7) in the chapter on human health benefits (Chapter 7). Here, the
ingenuity of the modeling effort encounters barriers that the Environmental
Health Committee has noted in some of its reports, the most recent of which is
entitled Superfund Site Health Risk Assessment Guidelines. February, 1993 (See
Appendix B; Reference 20). One of these barriers is the EPA practice of adopting
radically different approaches to cancer and noncancer risk assessment. Cancer
risk is given as a probability; systemic endpoints are described by Reference Doses
(RfDs). One example of the confusion this causes appears on page 7-15. Footnote
21 notes that cancer risk is calculated by averaging intake over a lifetime; that is,
dose distribution is ignored. For noncancer risk, intake is averaged over exposure
duration, with an averaging time of 9 years.
These assumptions can be disputed and might even be reversed in some
instances. Perhaps more important, the averaging of assumptions could easily
distort risk. For example, for both categories of risk, exposure during early and
prenatal development might be the crucial values, and exposure peaks the major
source of adverse consequences.
3.8.4 Inaccurate Identification of Critical Health Effects
Some of the other material also arouses suspicion about the adequacy of the
modeling effort. The table in Exhibit 7-19a (See Appendix B; Reference 7) lists
some of the agents driving noncancer effect levels. The critical health effects
noted there are perplexing. For example, the critical health effect for chromium
VI is given as central nervous system effects, which is contrary to the commonly
known effects ascertained by neurotoxicologists; for nickel, it is reduced body and
28
-------
organ weight rather than hypersensitivity reactions; for toluene, a prototypical
neurotoxicant, it is given as liver and kidney pathology. There are several other
peculiar entries.
3.8.5 Questionable Treatment of Different Waste Classes
Another difficult problem is how to handle the kinds of mixtures found at
RCRA sites. The properties of the Hazard Index, offered as a solution, cannot be
applied automatically. For example, if 101 agents are identified, each with a
Hazard Quotient of 0.01, the additivity assumption will yield a Hazard Index
above 1.0. Would such a situation really present a bothersome risk?
Another additivity problem arises with cancer agents. The MMRS disagrees
with the treatment noted on pages 7-56 to 7-57 (See Appendix B; Reference 7), in
which agents with different weight-of-evidence classes are combined by adding
risks, because the MMRS feels that it is misleading to add risks across class A
(known) and class C (suspected) carcinogens. Despite the warning contained
within the RIA that such a procedure may overstate the true cancer risk, the
presentation of such combined results is bound to have an impact.
3.8.6 Other Sources of Hazardous Wastes
Finally, going beyond the contents of the current documents, it would be
useful to point out, not just the intrinsic limitations of this rather ingenious and
extensive modeling effort, but also where it might coincide with the full scope of
EPA's responsibilities. Certain agents are not easily controlled and may pose
health risks beyond the substances discussed in these reports. Methylmercury, for
example, is partially a product of fossil fuel combustion. Inorganic mercury is
discharged when coal and oil are burned, ascends into the atmosphere, travels in a
global mercury cycle, returns to earth in rain, and is transformed to the toxic
methyl form by organisms in the bottom sediment of bodies of water.
Methylmercury travels up the food chain in a continuous cycle of bioconcentration
to lodge in fish that then reach human consumers. In keeping with the spirit of
Reducing Risk (See Appendix B; References 9 through 12, and particularly
Reference 10), such scenarios should be included in efforts such as the current
reports.
3.9 Comments on Use of MMSOHS in Corrective Action RIA
3.9.1 Faculty Selection Process
The facility sample used for the analysis is incorrectly characterized as
being a "stratified, random sample." The fact that various facilities were
eliminated from the analysis for various reasons, some of which are quite valid,
belies the concept of the sample being "random." The MMRS suggests that the
word "random" be deleted from any reference to the sample.
29
-------
3.9.2 Use for National-Level Screening
The model is emphasized as a screening tool. However, it is also clear that
the model is used beyond screening in estimating the fate and transport of
contaminants. If the model is used in a screening mode, then validation efforts
should focus on how well the model screens. If the model is to be used in
estimating spatial-temporal values of contaminant concentrations, then validation
requires comparisons with these kinds of data. Care must be taken in not
confusing the two different uses of the model. Similarly, the uncertainties
associated with model predictions must be evaluated in the context of model use.
If screening is the emphasis, then perhaps greater uncertainties on the model
outputs can be tolerated in making a coherent decision. If detailed
characterizations of contaminant concentrations are needed, for example, to feed
into a risk assessment, then it is likely that greater accuracy and precision will be
required if the model is to effectively contribute to these estimations. The SAB has
given modeling-related advice in a number of instances to the Agency, and the
Subcommittee refers the staff to this (See Appendix B; References 11, 12, 13, 18,
19, 21, and 22).
Instead of attempting to estimate a national average by aggregating the 38
site-specific applications of MMSOILS, it might be just as valid to use as much
data from the 5,800 sites to construct an "average" national waste site and apply
the model to this single hypothetical site. This may be particularly effective given
that the validity of each site-specific simulation is not held to be very accurate.
Analyzing the hypothetical site with the model in this fashion might be more in
line conceptually with the notion of screening. The Subcommittee wishes to
remind and caution the Agency that a screening model can estimate the spatial-
temporal values of concentrations; however, it is limited in the scenarios it can
consider. Further, it (the screening model) is not suitable for site-specific
applications where site details should be included in the model.
Subject to the reservations stated above, the MMRS agrees that MMSOILS
may be an appropriate to use as a screening-level model at the national level.
3.9.3 Presentation of Results in Corrective Action RIA
The Agency is to be commended for having drafted such a well-organized
and well-written report for such a highly complex issue as that for the Corrective
Action RIA.
The MMSOILS model adopted for the study, as clearly stated by the
Agency, was intended to be used as a screening tool. The assumptions made and
approaches followed in the development of this model make the use of it
appropriate for the screening exercise, but the current data base needs significant
improvement. However, the major goal of the RIA is to estimate the cost with
incremental benefit for corrective actions. Therefore a quantitative result is
required from the study. It is commonly recognized - and accepted as a reality -
30
-------
that the MMSOILS model application could derive exposure estimates no better
than "order(s) of magnitude". Although the results may still be valuable for the
purposes of screening, e.g., for assessing relative clean-up costs or cost versus
incremental benefits between various sites, their utility is brought into question
when the results are intended to be used for evaluating remediation costs, i.e., how
meaningful it is when a cost estimate is given with a built-in uncertainty of one or
more orders of magnitude, considering the total cost at the national level would
probably involve hundreds of billions of discounted dollars. In other words, how
can results at this level of confidence be used in the policy/regulation decision-
making process?
3.9.4 Presentation of Uncertainty Analysis in RIA
The major concern of the MMRS with respect to the contents of the RIA
relate to an inadequate representation of the magnitude of the uncertainties
associated with the cost and benefit estimates. For the RIA, how much
uncertainty can you live with and still make an intelligent decision regarding the
efficacy of RCRA clean-up. This issue again pertains to the use of MMSOILS as a
screening model, versus a realistic process model for estimating exposure and fate
concentrations.
3.10 Other User Groups for MMSOILS
3.10.1 Applicability to Other EPA Program Activities
The utility of MMSOILS for estimating ecological risks may loom in future
importance, for example hi relation to CERCLA. Therefore, the MMRS
recommends that this modeling construct should continue to receive attention,
both in terms of review and in resources, to ensure that it has utility beyond
RCRA.
3.10.2 Use for State-Level Screening
On a longer-term perspective, the MMRS recommends that the Agency
consider what might be its role in providing guidance to states as to the
appropriate types of models to use for state-level screening calculations.
3.10.3 Other User Groups
The documentation makes it clear that MMSOILS is meant to be used by
nc ^-specialists. The manual needs stronger statements to emphasize the model
limitations to such users, to recommend alternative models, and to emphasize the
inapplicability of the model to site-specific evaluations. With regard to model
limitations, the MMRS recommer J that each pathway model include a summary
table listing assumptions, and clearly stating the known parameter sensitivity
alongside site characteristics which might invalidate model results or which would
be expected to lead to results with order-of-magnitude uncertainties.
31
-------
APPENDIX A - BRIEFING AND REVIEW MATERIALS
Review Materials for the April 22. 1993 SAB/EEC/MMRS Review Meeting:
1) U.S. EPA/OSWER, A memo entitled "Charge for SAB Review of Regulatory
Impact Analysis Supporting the Corrective Action Regulation," signed by
Richard J. Guimond, Assistant Surgeon General, U.S. Public Health Service
and Deputy Assistant Administrator, Office of Solid Waste and Emergency
Response to Dr. Donald G. Barnes, Director, Science Advisory Board, March
26, 1993
2) U.S. EPA/OSWER & ORD, A jointly signed memo entitled "Request for SAB
Review of RCRA Corrective Action RIA," from Peter W. Preuss, Director,
Office of Technology Transfer and Regulatory Support, and Richard J.
Guimond, Assistant Surgeon General, U R Public Health Service and Deputy
Assistant Administrator, Office of Solid Waste and Emergency Response to
Dr. Donald G. Barnes, Director, Science Advisory Board, June 26, 1992
3) U.S. EPA/SAB, Environmental Engineering Committee, MMSOILS Model
Review Subcommittee, Notice of Subcommittee Open Meeting, Federal
Register. Vol. 58, No. 67, Friday, April 9, 1993, p. 18395
4) U.S. EPA, "MMSOILS: Multimedia Contaminant Fate, Transport, and
Exposure Model: Documentation and User's Manual," Office of Research and
Development [Prepared by the Exposure and Assessment Group, Office of
Health and Environmental Assessment and the Office of Environmental
Processes and Effects Research], Washington, D.C. 20460, EPA XXX/X-
XX/XXX draft document dated September 1992
5) U.S. EPA, "Draft Regulatory Impact Analysis for the Final Rulemaking on
Corrective Action for Solid Waste Management Units: Proposed Methodology
for Analysis," Office of Solid Waste, March 1993
6) U.S. EPA, "Draft Regulatory Impact Analysis for the Final Rulemaking on
Corrective Action for Solid Waste Management Units: Proposed Methodology
for Analysis," APPENDICES, Office of Solid Waste, March 1993
Briefing Materials from the April
7) U.S. EPA/OSW briefing entitled "Overview: Corrective Action Regulatory
Impact Analysis, Proposed Methodology," Presented to the MMSOILS Model
Review Subcommittee (MMRS) of the Environmental Engineering
Committee (EEC), Science Advisory Board (SAB), by the Office of Solid
Waste (OSW), April 22, 1993
A-l
-------
APPENDIX A - BRIEFING AND REVIEW MATERIALS: CONTINUED
Briefing Materials from the April 22. 1993 SAB/EEC/MMRS Review Meeting:
Continued:
8) U.S. EPA/OSW, briefing entitled "Application of the MMSOILS Model in the
Corrective Action Regulatory Impact Analysis," Presented to the MMRS of
the SAB's EEC by the Office of Solid Waste (OSW), April 22, 1993
9) U.S. EPA/OSW briefing entitled "Risk Assessment in the Corrective Action
"Regulatory Impact Analysis," Presented to the MMRS of the SAB's EEC by
the Office of Solid Waste (OSW), April 22, 1993
10) U.S. EPA/OSW briefing entitled "Simulation of Remedy Effectiveness in the
Corrective Action Regulatory Impact Analysis," Presented by the Office of
Solid Waste (OSW) to the MMRS of the SAB's EEC, April 22, 1993
11) U.S. EPA/ORD briefing entitled "RCRA Corrective Action RIA: ORD Input
on Significant Technical Issues," Prepared and Presented by Stephen G.
Schmelling of the Robert S. Kerr Environmental Research Laboratory to the
MMRS of the SAB's EEC, April 22, 1993
12) U.S. EPA/ORD briefing entitled "RCRA Corrective Action RIA: ORD
Participation (Fate & Transport), Presented to the MMRS of the SAB's
EEC, April 22, 1993
Briefing Materials from the June 29. 1993 SAB/EEC/MMRS Review Meeting:
13) U.S. EPA/SAB, Environmental Engineering Committee (EEC), MMSOILS
Model Review Subcommittee, Notice of Subcommittee Open Meeting for
June 29, 1993 on MMSOILS Review, as well as EEC Open Meeting of June
30 - July 1, 1993, Federal Register. Vol. 58, No. 108, Tuesday, June 8, 1993,
p.32122
14) U.S. EPA/OSW briefing entitled "Model Selection for the Corrective Action
Regulatory Impact Analysis," Presented to the MMRS and selected
specialists of the SAB's EEC by the Office of Solid Waste (OSW), June 29,
1993
15) U.S. EPA/OSW briefing entitled "Sample Selection for the Corrective Action
Regulatory Impact Analysis," Presented to the MMRS and selected
specialists of the SAB's EEC by the Office of Solid Waste (OSW), June 29,
1993
A-2
-------
APPENDIX A - BRIEFING AND REVIEW MATERIALS: CONTINUED
Briefing Materials from the June 29. 1993 SAB/EEC/MMRS Review Meeting:
Continued:
16) U.S. EPA/OSW, an untitled summary sheet, which lists SAB
Recommendations and the OSW/ORD Next Steps, one page, both sides, June
29, 1993
17) U.S. EPA/OSW briefing entitled "Facility Characterization in the Corrective
Action Regulatory Impact Analysis," Presented to the MMRS and selected
specialists of the SAB's EEC by the Office of Solid Waste (OSW), June 29,
1993
18) U.S. EPA/ORD briefing entitled "SAB Presentation on Corrective Action
RIA: MMSOILS Model Validation and RIA Uncertainty Assessment," a
presentation by Mr. Gerry Laniak of ORD-Athens, June 29, 1993
19) U.S. EPA/ORD untitled briefing comparing site-specific parameters by
modeling teams, using central tendency and high-end estimates, and plots
comparing the results of monitoring and modeling data by concentration
(ppm), as well as plots on modeling and monitoring minimum curve and
maximum error, a presentation by Mr. Gerry Laniak of ORD-Athens, June
29, 1993
A-3
-------
APPENDIX B - REFERENCES CITED
1) Kecly, J.F., 'The Use of Models in Managing Ground-Water Protection
Programs," EPA/600/8-87/003, ORD Ada, OK, 1987, 72p.
2) Kezsbom, A. and A.V. Goldman, 'The boundaries of groundwater modeling
under the law: Standards for excluding speculative expert testimony," Tort
& Insurance Law Journal. Vol. XXVII, No. 1, 1991, pp. 109-126
3) Konikow, Leonard F. and John D. Bredehoeft, "Ground-Water Models
Cannot be validated," Advances in Water Resources 15, 1992, pp. 75-83
4) National Research Council, "A review of ground water modeling needs for
the U.S. Army," Washington, D.C., 1992
5) National Research Council, Ground Water Models; Scientific and Regulatory
Applications, National Academy Press, Washington, D.C., 1990, 303 pp.
6) U.S. EPA, "MMSOILS: Multimedia Contaminant Fate, Transport, and
Exposure Model: Documentation and User's Manual," Office of Research and
Development [Prepared by the Exposure and Assessment Group, Office of
Health and Environmental Assessment and the Office of Environmental
Processes and Effects Research], Washington, D.C. 20460, EPA XXX/X-
XX/XXX draft document dated September 1992
7) U.S. EPA, "Draft Regulatory Impact Analysis for the Final Rulemaking on
Corrective Action for Solid Waste Management Units: Proposed Methodology
for Analysis," Office of Solid Waste, March 1993
8) U.S. EPA, "Draft Regulatory Impact Analysis for the Final Rulemaking on
Corrective Action for Solid Waste Management Units: Proposed Methodology
for Analysis," APPENDICES, Office of Solid Waste, March 1993
9) U.S. EPA, Expert Panel on the Role of Science at EPA (Loehr, Goldstein,
Nerode and Risser), "Safeguarding the Future: Credible Science, Credible
Decisions," EPA/600/9-91-050, March 1992
10) U. S. EPA, Memorandum entitled, "EPA Definition of 'Pollution
Prevention,"1 from F. Henry Habicht II, Deputy Administrator, to all EPA
Personnel, May 28, 1992
B-l
-------
APPENDIX B - REFERENCES CITED: CONTINUED:
11) U.S. EPA/SAB, "Review of the EPA Office of Solid Waste's (OSW)
Unsaturated Zone Code for the OSW Fate and Transport Model (FECTUZ),"
Report of the Unsaturated Zone Code Subcommittee of the Environmental
Engineering Committee (EEC), Science Advisory Board (SAB), EPA-SAB-
EEC-88-030, July 12, 1988
12) U.S. EPA/SAB, "Resolution on Use of Mathematical Models by EPA for
Regulatory Assessment and Decision-Making," Report of the Modeling
-Resolution Subcommittee of the Environmental Engineering Committee
(EEC), Science Advisory Board (SAB), EPA-SAB-EEC-89-012, January 13,
1989
13) U.S. EPA/SAB, "Review of the CANSAZ Flow and Transport Model for Use
in EPACMS," Report of the Saturated Zone Model Subcommittee of the
Environmental Engineering Committee (EEC), Science Advisory Board
(SAB), EPA-SAB-EEC-90-009, March 27, 1990
14) U.S. EPA/SAB, "Reducing Risk: Setting Priorities and Strategies for
Environmental Protection," The report of the Science Advisory Board (SAB),
Relative Risk Reduction Strategies Committee, EPA-SAB-EC-90-021,
September 1990
15) U.S. EPA/SAB, "Relative Risk Reduction Project: Reducing Risk, Appendix
A," The Report of the Ecology and Welfare Subcommittee of the Relative
Risk Reduction Strategies Committee (RRRSC) of the Science Advisory
Board (SAB), EPA-SAB-EC-90-021A, September 1990
16) U.S. EPA/SAB, "Relative Risk Reduction Project: Reducing Risk, Appendix
B," Report of the Human Health Subcommittee of the Relative Risk
Reduction Strategies Committee (RRRSC) of the Science Advisory Board
(SAB), EPA-SAB-EC-90-021B, September 1990
17) U.S. EPA/SAB, "Relative Risk Reduction Project: Reducing Risk, Appendix
C," Report of the Strategic Options Subcommittee of the Relative Risk
Reduction Strategies Committee (RRRSC) of the Science Advisory Board
(SAB), EPA-SAB-EC-90-021C, September 1990
18) U.S. EPA/SAB, "Review of OSWER's Draft Report on Usage of Computer
Models in the Hazardous Waste/Superfund Programs and Proposed Pilot
Study," Report of the Modeling Project Subcommittee of the Environmental
Engineering Committee (EEC), Science Advisory Board (SAB), EPA-SAB-
EEC-91-016, September 6, 1991
B-2
-------
APPENDIX B - REFERENCES CITED: CONTINUED
19) U.S. EPA/SAB, "Leachability Phenomena: Recommendations and Rationale
for Analysis of Contaminant Release," A Self-Initiated SAB Report on
teachability Phenomena by the Environmental Engineering Committee
(EEC), EPA-SAB-EEC-92-003, October 29, 1991
20) U.S. EPA/SAB, "Superfund Site Health Risk Assessment Guidelines: Review
of the Office of Solid Waste and Emergency Response's (OSWER) Draft
Risk Assessment Guidance for Superfund Human Health Evaluation
Manual," A report of the Environmental Health Committee (EHC) of the
Science Advisory Board (SAB), EPA-SAB-EHC-93-007, February 22, 1993
21) U.S. EPA/SAB, "Review of the Office of Solid Waste and Emergency
Response (OSWER) Assessment Framework for Ground-Water Model
Applications," A report of the Modeling Project Subcommittee of the
Environmental Engineering Committee (EEC), Science Advisory Board
(SAB), EPA-SAB-EEC-93-013, June 21, 1993
22) U.S. EPA/SAB, "Review of Draft Agency Guidance for Conducting External
Peer Review of Environmental Regulatory Modeling," A letter report of the
Modeling Peer Review Subcommittee of the Environmental Engineering
Committee (EEC), Science Advisory Board (SAB), EPA-SAB-EEC-LTR-93-
013, August 12, 1993
23) U.S. EPA, Risk Assessment Forum, "Framework for Ecological Risk
Assessment, EPA/630/R-92/001, February 1992
24) U.S. EPA, "Evaluation of Ground-Water Extraction Remedies," Office of
Emergency and Remedial Response (OERR), EPA/540/2-89/054, Volume 1:
Summary Report, 1989
25) U.S. EPA, "Evaluation of Ground-Water Extraction Remedies," Office of
Emergency and Remedial Response (OERR), EPA/540/2-89/054b, Volume 2:
Case Studies 1 -19, 1989
26) U.S. EPA, "Evaluation of Ground-Water Extraction Remedies: Phase II,
Office of Emergency and Remedial Response (OERR), Publication 9355.4-05,
Volume 1: Summary Report, 1992
27) U.S. EPA, "Evaluation of Ground-Water Extraction Remedies: Phase II,
Office of Emergency and Remedial Response (OERR), Publication 9355.4-
05A, Volume 2: Case Studies and Updates (on 24 Sites), 1992
B-3
-------
APPENDIX C - GLOSSARY OF TERMS AND ACRONYMS
ATFERM (Ad Hoc) Agency T_ask Force on Environmental fiegulatory Modeling
CANSAZ Combined Analytical-numerical Saturated Zone (Flow and transport
model for use in EPACMS)
Class A Known Human Carcinogen
Class C Suspect Human Carcinogen
Cwl Concentration of Chemical in Waste Layer
(milligrams/kilogram) (For instance, see Section 3.1.3)
GJ.i See Kdw|, below
Cv Contingent Valuation Methodology
DAF Dilution and Attenuation factor (For instance, see Section 3.3.2)
DF Fraction of Day During Which Exposure Occurs (hours/24 hours) (For
instance, see Section 3.1.3)
DNAPLs Dense non-Aqueous P_hase Liquids
EAG Exposure Assessment Group (U.S. EPA/ORD/OHEA)
EEAC Environmental Economics Advisory Committee (SAB/EEAC)
EEC Environmental Engineering Committee (SAB/EEC, also referred to as
'The Committee")
EHC Environmental Health Committee (SAB/EHC)
EPA U.S. Environmental Protection Agency (U.S. EPA, or "The Agency")
NATL EPA Composite Model for Landfills
EPACMS EPA Composite Model for Surface Impoundments
EPEC Ecological Processes and Effects Committee (SAB/EPEC)
FECTUZ finite-Element Code for Simulating Water Flow and Solute Transport
in the Unsaturated Zone (Variably saturated porous media)
K First order Coefficient, Which Measures Losses of Contaminant Due
to Pumping (Also referred to as Simplified Decay Rate Coefficient)
(For instance, see Section 3.6.3)
Decay Rate/Distribution Coefficient. Also Referred to as Soil-Water
Partition Coefficient (milliliters/gram)
Solid-Water Partition Coefficient Between the Solid Waste and the
Liquid Leachate (liters/kilogram). (Also referenced as Cdwl ) (For
instance, see Section 3.1.3)
Water-Phase Mass Transfer Coefficient. (Also referred to as the
Octanol-Water Partition Coefficient.) (For instance, see Section 3.2.5)
LNAPLs Light non-Aqueous Phase Liquids
MMSOILS Mathematical Model for Soils (A Multi-Media Contaminant, Fate,
Transport and Exposure Model.)
MINEQL A Chemical Speciation Model (For instance, see Section 3.3.2)
MMRS MMSOILS Model Review Subcommittee (U.S. EPA/SAB/EEC; Also
referred to as "the Subcommittee")
NAPLs non-Aqueous P_hase Liquids
NORM naturally-Occurring Radioactive Material
OEPER Office of Environmental Processes and Effects gesearch (U.S.
EPA/ORD)
K
•dwl
C-l
-------
APPENDIX C - GLOSSARY OF TERMS AND ACRONYMS: CONTINUED
OHEA Qffice of Health and Environmental Assessment (U.S. EPA/ORD)
ORD Qffice of Research and Development (U.S. EPA)
OSW Qffice of Solid Waste (U.S. EPA)
OSWER Qffice of Solid Waste and Emergency Response (U.S. EPA)
pH Negative Log of Hydrogen Ion Concentration
Qm Monthly Net recharge (cubic meters/month) (Also Referenced as qm)
(For instance, see Section 3.1.3)
qm Monthly Net Recharge (cubic meters/month) (Also Referenced as Qm)
(For instance, see Section 3.1.3)
RCRA Resource Conservation and Recovery Act
RfDs Reference Boseg
RIA Regulatory Impact Analysis
SAB Science Advisory Board (U.S. EPA)
SWMUs Solid Wjaste Management Units EPA-SAB-EC-LTR-94-OQ2
TC loxicity Characteristic Overview of SAB comments on
U.S. United States the proposed RIA for RCRA
corrective action rule
^ 1 r
-002
- r
- - - - 1"
t _ _.„ —-
ENVIRONMENTAL
PROTECTION
AGENCY
DALLAS, TEXAS
LIBRARY
C-2
DEMCO
-------
DISTRIBUTION LIST
Deputy Administrator
Assistant Administrators
EPA Regional Administrators
EPA Laboratory Directors
Deputy Assistant Administrator for Office of Research and Development (ORD)
Director, Center for Environmental Research Information (CERI)
Director, Office of Environmental Processes and Effects Research (OEPER)
Director, Office of Health and Environmental Assessment (OHEA)
Director, Exposure and Assessment Group (EAG)
Deputy Assistant Administrator for Office of Solid Waste and Emergency Response
(OSWER)
Director, Office of Emergency and Remedial response (OERR)
Deputy Director, OERR
Director, Office of Solid Waste (OSW)
Deputy Director, OSW
EPA ATFERM Co-Chairs (Agency Task Force on Environmental Regulatory
Modeling)
EPA Headquarters Library
EPA Regional Libraries
EPA Laboratory Libraries
------- |