United States
Environmental
Protection Agency
EPA Science Advisory
Board (1400F)
Washington, DC
EPA-SAB-05-003
November 2004
www.epa.gov/sab
EPA's Multimedia,
Multipathway, and
Multireceptor Risk
Assessment (3MRA)
Modeling System
A Review by the 3MRA Review Panel of the
EPA Science Advisoiy Board
-------
m *
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SHENCE ADVISORY BOARD
October 22,2004
The Honorable Michael O. Leavitt
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Re: SAB Review of the Multimedia, Multipathway, and Multireceptor Risk
Assessment (3MRA) Modeling System
Dear Administrator Leavitt:
A panel of the EPA Science Advisory Board (SAB) has reviewed the Multimedia,
Multipathway, and Multireceptor Risk Assessment (3MRA) modeling system. The
3MRA system is intended to be used by the Office of Solid Waste in evaluating wastes
for exemption from Subtitle C of the Resource Conservation and Recovery Act (RCRA).
The panel review of the 3MRA system finds in particular that:
The 3MRA modeling system is a major step forward in providing a flexible and
consistent tool for estimating the distributions of the probability of exceeding
adverse effect benchmarks that result from various choices of exit thresholds.
Used in conjunction with other factors, 3MRA provides a scientifically defensible
framework that gives reproducible results for determining national exit levels for
RCRA-listed hazardous wastes.
The manner in which 3MRA was developed, as a genuine cross-Agency effort
forming a formal partnership between the Office of Solid Waste and the Office of
Research and Development, is to be commended. It is clear that the developers of
3MRA were acutely aware of the need to address criticisms of previous modeling
attempts.
To maintain the value, utility, and credibility of 3MRA, the Agency should
support the continued development of the 3MRA modeling system; our comments
contained in the panel's final report offer a number of specific recommendations.
In order to maximize the long term utility and vitality of the model, the panel
recommends that the Agency articulate a plan for updating both the databases that
-------
-------
t
support the model, as well as the individual model components, as improved
information and models are developed.
The panel wishes to commend the EPA scientists, and especially Mr. Barnes
Johnson, Deputy Director, Office of Radiation and Indoor Air, and 3MRA team leader,
for their extensive and invaluable support for this review.
We look forward to your consideration of and response to the enclosed report.
Sincerely,
Dr. William Glaze, Chair
EPA Science Advisory Board
Dr. Thomas L. Theis
Chair, SAB Review Panel
-------
t
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory
Board, a public advisory group providing extramural scientific information and advice to
the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters related to
problems facing the Agency. This report has not been reviewed for approval by the
Agency and, hence, the contents of this report do not necessarily represent the views and
policies of the Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government, nor does mention of trade names or commercial
products constitute a recommendation for use. Reports of the SAB are posted on the
EPA website at: http://www.epa.gov/sab.
-------
U.S. Environmental Protection Agency
Science Advisory Board
Multimedia, Multipathway and Multireceptor Risk Assessment (3MRA) Modeling
System Review Panel
CHAIR
Dr. Thomas Theis, Director, Institute for Environmental Science and Policy, University
of Illinois at Chicago, Chicago, IL
Member: SAB Environmental Engineering Committee
MEMBERS
Ms. Andrea Boissevain, Principal Scientist, Health Risk Consultants, Inc., Fairfield, CT
Dr. Linfield Brown, Professor, Department of Civil and Environmental Engineering,
113 Anderson Hall, Tufts University, Medford, MA
Dr. John Carbone, Senior Scientist, Environmental Toxicology and Environmental Bisk
Assessment, Toxicology Department, Rohm and Haas Company, Spring House, PA
Dr. James Carlisle, Senior Toxicologist, Office of Environmental Health Hazard
Assessment, California Environmental Protection Agency, Sacramento, CA
Dr. Peter deFur, President, Environmental Stewardship Concepts, Richmond, VA
Dr. Joseph DePinto, Sr. Scientist, Limno-Tech, Inc., Ann Arbor, MI
Dr. Alan Eschenroeder, Harvard School of Public Health, Lincoln, MA
Dr. Jeffery Foran, President & CEO, Citizens for a Better Environment, Milwaukee, WI
Dr. Randy Maddalena, Scientist, Environmental Energy Technologies Division, Indoor
Environment Department, Lawrence Berkeley National Laboratory, Berkeley, CA
Member: SAB Integrated Human Exposure Committee
Mr. David Merrill, Principal, Gradient Corp., Cambridge, MA
Dr. Ishwar Murarka, Chief Scientist and President, ISH Inc., Sunnyvale, CA
Dr. Doug Smith, Principal Scientist, ENSR International, Westford, MA
Dr. William Stubblefield, Toxicologist, Parametrix, Corvallis, OR
Dr. Louis J. Thibodeaux, Jesse Coates Professor, Gordon A. & Mary Cain Department
of Chemical Engineering, College of Engineering, Louisiana State University, Baton
Rouge, LA
-------
t
Dr. Curtis Travis, Scientist, Quest Technologies, Knoxville, TN
SCIENCE ADVISORY BOARD STAFF
Ms. Kathleen White, Designated Federal Officer, Washington, DC
in
-------
U.S. Environmental Protection Agency
Science Advisory Board
CHAIR
Dr. William H. Glaze, Oregon Health & Science University, Beaverton, OR
VICE CHAIR
Dr. Domenico Grasso, Smith College, Northampton, MA
MEMBERS
Dr. Gregory Biddinger, Exxon Mobil Refining and Supply Company, Fairfax, VA
Dr. James Bus, The Dow Chemical Company, Midland, MI
Dr. Trudy Ann Cameron, University of Oregon, Eugene, OR
Also Member: COUNCIL
Dr. Deborah Cory-Slechta, Rutgers University, Piscataway, NJ
Dr. Maureen L. Cropper, The World Bank, Washington, DC
Dr. Kenneth Cummins, Humboldt State University, Arcata, CA
Dr. Virginia Dale, Oak Ridge National Laboratory, Oak Ridge, TN
Dr. Baruch Fischhoff, Ciimegie Mellon University, Pittsburgh, PA
Dr. A. Myrick Freeman, Bowdoin College, Brunswick, ME
Dr. James Galloway, University of Virginia, Charlottesville, VA
Dr. Linda Greer, Natural Resources Defense Council, Washington, DC
Dr. Philip Hopke, Clarkson University, Potsdam, NY
Also Member: CASAC
Dr. James H. Johnson, Howard University, Washington, DC
Dr. Meryl Karol, University of Pittsburgh, Pittsburgh, PA
Dr. Roger E. Kasperson, Stockholm Environment Institute, Stockholm,
Dr. Catherine Kling, Iowa State University, Ames, IA
Dr. George Lambert, Robert Wood Johnson Medical School/ University of Medicine
and Dentistry of New Jersey, Piscataway, NJ
IV
-------
Dr. Jill Lipoti, New Jersey Department of Environmental Protection, Trenton, NJ
Dr. Genevieve Matanoski, Johns Hopkins University, Baltimore, MD
Dr. Michael J. McFarland, Utah State University, River Heights, UT
Dr. M. Granger Morgan, Carnegie Mellon University, Pittsburgh, PA
Dr. Rebecca Parkin, The George Washington University, Washington, DC
Dr. David Rejeski, Woodrow Wilson International Center for Scholars, Washington, DC
Dr. Kristin Shrader-Frechette, University of Notre Dame, Notre Dame, IN
Dr. William H. Smith, Yale University, Center Harbor, NH
Dr. Deborah Swackhamer, University of Minnesota, Minneapolis, MN
Dr. Thomas Theis, University of Illinois at Chicago, Chicago, IL
Dr. Valerie Thomas, Princeton University, Princeton, NJ
Dr. R. Rhodes Trussell, Trussell Technologies, Inc., Pasadena, CA
Dr. Robert Twiss, University of California-Berkeley, Ross, CA
Dr. Lauren Zeise, California Environmental Protection Agency, Oakland, CA
SCIENCE ADVISORY BOARD STAFF
Mr. Thomas Miller, Designated Federal Officer, Washington, DC
-------
Table of Contents
Multireceptor Risk Assessment (3MRA) Modeling System Review Panel ii
Science Advisory Board iv
1.0 Executive Summary 1
2.0 Background and Charge Questions 4
2.1 Background 4
2.1.1 History of the 3MRA from HWIR to Development
of trie Integrated Research Plan 5
2.1.2 The 3MRA Modeling System 6
2.1.3 Peer Review of Modules within the 3MRS
Modeling System 6
2.2 Context 6
2.3 Charge 7
2.3.1 Assessment Methodology 7
2.3.2 3MRA Modeling System 8
2.3.3 Modeling System Evaluation 9
2.3.4 3MRA Modeling System Documentation 10
2.4 Procedural History of the Review 11
2.4.1 Request and Acceptance 11
2.4.2 Panel Formation 11
2.4.3 Panel Process and Review Documents 12
2.4.4 Review and Transmittal . 13
2.4.5 References .14
3.0 Responses to Charge Questions 15
Charge Question 1 15
3.1 Panel Commentary 15
3.1.1 Development of the 3MRA Modeling System 15
3.1.2 Additional Comments about 3MRA 22
3.1.3 References 23
Charge Question 2a 23
3.2 Panel Commentary 23
3.2.1 General Comments 23
3.2.2 Consistency 24
3.2.3 Reproducibility of Results 25
3.2.4 Potential Inconsistency in Model Uses 25
Charge Question 2b 26
3.3 Panel Commentary 26
3.3.1 General Comments 26
3.3.2 Site Specific Use of the 3MRA Modeling System 28
Charge Question 2c 29
3.4 Panel Commentary 29
vi
-------
Charge Question 3 a
3.5 Panel Commentary
Charge Question 3b
3.6 Panel Commentary
3.6.1 General Comments
3.6.2 Model-Data and Model-Model Comparison
3.6.3 Conservation of Mass
3.6.4 Peer Review
3.6.5 Sensitivity Analysis
3.6.6 References
Charge Question 4
3.7 Panel Commentary
3.7.1 General Comments
3.7.2 Recommendations
Appendices
1-1 Exposure Duration
1-2 Correlated Variables
1-3 Alternative Waste Management Options
2a-l Classification of 3MRA Submodels
2a-2 Comments on Embedded Assumptions and Default Values
2b Benchmarks of Human and Ecological Effects
2c-1 Discussion of 3MRA Monte Carlo Analysis
2c-2 Suggestions for Alternative MCA Data Synthesis
2c-3 Probabilistic Analysis of Chemical Toxicity
3b 3MRA Panel Review of the Generic Soil Column Model with
Recommendations for Improvement
4-1 Comments Regarding 3MRA Documentation
4-2 Candidate Outline for Improved "3MRA User's Manual"
4-3 3MRA Editorial Comments
Biosketches of Panel Members
32
32
33
33
33
34
37
38
39
40
40
40
40
41
VII
-------
1.0 EXECUTIVE SUMMARY
The panel concurs that the 3MRA modeling system is a major step forward in
providing a flexible and consistent tool for estimating the distributions of the probability
of exceeding adverse effect benchmarks that result from various choices of exit threshold.
Used in conjunction with other factors 3MRA provides a scientifically defensible
framework that gives reproducible results for determining national exit levels for
RCRA-listed hazardous wastes. It is clear that the developers of 3MRA were acutely
aware of the need to address criticisms of previous modeling attempts to the problem
posed by the HWIR. The panel supports the current approach for establishing exit
concentrations, and encourages its continued development for this and other uses.
The panel also commends the manner in which 3MRA was developed, i.e. as a
genuine cross-Agency effort that to a significant degree worked through the insular
nature of individual units in a large organization, forming a formal partnership between
the Office of Solid Waste and the Office of Research and Development, and encourages
the Agency to maintain and extend the collaborative nature of this process as 3MRA is
further developed. If the Agency does not continue to support the continued development
of the various, source, fate and effects modules, assessment data and integrated system
that comprise 3MRA, and the SuperMUSE computational system, the model will cease to
evolve and its future value and utility will diminish. In this context, the panel
recommends that the Agency develop and articulate a plan for future upgrades and
refinements of 3MRA and its databases.
The panel endorses the Agency's use of the Beck, et al. (1997) validation protocol
for evaluating the 3MRA modeling system. This approach represents a departure from
traditional notions of data matching as the only criterion, to an inclusive view of
validation as a process of model evaluation, rather than a state of model condition. This
is a bold step, but one the panel believes is appropriate, certainly for the national risk
assessment objectives of 3MRA, but in a broader context, for carrying the model
evaluation debate forward as it pertains to regulatory environmental modeling. The
Agency has provided, in 3MRA, perhaps the first case study of this model evaluation
protocol. While it carries some discomfort, e.g. limited data sets for module
evaluation, and has been constrained, e.g. inadequate resources for implementing
important peer review suggestions, the panel commends the adoption of this
evaluation process for 3MRA, and urges the Agency to continue with its plan for
3MRA modeling system evaluation, particularly data-model and model-model
comparisons.
The panel agrees with the adoption of a Monte Carlo analysis (MCA) framework
as an appropriate tool to use in examining a wide range of site, chemical, and exposure
scenarios when setting national exit levels. The MCA provides an established science-
based process to allow the Agency to identify a range of exit levels at defined levels of
protection. While the MCA is an appropriate and useful tool for identifying risk
management options to the decision-maker, it has important limitations. Even though the
MCA results provide quantitative estimates of the probability of protection, the implied
-------
level of confidence should be interpreted with caution. The Agency has recognized that a
quantitative evaluation of the uncertainty of the variable (and uncertain) model input
parameters (i.e., input sampling error, or ISE) is not feasible with available data. The
panel agrees that such an analysis is impractical for the complete MCA, but the
panel does make specific recommendations for a "focused ISE" uncertainty
analysis.
The panel acknowledges that 3MRA can be used today to support regulatory
decisions for establishing national exit concentrations. However, it must be recognized
that the model is built on limited data, pragmatic assumptions, and is the product of a
collection of submodels, most of them extant legacy models, thus any regulatory
decisions that rely on 3MRA will reflect the uncertainty and the limitations of these
models. The panel stresses the need for the Agency to make clear that 3MRA is to
be used in conjunction with other tools and factors that also affect the setting of
regulatory standards (e.g., economic implications, stakeholder input, etc.)*
The panel recognizes that the developers of 3MRA were required to balance the
need to include the most advanced science in the model against the reality of the
significant computational burden of the national assessment problem, forcing many
difficult choices with respect to the level of model sophistication to be included in the
3MRA system. The panel notes with concern, for example, the incorporation of the
ISCST3 air transport model (which does not distinguish among the physico-chemical
properties of different chemicals), and the non-legacy Generic Soil Column Model
(which contains questionable embedded assumptions). The FRAMES architecture of
3MRA makes it possible to swap out and/or update modules with relative ease; the
panel recommends that the Agency address these concerns before 3MRA is used to
support regulatory decisions.
The panel is also concerned about the lack of sophistication, in comparison with
transport, fate* and exposure, of the treatment of toxicity in 3MRA, and with policy
constraints placed on the application of 3MRA, i.e., lexicological parameters are fixed at
a single value rather than with a probability distribution, which the current 3MRA
technology supports. The panel strongly endorses the movement toward the
inclusion of such an approach, one that uses the capabilities of MCA, into 3MRA as
future versions are developed. Given the significant scientific limitations and
difficulties characterizing uncertainty and variability in toxicological parameters,
this goal can only be accomplished with a substantial commitment of resources for
research.
The application of 3MRA for site-specific purposes, in distinction to the setting of
national standards, will foster continued evolution of the model. With significant
expense and regulatory burdens at stake, stakeholders will seek to use 3MRA, and to
provide feedback to the Agency regarding model assumptions and outcomes. In this
respect, the panel finds that 3MRA omits certain pathways that may be important
contributors to exposures at specific sites or regions. For example, some human exposure
pathways (e.g., vapor intrusion, dermal exposure) are not included, nor is the potential for
-------
adverse effects beyond a two kilometer radius around WMUs (i.e., the attendant risks to
human health and the environment associated with long-range transport and
accumulation). Also, concurrent exposures to multiple contaminants in the waste are not
considered. The panel understands that many of these exposure pathways were screened
out of the modeling process because they were not thought to be significant contributors
to the national risk/hazard problem. In addition, 3MRA does not include disposal options
beyond land-based (e.g., incineration). Given the wide range of different chemicals and
release scenarios that the model was developed to assess, and probable site-specific
applications in the future, the panel believes that a more complete set of exposure
pathways, and eventually disposal options, be built into the model. If exposure
pathways that are acknowledged to be of potential importance are to be excluded,
the panel recommends that the Agency demonstrate, through appropriate analysis,
that the results will still achieve the level of protection intended at the site level. In
addition, the panel recommends that the implementation of the model for regulatory
purposes include the flexibility for interested parties to provide additional data and
new modeling approaches.
3MRA processes and outputs very large quantities of information. The panel
encourages the Agency to continue development of mechanisms for meaningful
interpretation of model output, currently underway for 3MRA version 1.x., and
believes it is necessary that the version 1.x tools be completed prior to adopting
3MRA for site-specific applications. Similarly, the panel urges that the Agency
complete the development and documentation of the Site Visualization Tool (SVT).
This tool shows significant promise for addressing panel recommendations to provide
"intermediate" model outputs such as chemical concentrations in various exposure media,
pathway-specific exposure, and so forth.
Finally, the panel recommends that the documentation for 3MRA undergo
significant reorganization and revision with respect to the need for a readable
summary, improved clarity of terms, concise descriptions of databases, and ease of
implementation of the modeling system (i.e., a User's Guide). Specific suggestions
are provided in the report.
-------
2.0 INTRODUCTION AND CHARGE QUESTIONS
This chapter of the report provides the background, context, charge for the review
and the procedural history. Specific responses to charge questions can be found in
Chapter 3.
2.1 Background
2.1.1 History of the 3MRA from HWIR to Development of the Integrated
Research Plan
There have been substantial efforts by Federal and State organizations and the
private sector to develop risk assessment tools that include the evaluation of contaminants
in different media and the integration of exposures across pathways to help establish an
integrated risk-based assessment.
In December 1995, EPA's Office of Solid Waste proposed to amend existing
regulations for disposal of listed hazardous wastes under the Resource Conservation and
Recovery Act (RCRA). The December 1995 proposal (60 FR 6634, December 21,1995)
outlined the Hazardous Waste Identification Rule (HWIR) that was designed to establish
constituent-specific exit levels for low risk solid wastes that are currently captured in the
RCRA subtitle C hazardous waste system. Under this proposal, waste generators of
listed wastes that could meet the new concentration-based criteria defined by the HWIR
methodology would no longer be subject to the hazardous waste management system
specified under subtitle C of RCRA. This would have established a risk-based "floor"
for low risk hazardous wastes that would encourage pollution prevention, waste
minimization, and the development of innovative waste treatment technologies.
In May and June of 1995, EPA's Science Advisory Board (SAB) reviewed the
proposed HWIR methodology for calculating exit concentrations and in May 1996
published its findings in Review of a Methodology for Establishing Human Health and
Ecologically Based Exit Criteria for the Hazardous Waste Identification Rule (HWIR)
(EPA-SAB-EC-96-002).
In addition to this review, EPA's Office of Research and Development (ORD),
and numerous industrial and environmental stakeholders, also reviewed the proposed
methodology. While the SAB concluded that the methodology' 'lacks the scientific
defensibility for its intended.regulatory use," the SAB also made the following
recommendations that, when addressed, should provide an adequate scientific basis for
establishing a risk-based methodology applicable at the national level for the waste
program:
a) Develop a true multi-pathway risk assessment in which a receptor receives
a contaminant from a source via all pathways concurrently, is exposed to
the contaminant via different routes, and accounts for the dose
corresponding to each route in an integrated way;
-------
b) Maintain mass balance;
c) Conduct substantial validation of the methodology and its elements,
against actual data derived from the laboratory or field, prior to
implementation of the model;
d) Conduct a systematic examination of parameters to ensure a consistent and
uniform application of the proposed approach, and further, the full suite of
uncertainties to be addressed for the final methodology;
e) Discard the: proposed screening procedure for selecting the initial subset of
chemicals for ecological analysis and instead require that a minimum data
set be satisfied before ecologically based exit criteria are calculated;
f) Seek the substantive participation, input, and peer review by Agency
scientists and outside peer review groups as necessary, to evaluate the
individual components of the methodology hi much greater detail; and,
g) Reorganize and rewrite the documentation for both clarity and ease of use.
As a result of the methodology reviews, the Office of Solid Waste (OS W)
collaborated with the Office of Research and Development (ORD) to develop and
document a sound science foundation, supporting data for an assessment, and related
software technology for an integrated, multimedia modeling system (entitled 3MRA)
following the recommendations of the SAB and other reviewers. This effort was initiated
with the peer review of an integrated research and development plan (ORD/OSW
Integrated Research and Development Plan for the Hazardous Waste Identification Rule
(HWIR), 1998, that describes the assessment methodology, the technical bases for the
integrated multimedia modeling system, and quality controls to be followed during the
developmental process.
2.1.2 The 3MRA Modeling System
The Multimedia, Multipathway, and Multireceptor Risk Assessment (3MRA)
modeling system represents a collection of science-based models and databases that have
been integrated into a software infrastructure that is based on the FRAMES (Framework
for Risk Analysis in Multimedia Environmental Systems) concept, which provides a
computer-based environment for linking environmental models and databases and
managing the large amounts of information within the system, including the visualization
of outputs. This integrated multimedia modeling system provides national-level
estimates of human and ecological risks resulting from long-term (chronic) chemical
release from land-based waste management units. The modeling system is described in
greater detail in Section 2.3.2.
-------
2.13 Peer Review of Modules within the 3MRA Modeling System
Over 45 experts participated in the peer review process of the underlying science
within the 3MRA modeling system. The EPA plans to use the modeling system to help
inform managers on a variety of decisions in the waste program, such as setting
concentration-based exit criteria for wastes in the hazardous waste management
regulations, or deciding whether technology-based standards are protective of human
health and the environment.
2.2 Context
The EPA Office of Solid Waste (OSW) is responsible for managing solid and
hazardous waste as specified by the Resource Conservation and Recovery Act (RCRA) of
1976 and subsequent legislation, such as the Hazardous and Solid Waste Act (HSWA) of
1984. These acts and the programs developed to implement them were designed to
protect human health and the environment. Thus, many of the regulatory decisions
within the RCRA programs are based, at least in part, on the human health risk and
environmental impacts of the regulatory options under consideration.
As the RCRA program has evolved, and as new risk assessment methods have
been developed, EPA's need for improved risk assessment models has greatly increased.
The RCRA programs initially addressed only releases to ground water from land disposal
operations and releases to air from waste incinerators and other types of boilers and
industrial furnaces. However, the RCRA programs have expanded in scope over the
years to encompass hundreds of constituents, thousands of waste streams, and many types
of waste management practices, ranging from recycling and reuse to disposal and
destruction techniques. Thus, new risk assessment models were needed to assess the
types and magnitude of risks that fall under the broad purview of the RCRA programs.
In addition, in the mid-1990s, several groups within and outside of EPA came
forward with recommendations or guidance for improving risk assessment methods. In
1996, EPA issued new guidelines for conducting exposure assessments and risk
assessments that focused on improving the science underpinning the risk or exposure
assessments that were being conducted, as well as improving the methods for
characterizing the uncertainty in the risk estimates that are generated. In 1997, the
Presidential/Congressional Commission on Risk Assessment and Risk Management
(CRARM) issued a report on improving risk assessment methods used by the federal
government. Also, EPA's Science Advisory Board reviewed and commented on a
number of EPA risk assessments and models, including the dioxin and mercury risk
assessments.
The 3MRA modeling system was developed as a predictive tool to provide risk
assessment support for the types of risk management decisions that are made within
OSW. OSW applies risk assessment modeling tools in a variety of situations; one
application is the conduct of site-based national-level risk assessments to support
rulemaking for the identification of hazardous waste. Consequently, the 3MRA modeling
-------
system needed to be able to model waste management environmental settings that are
representative of the range of environmental settings found in the United States, and
within this broad range of settings, to simulate the release, fate and transport of many
contaminants in waste undergoing a range of physical and biochemical processes. More
than 400 constituents are regulated under the RCRA programs. EPA needs to consider
the impacts of these released contaminants on humans and the environment within the
broad range of environmental settings. This requires a modeling tool that encompasses.
releases to all media, transport within those media, uptake in terrestrial and aquatic food
webs, and exposure of specific receptors to contaminated media and food items.
Together OSW and ORD intend to provide a base technology within which
assessments can be conducted and science-based modeling experiments can be
conducted.
2.3 Charge
The EPA asked the SAB to focus its review in the following four areas:
assessment methodology, 3MRA modeling system, modeling system evaluation, and
modeling system documentation. Some charge questions were modified slightly by the
Panel in August 2003; their final language is used in Section 3 of this report. The
original wording of the charge questions appears below.
2.3-1 Assessment Methodology
The 3MRA assessment methodology presents a strategy for estimating national
distributions of human and ecological risks resulting from long-term (chronic) chemical
release from land-based waste management units. The national distribution is
constructed by performing "site-based" assessments at a significant number of randomly
sampled hazardous waste site locations across the U.S. In the assessment methodology, a
pollutant is released from a waste management unit to the various media (air, water, soil)
according to its chemical properties and characteristics of the unit The pollutant is
transported through the media and exchanged between media via system linkages.
Receptors are exposed concurrently to the pollutant via multiple pathways/routes
resulting in an integrated dose.
The methodology describes a tiered approach for populating data files for each
site evaluation. The approach is referred to as "site-based1' because the assignment of
data values for the site being simulated occurs according to a tiered protocol. Data values
are filled first with data at a site level; when site data are not available, a statistically
sampled value from a geographically relevant regional distribution of values are used;
and lacking a representative regional distribution for the variable, a value from a national
distribution is assigned.
The 3MRA methodology was designed specifically to include Monte Carlo
simulation methods to address uncertainty and variability in the risk outputs. Statistical
distributions for many modeling parameters were developed and upon implementation
-------
provide a statistical measure of variability and uncertainty, i.e., the range and distribution
of potential exposures and risks occurring at a site. When applied to the sites in a
national assessment, the result is a statistical measure of variability and uncertainty, and
national distributions of risks. The sites currently in the database are randomly selected
from sites across the United States to represent the national variability in waste
management scenarios and locations. The methodology for selecting the sites allows for
measures of protection to be calculated at the site level and aggregated over all the sites
to develop the national distribution of risks.
Charge Question 1; While the EPA had the assessment methodology peer
reviewed prior to the development of the 3MRA modeling system, does the SAB
have any additional comments about the methodology as implemented?
2.3.2 3MRA Modeling System
To implement the 3MRA methodology, the EPA chose to develop a
comprehensive software-based modeling system, which facilitates the consistent use of
sound-science models through a framework that controls model sequencing, facilitates
data exchange, and provides data analysis and results visualization tools. Following
modem Object Oriented software design and development principles and honoring the
use of legacy models (i.e., fate and transport models that have a long history of use at the
EPA), the EPA has constructed a modern modeling system that facilitates the consistent
and reproducible application of the 3MRA modules and databases to problems requiring
a national-scale assessment of site-based risks. The 3MRA modeling system is
underpinned by a software infrastructure named FRAMES. FRAMES provides a
computer-based environment for linking and applying environmental models and
managing the large amounts of information within the system.
The 3MRA modeling system consists of: (a) 17 science-based modules that
estimate chemical fate, transport, exposure, and risk; (b) 7 system processors that select
data for model execution; manage information transfer within the system; "roll-up" site-
based results into distributions of risk at the national level; and provide a visualization of
the system outputs; and (c) multiple databases that (currently) contain the data for waste
managements sites across the country as well as regional and national distributions of
data values, (d) a software infrastructure (framework) based on FRAMES.
The 3MRA system was designed to provide flexibility in producing distributions
of hazards or risks at sites that may manage exempted waste because the final regulatory
decision framework for defining chemical-specific exit levels has not been formulated.
The system is designed to allow the evaluation of human health impacts to the general
population or selected subpopulations and the impact of varying the measures of
protection at different probability levels. The system has similar capabilities with respect
to evaluating the impacts on ecological systems.
-------
Charge Question 2a: Does the 3MRA modelin
performing national risk assessments that facilitates ci
and provides a mechanism for reproducing results?
Charge Question 2b: Does the 3MRA modeling
makers sufficient flexibility for understanding the irapn
exemption levels by allowing varying measures of prott
receptors and/or number of sites protected, types of bur
receptors, and distance?
em provide a tool for
tent use of the science
;em provide decision-
m potential chemical
in based on the number of
n and ecological
Charge Question 2c; Does the 3MRA modeling s> aem providi .ippropriate
information for setting national risk-based regulations for the waste program?
2.3.3 Modeling System Evaluation
In response to the SAB recommendation that substantial evaluation, of the
modeling system is essential to building confidence in the system, the EPA focused
significant efforts to ensure the scientific integrity of the 3MRA system and its results
during system development and post-development. The EPA designed and implemented
rigorous quality assurance and quality control procedures for software development, data
collection, verification testing, and peer review on the scientific components of the
system.
The EPA implemented specific steps to build a level of confidence in the system
to ensure that the system will present a reasonable estimate of nationwide risk for a
national-level assessment.
First, the overall technical approach and each science-based module included in
3MRA have been peer reviewed. Teams of peer reviewers (at least three per module)
provided critical feedback about the science-based modules. All told, over 45
independent experts reviewed the science modules to ensure that the theoretical concepts
describing the processes within release, fate, transport, uptake, exposure, and risk
components were adequate representations of the processes to be evaluated.
Second, all software components and databases underwent a series of tests to
verify that the software and data were performing properly. At the heart of this protocol
is the requirement that each component of the modeling system include a designed and
peer reviewed test plan that is executed by both the model developer and a completely
independent modeler (i.e., someone who did not participate in the original model
development). These procedures, test plans, test packages, and test results are fully
documented and available to the public.
Third, a comprehensive data collection approach was developed to parameterize
the modeling system in accordance with the site-based approach described in the
assessment methodology. This data collection plan described the general collection
methodology for the major types of data (for example, facility location, land use, soil
-------
characteristics, receptor locations), including quality assurance and quality control
procedures and references for data sources.
Fourth, the 3MRA modeling system is currently undergoing a comparison
analysis with EPA's Total Risk Integrated Methodology (TRIM) that is under
development. The objective of the model comparison effort was to increase confidence
that the 3MRA modeling system produces estimates consistent with other multi-media
models.
While complete validation of a modeling approach would be the ultimate proof
for a multimedia system like the 3MRA, the EPA did not find a multimedia data set to
compare with the system's predictive outputs. In addition, the model comparison study
was conducted using an actual industrial site where environmental monitoring data for
mercury representing the relationship between contaminant source and environmental
concentrations were available (albeit an incomplete set of observational data). Finally, a
formal program focusing on sensitivity and uncertainty analysis for high-order modeling
systems has been initiated at ORD. The early focus of this program is the investigation
of parameter sensitivities and system uncertainties within the 3MRA modeling system.
The SuperMUSE system has been configured to allow exhaustive experimentation with
the 3MRA system in Monte Carlo mode. Initial results of these efforts have been
documented.
Charge Question 3a: Is the software development and verification testing
approach implemented for the 3MRA modeling system sufficient to ensure
confidence that the modeling results reflect the modeling system design?
Charge Question 3b: Given the thorough evaluations that EPA has
implemented using the available data resources and technologies, while also
recognizing the real world limitations that apply to validating the 3MRA modeling
system, have we reasonably demonstrated through methodology design, peer review,
quality control, sensitivity analyses, and model comparison, that the 3MRA
modeling system will produce scientifically sound results of high utility and
acceptance with respect to multimedia regulatory applications?
2.3.4 3MRA Modeling System Documentation
In response to significant comments regarding the lack of clarity and transparency
associated with documentation of the earlier modeling system the EPA has devoted
significant time and resources to correcting this limitation. The 3MRA represents a
comprehensive risk assessment capability and as such integrates the science from all
contributing disciplines. Documentation is necessarily voluminous. In preparing the
current documentation our intent is to provide different levels of presentation depending
on the intended audience. The EPA has prepared a significant number of reports and
documents at various levels of technical complexity that describe the 3MRA modeling
system and the related HWIR application.
10
-------
The review documents consist of a four volume set of documents, providing a
comprehensive overview of the 3MRA modeling system. These documents are intended
to be the primary means by which the general public would become familiar with the
3MRA system and are also intended to provide the level of information necessary for a
risk assessor to make an informed decision regarding the applicability of the 3MRA
modeling system to specific risk assessment problems.
Charge Question 4: Has the EPA made substantive progress, relative to
1995, in designing and preparing documentation for the 3MRA modeling system?
Does the SAB have additional suggestions for improving the presentation of the
comprehensive set of materials related to this modeling system?
2.4 Procedural History of the Review
2.4.1 Request and Acceptance
In May 2002, the Office of Solid Waste requested that the Science Advisory
Board review the 3MRA modeling system in 2003. After considering all requests for
2003, the Executive Committee of the Science Advisory Board determined that the
review should be conducted by a specialized panel. The Director of the Science
Advisory Board Staff Office, in consultation with the Chairman of the Science Advisory
Board, selected Environmental Engineering Committee member Dr. Thomas L. Theis,
Director of the Institute for Environmental Science and Policy at the University of Illinois
at Chicago, as chair of the panel.
2.4.2 Panel Formation
The panel was formed in accordance with the principles set out in the 2002
commentary of the Science Advisory Board, Panel Formation Process: Immediate Steps
to Improve Policies and Procedures (EPA-SAB-EC-COM-02-003). A notice offering
the public the opportunity tq nominate qualified individuals for service on the panel was
published in the Federal Register on April 11,2003 (68 FR 17797-17800). Seventy-five
(75) individuals were considered for membership on the panel. On the basis of
candidates' qualifications, interest, and availability, the SAB Staff Office made the
decision to put 35 candidates on the "short list". On May 29,2003, the SAB Staff Office
posted a notice on the SAB Web site inviting public comments on the prospective
candidates for the panel.
The SAB Staff Office Director in consultation with SAB Staff (including the
DFO and the Acting SAB Ethics Advisor) and the Chair of the Executive Committee
selected the final panel. Selection criteria included: excellent qualifications in terms of
scientific and technical expertise; the need to maintain a balance with respect to
members' qualifying expertise, background and perspectives; willingness to serve and
availability to meet during the proposed time periods; and the candidates* prior
involvement with the topic under consideration. The final panel includes experts with
-------
experience in academia, industry, research organizations, state agencies, non-
Governmental organizations (NGOs), and consultant groups.
2.4.3 Panel Process and Review Documents
In summary, panelists were provided with, the review materials prior to the first
face-to-face meeting and asked to write down their preliminary individual responses to
the charge questions. After briefings and public comment at the first face-to-face
meeting, the panel articulated a set of consensus points to be used in drafting the report
and coordinators were assigned to prepare responses to each of the four major charge
questions using input from their colleagues on the panel. Although a draft was discussed
at the second face-to-face meeting, much of the meeting was spent on additional Agency
presentation and public comment, with a second draft being discussed by conference call
on December 15 and January 16. New material was provided to the Panel during this
time. Discussion of a third draft February 6 led to some further analysis and writing in
specific areas to achieve clarity. The panel approved the final wording of its report
March 18,2004 after which it was forwarded to the Board for review and approval prior
to transmittal to the Administrator.
The 3MRA modeling system is complex, the documentation extensive, and the
review intense and time consuming. The Panel had two face-to-face meetings. These
were held August 26-27 and October 28-30,2003. These open meetings were
supplemented by ten open conference call meetings: July 21, August 15, September 16,
October 9, November 24, December 15, January 16 (2004), February 6, February 27, and
March 18. Opportunities for written and oral public comment were provided at all of
these meetings.
From time to time, a subset of the panel met with the Designated Federal Officer
(DFO) to do planning, fact-finding, or other work preparatory for a subsequent open
meeting. Each occasion was acknowledged at the following open meeting; a participant
would summarize and answer questions and the DFO's notes were included with the
relevant minutes. Such calls were organized around particular technical issues and
include calls on validation on September 11 and 18; uncertainty September 12,
September 19, December 4, and January 12; ecology and health October 8 and 15; on
soils and source terms October 10; and the 2 km radius on January 9. In addition, the
coordinators of the responses to the various charge questions met October 11 to discuss
issues such as organization and format of the responses.
The primary review materials included a CD with four volumes of review
materials and a user's guide for the model, a CD with the model on it. These were
provided in July 2003 along with the website where the results of more than 45 previous
peer reviews of parts of the modeling system are available, an e-mailed "Roadmap" from
the Agency relating materials in the four volumes to the public comment received at the
previous conference call.
12
-------
The primary review materials were supplemented with additional information,
almost always in response to requests from Panelists. In September, one panelist had
requested and been sent the document, Quality Assurance of Multi-Media Model for
Predictive Screening Tasks (EPA 600/R-98-106, August 1999). The Panel also requested
and received On The Problem of Model Validation For Predictive Exposure Assessments,
published in Stochastic Hydrology and Hydraulics, Vol. 11, pages 229-254,1997, by
M.B. Beck, J.R. Ravetz, LA. Mulkey, and T. O. Bamwell and Model Evaluation and
Performance by B. Beck, published in Encyclopedia of Environmetrics (ISBN 0471
899976) Volume 3, pp 1275-1279, edited by Abdel H. El-Shaarawi and Walter W.
Piegorsch, John Wiley & Sons, Ltd. Chichester, 2002. In October, Robert Ambrose of
EPA provided the Panel with a write-up on water balance. The Agency also provided a
compilation and analysis of the 45 prior peer reviews. By December, the Agency had
also provided a CD of new material with uncertainty analyses tor seven chemicals,
additional material relating to the General Soil Column module. In January, the Panel
received additional material on uncertainty.
2.4.4 Review and Transmittal
During the course of the 3MRA review, the Science Advisory Board underwent a
reorganization. The membership of the Executive Committee was broadened and it was
renamed the Board. The mechanism for review of final products before transmittal to the
Administrator was also modified. Under the previous organization, the Executive
Committee had assigned vsttors to review reports before transmittal; under the new
organization a separate and specialized Quality Review Panel would be formed for the
review of important reports. This Panel would meet separately, then report to the full
Board. The 3MRA report was the first report to go through this new process.
The Vice Chairman of the Board, Dr. Domenico Grasso, Rosemary Bradford
Hewlett Professor and Chair of the Picker Engineering Program at Smith College and
former chair of the Envinxamental Engineering Committee, formed a Quality Review
Committee (QRC) to review the 3MRA Panel's report. The review considered whether:
a) the original charge questions were adequately addressed;
b) there were any technical errors or omissions in the report or issues that
were inadequately dealt with in the Panel's report;
c) the Panel's report were clear and logical; and
d) the conclusions and recommendations were supported by the body of the
Panel's report.
After a review by the SAB's Quality Review Committee, the Board considered
this report together with the evaluation of the QRC and decided and approved it for
transmittal to the Agency. The Board expects that the Agency will provide a written
response to this report.
13
-------
2.4.5 References
Review of a Methodology for Establishing Human Health and Ecologically Based Exit
Criteria for the Hazardous Waste Identification Rule (HWIR) (EPA-SAB-EC-96-002),
Science Advisory Board, U.S. Environmental Protection Agency, 1996.
Panel Formation Process: Immediate Steps to Improve Policies and Procedures (EPA-
SAB-EC-COM-02-003), Science Advisory Board, U.S. Environmental Protection
Agency, 2003.
Integrated Research and Development Plan for the Hazardous Waste Identification Rule
(HWIR), Office of Research and Development and Office of Solid Waste, U.S.
Environmental Protection Agency, 1998.
http://www.epa.gov/epaoswer/hazwaste/id/hwirwste/risk.htni
14
-------
3.0 RESPONSES TO CHARGE QUESTIONS
Charge Question 1. While the EPA had the assessment methodology peer reviewed
prior to the development of the 3MRA modeling system, does the SAB have any
additional comments about the methodology as implemented?
According to the Agency, the 3MRA assessment methodology as implemented contains
several elements:
Statistical sample of industrial sites
Site-based human and ecological exposure/risk assessment
Multi-contaminant, -media, -pathway, -receptor
Tiered Data (site-specific, regional, national)
Population-based site level risk estimates
National roll-up of risks
Alternative measures of protection
* Pseudo two-stage Monte Carlo
Probability-based design to facilitate uncertainty analysis and sensitivity analysis
Externally peer reviewed and independently tested
3.1 Panel Commentary
3.1.1 Development of the 3MRA Modeling System
The panel concurs that the 3MRA modeling system is a major step forward in
providing a computer-based tool for estimating the distributions of the probability of
exceeding an adverse effect benchmark that result from various choices of exit threshold,
and provides a scientifically defensible framework for determining national exit levels for
RCRA-listed hazardous wastes. The panel recognizes the rationale of a tiered set of data
for conducting screening Level assessments, and the use of statistical sampling and
analysis that together define the approach for developing a national assessment
methodology. In addition, the panel agrees that 3MRA is truly a multi-media, multi-
pathway, and multi-receptor model that produces consistent and reproducible results.
The panel supports the current approach for establishing exit concentrations, and
encourages its continued development for this and other uses.
The panel commends the manner in which 3MRA was developed, i.e. as a
genuine cross-Agency effort that to a significant degree worked through the insular
nature of individual units in a large organization, forming a formal partnership between
the Office of Solid Waste and the Office of Research and Development, and encourages
the Agency to maintain and extend the collaborative nature of this process as 3MRA is
further developed. The complexity of the technical and scientific issues involved makes
an undertaking of this type extremely difficult If the Agency does not continue to
support the continued development of the various, source, fate and effects modules,
assessment data and integrated system that comprise 3MRA, the model will cease to
evolve and its future value and utility will diminish. In this context, the panel
15
-------
recommends that the Agency develop and articulate a pJan for future upgrades and
refinements of 3MRA and its databases.
From a regulatory perspective 3MRA is a valuable tool and an important step
forward for understanding the fate and effects associated with the disposal of chemicals
in the environment. The panel acknowledges that 3MRA can be used today to support
regulatory decisions for establishing national exit concentrations. However, it must be
recognized that the model is built on limited data, pragmatic assumptions, and is the
product of a collection of submodels, most of them extant legacy models, thus any
regulatory decisions that rely on 3MRA will reflect the uncertainty and the limitations of
these models. While the panel recognizes the benefits of building 3MRA on legacy
models, it nevertheless stresses the need for the Agency to make clear that 3MRA is to be
used in conjunction with other tools and factors that also affect the setting of regulatory
standards (e.g., economic implications, stakeholder input, etc.).
An example of the panel's concern about the general applicability of legacy
models is the ISCST3 submodel. ISCST3 is a steady-state Gaussian plume dispersion
model originally designed for application to criteria air pollutants (CO, NO& Pb» PMio,
SOi and PM2.5) for which the primary factors influencing atmospheric fate are advective
flows and irreversible deposition. Such a model may not be ideal for chemicals mat are
not typically thought of as air pollutants. The panel agrees that the algorithms in ISCST3
have been extensively reviewed and evaluated for the criteria air pollutants and that the
model has a long history of use by the EPA, but the panel also notes that because the
model was developed for criteria pollutants, it does not account for differences in the
physicochemical properties of volatile and semi-volatile organic pollutants. The panel
cautions the Agency that "legacy" status does not necessarily mean a model is
appropriate for all chemicals, particularly when the legacy model was designed for a
specific purpose or chemical class. The Agency should demonstrate the adequacy of the
air dispersion model for a wider range of physicochemical properties (see response to
question 3b). This might be done by comparing results from 3MRA (over a relevant time
period) with alternate models that have been developed specifically for multimedia fate
modeling such as the Agency's TRIMFaTE model or other Mackay-type fugacity
models.
An exception to the use of legacy models in assembling 3MRA is the Generic Soil
Column Model (GSCM), which is embedded within several of the transport modules of
3MRA. This model was written exclusively for 3MRA and does not appear to have been
extensively reviewed or validated. The Agency has made a number of pragmatic
assumptions regarding boundary transfer, local equilibria and solution methodology in
order to ease the computational burden associated with this important module. It is
incumbent upon the Agency to continue to test and evaluate the suitability of these
approximations, as well as to explore more mechanistic treatments of the GSCM
processes as they affect constituent fate and transport, in order to build confidence that
the module is operating adequately and retains needed accuracy. Options for
accomplishing this include additional data matching, comparison of results with other
16
-------
accepted models, theoretical analysis, or error analysis. The panel's review of GSCM is
presented in its response to question 3b and in appendix 3b.
The ultimate results from 3MRA are expressed in terms of allowable
concentrations of particular chemicals in a waste stream that may "exit" RCRA Subtitle
C hazardous waste management facilities. Yet, the transport models used in 3MRA
require as input the chemical concentration that enters a particular WMU (e.g., a
concentration in soil for lard-based units). In 3MRA, the chemical concentration that
actually is "applied" to a particular WMU is a function of two parameters: the modeled
concentration in the waste (Cw) and the "fraction of waste," or fwMU, term that defines the
relative amount of waste in the waste stream applied to the WMU. The actual initial
chemical concentration in the WMU is not ,, but Cw reduced by a random fraction
(fwwu) between 0.01 and 1.0. For example, if an fwMU value of 1.0 is randomly selected
for any given simulation, this would equate to a "monofill" which receives 100% of a
given waste compound. Because the value of fwMU is selected randomly within the
3MRA Monte Carlo simulation structure, there is no way to determine the actual initial
concentration that enters a WMU. The panel recommends that the Agency conduct an
analysis of the 3MRA results in order to document the range of fwMU values that
ultimately are associated with the exit level results, for example are the exit levels
typically associated with fwMu values at the upper end of the range (e.g., values near 1.0)?
In addition, there does not appear to be any discussion or rationale in 3MRA for why the
fwMu term should be considered as a random variable, nor a justification for assigning it
as a "uniform" random variable (a selection which implies very little knowledge of this
parameter). As a direct scalar of the applied waste concentration, this single factor could
potentially have a large impact on the 3MRA results. The panel suggests considering
fwMU as a decision variable, and modeling several discrete values.
During its development, 3MRA has become a sophisticated, computationally-
intensive program. This has led to some confusion on the part of the panel about the
intended users of 3MRA. On the one hand the Agency, in its regulatory role, can be
viewed as the only valid user as they fulfill the requirements of HWIR in setting national
risk-based exit values for subtitle C facilities. On the other hand, with significant
expense and regulatory burdens at stake, stakeholders will seek to use 3MRA for various
site specific purposes, and to provide feedback to the Agency regarding model
assumptions and outcomes. The ready availability of extensive new data sets for
incorporation into 3MRA is unlikely unless EPA seeks out appropriate data from the
stakeholder community. As newer and more reliable data become available, and
uncertainty is reduced, more realistic assumptions regarding the fate and effects of
chemicals of concern on a site-specific basis should be incorporated into 3MRA. The
application of 3MRA for site-specific purposes will foster continued evolution of the
model. Therefore, the panel suggests that the implementation of the model for regulatory
purposes include the flexibility for interested parties to provide additional data and new
modeling approaches.
Early in its development the Agency made the decision to implement 3MRA on a
Windows platform in order to facilitate its use in a PC-based computational environment.
17
-------
(Indeed, during the familiarization period, panel members both individually and
collectively at face-to-face meetings made several runs of the model on single PCs).
However, actual use of 3MRA to assist in the setting of national exit levels is a much
more complex application that greatly magnifies computational demands, thus
necessitating the assembly and use of the SuperMUSE system. The panel recognizes the
significant achievement that this represents, and expresses its support for maintaining this
resource. The panel also recognizes concern on the part of the stakeholder community
regarding the ease of use of 3MRA, and suggests that the Agency be cognizant of these
concerns as future versions of 3MRA are developed and made available.
Although the national distribution of risks is clearly an important factor for
decision makers who are responsible for setting exit levels, the panel is concerned that
the exclusive focus on ecological and human receptors at specific points in space may not
provide reasonable assurance that natural resources (aquifers, surface water bodies and
agricultural soils) will be protected. To illustrate this concern, the Panel notes that the
3MRA modeling system only includes the groundwater pathways for receptor locations if
the 1990 Census data indicate the presence of private wells in the particular Census block
group. As a result, only '-35% of the population in the national assessment is exposed to
groundwater. In addition, public water supplies, even those that originate from
groundwater, are assumed to be treated so that exposure to groundwater is reduced even
further. Although these may all be valid assumptions, the final result is that the spatial
coverage around the national set of WMUs for exposure to groundwater is potentially
small. As a result, an exit level Cw that is protective based on the national distribution of
risk may result in a significant traction of sites where contaminant levels in groundwater
exceed levels set out in the National Primary Drinking Water Regulations. Similarly,
contaminant levels in surface water bodies within the AOI may exceed the National
Recommended Water Quality Criteria.
The panel notes that 3MRA already outputs specific media concentrations and the
Site Visualization Tool (SVT) provides a means to access this data. These media-specific
concentrations could be used to communicate to decision makers the potential for
contamination of media around the WMUs. The panel recommends that the Agency
include a summary table of abiotic media concentrations around the WMUs in the final
model output. This will require continued development and documentation of the Site
Visualization Tool (SVT), which is briefly described in Volume V, Section 4.3.2.1. Even
in its "beta version" this tool shows significant potential for addressing panel
recommendations to provide "intermediate" model outputs (this refers to such outputs as
chemical concentrations in exposure media, pathway-specific exposure and so forth as
given in the response to question 3). To be consistent with thejother model outputs, the
media concentrations might also be rolled up as percent of sites where NPDWR levels for
groundwater, NRWQC levels for surface water and/or existing Soil Screening Guidance
levels for soil are exceeded. Lacking a fully functional and documented SVT, the model
results for the numerous output variables stored in the "GRF" files cannot be readily
interpreted by anyone except the model developers. The panel recommends that
completion and enhancement of the SVT should receive high priority (see additional
recommendations in Appendix 4-1).
18
-------
The panel is concerned about the lack of sophistication, in comparison with
transport, fate, and exposure, of the treatment of toxicity in 3MRA, and with policy
constraints placed on the application of 3MRA, i.e., lexicological parameters are fixed at
a single value rather than with a probability distribution, which the current 3MRA
technology supports. In many cases, a significant degree of uncertainty in both the
human health and ecological risk assessment protocols is associated with dose-response
relationships. The panel feels that the Monte Carlo analysis should recognize these
uncertainties as well as species response variability. The panel strongly endorses the
movement toward the inclusion of such an approach into 3MRA, to the extent possible,
as future versions are developed. Given the significant scientific limitations and
difficulties characterizing uncertainty and variability in lexicological parameters, this
goal can only be accomplished with a substantial commitment of resources for research.
Lacking this, the panel recommends that sensitivity analysis include the dose-response
for candidate chemicals. Another area where the modeling system appears to lack
sophistication is with the assessment of potential ecological impact The current
approach in 3MRA, which uses simple protective criteria as benchmark comparative
values, may not adequately characterize risk in ecological populations (see appendix 2b
for additional details).
On September 14,2004 the Chair of the Integrated Human Exposure Committee and the
Environmental Health Committee presented a draft letter to Administrator Leavitt to the
Board. After hearing a presentation on An Examination of EPA Risk Assessment
Principles & Practices fEPA/100/B-04/001t March 20041. the Committees wished to
convey two messages to the Administrator. The second of these, the application of
probabilistic methods for performing hazard and dose-response assessment, may be of
interest to readers of this report. The 3MRA Panel did not consider this letter in its
deliberations as it was drafted after the Panel had finished its work. The letter, when
finalized, will be available at the SAB Web site: http://www.epa.gov/sab
The panel also finds that 3MRA omits pathways that may contribute to exposures.
For example, some human exposure pathways (e.g., vapor intrusion, dermal exposure)
are not included. Also, concurrent exposures to multiple contaminants in the waste are
not considered. The panel understands that many of these exposure pathways were
screened out of the modeling process because they were not thought to be significant
contributors to the national risk/hazard problem. However, given the wide range of
different chemicals and release scenarios that the model was developed to assess, and
probable site-specific applications in the future, the panel believes that a more complete
set of exposure pathways be built into the model. If specific exposure pathways are to be
excluded, the panel recommends that the Agency demonstrate, through appropriate
analysis, that exclusion of these pathways will still achieve the level of protection
intended at the site level. Appendix 1-1 contains additional amplification from the panel
on its concerns about exposure values. The panel also notes that several exposure
parameters in 3MRA that co-vary with body weight are treated as independent. This may
make the exposure appear more variable than it really is. Appendix 1-2 provides more
detail on this matter.
-------
The panel notes that the 3MRA modeling system does not address the potential
for adverse effects in humans or ecosystems (and their components) beyond a 2 kilometer
radius around WMUs. Therefore, it does not predict transport of chemicals beyond this
region, nor was it designed to address the attendant risks to human health and the
environment associated with long-range transport and accumulation (such as, for
example, the atmospheric transport and deposition of chlorinated hydrocarbons in the
Great Lakes). Thus care must be taken in the use of 3MRA as a regulatory tool to ensure
that the risks associated with chemicals from medium to long-range transport beyond the
2 km region near a WMU are addressed via other means. The panel recommends that the
agency account for those chemicals that are known to have risks of this nature, or are
strong candidates for such risk pathways, and identify additional ways of assessing their
potential for environmental harm in addition to the 3MRA analysis.
The 3MRA system is intended to rest on sound scientific principles, among them
the conservation of mass. The panel is convinced that precautions have been taken to
ensure that mass is conserved within the individual modules of 3MRA and during the
transfer of information among linked modules. The panel notes in particular mass
conservation within the source modules. Still, the panel is concerned that secondary
sources of contamination, which are not presently modeled within 3MRA, may result in
significant mass imbalances for certain chemicals, particularly over the long time scales
used in the setting of national exit levels. Definitive demonstrations of acceptable mass
balance are desirable, both for purposes of scientific integrity, and to promote confidence
in the 3MRA system within the stakeholder community. Such demonstrations should
include a suite of chemicals, particularly those that are highly partitioned between
different media, and may take several forms, including summative inter-media mass
calculations for the modules of 3MRA, comparisons among point estimates, comparisons
with TRIM-Fate (a compartmental model in which mass is conserved), and heuristic
calculations and arguments. The panel is aware that activities aimed at such
demonstrations are underway, and encourages the Agency to complete these and make
them public in a timely fashion.
The panel endorses the adoption of a Monte Carlo analysis (MCA) framework as
an appropriate tool to use in examining a wide range of site, chemical, and exposure
scenarios when setting national exit levels. The MCA provides an established science-
based process to allow the Agency to identify a range of exit levels at defined levels of
protection. In this manner, it provides useful results for risk management decision-
making that provides an approximate quantitative estimate of the degree of protection
(e.g., 99% of population, at 95% of sites as one example) associated with alternative exit
levels.
While the MCA is an appropriate and useful tool for identifying risk management
options to the decision-maker, it has important limitations. Even though the MCA results
provide quantitative estimates of the probability of protection, and thereby provide an
associated degree of "confidence," the implied level of confidence should be interpreted
with caution. The Agency has recognized that a quantitative evaluation of the uncertainty
of the variable (and uncertain) model input parameters (i.e., input sampling error, or ISE)
20
-------
is not feasible with available data. The panel agrees that such an analysis is impractical
for the complete MCA, but the panel does make specific recommendations for a "focused
ISE" uncertainty analysis (see question 2c). In addition, the Agency also recognizes that
the MCA does not address model error (ME), another significant source of uncertainty.
Perhaps the most complex issue that the panel has faced in evaluating the 3MRA
modeling system has been that of validation. 3MRA is a complex higher order model
that does not lend itself to traditional methods of validation, i.e. in die sense of data
matching. While such an approach can be achieved for some of the model components,
such as waste management unit and fate and transport models, it is not possible to
perform such a validation on the model as a whole for two reasons: (1) because a
complete dataset that stresses all seventeen of the sub-models simultaneously does not
exist and is unlikely to become available soon, and (2) because, ultimately, the purpose of
3MRA is to perform a national risk assessment. The Agency's approach to this has been
to develop a tiered validation protocol, based heavily on the work of Beck et al. (1997).
In this scheme, validation is seen as a design problem with several elements:
* Quality of input data (volume 2 -of the 3MRA material)
Quality of model components (volumes 1 and 3)
Quality of the modeling system (also in volume 3)
Performance of the model as a reliable instrument for its assigned task
(performance validity). Uncertainty and sensitivity analysis are central to the
concept of performance validity, as is comparison with other models (e.g., TRIM
fate), and matching against available but limited datasets (a chlor-alkali site).
These are the subjects of volume 4.
The panel believes that the protocol that the Agency has developed and is
following to gauge the acceptability of the 3MRA modeling system represents the state of
the art for evaluating complex regulatory environmental models. Validation is achieved
through completion of a series of well-defined tasks that must meet rigorous quality
assurance evaluations of their outcomes. This approach represents a shift away from
equating model validity with its ability to correctly predict the future, a future that in a
scientific and policy context is fundamentally unknowable, to a focus on the quality and
reliability of model forecasts (minimum risk of an undesirable outcome). In this context,
the Agency has described in detail the problem that needs to be solved (national risk
assessment), has designed a method for obtaining a solution (the 3MRA risk assessment
methodology), and has generated a "solution" (the 3MRA model system). At present
they are in the early stages of evaluating the performance validity of the modeling system
for generating reliable forecasts. Thus in terms of the steps above, they have
accomplished the first three and are engaged in the fourth.
The panel endorses the Agency's use of the Beck, et al, (1997) validation protocol
for evaluating the 3MRA modeling system. It represents a departure from traditional
notions of data matching as the only criterion, to an inclusive view of validation as a
process of model evaluation, rather than a state of model condition. This is a bold step,
but one the panel believes is appropriate, certainly for the national risk assessment
21
-------
objectives of 3MRA, but in a broader context, foi carrying the model evaluation debate
forward as it pertains to regulatory environmental modeling. The Agency has provided in
3MRA, perhaps the first case study of this model evaluation protocol. While it carries
some discomfort, e.g. limited data sets for module evaluation, and has been constrained,
e.g. inadequate resources for implementing important peer review suggestions, the panel
commends the adoption of this evaluation process for 3MRA, and urges the Agency to
continue with its plan for 3MRA modeling system evaluation.
It is clear to the panel that in each of the stages of model validation the Agency
set forth extensive quality assurance procedures that include consensus on the model's
intended use and performance criteria; incorporation, whenever possible, of legacy
models with which the scientific community has considerable experience; independent
peer reviews of model architecture and components; and verification of computer code
and inter-model communication. Thus in evaluating 3MRA, the panel has had to first
grasp the basis of the validation protocol, and then assess the degree to which the Agency
has achieved what it set out to do. The panel believes that the final stage of the Agency's
protocol, i.e., the performance evaluation, will be the most demanding and also the most
informative. The Agency is still engaged in this effort so it is premature for the panel to
make judgment about the ultimate acceptability of the modeling system at this stage.
However, the panel believes the steps identified by the Agency for accomplishing this
task are appropriate and the panel strongly encourages the Agency to continue these
efforts with particular emphasis on evaluating mass balance, completing the development
and application of the sensitivity analysis procedure, and continuing the data matching
and inter-model comparisons activities.
The issues raised above are addressed in greater detail in the responses to charge
questions 2,3, and 4 below.
3.1.2 Additional Comments about 3MRA
The 3MRA system is based on the concept of acceptable risk. As a result, the
model allows a contaminant to enter ecosystems, with some potential to adversely affect
ecosystems and human health. Further, the system is based on the concept that the
environment has an inherent assimilative capacity; that is, degradation, metabolism,
transfer, or storage of contaminants within or outside of a WMU will occur and, as a
result, will contribute to risk reduction.
The principal problem the designers of 3MRA set out to solve was that of the
migration of listed hazardous waste streams from RCRA subtitle C to "ground based"
subtitle D facilities (WMUs). This excludes other obvious waste management alternatives
by adopting a conservative view that encompasses a limited range of final disposal
options. This grows from the dependence on the inventory of candidate facilities dating
back in the decade of the mid 1980s, and reflects the underlying influence of the
Resource Conservation and Recovery Act on the motivation for developing 3MRA.
Indeed, the Agency has embarked upon a thought process to reconsider the basis and
22
-------
procedures of RCRA to make it more congruent with its original goal to encourage
recycling and reuse of materials (Office of Solid Waste, 2003).
Because the FRAMES architecture allows for plug-in applications to suit specific
needs, 3MRA can potentially have many other uses. The panel's view is that the present
assessment methodology overlooks at least five strategies for releasing a waste stream
from the rigors of Subtitle C: support for delisting of hazardous wastes, municipal waste
combustors, detoxification of wastes, and pollution prevention and industrial ecology
alternatives. By omitting such options, the 3MRA assessment methodology needlessly
restricts the decision-maker's thinking by offering only the five classes of WMUs
included in the simulation, when in reality the missing alternatives are readily
implemented and officially encouraged under available contemporary practices.
Appendix 1-3 contains more details about these options. The panel recognizes that the
3MRA modeling system has the capability of incorporating such alternatives, and
recommends that Agency regulatory strategies take full advantage of this capability.
3.1.3 References
Beyond RCRA: Waste and Materials Management in the Year 2020
(EPA530-R-02-009), Office of Solid Waste, U.S. Environmental Protection Agency,
April 2003.
Charge Question 2a. Does the 3MRA modeling system provide a tool for
performing national risk assessments that facilitates consistent use of the science
and provides a mechanism for reproducing results?
3.2 Panel Commentary
3.2.1 General Comments
The panel finds that the 3MRA modeling system produces internally consistent
and reproducible results. The 3MRA model development team has clearly succeeded in
developing a national risk assessment tool that facilitates consistent use of the science
incorporated in the 3MRA modeling system.
The panel recognizes that as with any model, the developers of 3MRA faced
difficult choices in balancing the degree of scientific sophistication in the models adopted
with practical real-world limitations due to computing power and data availability. The
extensive peer reviews of the 3MRA modeling components have made it clear that
scientific consensus about which models, modules or modeling components represent the
"best" state of the science for fate, exposure and risk analysis is difficult if not impossible
to fully achieve. Nevertheless, the panel believes that the choices made about the degree
of scientific complexity to include in 3MRA modeling system were consistent and
reasonable. Further, the modular "plug-and-play" design in 3MRA recognizes the fact
that science, data quality and computing power will continue to increase and unlike most
models, the 3MRA modeling system facilitates immediate access of new and/or alternate
23
-------
components so that the model can systematically move forward with the advances in
science.
As part of the model-model comparison process, one of the challenges in
producing scientifically sound results is assessing how well the seventeen science
modules that comprise 3MRA work together. As has been previously noted, not all
models have been in use as long as others and thereby have not undergone the same
degree of operational testing and peer review. The models employed by 3MRA range
from the simplest of "screening" models, to advanced regulatory guidance modeling
systems used for site-specific decision making, to more elaborate research-grade models.
Therefore, when called upon for inclusion in a nationally applied comprehensive
analysis, any of these models which are postulated as falling into a more advanced class
of detail and sophistication may actually require application as a simpler model, due to
the loss of key site-specific information normally relied upon to improve their scientific
representation of the chemical and physical processes addressed. The concern, therefore,
is that multi-part systems operate as efficiently and as effectively as their weakest
component.
In order to clarify the relative strengths of the submodels used in 3MRA, the
panel undertook an example model characterization/ranking exercise for 3MRA's readily
identifiable submodels. The exercise reinforced the impression of many panelists mat
one's assessment of a model depends on the end-use. EPA has represented that 3MRA is
currently being developed primarily for national regulatory policy analysis and
implementation by regulatory specialists. However, the charges to the 3MRA review
panel include a specific request for opinions and suggestions on "the best science" that
may be presently included or readily added in the near future.
As such, two distinct rankings were created by the panel for each of the models
employed by 3MRA with respect to the state of science (the scientist perspective)
embodied in the module, and the level of regulatory practice with which each module is
applied (the regulatory specialist perspective). A summary table, combining these
findings along with background and details of the exercise, is presented in Appendix 2a-
1, Table A2a-l.
While not a statistical assessment of model rankings (mere were insufficient
numbers of panelists), a pattern emerged when viewing the models from a regulatory
perspective: voters biased toward the right (i.e.t they tended to rank models as more
advanced) as compared to when panelists were wearing their "scientist" hat. From a
"best science" perspective, the models tended to be ranked as less sophisticated. This
qualitative exercise served only to highlight the challenge faced by the modelers when
using models at different stage of maturity and linking them together.
3.2.2 Consistency
Consistency of scientific approaches is difficult to attain because of the disparate
intrinsic time steps governing the chemical migration in different media; e.g., in air
24
-------
changes are tracked over hours, but in groundwater, where migration is much slower,
typical time frames of interest run into years. In a coupled multimedia-modeling system,
any choice of time step results in significant compromises. The developers attempted to
overcome this by altering the legacy models; for example, the sampling of hours in the air
model instead of using the entire hourly meteorological record set, as well as the offline
generation of short time-scale data (or intermediate calculations) to avoid rerunning the
fully expanded model algorithms hundreds or thousands of times to represent the
behavior of every single hour. For SMRA's "sampling" approach to use of
meteorological records as input for the ISCST3 air model, the EPA has presented results
of specific sensitivity testing that demonstrates the effectiveness of this new method for
reducing model run time without adding any significant margin of uncertainty to the
analysis. This serves as an example of the type of continuing effort that should be
employed to assess the additional uncertainties attendant to these necessary model
changes (operational compromises). Perhaps an unavoidable set of inconsistencies
occurs, because the degrees of advancement and validation differ widely among the
module algorithms (as illustrated in Appendix 2a-2).
3.23 Reproducibility of Results
The panel also believes that the 3MRA modeling system provides a mechanism
for reproducing results, particularly when used by trained technicians and scientists who
are familiar with the system. The panel recognizes that for every model run a very large
amount of information/data is generated, transferred and consumed by the various
modules in 3MRA and the panel commends the agency for the approach that they
developed using dictionary files of metadata to insure that various attributes of the
information being passed through the modeling system remain consistent and correctly
applied.
3.2.4 Potential Inconsistency in Model Uses
While the panel recognizes that 3MRA was developed specifically for Agency
use, we note that with significant expense and regulatory burdens at stake, stakeholders
will also seek to use 3MRA to confirm agency results, apply it for site-specific
assessments, and provide feedback regarding model assumptions and outcomes. Even
with the moderate amount of user guidance that is already provided in the documentation,
users who are new to the model will make mistakes.1 Thus, the greatest potential source
of differences and inconsistencies in modeling results will be due to mistakes made
during model setup and execution. This is not unusual for a model that is as complex as
3MRA, but to help minimize this type of inconsistency the panel recommends that the
revised user manual be explicit in describing the steps in conducting a model run, e.g.
clearly identify steps thai: are required versus those that are recommended. Furthermore,
the panel recommends that training workshop be developed and presented at select
1 As an example, refer to the "Second Evaluation of 2003 3MRA" prepared by AMEC where the modeler seemed to be
getting different results from run to run and it was later determined that the model user was not running the batch file to
clear results from previous runs before performing a new run. This inconsistency occurred even though the user was
experienced and had reviewed the user guidance material.
-------
scientific meetings (e.g., Society of Environmental Toxicology and Chemistry; Society
for Risk Analysis; Society of Exposure Analysis and Environmental Epidemiology, etc.).
The panel further suggests that the Agency should extend the existing benzene example
to several other chemicals and scenarios so that interested model users can start with a
realistic amount of information, setup and run the model, and then verify their results
against simulation outcomes provided by the Agency.
Charge Question 2b. Does the 3MRA modeling system provide decision-makers
sufficient flexibility for understanding the impacts on potential chemical exemption
levels by allowing varying measures of protection based on the number of receptors
and/or number of sites protected, types of human and ecological receptors, and
distance?
3.3 Panel Commentary
3.3.1 General Comments
The panel believes that 3MRA provides sufficient flexibility to model the local
impacts of waste management units with a reasonable level of detail sufficient for its
primary intended use - to develop national concentration thresholds for wastes that
would be exempted from the hazardous waste regulatory rigors of RCRA. The decision
as to the appropriate level of detail to include in a modeling effort is a trade-off between
increased complexity (and flexibility) and manageability of the modeling effort. The exit
level processors of 3MRA are especially important innovation that should assist the user
in interpreting results.
The 3MRA modeling system uses more than 700 variables to describe a site's
setting. These include human and ecological receptor locations and physical
characteristics of WMUs that are site-specific, as well as regional and national input
parameters relating to hydrogeologic factors, human exposure factors, etc. The spatial
input parameters included in the 3MRA modeling system consist of the location and type
of the waste management unit, the surrounding environment (including lakes, streams,
and wetlands), and the location and type of human and ecological receptors. The model
incorporates site-specific data on the location of human receptors and local land uses.
These are based on U.S. Census block data for residents and home gardeners, and urban
and agricultural census data for land use. Census data include the number and location of
households with private wells. These locations are included in the modeling effort. Farm
number and sizes are based on agriculture census data. In addition, every farm is
assumed to be on a private well. Census data provide estimates of urban, rural-farm, and
rural-non-farm recreational fishers as a percentage of the total state population. These
percentages are used to calculate the recreational fisher population within the study site.
Ecological habitats and receptors are based on site-specific land use. Fourteen
representative terrestrial, wetland, and margin habitats have been developed for use in the
3MRA modeling system. Ecological receptor species can be selected to represent
ecological regions throughout the United States. Other inputs used are watersheds and
26
-------
water sub-basins; local lakes, streams, and wetlands; and information regarding surficial
aquifers (unconfmed ground water sources near the surface) and vadose zone data. These
parameters provide considerable flexibility for modeling the migration of chemicals from
waste management units and their subsequent uptake by ecological and human receptors.
While 3MRA appears to incorporate significant flexibility in the derivation of exit.
levels, the panel cautions that this doesn't necessarily lead to an adequate understanding
of the impacts that these levels may exert on human health or ecological systems. It is
possible to incorporate a great deal of flexibility in the selection of protection levels,
whether based on the number or types of receptors, the number of sites protected, or
distance from the WMU. However, the selection of an exit level cannot be rationalized
by flexibility. Rather, it must be based on the adequacy of the underlying biology,
ecology, and toxicology and on an appropriate level of confidence that exit levels will be
fully protective of human health and the environment.
For the purpose of generating an exit level, a model user or risk manager must
select a suite of choices in the exit level processor, e.g. percent sites protected, population
protection, risk level, hazard quotient, receptor, cohort, pathway, radius of the area of
interest, etc. It would expected that by selecting, for example, 99% population protection
at a risk level of 10"* and a hazard quotient of 1, the processor would return an exit
concentration that would result in 99% of the selected receptors in the specified area of
interest having a calculated risk of 510*6 and a hazard quotient of <1 at the specified
percentage of sites. However, that would not necessarily be the case. Instead, those
selections would mean thai: 99% of the selected receptors in the specified area of interest
would have a calculated risk of <2.5 x 10"6 and a hazard quotient of
-------
may be under-protective in delisting of wastes. The use of average annual values tends to
"smooth" the estimated exposure concentrations used in the risk assessment.
Instantaneous exposure concentration estimates will vary around the mean in terms of
magnitude, duration, and frequency and may be high enough to cause acute or sub-
chronic toxicity even though the average annual concentration is below a level of
concern. Therefore, the panel recommends that the Agency consider and if possible
demonstrate, through appropriate analysis, that the approach implemented in the 3MRA
modeling system achieves the desired level of protection to ecological receptors.
Additional detailed technical comments regarding effects on ecological systems
and human health are provided in Appendix 2b.
3.3.2 Site Specific Use of the 3MJRA Modeling System
The panel believes that additional factors that should be incorporated into 3MRA
before it is applied to site-specific assessments. One exposure pathway that is not
considered, and for which the Agency and a number of States have begun to take into
account with respect to environmental impact, is volatilization of groundwater
contaminants into indoor air. Because 3MRA considers groundwater as a potential
source only when drinking wells are found to be in use, the drinking water and shower
inhalation exposures are the only resultant pathways considered. However, if
groundwater is impacted it is possible that even if it is not used as a source of potable
water, vapor intrusion can be a potential source of exposure. There are, of course,
numerous factors that contribute to the vapor intrusion pathway that will only add to the
complexity of the model. At the moment, it is unclear how to strike a balance between
comprehensive consideration of exposure pathways and modeling burden. At the very
least, this should be addressed in the text.
The 3MRA documentation clearly acknowledges that dermal exposure is not
considered in the model. Yet, efforts at EPA and elsewhere have been conducted since
1995 to assess and predict dermal exposure and its effects (its contribution to aggregate
or cumulative exposure and risk). Olin (1999) concluded mat "it is fairly easy to develop
estimates of body burden from a dermal exposure." The 3MRA system should be updated
to address this deficiency by including dermal exposure in the human exposure
component of the model before assessment of specific sites.
Charge Question 2c. Does the 3MRA modeling system provide appropriate
information for setting national risk-based regulations for the waste program?
3.4 Panel Commentary
The panel concurs that the Monte Carlo Analysis used in 3MRA is an established
science-based process that allows the Agency to identify a range of national exit
concentrations at specified levels of protection. In using this process, 3 MRA provides
useful information for risk management decision making in the form of an approximate
quantitative estimate of the degree of protection (e.g., 99% of population, at 95% of sites)
28
-------
associated with alternative exit levels. In addition, the panel wishes to emphasize that the
framework represented by 3MRA serves as an excellent foundation for integrating new .
science and information as it becomes available.
While the MCA is an appropriate and useful tool for identifying risk management
options to the decision-maker, it has important limitations. Even though the MCA results
provide quantitative estimates of the probability of protection, and thereby provide an
associated degree of "confidence," the implied level of confidence should be interpreted
with caution. The Agency has recognized that a quantitative evaluation of the uncertainty
of the variable (and uncertain) model input parameters (i.e., input sampling error, or ISE)
is not feasible with available data. The panel agrees that such an analysis is impractical
for the complete MCA, but does make specific recommendations for a "focused ISE"
uncertainty analysis (see below). The MCA also does not address variability/uncertainty
associated with the toxicity component of the risk analysis (a component of ISE), which
the Panel believes is a significant limitation in the 3MRA analysis. In addition, the
Agency also recognizes that the MCA does not address model error (ME), another
significant source of uncertainty. Appendix 2c-l provides greater detail on the use and
interpretation of MCA within the 3MRA modeling system.
The MCA provides an efficient mathematical means to iterate the model
outcomes many times to generate "probabilities" of associated outcomes, however these
probabilities must be inteipreted within the narrow context of the system that is modeled.
The magnitude of the uncertainties that are not modeled remains undefined. It is the
sense of the panel that these unaccounted for uncertainties may in fact be more significant
than the range of uncertainty currently modeled, in which case the probability of
achieving the desired levels of protection remain unknown. Furthermore, model
uncertainty is not accounted for in the MCA, and is unlikely that it can be addressed in
any quantitative fashion. As a result, it is potentially misleading to interpret the exit
levels as though they provide A% protection at G% of the sites with H% certainty
without carefully qualifying such statements. Without a rigorous sensitivity/uncertainty
analysis, the degree to which the exit levels are "protective" will remain unquantifiable.
While the panel supports the use of MCA in 3MRA, it is not convinced that the
procedures adopted in 3MRA represent a discemable "second dimension" of uncertainty
analysis, as the "pseudo 2-D" terminology implies. In the panel's assessment, the
regional and national data distributions do not represent only variability as suggested in
3MRA. Many fate, transport and exposure parameters, for example, are uncertain as well
as variable. Yet, the uncertainty in the selected distributions is not modeled or quantified
in 3MRA. Statements regarding the "confidence" or "level of certainty" in the outcome
can be misunderstood because the 3MRA analysis ignores significant contributors to the
overall uncertainty - ISE and ME (and uncertainty in the toxicity component which is a
component of ISE). The panel believes that a more accurate statement of the MCA
results would be:
"For the sites and conditions modeled, the exit levels ensure that no more than
A% of the population near a WMU will have a specified incremental human
29
-------
lifetime cancer risk above CR* or non-cancer Hazard Index above HI* (where
CR* and HI* are both Agency defined guidelines), and these levels of protection
are met for G% of the WMUs modeled. Similarly, for the conditions modeled, no
more than D% of the estimated exposure concentrations exceeds ecological
benchmark concentrations for G% of the WMUs modeled."
Accordingly, the panel has developed the following recommendations on the use
of MCA in the 3 MRA modeling system:
The panel recommends reconsidering the use of the "pseudo 2-D"
terminology. Given that the "two dimensions" of risk analysis in 3MRA
appear to revolve around dual protection metrics of population protection and
percent of sites meeting this population protection, the Panel suggests using
terminology that more accurately reflects these dual protection criteria (e.g.,
"Dual Population Protection" MCA). Furthermore, as described in response
the Charge Question 4, the panel recommends that the documentation
describing the Monte Carlo Analysis in 3MRA be significantly revised in
order to describe the analysis more succinctly.
It is suggested that the Agency modify the method of processing the MCA
results. The panel finds that the current approach lacks transparency, and
appears to discard valuable information. The panel understands that this
proposed change may impact the storage requirements for the MCA results,
depending on how the results are stored, and has offered a possible means to
address this issue. Appendix 2c-2 contains further details.
The "resolution" of the modeled Cw range should be addressed. In some
instances, two orders of magnitude separate Cw intervals, which will
inevitably lead to crude interpolation of the exit levels (with unknown biases).
One possible approach to do this would be to conduct an initial set of model
runs in order to determine an approximate Cw range for the exit
concentrations. Once an approximate concentration range is identified, the
range of Cw's within this narrowed range could be selected such that the
interval between each successive Cw is less than an order of magnitude.
Sensitivity analyses planned for 3MRA model/exposure parameters will no
doubt reveal those that have very large impacts on model output. The panel
recommends that the Agency conduct a 2nd order MCA analysis using a
manageable number of such key parameters (e.g., say less than 20 input
parameters for example), for a subset of chemicals, one or two WMU types,
and a reasonable subset of sites. Such an analysis would provide a more
quantitative assessment of the degree to which uncertainty in key input
distributions in turn impacts the exit levels. To be clear, the panel is not
recommending that this 2nd order, or 2-D, analysis be conducted on a
"national" scale but rather for a manageable number of sensitive model
parameters for which sufficient data (or professional judgment) allow for the
assignment of a PDF representing uncertainty. This analysis could be
conducted by conducting a "1-D" analysis for a particular WMU and a
-------
particular site using the current 3MRA distributions. The "focused 2-D"
analysis would apply to the same site/WMU, simply adding the 2nd dimension
uncertainty PDFs. If this were done for a small number of site/WMU
combinations, and on the order of three chemicals, the results would provide
insight into the degree to which input parameter uncertainty gives rise to
large, modest, or relatively small changes in the upper percentiles of risk for
the modeled scenario. This information would then provide a semi-
quantitative context for interpreting the degree to which the inability to model
ISE impacts the overall degree of "confidence" that the national exit levels are
"protective."
The panel recommends extending the MCA to include the uncertainty/variability
of chemical toxicity factors within the analysis. As has been noted previously, toxicity
parameters are treated as fixed when, in fact, they are both variable (not everyone's
threshold is the same) and uncertain (most criteria are based on laboratory animal data).
On the one hand, this has the effect of artificially narrowing the distribution of risk and
percent population protection. However, because the fixed values are upper-end
estimates, the distribution of risk versus probability is artificially shifted to the right.
Ideally, toxicity parameters should be entered as distributions, like other variable and/or
uncertain parameters. It should be a long-term goal to develop distributions for toxicity
parameters. While the Agency has indicated it does not intend to adopt such an
approach, at the very least, the documentation should make it clear that the risk and
hazard estimates corresponding to the exit levels are exaggerated on the basis of the
selection of high-end toxicity factors. However, even if the Agency will not incorporate
toxicity variability/uncertainty in the derivation of the exit levels, such an analysis could
and should be conducted on a subset of the chemicals that are being modeled as part of
the uncertainty/sensitivity (UA/SA) analysis. The panel recognizes that probability
distributions for toxicological parameters cannot always be characterized with confidence
and precision. Published studies describing possible approaches that could be considered
for this analysis are listed in appendix 2c-3.
Charge Question 3a. Is the software development and verification testing
approach implemented for the 3MRA modeling system sufficient to ensure
confidence that the modeling results reflect the modeling system design?
3.5 Panel Commentary
This question asks, whether the 3MRA modeling system code implements a
quantitative calculation that is consistent with the model conceptual design and whether
EPA has "verified" that the code computes what it is intended to compute. The panel
agrees that the Agency has made a significant effort to verify that the 3MRA modeling
system functions according to its design. The special attention given to the development
team communication and "top-down" code design, as well as conduct of QA/QC testing
according to a pre-planned testing strategy (as presented in Vol. 3 - Section 3.1 of the
documentation) are particularly notable. Also, the individual modules and the feed-
-------
forward connections between modules have been verified with respect to data and
information transfer.
Two advancements in model development and application that the agency has
made during the 3MRA development and verification process represent a major
contribution to the modeling community and warrant special commendation. First, the
use of the FRAMES architecture is a remarkable mechanism for making the model more
adaptable to future modifications, with less repetition of structural testing. Second, the
significant efforts that the Agency expended to develop hardware and supporting
software tools for the SuperMUSE windows-based parallel computing framework have
greatly facilitated the verification process, not to mention the capability to conduct
sensitivity and uncertainty analyses heretofore impractical with a model of this
complexity. Also, the fact that the SuperMUSE system is scalable to any number of
networked computers allows stakeholders with a range of available resources to conduct
national scale analyses with 3MRA.
There are two concerns relative to verification of the modeling system that the
panel would like to suggest be investigated during the ongoing sensitivity and uncertainty
analysis. As the current verification process now stands, each of the algorithms and
calculations for the human risk module have been checked with respect to their intended
functionality individually. The first concern arises out of a desire by the panel to have
data consolidation understood with respect to its impact on under- or over-estimating
risk. The ELP1 and ELP2 capture all the different combinations of human cancer risk
and non-cancer health impact producing up to 21,840 separate exit levels for a given
population percentile and percent sites protected. As pointed out in Section 4.6.2,
Volume 4, the decision-maker will need to narrow his or her focus to a smaller set of
national exit levels. Selection of the specific exit level construct or scenario is a matter of
policy. It is here that 3MRA output, in the form of preliminary analyses, can provide the
decision-maker with adequate background information to determine what the driving
concerns for each chemical and each WMU are. In 3MRA's effort to be manageable
with respect to data storage and run-time for PC-based applications, these preliminary
analyses are designed so the system can aggregate results into four composite receptor
categories (resident, resident gardener, fisher, and farmer) to develop cumulative
population frequency histograms and critical years (of maximum risk). 3MRA has a dual
classification capability and can provide risk/HQ information on individual pathways as
well as aggregation across pathways. The Tent max (for aggregated data) and Tent value
(for individual pathways) may not be the same values. The panel recommends that these
values be cross-checked to ensure that cumulative risk is being adequately captured. It is
conceivable that the "trueness" of risk of the specified percent population protected,
because it is population based, in actuality relies on the quality of the census data.
The second verification issue is related to the quantification of biases in model
results based on the propagation of module assumptions/limitations (i.e., process/loading
assumptions, module structure, etc.) through the system. Limitations and potential biases
of individual modules have been qualitatively described for each module and in some
cases the direction and approximate magnitude is presented. It would be desirable to
-------
attempt to make estimates of biases for all modules more quantitative. Also, it is
important for model developers to estimate how the module biases are propagated
through the integrated system. In other words, does the risk bias inherent in the known
module limitations tend to accumulate (positive or negative direction) or do some biases
increase risk while others decrease it, thus offsetting each another? This concern could
be evaluated through an appropriately designed sensitivity analysis.
Charge Question 3b. EPA has implemented thorough evaluations using the
available data resources and technologies, while also recognizing the real world
limitations that apply to validating the 3MRA modeling system. Have we
reasonably demonstrated through methodology design, peer review, quality control,
sensitivity analyses, and model comparison, that the 3MRA modeling system will
produce scientifically sound results of high utility for use in multi-media regulatory
applications?
3.6 Panel Commentary
3.6.1 General Comments
This question deals with the ability of the 3MRA modeling system to reproduce
actual system responses relative to risks to human and ecological receptors caused by
chemical releases from WMUs. This capability is necessary for decision-makers to
confidently use the model to inform decisions regarding national-scale, contaminant-
specific solid waste risk assessment. In recognition of the virtual impossibility of
conducting a traditional validation for 3MRA, the panel agrees with the adoption by the
3MRA modeling team of the validation protocol suggested by Beck et al. (1997) for
evaluating higher order models such as 3MRA. This protocol recognizes other means of
model performance evaluation. It includes two basic model features: 1) assessment of the
theoretical and numerical construction of the model (essentially an internal measure of
validity); and 2) the demonstrated performance of the model in terms of its design
purpose (essentially an external measure of validity). The Beck et al, (1997) paper also
correctly contends that the question of "model validation" in the context of predicting
risk to natural environmental systems does not really have a "yes or no" answer. The
best we can do is gain sufficient confidence in the model to support decision-making in
the specific domain for which it was developed. Part of the "internal" measure of model
validity rests in its design and code verification, which has been dealt with in question 3a
above. With regard to the other aspects of model validation, the panel believes that the
Agency has made every attempt during the developmental phases of 3MRA, given the
limitations imposed by available resources and programmatic considerations, to follow
the set of principles for validation discussed by Beck, et al. (1997).
In spite of the panel's general agreement with the process being followed for
3MRA validation, there are several concerns and recommendations relative to what has
been done and what is ongoing with regard to this question.
3.6.2 Model-Data and Model-Model Comparison
33
-------
The panel believes that careful comparison with as many actual datasets as
possible, even if they are incomplete, is an important way to build confidence in use of
the 3MRA modeling system for its stated purpose. The panel applauds the effort that has
been initiated for mercury at a former chlor-alkali site, but encourages the location of
additional, albeit partial, site-specific datasets involving other contaminants that can be
used for model-data comparison. Perhaps a good source of such datasets would result
from a post auditing of sites where 3MRA has been applied and a decision has been
made. Of special concern are data that stress the source modules of 3MRA. Also, waste
disposal sites for which data are available on a single major exposure pathway would
provide a useful field test for 3MRA. Such an approach is useful for determining biases
of 3MRA and its databases, and will assist in the interpretation of results. The more of
these data comparison exercises that EPA can perform, the more transparent the model
will become and the more confidence the public will have that 3MRA produces
reasonable predictions of risk on a national basis.
The chlor-alkali facility appears to be one of the few cases where a site exists with
available information for both model-model and model-data comparisons. The panel
supports the efforts of the Agency thus far in comparing 3MRA with TRIM.fate and with
site data on mercury exposure and effects. However, it is apparent that the model to data
comparison at the chlor-alkali site is not expected to be very meaningful given the long
(and unknowable) history of releases from the site, and the fact that mercury can be
transported long distances in the environment making it difficult to determine the original
source loading to the area.
Making model-model comparisons and model-data comparisons and explaining
differences using knowledge of model process formulations and assumptions is another
valuable exercise in building confidence in 3MRA. For example, in the 3MRA -
TRIM.fate comparison the 3MRA modeling system takes a linked media receptor-based
approach while the TRIM.fate model takes a folly coupled media area-based approach.
Recognizing these model differences provides a unique opportunity to test and compare
some of the underlying assumptions that these models are based on. For example, the
higher concentration of mercury in soil worms in 3MRA versus TRIM.fate at the chlor-
alkali site might be explainable by the deeper mixing (and thereby more diluted
exposure) of surface soils in TRIM.fate. The agency should endeavor to understand other
comparison differences, such as the surface water mercury concentrations and chemical
speciation.
Regarding the appropriate chemical space over which 3MRA should be tested for
multimedia modeling performance, the panel would certainly recommend retaining the
metals and pH dependent chemicals that are currently among the 46 constituents selected
to develop and test the 3MRA modeling system. However, among organic constituents,
it is important to have chemicals representing the four general areas of solubility
parameter space. One approach for describing this solubility space is illustrated in Figure
3-1 where a set of 300+ chemicals are plotted based on their octanol/water partition
coefficient and air/water partition coefficient. Regions of the plot are identified where
34
-------
chemicals partition primarily into a single media (air, water, solid)2. The fourth
dimension that should be considered when constructing a test set of chemicals for a
multimedia model is environmental persistence, The panel notes that the existing set of
chemicals in the dataset that is planned for use in testing the model is clearly biased
towards chemicals that partition into the air. The panel recommends that the existing set
of chemicals be augmented to more fully represent both the potential solubility space and
environmental persistence for organic chemicals; and especially recommends adding
chemicals that do not partition greater than 90% in any single medium.
The panel recognizes other model analysis efforts to address various validation
questions that have arisen. For example, the panel acknowledges EPA's analysis of
ISCST3 with regard to its sensitivity to distance from the source. This was an attempt to
address the question regarding implications of the 2 km boundary for site definition. The
analysis showed that annual average vapor concentrations approach zero within a range
of 1-4 km from the edge of each of 20 HWIR sources tested at 10 sites. However, the
panel cautions that 3MRA will not capture the cumulative risk potential of chemicals
with long-range atmospheric transport potential. While the panel recognizes that the 2
km radius was chosen largely on the basis of limiting the cost of data acquisition, it is
recommended that the Agency identify those classes of chemicals for which other tools in
addition to 3MRA may need to be applied to assess cumulative risks at longer transport
distances.
»>90%AIR
°> 90% AQUEOUS
> 90% SOLID
- Multimedia
3MRA data set
13 5 7
bg octanol/water partition coefficient
2 The idea of using solubility space to evaluate multimedia behavior comes from Wania, F. (2003) "Assessing the
Potential of Persistent Organic 'Chemicals for Long-Range Transport and .Accumulation in Polar Regions*' ES&T.
2003,37,1344-1351 although earlier papers and authors have also used this approach.
35
-------
Figure 3-1.
Example of solubility parameter space and bow it influences partitioning .in the
environment (generated with the CalTOX model). Region A of the plot includes
chemicals that partition primarily into air. Region B includes chemical that
partition primarily into water. Region C includes chemical that partition to solids
(soil or sediment) and Region D includes chemicals that partition into multiple
environmental media. The plot also shows the current list of 3MRA chemicals (solid
dots).
While the preliminary results of the ongoing model-data and model-model
comparison exercise seem promising, the panel recommends that EPA look for
opportunities for additional such validation exercises in order to provide increased
confidence in using 3MRA. In looking for additional comparisons, the panel
recommends selecting simpler site layouts so that a better site characterization is
available. In conducting these exercises, the panel urges that quantitative criteria for
evaluating the performance of 3MRA be established. For example, "satisfactory" model-
data comparisons might mean that model state variables are within a factor or 2-5 of field
observations. Finally, the panel also suggests that any model-model comparison exercise
should also include a comparison of model sensitivities for each of the estimation
endpoints.
Given the complexity and broad scope of the 3MRA framework and the resulting
difficulty in performing traditional data-matching validation of the model prior to using it
as a management tool, the panel suggests that a complimentary approach be considered,
i.e. to ask if the use of the model leads to "correct" management decisions. In fact, this
assessment is consistent with the principle stated by Beck, et al. (1997) that the best
validation of a policy model is "whether the model can perform its designated task
reliably, i.e., at a minimum risk of an undesirable outcome." Given the long history of
RCRA and Subtitle C (> 25 years), it may be possible to pose questions where the
answers are actually knowable. For example, are there chemicals where a consensus on
exit levels has been reached through some other process? Of course, if these chemicals
are not already in the 3MRA database, it would be necessary to determine their physical-
chemical and toxicity properties and potential source rates so that they could be modeled
by the 3MRA system to compute exit levels.
Another possible confidence building exercise would be to show that the relative
partitioning of chemicals into different environmental media predicted by 3MRA is
reasonable. This can be accomplished by comparing 3MRA model results with those
obtained using fugacity models.
3.6.3 Conservation of Mass
An important internal measure of model validity is its ability to conserve both
water and chemical mass. As noted in charge question 1, the panel agrees that
precautions have been taken to ensure that mass is conserved within the individual
36
-------
modules of 3MRA. The panel notes in particular that mass is conserved within the
source modules, a response to previous criticism.
Still, the panel is concerned that feedback (secondary) sources of contamination,
which are not presently modeled within 3MRA, may result in significant mass
imbalances for certain chemicals. The panel recognizes that 3MRA contains strictly
feed-forward transport of mass from one module to the next. This assumption, although
it reduces runtime, may be invalid for certain chemicals and certain media configurations.
For example, in 3MRA volatilization loss from surface water or terrestrial plants is a loss
of mass from the entire system rather than a contribution to the air compartment. At the
same time, volatilization loss is not balanced by gas phase absorption (volatilization is
treated effectively as a first-order loss rate based on the single medium concentration). It
must be demonstrated using mass conservation analyses that computing air-water or air-
plant exchange on the basis of the inter-media concentration gradient (this would require
some level of module coupling that allows inter-media feedbacks that are time-
dependent) is not a necessary part of the 3MRA system for all chemicals of concern. For
example, the screening calculations of air-leaf transfer of chemicals conducted by
Ambrose (personal communication, November 20,2003) does a reasonable job of
demonstrating that this partitioning does not impact the air compartment mass balance by
more than 5%. The panel is appreciative of this sort of analysis, and recommends
continued thinking along these lines to address mass balance questions.
Another mass balance concern in 3MRA deals with flic assumption that biota in
the system are not included in the mass balance for a given medium; that is, chemical
taken up by biota in the food web bioaccumulation modules for water and land becomes
an unaccounted-for loss from the entire system. If, in reality, this is a significant transfer
of mass, then 3MRA is failing to balance mass by whatever fraction of the total mass is
entering the biota. This would be a fairly simple assumption to check; at a given point in
time one could multiply the 3MRA-computed biota chemical concentration by the model-
assigned biomass for that organism within a given media segment (or segments). One
would then compare this mass with the mass of chemical (computed in the same way) in
the abiotic portion of that media segment. Probably the simplest media-biota check
would be to look at fish in surface water segments.
3.6.4 Peer Review
The panel finds that the level of peer review that the individual science modules
received has been impressive, however it is noted that many of the concerns identified by
the reviewers, although acknowledged by the Agency, have not been implemented in
3MRA. The panel recognizes that resource limitations may have prevented the Agency
from implementing many of the peer review suggestions. We further concur that overt
model errors identified through peer review have been corrected through the verification
process. Still, concerns persist about the continued implementation of reviewer
suggestions such as those related to the GSCM module, secondary sources, and human
and ecological exposure issues.
37
-------
With regard to the GSCM the panel recognizes the key role it plays in three of the
five 3MRA source modules. The land-based source modules that use the GSCM are the
following WMUs: landfills, waste pits, and land application units. Those not dependent
on the GSCM are surface impoundments and aerated tanks. For the 3MRA predictions to
be reliable these five source modules must accurately launch the chemical masses on
their multi-pathways through the ecosystem delivering quantities that eventually impact
biological targets. The panel is concerned that this module, which is not a legacy model
with a long history of peer review and field-testing, has not received the same degree of
scientific scrutiny as other modules. Of particular concern with this model are its method
of mass transfer from wastes to environmental media (air and vadose zone) and the
assumption of local equilibrium among phases.
Given that the GSCM is relatively untested and has some potential theoretical
inadequacies it is incumbent upon the agency to demonstrate that the GSCM affords a
level of accuracy commensurate with the legacy models being used in the 3MRA system.
This analysis should be done using data matching under a wide range of conditions and
chemicals, comparison of results with more robust models, and theoretical or error
analysis. The LAU Module, which contains GSCM, has been compared to experimental
data obtained on organic chemicals during application of municipal wastewater onto soil
(Schmelling et al. 2003). Five factors were tested: volatilization, first order chemical
decay, appropriateness of the quasi-analytical solution, LAU thickness, and temperature.
The volatilization rate was reported to be in the "right order of magnitude" for all
categories of compounds. However, for the highly volatile chemicals the model was
consistently lower than observed. In another validation exercise for the LAU module,
measured half-lives of dioxin in sewage sludge were compared. Remaining
concentrations at equivalent human health risks were calculated for the LAU in order to
estimate the half-lives. Results were stated as: "The range of half-lives over the selected
percentiles was 20 to 48 years, which is in reasonable agreement with the observed half-
lives at several monitored sites." However, the numerical range was not reported; and the
number of monitoring sites not in agreement was not reported. The panel also recognizes
the model comparison effort between GSCM and MODFLOW-SURFACT that has
recently been made by EPA (December, 2003 communication). Several insights were
gained through this exercise, including an explanation for why GSCM gives higher
volatilization fluxes in the first several years of a simulation - it solves for fluxes
sequentially with volatilization computed prior to leaching. While these validation
exercises are the type of activities that the panel believes are needed for GSCM, the panel
recommends a more rigorous model-data validation analysis, perhaps using well-
controlled lysimeter experiments, that identifies the conditions (chemical and site-
specific) under which GSCM fails. As part of this analysis, quantitative criteria need to
be established that are consistent with those used for assessing the WMU legacy models.
If the GSCM does not meet these criteria, the panel strongly recommends that the model
be revised to remedy the shortcomings. The outcomes of this validation process and
model revisions should be documented.
A more detailed discussion, review, and recommendations for improvement of the
GSCM module are presented in Appendix 3b.
38
-------
3.6.5 Sensitivity Analysis
The panel believes that sensitivity analysis is a critical part of the validation
process for 3MRA. Although a model sensitivity analysis is not a direct measure of its
reliability, there are indirect indicators of problems with model validity that can be
obtained through a sensitivity analysis. The sensitivity analysis results can identify and
prioritize areas of concern that require model modifications or additional data collection.
For example, a model whose predictions vary greatly in response to minor changes in key
parameters, especially parameters with high uncertainty, would be a cause for more
detailed investigation of its validity. Although the Agency has demonstrated their
understanding of existing techniques, the documentation fails to adequately explain - or
demonstrate through an illustrative case study - how an actual sensitivity analysis of a
3MRA application will be performed and how the outcome will be presented to the
decision-makers and stakeholders. The panel strongly encourages the Agency to
complete a sensitivity analysis in 3MRA that covers the chemical space displayed in
Figure 3-1 so that each "national assessment" for the list of chemicals for which data are
available can be accompanied by a sensitivity analysis using a stochastic basis for risk
parameters.
In summary, the panel agrees with and commends EPA's defined series of model
validation tests that 3MRA must pass prior to acceptance for use in the HWIR national
risk assessment. The panel recognizes that completion of the very important performance
validity tests is ongoing. Indeed, the panel understands that models and sub-models are
continually evolving and improving as we expand our knowledge base and acquire new
data, made easier in 3MPA by its innovative design and construction within FRAMES.
In this regard, the panel recommends use of 3MRA for its stated purpose of supporting
the establishment of national exit levels pending completion of the Agency's planned
performance validity tests. However, the panel strongly recommends that the 3MRA
modeling team continue to evaluate and upgrade the 3MRA model and to clearly
communicate the results of this work to the stakeholder community along with
commensurate statements regarding model limitations and caveats for its use.
3.6.6 References
Ambrose, R.B., Jr. "Screening Analysis of Air to Canopy Pollutant Mass Balance,"
personal communication to the 3MRA panel, November 20,2003.
Beck, M.B., J.R. Ravetz, L.A. Mulkey and T.O. Bamwell. 1997. On the problem of
model validation for predictive exposure assessments. Stochastic Hydrology and
Hydraulics. 11:229-254.
Schmelling, S., M. Wang and K. Liu. 2003. Proceedings The Air and Waste
Management Association: National Mtg., Washington, D.C.
39
-------
Charge Question 4. Is the documentation for the 3MRA Modeling System
adequately designed and prepared? Does the SAB have additional suggestions for
improving the presentation of the comprehensive set of materials related to this
modeling system?
3.7 Panel Commentary
3.7.1 General Comments
In general, the panel finds the documentation for 3MRA (five volumes) to be well
presented, and reasonably well organized. Several panelists with familiarity with the
'earlier 1995 HWIR documentation that preceded 3MRA have indicated that the 3MRA
documentation is a significant improvement over the HWIR materials prepared in 1995.
It seems clear that many earlier criticisms about the clarity and completeness of the
deficiencies in the HWIR documentation have been taken as constructive criticism by the
EPA authors.
Given the challenging volume of material included in the 3MRA modeling
system, it is generally readable if taken in modest doses. The level of detail provided
helps the reader to understand both the strategic thinking that went into its planning,
development, and testing. The organization of the material, with detailed tables of
contents, makes it relatively easy to limit reading to the subjects of greatest concern. The
document also provides numerous helpful graphical depictions of model components to
aide the reader's understanding of this complex modeling system.
For the reader who is deeply interested in the model framework, the development
and verification history, or the specific modeling algorithms used in the 17 simulation
models, the documentation is well designed. However, for those most interested in
applying the model, the current documentation could be improved. The panel
recommends reorganization and revision with respect to the need for a readable
summary, improved clarity of terms (especially those related to the treatment of
uncertainty in 3MRA), and more concise descriptions of databases. In addition, for users
of the model (beyond the model developers), a revised User's Guide should be
considered that better focuses on implementing the model and processing the results,
rather than the model theoretical framework.
There are several areas that require clarification, and even significant revision, in
order to make the 3MRA documentation clear, transparent, and more understandable in
order to facilitate stakeholders outside the Agency in their ability to understand and run
the model. The panel provides the following general recommendations.
3.7.2 Recommendations
The panel recommends that the Agency develop a more "digestible" summary
that describes the 3MRA in more understandable terms. The sheer volume of material,
40
-------
combined with technical jargon covering many disciplines, makes for a "dense" read for
even a technical audience.
The concept of "population protection" is central to the development of national
exit levels, yet this concept does not receive adequate attention in the documentation.
The document should provide a very simple and clear example of how population
protection is calculated, using a graphical depiction, and then how these results for a
particular site/simulation are "rolled up" into a calculation of the percent sites protected.
As it stands, the documentation tends to describe the mechanics of the approach relying
on 3MRA "jargon" (such as ELPI, ELPII, RSOF, etc."), which does not provide the
typical reader with an intuitive understanding of the approach. The document should
provide very concrete examples of a probability distribution of population protection with
well-labeled graphs, using an example of a single chemical/concentration simulation.
The figures in Volume 1 describing these concepts are not clearly labeled and do not
convey the concept of population protection adequately. Not only does population
protection (and percent sites protected) require better definition, at present the text gives
the impression that 3MRA will provide outputs of the "nationwide distribution of risks"
for receptors (see, for example, pp. 1-15). Yet, it is the panel's understanding that the
risk results themselves are not stored in 3MRA, and instead "risk bins" are used to
estimate the distribution o:f nationwide WMU sites that achieve a specified value of
population protection (in question 2c, the panel recommends that 3MRA actually store
the calculated risk outcomes at each site.)
From public meetings/conference calls with the Agency, the panel understands
that there are internal "check points" in the 3MRA calculations such that particular sites
are "excluded" from the percent population protected calculations if an adequately sized
human population does not exist within the radius of interest examined in 3MRA (the
Agency indicates that all sites have ecological receptors within the specified area of
interest). The documentation should include a discussion of this, and also indicate for
how many site/WMU combinations this in fact occurs. The documentation should be
very clear in terms of the minimum threshold population size within the radius of interest,
or within census blocks if that is how the calculations are performed, that is required to
be included in the 3MRA percent population protection and percent sites protected
calculation.
As noted previously, the number of operational input parameters that go into the
3MRA is very large, such that it is quite difficult for someone not fully versed in the
mode) details to grasp those that are based on empirical data, those that are based on
professional judgment, and those that are "operational" assumptions. It appears there are
key variables that are based on operational assumptions (for example the "fraction
hazardous waste," or fwmu, term) that are not clearly articulated in the documentation. It
is essential that EPA summarize these more concisely, perhaps developing a parameter
matrix and categorizing them as suggested here (empirical, professional judgment, and
operational), in order to provide a more intuitive understanding of how the operational
parameters influence the model formulation and results.
41
-------
The discussion of uncertainty, variability, and sensitivity concepts relating to the
Monte Carlo analysis (Volume IV) fails to explain adequately what was done to address
variability and uncertainty in the 3MRA. The explanations in Volume IV require
significant revision in order to make the actual Monte Carlo Analysis implementation of
the 3MRA understandable and transparent (see Appendix 4-1 for additional suggestions).
In addition, for reasons further explained in question 2c, the panel recommends that the
document avoid the "pseudo 2-D" terminology used in reference to the MCA, as the
terminology invites misinterpretation of the results.
The panel encourages the Agency to continue development of mechanisms for
meaningful interpretation of model output, currently underway for 3MRA version 1.x.
While the Agency has indicated that Version 1.x provides the ability to store detailed
output results for some of the underlying governing model parameters, only the Agency
has access to the version 1 .x "tools." The panel believes it is incumbent upon the Agency
to complete the version 1 .x tools prior to adopting 3MRA for site-specific applications, as
without these tools, the public is left without the ability to examine the results in a
meaningful fashion.
The adoption of the FRAMES modular architecture with the inclusion of legacy
modeling codes, offers the flexibility to "swap out" modules as improved models/data are
available. In addition, the means of passing input and output parameters from one
module to another in the form of SSF and GRF files is one of virtues of the
3MRA/FRAMES construct. As recognized by the Agency, this structure offers the
flexibility and ability to turn certain modules "on" or "off' and even substitute certain
components (e.g., source terms) as information allows. While the panel strongly
endorses this architecture and believes the Agency should be commended for its design,
at present the documentation necessary to take advantage of this flexible design is
insufficient for anyone but the model developers. The panel therefore encourages the
Agency to provide adequate documentation of (1) how to substitute or turn off certain
modules, and (2) provide a more detailed data dictionary and description of the SSF and
GRF data files. Without these two critical elements, it is at present infeasible for the
public and stakeholder community to harness or test these elements of the 3MRA.
The panel is confused over the Agency's use of the term "screening level" with
respect to 3MRA. Is this meant to convey that the mathematical models within 3MRA
are considered "simple" based on conservative assumptions? Alternatively, the term
screening level in the 3MRA context could be viewed instead as a collection of models
(of varying complexity) that are assembled in 3MRA for excluding ("screening") listed
hazardous wastes. It is recommended that the 3MRA documentation provide a clear
definition and consistent use of this term.
The remaining panel comments on the 3MRA documentation are provided as
"specific comments" in Appendix 4-1. Several that are not specific to a particular
volume are presented first, followed by comments that are directed toward
recommendations for specific volumes. Appendix 4-2 contains a candidate outline for a
possible revision to the 3MRA User's Manual. Finally, while it was not the panel's
42
-------
intent to review the document in terms of style, grammar, or typographical issues, to the
extent we have input, these comments are noted in Appendix 4-3.
43
-------
APPENDICES
Appendix 1-1
Exposure Duration
Exposure duration is input as a constant value in 3MRA, neither variable nor
uncertain. This fixed value fails to capture the upper end of the distribution. For
example farm families in particular often spend their whole lives at one residence. This
lack of variation may distort the output distribution, making it appear narrower than it
really is. This may tend to underestimate risk or hazard at the upper end of the
distribution. Since 3MRA addresses farmers specifically, a distribution of exposure
duration applicable to farm families should be used or, at a minimum, the general
exposure duration distribution should include farm families. Inclusion of upper end
values will ensure that subpopulations that that tend to reside in one location for an
extended time are protected.
Appendix 1-2
Correlated Variables
Several exposure parameters in 3MRA that co-vary with body weight are treated
as independent. This makes the exposure appear more variable than it really is. For
example, respiration rates are not normalized to body weight, nor are they apparently
correlated to body weight. This means that body weights and respiration rates are
selected randomly and independently from their distributions for each model realization.
The result of this approach is that the largest adult (660 Ibs) is just as likely to be paired
with the minimum breathing rate (1 m3/day) as with the maximum breathing rate (50
m3/day). This is also true for the smallest adult (33 Ibs). This means that the breathing
rate can cover the implausible range from 3.3 L/kg/day to 3,300 L/kg/day. (The
occurrence of such implausible combinations can be studied by storing individual
iterations of the model Ranking them according the resulting risk or hazard and
examining the results in the tails of the distributions.) Ideally, respiration rates would be
expressed as a function of body weight to the 0.7 power. Other examples of variables
incorrectly treated as independent of body weight include fish consumption and drinking
water consumption. Again, the problem is similar: when the exposure to contaminants in
the fish or in drinking water is expressed in mg/kg/day, the range of exposure rates is
exaggerated because the model allows, for example, a 10 kg child to eat 1500 g fish or
drink 2100 ml of water per day.
The panel recognizes that extremes in the tails of the distributions are not picked
very often, so absurd pairings will be rare. Still, 3MRA exposure factors are based on
sources (e.g., the 1997 Exposure Factors Handbook) that were not designed with
stochastic analysis in mind. If mid-range body weight and breathing rates for an adult are
chosen for a deterministic analysis, then it does not matter if normalization to body
weight is done or not. But when parameter values are allowed to vary within their
ranges, it is essential to control how they co-vary. The assumption of independence is
not justified in this case.
-------
Appendix 1-3
Alternative Waste Management Options
' The focus in 3MRA on only land-based facilities has some intrinsic limitations;
for example, the landfill prototype and the surface impoundment WMU are put down on
native material without the benefit of liners, gas collection systems or leachate treatment
systems. It is unclear to :he panel how many facilities fitting these descriptions are still
allowed to operate even considering the range of regulatory oversight under state
jurisdictions, but in any case the panel feels that some representative range of modem
technology should become available to the 3MRA user. This becomes especially
important as the uses of 3MRA are extended to include evaluation of individual facilities
(for example for delisting purposes), or designs for proposed facilities. In addition, the
consequence of allowing relatively unsophisticated protective designs is the influence on
exit levels, which may turn out to be unrealistically restrictive thereby defeating one of
the major purposes for developing 3MRA, the perception of overly stringent regulation.
In order to treat the case of a modern Subtitle D landfill, there will need to be sub-models
to determine gas generation, collection and utilization, contaminant levels in fugitive
emissions of uncollected gas, pollutant partitioning in cover soil, and failure mode
analysis of these processes.
The panel's view is that the present assessment methodology overlooks at least
five strategies for releasing a waste stream from the rigors of Subtitle C: support for
delisting of hazardous wastes, municipal waste, combustors, detoxification of wastes, and
pollution prevention and industrial ecology alternatives. By omitting such options, the
3MRA assessment methodology needlessly restricts the decision-maker's thinking by
offering only the five classes of WMUs included in the simulation, when in reality the
missing alternatives are readily implemented and officially encouraged under available
contemporary practices. These alternatives are amplified below.
An immediate application of 3MRA would be to support de-listing petitions. For
this use it needs to be set up in such a way that site-specific data can be readily entered to
supplement the existing databases, and enough iterations run for a single site to give
reproducible results. Beyond this, there is potential for assessment of risk at
contaminated sites, and risk-based support of permitting decisions. Because the
FRAMES architecture allows for plug-in applications to suit specific needs, it can have
many other uses. The panel supports the use of this model with some enhancements for
the intended purpose and its continued development for other applications.
Considering that, after materials recovery, 21% of US municipal waste is handled
by municipal waste combustors (MWCs), it is surprising that this diversion alternative is
not included in 3MRA. Preliminary studies have suggested the favorable feasibility of
destroying household hzizardous waste in MWCs considering the temperature-time
characteristics of MWC furnaces Very effective destruction and removal efficiencies are
available in the MWC even for such refractory compounds as CFCs. In its present form,
the 3MRA modeling system has all of the modules needed to assess risks of air emissions
and ash disposal from MWCs, and sufficient data exist to support a combustion
-------
alternative that can be evaluated on a national scale. Emission rates and configuration
parameters (e.g., stack characterization: height, diameter, temperature, velocity and base
elevation) are available for the US population of MWCs, although the range of receptor
domains would need to be enlarged for each source because of emissions at higher
elevations. All of the algorithmic mechanisms for handling deposition and indirect
pathways are already embodied the present version of the 3MRA so that only the source
and receptor files need to be modified.
A second type of WMU alternative that 3MRA might address are pollution
prevention/detoxification schemes involving stabilization of a hazardous waste in a
product stream. An example of this is the exemption of petroleum coke quenched with
oily refinery sludges from the standards, record keeping and labeling requirements of
RCRA. Since the early 1970s some refineries have blended API separator sludges, tank
bottoms and biological solids in the water stream used to cool petroleum coke at the end
of a delayed coker cycle. Presumably contaminants such as metals or polycyclic
aromatic hydrocarbons bind to the carbonaceous substrate of coke particles. This
technique has been embraced by the European Commission in its catalog of Best
Available Technologies, but it is doubtful that any occupational or community health risk
assessment was ever performed. Evaluation of exemptions such as this should have a
clear place b the 3MRA assessment methodology. A generic module with adjustable
input/output structure might be contemplated in anticipation of problems like this. As
further encouragement to the user, some synthetic case studies might be packaged in with
the software as a means of demonstrating the flexibility of 3MRA.
The third example is taken from the industrial ecology field: the use of a waste
stream from one process as a raw material for another. Many industries, long accustomed
to seeking out profits from their own waste products, have embraced this concept
enthusiastically. There are numerous present-day examples of such approaches ranging
from the use of more environmentally benign materials in processing, pollution
prevention, products designed for reuse, and materials recycling. Such approaches are
viable when they retain embodied value within the manufacturing system, or when they
help in the avoidance of waste management costs. On a larger scale, the co-location of
compatible (in terms of waste reuse) industries in eco-industrial parks has gained wide
recognition as a viable waste management option in many areas, including the United
States. A recent report by the SAB has documented the advantages of the industrial
ecology approach, and summarized research needs in this area (Thomas et al, 2003).
Flexible access to a tool such as 3MRA, suitably modified to incorporate industrial
ecology strategies, could be quite useful in quantifying comparative risk among
traditional and forward-looking management strategies. A synthetic demonstration study
would encourage users to consider other possible waste management scenarios.
-------
Reference
Thomas, V., T. Theis, R. Lifset, D. Grasso, B. Kim, C. Koshland, and R. Pfahl (2003).
Industrial Ecology: Policy Potential and Research Needs. Environmental
Engineering Science, 20(1): 1-9.
Appendix 2a-l
Classification of 3MRA Submodels
The 3MRA review panel and the Agency both recognize that it is common risk
assessment practice to consider a series of graded steps or "tiers" to distinguish between
models used to make judgments about output information applied to environmental risk
analyses. These range from the simplest of "Screening" models to advanced regulatory
guidance modeling systems used for site-specific decision making to more elaborate
research-grade models. The panel also recognized that the use of the terminology "tiers"
could often mean different things to different model users depending upon the intended
application. For instance, those who are most concerned about the use of the "best
current science" might have a different perspective about the needs for relative ease of
use, understandability to the broader public, or verification history than someone who is
tasked with using the information to formulate policy. The regulatory officials are
charged (and challenged) with making credible decisions in an "immediate" time frame,
based on modeling results likely to have a reasonable level of acceptance by the
stakeholders in the pending decision.
A further challenge arises when multiple models, of varying complexities, are
linked together, as is the case with 3MRA. Often, a multi-part model may be only as
strong as its weakest component in terms of the validity, reliability, and reproducibility of
its output. Therefore, to help clarify understanding of the 3MRA system of models, the
panel undertook a simple characterization exercise. Panel members were asked to rank
the relative level of sophistication and validation experience observed for each of
3MRA's sub-modules, considering both the "state of science" embodied in the module
and the level of "regulatory practice" with which each module has been applied.
To support this effort, outlined below are some common characteristics used to
classify models according to their varying "Tiers" of sophistication, and demonstration of
performance acceptability by those in the regulatory and risk analysis research
communities. Initially, two separate model characterization matrices were proposed for
panel members. Each of these tables contains the same set of models and Tier
classifications to register panel member opinions. However, one version of the table was
to characterize the rela;ive Tier levels on the basis of "best science" employed by the
individual components; while the other table was to characterize how the individual
components typically are used in the regulatory arena.
Any such listing is necessarily affected significantly by the experience of the
authoring group. For that reason, the panel has made this model ranking demonstration a
-------
"group effort" to ensure that they would reflect some of the diverse experience of panel
members. As this model characterization effort developed, it became evident that the
detail level presented below in order to help standardize the assessment framework could
only be maintained when a member was intimately familiar with a model's design and
application history. Therefore, most participating panel members simplified their
representation of the detailed characteristics and presented assessments that clearly
categorized a submodel as falling into one, or possibly two Tiers. After the assessments
were submitted, the panel's task force editors tallied the {sometimes split) votes and
further simplified the presentation of results to present them in a single tabulation (Table
A2a-l). At the end of the table are listed the three versions (1.0, 1.X, and 2.0) of the
entire 3MRA modeling system that the EPA now has under development. Without
presenting details of version differences in this appendix, the assessment of panel
members is presented to reflect their overall opinions of the Tier-status of each of these
versions. As noted in the table, the assessments for "best science" are labeled as "s", and
for "regulatory application" as "r". To give readers a sense of the "mode" of their
distributions, these symbols are capitalized when two or more total assessments
(summing split and whole assessments) were tallied for a particular Tier classification.
This model classification effort commenced early in the panel's review efforts.
As the review continued, a number of the steps already taken by the Agency earlier in its
3MRA development planning became more evident. The panel appreciates the very
significant progress already made by the Agency in reaching the present level of 3MRA
development. The panel trusts that the characterization scheme identified here will not
only have been useful to the panel in its own deliberations about the state of 3MRA
development; but may also assist the Agency in establishing priorities for the continuing
that progress as it strives to make the best use of the present version(s) of this landmark
risk assessment modeling system.
The 3MRA Submodel Classification Process
Beyond the simplicity of the present characterization, the models/modules of
multi-pathway risk modeling systems range widely in their level of sophistication and
applicability. They may start with a very simple screening model (Tier 1) used to make
initial "back of the envelope" estimates. These Tier 1 models predict chemical
concentrations in various media for instant comparison with benchmark concentration
levels or risk levels; and they are often based on use of generic national or "worst case"
input parameters.
At the next, more refined level, (Tier 2), more sophisticated and complex
models/modules consisting of "stand-alone" environmental release, dispersion,
environmental transport and fate, and exposure/uptake/risk models are frequently used by
the EPA for advanced screening assessments, or basic site-specific studies. Finally, for
advanced risk assessments (Tier 3), usually performed on a local or site-specific basis,
some of the same models, employing more representative local (or regional) input
parameters will be applied to more advanced risk assessment studies. For many of the
most sophisticated models, however, the detail and quantity of input data may be
-------
demanding. In these latter cases, the unavailability of representative and detailed local
data can make a Tier 3 model inaccurate and ineffective; and in those cases, either the
Tier 1 or Tier 2 models may be preferred. That is why many of the Tier 1 and Tier 2
models are most often broadly recommended by regulatory agency "guidance."
Initially the pane" also discussed the potential role of research-grade models, of
which there are many, to allow analysts to address special chemical fate and transport or
environmental transformation and uptake processes. Although these models can be
considered when needed to address information gaps that are recognized in the 3MRA
system, particularly mocel verification comparisons; the selection, application and
interpretation of this last set of modeling tools relies heavily upon professional judgments
made by the model users, most often for site-specific rather than national "screening"
contexts. (The panel has seen EPA's consideration of the many alternatives in its careful
selection of a set of models that will, in the Agency's view best meet the regulatory
application objectives established for the 3MRA system).
For these reasons, the following outline defines, for the present 3MRA submodel
classification context, a set of characteristics that the panel members have used for
determining the herein-defined "Tiers." (Note that, for considerations of "best science",
these Tiers necessarily differ from the Tier systems specified in various regulatory federal
and state agency guideline documents; but for the current listing of "Regulatory
Application" tiers, they could be tailored in the future to match a particular reference
guideline document if desired). As defined below, Tier characteristics are separated into
traits that are usually important for judging the level of achievement/applicability based
on "be.st science" or "regulatory application." In all cases, it is assumed that the output
information is either exposure point concentrations in various media, or the risks
calculated by immediate application of widely published risk (or Hazard Quotient/Index)
factors to these media concentrations.
TIER I - SIMPLE SCREENING MODELS
Qualitatively
(a) Box Models - based upon experience with more refined models applied to a range
of site specific cases, but in present case applied to a geographical, physical, or
chemical enviranment setting about which very little information is available.
(b) Environmental Compartment (e.g., fugacity) Models.- basic versions, when
sufficient information about application environment is known to derive estimates
of potential fluxes between compartments, but scale of application may lead to
very low precision due to inclusion of numerous regimes or ranges of variation of
key variables for the environment of the particular screening application.
(c) Simple Dispersion or Linear Transport Models - based on models with either
(1) modest history of application and little verification/validation data; or (2)
better history of validation and acceptance, but applied in a situation in which key
-------
underlying theoretical assumptions are known to be violatedmaking the
representativeness/accuracy/precision historically associated with the model all
highly suspect.
(d) Advanced Dispersion or Complex Transport and Fate Models - based on use
of scientifically advanced (perhaps newest) models that include much of the
known science on transport and fate processes, but applied in situations for which
many key model input parameters must be based on "generic" default values
because no comparable data are available to represent/describe the particular
location of the application. [For example use of national average values for all
input parameters without any information available to judge how to potentially
"adjust" the results to the specific situation(s)].
(e) General Concerns: (1) validation/verification history of less concern at this level
due to inherent questions raised by magnitude/impact of simplifying assumptions;
(2) reproducibility is enhanced by simplicity of formulation; (3) history of use,
especially in regulatory context, can add importantly to credibility of screening
decision outcome, even at a Tier 1 level of application.
Quantitatively
(a) Typically deterministic "poinf-estimate models may provide estimates for a
number of case-specified locations or situations, but normally do not include any
stochastic sampling routines/elements.
(b) May be "pre-solved" lookup tables based on distillations of analysis results from
families of existing or hypothetical situations. (E.g., engineering nomographs for
range of typical situations).
(c) Large Margin of Uncertainty due to anticipated wide range of variability in model
input data and large uncertainties about model representativeness. (Depending on
application uncertainty may be 10 to 10,000-fold).
(d) Use in decision-making virtually always "one-way": If results, with consideration
of all conservatively-biased input data estimates, indicate "no possible problem",
then no further analysis is necessary and results are considered "acceptable."
(e) Simplicity ensures quantitative reproducibility and easy understanding of
limitations of application results.
TIER II - REFINED SCREENING MODELS
Qualitatively
-------
(a) Box Models - advanced versions, more in the sense of their calibration or
verification history and their representativeness of the particular
application/location; may also be derived from statistical (e.g., epidemiological)
studies in which the coefficients for the particular application have been estimated
from an empirical study that include relevant parameters.
(b) Environmental Compartment (e.g., fugacity) Models,- more advanced
versions, when sufficient information about application environment is known to
derive estimates of potential fluxes between compartments, and scale and
specificity of application allows "reasonable" (moderate) precision due to
availability of environmental media concentration data {supplemented by
"proven" models) and limitation to known regimes or ranges of variation of key
variables for the environment of the particular screening application.
(c) Simple or Advanced Dispersion or Linear Transport Models - based on
models with "solid history" of application, including moderate to good
verification/validation data; but applied in a situation in which key underlying
theoretical assumptions are known to be reasonably applicable, or at least not
seriously violated, so that the representativeness and accuracy/precision
associated with the model are more acceptable to both scientists and regulators.
(d) Advanced Dispersion or Complex Transport and Fate Models - based on use
of scientifically advanced (including perhaps the newest) models that include
much of the known science on transport and fate processes, but applied in
situations for which many key model input parameters can be supported by a
combination of parameter values demonstrably specific to the application, with
modest reliance upon "generic" default parameter values when necessary. [For
example, replacement of national average values for all key input parameters with
regional or local (site-specific) values to potentially "adjust" the results to the
specific situations)].
(e) General CoQcerns: (1) validation/verification history of more concern at this
level, but due tc inherent questions raised by magnitude/impact of simplifying
assumptions; and the typical "one-way" decisions made, not necessarily crucial
(2) reproducibility may be a significant issue for the more complex models, as
more technical skill and frequent use of "professional judgment" is required; (3)
history of use, especially in regulatory context, can also add importantly to
credibility of screening decision outcome .
Quantitatively
-------
(a) Typically deterministic "point" estimate models may provide estimates for a
number of case-specified locations or situations. However, more advance
screening models but may include a stochastic "shell" sampling routines/elements
to enhance analyst's ability to evaluate uncertainty of model predictions by
allowing repeated simulations to vary input values over anticipated ranges that
could be applicable to the particular situation of interest. [Normally Monte Carlo
or Latin Hypercube sampling is performed on a One-dimensional basis at this
level and thus does not attempt to separate effects of parameter "variability" from
measurement or estimation "uncertainty"].
(b) Unlikely to be "pre-solved" lookup tables based on distillations of analysis results
from families of existing or hypothetical situations, unless the method is
supplemented by algorithms that can be evaluated with site-specific information
to over-ride the original "generic" results. (E.g., engineering nomographs for
interactive range of specific situations).
(c) Reduced Margin of Uncertainty due to better-understood and more limited range
of variability in model input data and smaller uncertainties about model
representativeness. {Depending on application uncertainty may still be 10 to
1,000-foldindividual media concentrations may be within 5 to 500-fold).
(d) Use in decision-making usually "one-way": If results, with consideration of all
conservatively-biased input data estimates, indicate "no likely problem", then no
further analysis is necessary and results are considered "acceptable".
(e) Greater complexity suggests specification of benchmark problems or test cases to
be run by model user to ensure quantitative reproducibility and to promote better
understanding of limitations of application results.
TIER III - ADVANCED MULTIPATHWAY RISK iMODELS
Qualitatively
(a) Box Models - advanced versions may still be used to fill in gaps not addressed by
other modules. In this case, simplicity and direct specificity (and perhaps
comparison to site-specific data gathered for the purpose) for the particular
application are essential to replace the absent calibration/verification historyto
demonstrate representativeness and likely accuracy/precision for the particular
application/location; may also be "supported by" data/results derived from
statistical (e.g., epidemiological) studies in which the coefficients for the
particular application have been estimated from an empirical study that include
relevant parameters. In general, usually replaced or augmented by more complex
deterministic and/or stochastically enhanced modules.
(b) Environmental Compartment (e.g., fugacity) Models - maybe compared with
more detailed deterministic model results by regulators, but less likely to serve as
-------
primary tool for regulatory decisions, except at the global, or perhaps national
policy level. In this latter situation, the most advanced model versions (along
with their validation history for situations in which they have been optimally
applied and compared with available measurement data) would be used to judge
whether the predictions of future balances between environmental compartments,
and estimates of potential fluxes between compartments, are credible for the
application of interest.
(c) Simple or Advanced Dispersion or Linear Transport Models - based on
models with "solid history" of application, including previous regulatory
application precedent and moderate to good verification'validation data. Once
again, acceptability depends upon model application in a situation in which key
underlying theoretical assumptions are known to be reasonably applicable, or at
least not seriously violated, so that the representativeness and accuracy/precision
associated with tne model are more acceptable to both scientists and regulators.
[Often the use of a more advanced model, especially one which is based on
stochastic procedures for producing distributions of outcomes, will be required to
be compared with results from a simpler "legacy" regulatory model to provide a
regulatory context that is best understood by all stakeholders in the particular
regulatory decision].
(d) Advanced Multi-pathway Dispersion and Complex Transport, Fate, and
Exposure Models - based on use of a combination of scientifically advanced
(including perhaps the newest) models that include much of the known science on
transport and fate processes, but applied in situations for which many key model
input parameters can be supported by a combination of parameter values
demonstrably specific to the application, with modest or minimal reliance upon
"generic" default parameter values when necessary. [For example, replacement
of national average values for all key input parameters with regional or local (site-
specific) values to potentially "adjust" the results to the specific situation(s), to
the limits of application-specific data].
(e) General Concerns: (1) validation/verification history of highest concern at this
level, but due to inherent questions raised by the complexity of the combination of
models/modules it is likely (as stated in the present 3MRA documentation) that it
may never be possible to test all of the components of any available multi-
pathway environmental risk assessment model.at least, not while they are
functioning in an integrated manner, unless a major verification experiment is
undertaken with that goal as the organizing principal for the experiment. The
extraordinary cost and complexity of such an undertaking would suggest that it
would likely take at least several years to accomplish. In the meantime, the
regulatory agencies have many risk management decisions requiring decisive
action in a shorter time frame. Thus, in the interim the best situation that may be
achievable would be that resulting from critical review of the currently available
modeling tools, with an ongoing commitment to a maintaining a "best practical
application of the best science" culture; (2) reproducibility will certainly be a
10
-------
significant issue for the most complex models, as much technical skill and
frequent use of "professional judgment" is required (see need for benchmark
testing, below); and (3) history of use, especially in regulatory context, but also
with independent verification of important results by other stakeholders will
determine the ultimate credibility of risk management decisions based on
employment of these most complex regulatory modeling tools .
Quantitatively
(a) Typically deterministic "point" estimate models may provide estimates for a
number of case-specified locations or situations. However, the most advanced
multifunctional multi-pathway models like 3MRA will increasingly rely upon the
supplementary perspectives provided by the ability to properly apply stochastic .
"shell" sampling routines/elements. Eventually the availability of 2-D (and
perhaps 3-D) stochastic modeling routines as shells will not just provide an
important "diagnostic" tool for the model development team. Instead, these tools
will also allow the well-trained user to demonstrate the differences between
uncertainties that are due purely to "variability" and the relative magnitude of the
residual "uncertainties due to measurement or estimation or model
representativeness (from 3-D tests).
(b) Potential to reduce, or at least better diagnose, Margin of Uncertainty due to
better-understood and more limited range of variability in model input data and
smaller uncertainties about model representativeness. (Depending on application
residual uncertainty in components of primary interest may still be 10 to 1,000-
foldindividual media concentrations may be within 3 to 100-fold).
(c) Use in decision-making is improved for drawing more reliable conclusions about
alternative situations that yield competitive risk results; comparisons can suggest
which of potential future situations is likely to be associated with reduced risk to
the designated parties. (E.g., more confidence (with quantitative statement of
level) that one result differs by a quantifiable magnitude from the alternative).
(d) Greater complexity suggests specification of benchmark problems.or test cases to
be run by model user to ensure quantitative reproducibility and to promote better
understanding of limitations of application results,
Detailed Model Characteristics to Consider for Tier Selection
Table A2a-l presents the combined matrix for both the "Best-Science" (S) and
"Regulatory Application" (R) attributes for the 3MRA submodels, and the entire model
(three versions). Capitalization of symbol S or R indicates that 2 or more panel members
voted for the particular classification of the sub-model. Lower case = < 2 votes, some
members split votes. This summary tabulation notation differs from the detailed notation
method initially recommended to Panel members for the first draft of the model
characterization/ranking procedure. That initial procedure was quite detailed (see below),
and then streamlined slightly to make it easier for those relying on 3MRA documentation
-------
as much as their own experience to draw conclusions on model features. These detailed
attributes are the descriptors of process simulation details included in each model.
I) PI) = Process Understanding (detail)
2) SR = Spatial Resolution (1-D, 2-D, 3-D)
3) TS = Temporal Structure (static equilibrium or dynamic)
4) MS = Mathematical Sophistication (e.g. box, multiple layer, Gaussian or
Lagrangian model)
5) H = History of Use (validation, verification, and regulatory use)
6) ID = Input Data (age, quality, & representativeness of information)
7) UR = Uncertainty of Results (model output)
Initially, a blank "A" and "B" versions were circulated to panel members to
acquire separate votes for 'Best Science" and for expected best "Regulatory Practice",
respectively. The Agency has represented that 3MRA has been developed primarily for
national regulatory policy analysis and implementation by regulatory specialists.
However, because the utility of this set of modeling tools may be attractive for more
extensive applications, some future users are likely to be by non-Agency personnel, and
the context may be different. Thus, the 3MRA review panel was charged with a specific
request for suggestions on the "best science" that may be presently included or readily
added in the near future. (It was assumed that some of the descriptors might not be
applicable in both matrices, and panel members only voted on models for which they felt
they had enough information to support an opinion).
Streamlined Alternative Model Ranking Scheme
Many panel members considered the use of the mnemonic (two letter) coding
scheme identified above for all 7 listed characteristics a bit too complicated for the
current demonstration exercise, and the following improved summary method was
proposed. As noted above, all votes were ultimately translated into integrated scores that
could be summed and represented by either an "S," "s," "R," or "r" symbol on the final
Table A2a-l, with capitalized symbols indicating two or more total votes for the
particular table entry.
Tier Ranking
Level:
Process
Understanding
Space, Time
And Structure
Tier I
Low
Steady State,
Equilibrium
Single Box.
Mathematical
Level
Algebra,
Deterministic (no
Tier II
Medium
Space, or. Time
Resolution,
Multiple Boxes.
Ordinary Diff. Eqns.
Some Statistics
Tier UI
High
Space and Time
Resolution
Continuous Coord.
Partial Diff. Eqns.,
Multiple Statistics
12
-------
History of
Use/ Acceptance
statistics)
New and Untested
(e.g., 1-D)
Emerging, Incomplete,
Promising
(e.g., 2-D)
[Time tested, Much
used, Extensive
validation
Key: Enter alphabetic codes that correspond to the attribute that exists for the
model at the corresponding tier level. Some models can span more than one
tier, depending on the layers and corresponding level of sophistication. See
below for further notes on definition of the streamlined attributes for model
characterization:
P = The level of Process understanding attribute ranks the degree/state of
scientific development of the combined physical, chemical and / or biological
mechanisms within or across the natural media systems.
ST = The Space fx. v & z\ time ft) and structure attribute ranks the degree to
which the state variable(s) need to be described in the respective media in
order to capture the most realistic behavior patterns of the processes.
M = The Mathematical level attribute reflects the types of coupled
deterministic and stochastic algorithms that are needed to capture the process
space, time and structure elements in order to realistically quantify the state
variable(s).
H = The History attribute reflects an integrated, weight-of-evidence ranking
that combines the time-period of use, data/validation issues and
consensus/acceptance of the final algorithm.
Results and Conclusions
Review of Table A2a-l indicates the diversity of opinion among the participating
panelists. Due to the small number of individuals casting opinions for any individual
model, this may not be too surprising. The panel members were selected by the EPA
SAB to represent a diverse set of experienced scientists who have experience with either
model development or model application. Most often panelists have modeling
experience primarily in subject areas most closely associated with a particular
environmental science, or with the biological aspects of risk assessment, without equal
experience in other specialty topic areas, thus the sum of the team responses often
includes between two and five individuals with special expertise in any given model or
submodel topic area. This limitation of sample size must be remembered when reviewing
the overall results.
-------
The results in Table A2a-l indicate a few general trends, in spite of the small
sample sizes. Although there are a few elements in which the panelists agreed that the
science was already effectively at an "advanced" Tier 3 level, the majority of the
transport models varied between Tier 2 and Tier 1. Generally, the opinions on
"regulatory applicability" and "best science" did not differ very much. One minor trend
was noted: the "center o:;" mass" of the R, r scores tended to lie slightly to the "right"; that
is, toward the more advanced analysis cases, than the S, s scores. More noticeable,
however, was the fact that "best science" scores were higher (more advance) for the
models that were devoted to characterizing basic chemistry, air modeling, and basic
hydrological processes compared with more complex exposure modeling and risk
assessment details. That trend seemed consistent with many of the panel discussions in
meetings. The results, perhaps supplemented by further EPA analysis of submodel
precision and accuracy, may help the EPA decide on which improvements in the science
may offer the greatest enhancement in ultimate accuracy and precision of the final risk
estimates. In turn, these continuing improvements will positively affect the value of the
other statistical decision-support parameters calculated by 3MRA.
TABLE A2a-l
Tier-Ranking Matrix for 3MRA Submodel Assessment
3MRA Constituent Models:
Tier!
Simple
Screening *
Tier II
Refined
Screening *
Tier III
Advanced
Risk Analvsis *
* S = best science; R = good for regulatory applications
Capitalization or symbol "Si** or "R" indicates that 2 or more panel members voted for the
particular classification of rhe sub-mode!. (Lower case = < 2 votes, some members split votes).
Chemical Prooerties Data:
SPARC
MfNTEQ 2
Sources:
Wastewater Modules:
(SI and AT)
CHEMDAT8
EPACMTP
Land-based Source Modules:
(IF. WP.andLAU)
GSCM
Local Watershed Model
Participate Emissions Model
Hydrology Model
Regional Baseflow
(STORET-30Q2)
Media Fate & Transport:
s
s
S
s
s
s
s
r
s, r
S, r
r
s, r
r
S,r
S,r
s,r
S
S
s
R
s
S
14
-------
3MRA Constituent Models:
Tierl
Simple
Screening *
Tier II
Refined
Screening *
Tier III
Advanced
Risk Analvsis *
* S = best science; R = good Tor regulatory applications
Capitalization of symbol "S" or "R" indicates that 2 or more panel members voted for the
particular classification of the sub-model. (Lower case = < 2 votes, some members split votes).
Air Module:
ISCST3
Enhanced by: area sources, SCIM input,
dry/wet deposition, terrain & new plume
depletion.
Watershed Module:
GSCM
30Q2
MUSLE
Surface Water Module:
Waterbody Network
EXAMS [I
Vadose Zone & Aquifer Modules:
BPACMPT
1-D Vadose Zone Model
-3-D Aquifer Mode!
Food Webs
Farm Food Chain Module:
Air-to-Plant
Plants in Soil
Beef and Milk
Terrestrial Food Web Module:
Soil Concentrations
Plant Concentrations
Soil Invertebrates
Vertebraie Prey
Aquatic Food Web Module:
Food Webs for Waterbodies
Dietary Matrix
Concentrations in Food Web
Consumed Fish (Concentrations)
Exposures (Doses}
Human Exposure Module:
Ingestion of soil
Ingestion of groundwater
Inhalation of shower air
Inhalation of volatile emissions
Inhalation of paniculate
Ingestion of produce
s
s
s
s
s
s
S
s,r
s
S,T
S
S
s
s.r
s
!
s, r
S,r
s, r
s
S, R
S,R
S, R
s
s
s
s
S.r
S
r
s, r
s, r
s
s, r
s.r
s,R
r
s,R
s, r
S.R
S
r
S,T
S,r
S.r
S.R
R
S
S
S
S.R
R
R
R
15
-------
3MRA Constituent Models:
Tier!
Simple
Screening *
Tier II
Refined
Screening*
Tier III
Advanced
Risk Analysis *
* S = best science; R - good for regulatory applications
Capitalization of symbol "S" or "R" indicates that 2 or more panel members voted for the
particular classification of the sub-model. (Lower case = < 2 votes, some members split votes).
Ingestion of beef & milk
Ingestion of fish
Breast Milk Exposure to PCDD/F
Ecological Exposure Module:
Direct Exposure: surface water
Direct Exposure: sediment
Direct Exposure: soils
Indirect Exposure: surface water
Indirect Exposure: sediment
Indirect Exposure: food
Risks/Hazard Indices
Human Risk Module:
Risks
Weighted Population Risk
Hazard Index
Ecological Risk Module:
Benchmarks
Srressor Limits
Hazard Quotients
3MRA
VERSION 1,0
VERSION I.X
VERSION 2.0
s
S
s, r
s, r
s, r
S, R
S,R
S, R
s, r
s, r
s, r
s
s,R
s
S
S,r
s
r
s,r
s
s, r
r
s, r
s,r
s,R
R
r
r
s
S
S,r
Appendix 2a-2
Comments on Embedded Assumptions ami Default Values
The level of peer review that the individual science modules received while the
model was under development was impressive and the table of comments that the review
produced represent a significant resource and guide for continued development and
enhancement of the model. It is unfortunate that time and resources did not allow more
of the comments to be explored in more detail and implemented where appropriate. The
panel recognizes the need for making tradeoffs or simplifications during the model
development process but we caution that some of these simplifications may lead to
unexpected inconsistencies in model performance.
For example, there are currently two alternate methods used in 3MRA to estimate
the concentration of constituents in aboveground vegetation, Cp. A deposition-based
approach is used for chemicals with log Kow less than 5 and a partitioning-based
16
-------
approach is used for the more lipophilic compounds. The partitioning-based approach
uses an empirical air-to-plant bio-transfer factor (ChemBv) to relate the concentration of
the constituent in air to that in aboveground vegetation. As discussed in Volume 2
Section 10.0, there are currently three chemicals classified as "special chemicals" and of
these only Benzo(a)pyrene is assigned an empirical ChemBv value (4.7E+4). The other
two special chemicals, Bis (2-ethylhexyl) phthalate and Dibenz (a, h) anthracene, are
assigned ChemBv values of 1 because, as the documentation points out, "There are few
experimental determinations of ChemBv."
The panel is concerned that arbitrarily assigning a value of 1 for the ChemBv of
lipophilic compounds can lead to inconsistencies in the modeling results both across
chemicals that have similar physicochemical properties and even from run-to-run for the
same chemical if uncertainty in Kow is considered. To illustrate this point, consider that
Bis (2-ethylhexyl) phthalate has a log Kow reported in the 3MRA documentation of
around 4 but other sources put the values above 5. Thus, it is conceivable that a user
might select a log Kow that is on either side of the threshold that is used to change
calculation methods. It is also feasible that a distribution might be assigned to this input
that crosses the threshold.
If a log Kow value of around 4 is used then the model would default to the
deposition method (Eq. 10-2). Using forage as the vegetation type and assuming wet
deposition is negligible for lipophilic chemicals as stated in the first paragraph on page
10-7 then the resulting Cp is about 4*Cva. (Note that Cva is the vapor concentration in
ug/m3 and Cp is in mg/kg[DW]). If the model user selects a log Kow value of 5 and the
ChemBv of 1 is used then the model would default to the bio-transfer factor approach
(Eq. 10-3) resulting in a Cp of approximately 8.4E-4*Cvap. Further, if the ChemBv was
assigned a value similar to that of BaP (4.7E+4) then the resulting Cp comes back up to
approximately 40*Cvap. Thus, there is potential for almost 5 orders of magnitude
variation in Cp cause by a relatively small change in Kow. It is not clear how this might
influence the overall model outcome because the chemicals in question may not be
present in the vapor phase of the air but this would need to be confirmed using sensitivity
analysis.
Another imbedded assumption that may lead to discontinuities in the results can
be found in the calculation of bio-concentration factors for milk and beef (ChemBa), The
values are calculated using an empirical function on Kow but the calculation defaults to a
value of 1 outside the range of the original data (see Figure A2a-2-l). The panel
appreciates the concern for not wanting to extrapolate beyond the range of an empirical
relationship but simply defaulting to a value of 1 may not be appropriate. For chemicals
with log Kow between 1 and 3 or between 6 and 7, a small change in Kow can result in a
significant change in the resulting ChemBa value.
17
-------
risk in the model, and ultimately by incorporating a more thorough assessment of
cumulative risks from multiple chemical exposures in the model.
Finally, the 3MRA system appears to discount cancer risk where exposure occurs
only for a portion of the lifetime. However, less than full lifetime exposure should not be
discounted for some chemicals such as vinyl chloride, particularly where exposure occurs
in childhood. Ginsberg (2003) has developed a list of chemicals for which this approach
is appropriate and a thorough discussion of why this approach is appropriate. The 3MRA
system must incorporate flexibility that allows the calculation and use of non-discounted
cancer risks for vinyl chloride and other chemicals as appropriate.
References
Brungs WA, TS Holderman, MT Southerland. 1992. Synopsis of Water-Effect Ratios for
heavy metals as derived for site-specific water quality criteria. U.S. EPA contract 68-
CO-0070.
Ferenc, S,A. and J.A. Foran. 2000. Multiple Stressors in Ecological Risk and Impact
Assessment: Approaches to Risk Estimation. SETAC Press. Pensacola, FL. 251pp.
Foran, J.A. and S.A. Fersnc. 1999. Multiple Stressors in Ecological Risk and Impact
Assessment. SETAC Press, Pensacola, FL. 100pp.
Ginsberg, G.L. 2003. Assessing cancer risks from short-term exposures in children.
Risk Analysis, 25:19-34.
Olin, S,S. 1999. Exposure to contaminants in drinking water: Estimating uptake
through the skin ana' by inhalation. ILSI Press. Washington, D.C. 232pp.
Suter, G.W., A.E. Rosen, E. Linder, and D.F. Parkhurst. 1987 Endpoints for responses
of fish to chronic toxic exposures. Environ. Toxicol. Chem. 6:793-810.
i
Spehar RL, AR Carlson. 1984. Derivation of site-specific water quality criteria for I
cadmium and the St. Louis River Basin, Duluth, Minnesota. Environ. Toxicol. Chem.
3:651-665.
Appendix 2c-l
Discussion of 3MRA Monte Carlo Analysis
The 3MRA system relies upon a Monte Carlo simulation framewotk as a critical
component of developing chemical-specific national exit levels. As defined in Volume 4
of the 3MRA documentation, the general national risk assessment problem statement is
designed to establish exit levels such that a given exit level (Cw, EXIT) will:
1. Protect A% of the human population (within specified distance and at
defined risk/hazard thresholds).
20
t
-------
*
2. Protect D% of the ecological habitats {within specified distance and
defined hazard quotient).
3. Provide these levels of protection at G% of the facilities nationwide.
4. Provide H% confidence that the exit levels accomplish the stated goals
(while also minimizing any simulation uncertainty (1%) that arises as an
artifact of the number of Monte Carlo iterations performed).
In order to develop exit levels meeting these criteria, the 3MRA implements a
series of multimedia deterministic source and transport/fate modules and
human/ecological exposure models, combined with a Monte Carlo sampling/simulation
strategy that draws samples from distributions of input parameters used by the fate and
risk modules. The input parameters modeled as random variables include those that
describe the physical and operational characteristics of the WMU, local hydrogeology
and habitat, human and ecological population characteristics affecting chemical intake
and exposure, etc. The panel believes that the MCA as structured is capable of
addressing the first three of the above-posed objectives (with reservations that many
sources of variability/uncertainty are not characterized and setting aside issues relating to
model validation), but the MCA is more limited in its ability to provide quantitative
statements regarding the degree of confidence in the results (i.e., the fourth objective
above).
Given the complexity of the MCA, it is helpful for this discussion to render a
simplified diagram of its basic components.3 Figure 2c-l-l provides a depiction of the
MCA in 3MRA using for illustration a single chemical for a single initial chemical
concentration in the waste (Cw).4 Fundamentally, the MCA contains two sampling
"loops" from which model parameters are drawn:
1. Site-Based Parameters: The site-based parameters are defined by a sample
of 201 Subtitle D waste disposal facilities included in the 1985 Westat
survey. In the case of the Landfill WMUs used in this illustration, the
sample size is N=56 landfills. Input parameters describing site-specific
WMU characteristics (e.g., size of WMU, loading rates, etc.) are drawn
from this site based database. Additional site-based information includes
the population within 2-km, local watershed and habitat information, etc.
At any particular site, these parameters are considered fixed or "constant,"
whereas the "variability" across WMUs arises from the empirical
differences in site conditions from one site to the next throughout the U.S.
3 The Panel's interpretation of the basic structure of the MCA was developed from reading Volume 4 and from
discussions during several "fact-finding" calls with the Agency. The basic structure as depicted here was confirmed to
be a correct interpretation of the MCA as implemented during the public meeting with the Agency that took place
October 28 - 30. This figure is a modification/simplification of Figure 2-10 in Volume 4.
4 The full MCA in 3MRA simulates a range of five C» values, spanning many orders of magnitude. From the MCA
results for the five initial Cw values, an interpolation of the results allows the determination of the single exit level
(C..EXIT) that meets the stated protection goals.
5 Examples of the site-based input parameters for the Landfill WMU are given in Volume 4, Table g-9b and Table 8-9f
where the site based "certain" parameters are those indicated as site based in the column "Site" and the corresponding
distribution type is "constant," indicating constant at a particular site but variable from site to site.
21
-------
In this sense, the 56 Landfill WMUs in the 3MRA database represent a
specific sub-sample from the "universe" of possible Landfill WMUs.
Once sampled, certain parameters for a particular site are no longer
random, bat instead represent those particular combinations of site-
specific parameters for the particular site in the database,
2. National/Regional Parameters: These include variables describing human
exposure factors, ecological exposure factors, regional hydrogeology
factors, etc. and are based on information that is not site-specific, but
rather gathered from a variety of regional and national sources.6 Although
discussions between the panel and the Agency indicate there is agreement
that these distributions inherently represent a combination of variability
and uncertainty in characterizing the underlying model parameter, in
3MRA these input distributions are assumed implicitly to reflect primarily
parameter variability. The panel generally views the PDFs that contribute
to the distribution of sites protected (e.g., G%) to be a blend of uncertainty
and variability. No analysis has been presented in the 3MRA to quantify
the degree of uncertainty in these parameters.
The MCA process then requires running a sufficiently large number of
simulations to generate a model result that is unaffected by the specific number of
iterations performed (1,000 simulations for illustration here). It is important to recognize
that the WMU database consists of 201 Subtitle D facilities surveyed by the EPA in 1985.
For any specific WMU type, the number of WMUs for which site-specific data are
available varies: Land Application Units (28), Landfills (56), Waste Piles (61), and
Surface Impoundments (137). The Landfill WMU (with 56 sites modeled) is used for
discussion purposes here.
* Examples of these parameters are given in Vol. 4, Table 8-9q, and using particular examples for human exposure
include factors such as soil ingesiion rate, food intake rates, drinking water intake, etc.
-------
Fixed
(Ring, of 5 C.
only \ shown here)
Chemical Conccniranm (C.I
VHMU Paranwton,
Hydroto^c wtbng,
acoiogical »Hing,
human popuiatcn...
L
N
I
I
"National ItaslualuwT
SUr-Baml DaLubaM lapuii
in- I lo56fi«I.K WMU)
I Varcthle Nil "Certain" ut Paiticiii:* Silc)
Sample Random Input Parameter*
(0-U-. Fknw. hh J. Cunl.. Anmi-Kib. D«. ale
- budy v»««i$k.
Site Risk Outcome - SR(n)
NulSlfai
514(2), SRCS6);
{*, J - J* im cl -wlixl i»llilll«|-)
Pcnreol Sin Pranctnl (C%)
* ^ Si 36uhwc N* couiil ufiitet pnuvled 9t
Noll National Rcaliuoon
cor.rr
encral Sin fn»mnl(C%)
[*
i:
/
NalC.
Figure A2c-l-l
3MRA Monte Carlo Schematic - Landfill WMU Example, Single Cw
According to the problem definition as posed by EPA, the "experimental unit" for
developing exit levels consists of the set of WMU sites around the country (e.g., 56 for
Landfills). Thus, a single outcome of the "national experiment," or a single "national
realization" in the parlance of 3MRA, is given by running a single MCA iteration across
all sites (e.g., 56 Landfills) for a given value of Cw. This single iteration selects random
samples from the national/regional distributions of input parameters and applies these
23
-------
random samples to the site-specific input parameters for each o: WMUs. Each such
single iteration for the Landfill WMU example gives the following model outcomes:
56 site-specific (independent) values of the percent population above/below
specified cancer risk and non-cancer hazard quotient benchmarks within 2-km of
each site.
56 site-specific (.ndependent) values the percent of the habitats which fall
below/exceed ecological hazard quotients within 2-km of each site.
A single estimate of the percent of the sites (e.g., percent of the 56 sites for
Landfill WMUs) that are protected, or in other words a single value of "G%."
Depending on how the results are tallied from these 56 simulation outcomes, it is
possible to construct a cumulative distribution function (CDF) of the population risk, or
percent sites protected. In 3MRA, the individual or the population risk results themselves
are not stored to create a CDF of risk or non-cancer hazard indices. Instead, the risk
results are first tallied in a series of "bins" associated with specified "risk ranges." From
these risk bins, the percent of the population at a particular site falling within the
specified risk range bins is then tallied (the risk bins are described in Volume I, Section
14.2.2). For example, for a particular simulation if 94% of the population at a particular
site falls within the risk range of 2.5 x 10*6 to 7.5 x 10"6, then this result would be tallied
in the "population protection bin" corresponding to the range ">90% to <95%".
In order to determine the percent sites that achieve a specified level of population
protection, it is a sirnplo matter of tallying the results for particular population protection
"bin" of interest (e.g., >90% to £95%). In this fashion, the Landfill WMU simulation
outcomes for a single national realization yield 56 "pass/fail" values for a particular level
of protection simply by tallying the number of sites that fall within the specified
protection bin of interest. Thus, G% is given by N/56 where "N" is the number of sites
that fall within the population protection bin of interest. In this manner, 3MRA calculates
a single estimate of the "percent sites protected," or "G%," corresponding to a specified
population protection threshold for each "national realization."
Clearly, for each "national realization" the single value of G% that is calculated
has little significance by itself. It represents the modeled outcome for 56 sites that is but
one particular combination (of essentially infinite combinations) of the set of hundreds of
model input parameters treated as random variables. Thus, in order to generate a
meaningful result, one that develops a distribution of the number of sites protected (a
distribution of G%), the MCA must be performed a large number of iterations. The
actual number of iterations should be sufficiently large such that model simulation errors
are minimized. The "outer" loop in Figure 2c-l-l shows this repeated iteration process,
using for illustration 1,000 iterations.
When this MCA is completed for a single chemical concentration (Cw) and WMU
type, the result is a CDF of G%. An illustrative example of the CDF for G% for a single
24
-------
of Cw is provided in Figure 2c-l-2 (in this illustrative example it is noted that the
presumptive percent population protected, or P%, is 99% at each site). Note that the x-
axis for this example CDF is plotted in reverse order. This is because the probability of
site protection decreases as the percentage of sites protected increases.7 In this example
approximately 38% of the sites were "protected" in 90% of the modeled outcomes,
whereas approximately 58% of the sites are protected in 50% of the modeled outcomes
(e.g., the median outcome is that 58% of the sites are protected. Note that this example
CDF of G% is hypothetical and was not generated using actual 3MRA model runs.
Example Cumulative Distribution Function
Percent Sites Protected (G%) at Given Waste Concentration (Cw)
(AJaunwig 9«>" ' Popubuon Protection at Each Sic)
It
/
/
50% probability of-58%
S*e Protection
,-"
0 90 80 70 60
^-
/
1. .)
90th Percent* of G%
30th Pcrccntilc of G% (mcdan)
90% probabity oC-38%
Site Protection
,
100%
80% -
-70% *
- 60% u
40V. '§
30% "i
- 20% fa
10%
SO 40 30 20 10 0
Percent Sites Protected (G%)
Figure A2c-l-2.
Illustrative Example of 3MRA CDF of G%.
Evaluation ofSMRA MCA Results Provided to Panel
The panel was provided with actual MCA simulation results for several
compounds (information provided 11/27/2003). In addition, a subset of the panel
participated in an informational call with the Agency to review these materials on
December 4, 2003. This discussion provides the panel's observations and comments
regarding the example results provided.
Results of percent sites protected (G%) were provided for two scenarios (for
brevity, the discussion here addresses only human health protection, recognizing that
ecological habitat protection measures were included in the results provided):
95% human population protection
99% human population protection
7 Alternatively, the CDF could be plotted as a "complementary CDF" as discussed in the 3MRA documentation.
-------
As an example, Figure A2c-l-3 provides a plot of the MCA results of G% for a Landfill
WMU for arsenic, for a particular waste concentration (Cw = 1,000 mg/kg). The figure
plots the results for both 95% and 99% population protection. As these results show, for
a given waste concentration, fewer sites meet the population protection criteria as the
criteria increases from 95% to 99% protection (note the x-axis is plotted in reverse order).
Several things are worth noting from these CDFs. The percent sites protected do not
increase in a continuous fashion, but instead increase in discrete "bins." This is a direct
result of the fact that the WMU database constitutes only a discrete number of sites. In
the Landfill WMU example there are 56 sites in the EPA database. Thus, if 1/56 of the
sites meet the populatior protection criterion, then G% is 1.79%, if 2/56 sites meet the
population protection criterion then G is 3.57%, if 55/56 sites meet the criterion then G%
is 98.21%, and so forth.
3MRA Cumulative Dtetribution Function
Percent Sito Protected (Git)
Landfill WMU - AriMlo Cw 1,000
,1 I1'
1 1
1
1 1
,1
.
1
,r .i1
30 90 90 70 M SO 40 30 10 « C
0.9
O7
aft a
0.4 *
O3
0.1
0
P«e«lt Vta« Pratactod (O%)
FIGURE A2C-1-3
3MRA Cumulative Distribution Function
Percent Sites Protected (G%)
In Figure A2c-l -4, the 3MRA MCA results for nickel are plotted for a Landfill
WMU for the 99% population protection scenario. Three values of Cw are shown,
Cw=l 0,000 mg/kg, CW~1,000 mg/kg, and Cw=100 mg/kg. One immediate observation
from these results is that for Cw^lOO, for all values of H% up to approximately H% ~
26
-------
90%, 100% of the sites achieve 99% population protection. At the other end of the scale,
for Cw= 10,000, none of the sites achieve 99% population protection. In order to
calculate exit levels from the discrete ranges of Cw values modeled, the Agency
interpolates between the results for particular values of Cw. Illustrative "exit levels" are
shown on the figure corresponding to H=95% (Ce»it = 174), H=80% (Crai[ = 355), and
H=50% site protection (Cexit = 541). These "exit levels" correspond to different
percentiles of the G% distribution, namely the 95th, 80th, and 50th percentile of G%.8 The
exit levels are determined by interpolating between the CDF results for Cw=l,000 and
Cw=100.
3MRA Cumulatlv* Dl*Utbutk>n of Pirmnt Site* Protected (G%)
9«% Population Protection (Sc.oirto 1)
Solution 3p« Nlcktf - Landnil WMU
CV»ying*H)
U
] ."<*
4&
^
II*O«1000I) BCwIOOO »Cw1«
II 1
1. 1
y~ . . 1
1
1-L
rl
1
,
1 '
. 1
. .-1 '
Eilt (.»<« IW* Poo ?rataethin)
C«174
C-35!
c«=
nw
<*
S*.
lua
^til«
Kk
0.0 9S.o sao as.o w.o 75.0 ro.o
«50
(»S»O, 95% H)
(95*0,WSH)
(M%O,iO»H|lM
-------
appears that the method adopted in 3MRA will tend to force the interpolation of the exit
level to a lower concentnition threshold than would be the case if a larger number of Cw
values (e.g. , reduce the concentration range between Cw values) were modeled.
The panel's initial evaluation of the results also raises further questions regarding
the interpretation of the 3MRA results that has been suggested in the 3MRA
documentation and that has been presented to the panel during meetings and telephone
conference calls with the Agency. The panel understands that the Agency will select an
upper percentile from thi: G% distribution as the basis of setting exit levels (this upper
percentile corresponds to H% in the 3MRA terminology). The panel agrees that selecting
an "upper percentile" in this fashion is consistent with stated Agency risk assessment
practice and policy relating to public health protection (e.g., analogous to protecting the
"reasonably maximally exposed" individual, or in this case population). However, the
panel does not necessarily agree with the interpretation of the CDF of G% as representing
exclusively the "uncertainty" of site protection resulting from the finite number of sites in
the WMU database.
The G% distribution is derived as the iterative result of sampling from input
parameter distributions that are "single dimensional." In the panel's view, these input
distributions represent a hybrid distribution of natural variability and an unqualified
component of uncertainty. Thus, for each simulation for a given WMU (e.g., each
iteration through a single site in Figure 2c-l-l), the value of population protection (P%)
that is calculated reflects variability and uncertainty associated with the input parameters.
As the simulation proceeds to the next site, this variability/uncertainty associated with
input parameters gets applied at each site independently. The results in the P% at
separate sites reflect different random combinations of site-specific/regional/national
input parameters reflecting natural variability of population density, site characteristics,
WMU characteristics, e-/c. from site to site. The differences in P% outcomes at each site
inherently reflect the uncertainty imbedded within the "hybrid" distributions of
variability/ uncertainty that form the basis of the site-based/regional/national parameter
base. Each estimate of percent sites protected (e.g., each value of G%) thus reflects both
parameter variability, parameter uncertainty, and differences based on different site
conditions. The degree to which input parameter variability or uncertainty dominates the
outcome cannot be detsrmined (higher order ISE and/or SME dimensional uncertainty
analysis will offer insights to this question). Regardless of whether the CDF of G% is
interpreted as an expression solely of the uncertainty of sites protected, or a hybrid
representing both variability and uncertainty, the Panel emphasizes its concurrence that
selecting an upper percentile of the CDF of G% (i.e., H% is this upper percentile) is
appropriate. This approach is a reasonable means of meeting the Agency's stated goal of
establishing national exit levels such that a specified percentage of the sites achieve the
defined levels of population protection.
The panel recognizes that the number of sites in the WMU database influences the
distribution of G%, but the influence appears to be simply the inevitable result of the
particular number of sites modeled. That is, the fact that only 28 Land Application Units
(LAUs) are modeled, constrains the G% "bins" to certain fixed percentages (e.g., 3.57%,
-------
7.14%, 10.71%, etc.) corresponding to 1/28, 2/28, 3/28, etc. sites protected. Likewise, for
Landfill WMUs the G% bins are restricted to 1.79%, 3.57%, etc. simply because the
number of landfills modeled is 56. A comparison of Figure 2c-l-4 (e.g., LF for nickel),
with 2c-l-5 (LAU for nickel) illustrates the differences in the G% bins. Yet, these G%
bins have no bearing on the statistical interpretation of the MCA results. Take for
example the model simulations where 100% of the sites are protected. In such instances,
the percentile of the G% distribution is 100% for the Landfill and LAU WMU regardless
of the fact that N=28 for LAUs and N=56 for Landfills. Thus, the percentage of
simulations (e.g., the probability axis on Figure 2c-l-4 and Figure 2c-l-5) that fall into a
particular G% bin has no relationship to the number of WMU units simulated, but instead
is a function of the random combination of particular input parameters/site parameters
coupled with the particular waste concentration modeled.
3MRA CumuUtiv* Dlitribulion of P*rc«nt SttM Protected (G%)
99% Population Protection (Scwurio 1)
I - Urnd Application Unit WMU
9olllltO« Sp*M
CVvyingtlH}
" I ' I "
©1 ' 1
s£J|
1
ISSS^
SiS?
El* Lonta HI* Poo Protection)
C-171
C - 4.47
tw-
IMttfl
ikolun
5--HW
-Ult
(05*aas*H)
<«%a.ao*M)
(«5«0,MSr<)in>d«
1 10 lut)
Itti.l) V.2 K.i
1(X] 0 9A.2 UI.S
IOU.U Mi '<')
Klfl Q [1X1.0 HX.j
, . .
100 850 90.0 950 80-0 75.0 700 «iO
00.0 55.0
50
5 P 1
* VI I
p [^cumu
,<
Q.J ^
0.1
0
0
P«c*«t SIM hotocM (0%)
FIGURE A2c-l-5
Stated differently, there is one "true" (but unknown) distribution of G%, with only
one "true mean" and "true variance." It stands to reason then that as more sites are
modeled, the ability to approximate this "true" distribution improves (ignoring other
sources of error/uncertainty). Thus, when the 90'h percentile of G% is selected from a
distribution derived from a large number of modeled sites, there is greater confidence in
this estimate of the 90lh percentile than a situation where fewer sites are modeled. Yet,
the number of sites modeled says nothing about the actual value of the 90th percentile
itself. That is, modeling a larger number of sites does not translate into a reduction of the
variance in G% (there is but one true value). Instead, including a larger number of sites
29
-------
in the experiment reduce;; the uncertainty in estimating the true mean and true variance.
The actual variance of G% is unrelated to the number of sites modeled (again there is
only one fixed but unknown variance ot"G%).
Informational materials provided by the Agency to the panel on January 9,2004
indicated that the Central Limit Theorem (CLT) offers an explanation for the underlying
connection between the number of WMUs modeled, and the variance of G%. It is
unclear to the panel how the CLT offers such an explanation. The CLT speaks to the
distribution of means. The CLT states that for any arbitrary distribution type, the
distribution of the means of these distributions tends to normality and that the variance of
the mean reduces as a function of 1/N (e.g., a /N, with N being the number of samples).
Yet the distribution of G% in 3MRA is not a distribution of the means, it is the
probability distribution of sites protected.
The panel notes ihat this statistical question (the connection between the number
of WMUs modeled and the variance in G%), is not an issue that raises fundamental
concerns about 3MRA. It is an issue of clarity and transparency in terms of describing
the MCA methods in 3MRA. As noted in the panel's recommendations for the MCA
(presented in the body of the panel report), the larger question is whether a sufficiently
robust sample size of WMUs has been modeled to allow statements about the degree of
protection offered by the exit levels when they are adopted on a national basis (e.g.,
extending to the thousands of waste management facilities across the country).
Monte Carlo Simulation Error/Precision (1%)
The panel would like to also comment on the issue of MCA precision, or output
sampling error (OSE) is; 3MRA terminology. The approach that is proposed in the
3MRA (e.g.. Vol. IV, Section 2.5.3) is an appropriate and useful method for determining
the number of MCA iterations need to bound OSE within acceptable limits. The
approach outlined, and the results provided to the panel, provide reasonable assurance
that the 3MRA results (e.g., selection of a particular percentile of the G% distribution to
establish exit levels), will not be unduly affected by numerical simulation uncertainty.
The panel does not see this as an issue that is a fundamental component of the decision
framework for developing national exit levels. Instead, it represents good scientific
practice and the algorithm for establishing the appropriate number of MCA samples to
minimize OSE is reasonable. This algorithm could be easily used to establish the
appropriate number of MCA samples under alternative MCA formulations, including the
alternative approach to the MCA that the panel has offered.
Appendix 2c-2
Suggestions for Alternative MCA Data Synthesis
As described in the responses to Charge Question 2c, and illustrated previously in
Figure 2c-l-l, the MCA in 3MRA consists of an iterative calculation process that
contains two fundamental "looping" routines:
30
-------
1. A site-based loop that holds site-specific parameters "constant" at a
particular site, and
2. A national/regional loop that draws from probability distributions of
model input parameters.
The "outer loop" in this MCA as proposed in 3MRA consists of sampling first from the
national'regional/site-based input parameters, then applying these to each of the WMUs
in succession (independent samples at each site). Thus, for the Landfill WMU example,
the inner site-based loop results in 56 outcomes for each pass through the "outer loop."
The outcome that is stored in 3MRA is not the actual calculated risk, or percent
population protected (weighted by population at each centroid) at the site, but rather a
series of "pass/fail" results for each of a series of population protection bins (e.g., 90-
95%, 95%-98%, etc.) for a particular site and iteration. By storing the information in this
fashion, the MCA in 3MRA is discarding valuable information, namely the value of the
calculated population protection for each simulation. Currently there is no way to
determine whether in meeting the goal, the majority of the sites achieve the percent
population protection criteria by a wide margin, and a subset "just meet" the target. The
actual degree of protection at any particular site is "concealed" by the current method of
calculating (or summarizing) the percent sites protected.
To illustrate the panel's suggestion of preserving the percent population
protection on a site by site basis, we have presented an alternative MCA formulation that
reverses the looping order in the 3MRA. That is the site-based loop is the "outer loop"
and the national/regional loop is the inner loop. [The panel notes that the actual "order"
of the loops is immaterial, and it is only a question of how the results are aggregated that
is important. This suggested "re-ordering" of the looping structure is retained only for
clarity and to distinguish it from the current method of aggregating the results.] If this
approach is adopted, then a probability distribution of percent population protection can
be constructed at each site. This process is straightforward and would require only a
change in the way the results are processed, without modifying any fundamental model
components. An illustration of how the site-based distribution of P% could be derived is
shown in Figure 2c-2-l.
31
-------
Fixed
(Rang* of 5 Cw
only 1 »hown h«
I
lie) I
CVwical CoAcenniDua (Cw)
WMU Param«t«ra,
Hydfolcpc letting,
ecological aiftng,
human population...
Site-Bawd DatultwM Input Limp
(n= 1 10 56 t'.v LF WMU)
anublc bui "Certain" m Pimvui* Sue)
iX»lloBJ*R«iioa.l RV Swipto Loop
(PsraTOtoi rgprcscnluiy
Simple Rudom Parameter)
(.g.. Fwnu. HyJ. Cuml.. Aiuumt. Ev«,«te.
Huiiua Expujur. - balv v»t|$l«. Juil ingnlum. £«.;
Site Risk Outcome - SR
Nan NdwraVRcfioittJ RV Sample
0 -1. x,., icon)
Next Silt
Funljr
-------
I
Inspecting the results of all sites would then reveal the percent of sites (e.g., G%) that
achieve the defined level of P%. If the CDF of P% indicates 95% population protection
(for a given Cw)» that site "passes" the decision rule, otherwise it does not. The
percentage of the sites meeting this goal is then calculated in a manner analogous to the
approach in 3MRA (e.g., N/56, where N is the number of sites meeting the protection
criterion at a particular Cw and P% threshold).
The panel does not suggest that the moments or percentiles of the site-based P%
distributions would be "added" or otherwise combined into a "derived" distribution of
*
means. The panel believes this alternative approach does provide decision-makers the
information to achieve G% site protection with a specified degree of confidence (e.g., the
confidence is derived from selecting the upper percentile of P% distribution). Also, the
algorithms adopted to minimize OSE (e.g., 1%), apply to this approach in the same
manner in which they apply to the current 3MRA formulation.
If data storage requirement for such an approach would become unmanageable, it
could be possible to store a subset of the percentiles of the P% distribution for each site,
rather than the entire distribution.
Advantages of proposed approach:
The proposed method preserves data. This is useful to further characterize
protectiveness. For example, a stakeholder may ask, "What is the P% for the 5*
percentile (for example) of sites for which P%<95%? Retaining the original P%
values will allow that question to be answered.
P% incorporates both site-based spatial variability of exposure (e.g., concentration
term), and "standard" distributional approaches for input parameters as would be
applied in a l-D analysis.
The distribution of P% at individual sites is amenable to common distributional
analyses, and does not imply a separation of uncertainty and variability.
This approach of considering each site as the basic "experimental unit" preserves
the ability to evaluate inter-site variability and is more intuitive. Because the
national analysis is nothing more than the "roll-up" of the site-based analyses,
retaining the site as the basic unit for analysis does not hamper national decisions.
The disadvantage of the current approach in 3MRA is that it does not allow an
analysis of general site conditions that give rise to "high risk" scenarios, in
essence completely removing any "site-based" inquiry from the results.
The approach recommended here is amenable to adding a 2nd order analysis, and
is consistent with EPA's stated intent of using the 3MRA modeling framework for
both national and site-based analyses.
33
-------
Appendix 2c-3
Probabilistic Analysis of Chemical Toxicity
Because of Agency policy decisions, the current version of 3MRA treats several
parameters as fixed and certain when, in fact, they are variable and/or uncertain. Among
these are the human toxkity criteria: cancer potency factors (CPF) and reference doses
(RfD). Each of these parameters has several identified sources of variability and
uncertainty, and each is related to the outcome in a linear 1:1 manner. Current values for
these parameters are conservative estimates, intended to be reasonably certain of
protecting most humans when combined with conservative exposure parameters in a
deterministic risk assessment. However, using these deliberately biased estimates in a
stochastic risk assessment violates a basic tenet of this practice, namely that distributions
are unbiased.
The following discusses sources of variability and uncertainty (primarily
uncertainty) and some suggested ways of incorporating the range of uncertainty into
probability distribution functions (PDF)s for these parameters. Factors that contribute to
variability inherent in how a heterogeneous human population responds to chemical
exposure include pharmacokinetics, physiology, disease status, and age of exposure
some of which may be amenable to quantification. For instance, the Agency has begun
to quantitatively address; variability associated with age of exposure in its recent re-
assessment of vinyl chloride (EPA, 2000). Numerous research groups are trying to
' develop ways to quantitatively account for the pharmacokinetic and physiological
differences in which children handle toxicants (Ginsberg, 2003; Ginsberg et al., 2004;
Landrigan et al,, 2004.). However, for other factors such as disease status, nutritional
status, age at exposure, and sensitive or susceptible populations where the variability is
very large, it may prov« much more difficult. A more complete discussion of this topic
can be found in the references at the end of this section.
Cancer Potency Factor (CPF)
Most CPF are derived from rodent bioassays usually conducted for the majority of
the animal's lifetime. A typical National Toxicology Program or similar protocol would
involve rats and mice of both sexes exposed to two dosages and controls. The highest
dosage is supposed to be a maximum tolerated dose, just below incipient overt toxicity,
and the other dose usually one-half that amount. Several steps are involved in applying
the results of these animal bioassays to estimate carcinogenic effects in humans, and each
step involves uncertainty. In general, variability is reflected in the fact that some test
animals in a given dose group develop cancer and some do not, and the same is
presumably true of humans, although we seldom have good quantification of the human
dose.
Selection ofbioassay
If both species show positive results, a choice must be made which bioassay
results to use in the calculation of CPF. Generally the species and gender that gives the
highest CPF is used. If more than one satisfactory study is available for a given species
\
34
-------
I
and gender, a median of the available results may be used. Study selection could be the
basis for PDFs of rat and mouse potency by using the calculated potency from each
satisfactory study to form a finite distribution.
Dose response modeling
If humans were typically exposed to dosages in the same range as the animal
bioassays are conducted, there would be no need for extrapolation from high-dose
observations to low dose predictions. That not being the case, curves are typically fit to
the experimental data and extended through the origin. Although different types of
curves make little difference in the observable range, they can make a very large
difference at dosages a few orders of magnitude below the observable range, where the
level of human exposures are often found. Although there are many approaches to curve-
fitting, a common approach is the linearized multistage model, which generates curves
described by a polynomial with various coefficients on the various exponents of dose.
These coefficients are constrained only in that they cannot be negative. A family of
curves is generated and they are ranked according to the magnitude of the coefficient on
the linear term (Q). Typical Agency practice is to select the 95th percentile value of Q
(Q*) as the CPF for the test species. A PDF for CPF could be generated by selecting all
percentile estimates of Q and entering them into a finite distribution.
Dose scaling
Because of the large difference in body size between humans and laboratory
rodents, dose scaling becomes an issue, i.e. what is the human equivalent of a 1
mg/kg/day rat or mouse dose? There are several factors that may affect this equivalency,
the most important of which is kinetics of the compound. For example, if the test
compound is the active carcinogen, and the compound is broken down to a non-
carcinogenic metabolite in the liver, then the effective dose may scale according to liver
function, which is generally related to body weight0'7. In this case humans will be about
6 times as sensitive as rats on a mg/kg/day basis and about 12 times as sensitive as mice.
On the other hand, if the test compound must be metabolized to the active form, rats and
mice, with their higher metabolic rate, may be as sensitive as humans. In this case,
effective dosage would be a function of body weight1 °. Although compound-specific
interspecies conversion factors would be ideal, development of these data would be a
long-term project. In the interim, a uniform distribution for interspecies sensitivity ratio
ranging from 1 to 12 for mice and 1 to 6 for rats could be used. For each iteration, the
value selected from this PDF would be multiplied by the value selected from the PDF for
rodent CPF and the result would be the human CPF. However this may substantially
under-characterize the interspecies uncertainty in extrapolation, given the substantial
qualitative species differences in pharmacokinetics and lack of site concordance that have
been observed for some carcinogens.
35
-------
Reference doses (RJJ))
The majority of RfDs are derived from rodent bioassays usually conducted for the
majority of the animals' lifetime, although many are based on subchronic studies or
studies in dogs, humans cr other primates. As with CPFs, several steps are involved in
applying the results of these animal bioassays to estimate chronic toxicity in humans, and
each step involves uncertainty. This uncertainty is compensated for using uncertainty
factors (UF). If no-adveise-effect-levels (NOAEL) are based on adequate, long-term
studies in sensitive humans, then UF would be unnecessary. For each step away from
this ideal, the NOAEL is divided by a UF. Although uncertainty may be unidirectional or
bidirectional, UF are unidirectional, i.e. they are only used to lower the RfD. Despite the
fact that they are, by definition, uncertain, they are treated as certain in Agency risk
assessments, including the current 3MRA analysis.
Inter-species extrapolation
Typically a UF of 10 is applied to compensate for uncertainty in extrapolation
from laboratory animals to humans. This implies that humans are 10 times as sensitive to
the effects of the chemical as the test species. However, humans may be more or less
sensitive to the effects of the chemical than rodents are, depending on absorption,
metabolism, excretion, whether the chemical has to be metabolized to the toxic form, the
mechanism of toxic action, etc., i.e. the uncertainty is bidirectional. If the RfD is based
on epidemiological data, this UF is not used. Although compound-specific interspecies
conversion factors would be ideal, development of these data would be a long-term
project. In the interim, published and unpublished data comparing similar endpoints in
humans and laboratory species for various classes of chemicals could be used to develop
a distribution for this parameter.
Intra-species extrapolation
Although the test animals exhibit variability (not all of the animals in a given dose
group are adversely affected), it is thought that humans are likely to be genetically more
heterogeneous than inbred laboratory animals. For that reason, a UF of 10 is typically
included to ensure that the most sensitive human is protected. This implies that the most
sensitive human is 10 dmes as sensitive to the effects of the chemical as the average
human. The uncertainty is unidirectional, but the magnitude of the difference between
average and sensitive humans may vary considerably. Epidemiological data may be
helpful in estimating a range for this parameter.
LOAEL to NOAEL extrapolation
If an adverse effect was observed in every dose group tested in the definitive
study, it is uncertain how much lower the dose would have to be to produce no adverse
effect. The UF of 10 that is typically included to compensate for this uncertainty implies
that the NOAEL is 1/10 of the LOAEL. This type of uncertainty is unidirectional. Other
information such as the slope of the dose-response curve, data from other members of the
I
36
-------
same class of chemicals, range-finding or shorter-term studies, or estimates based on the
magnitude of the effect at the LOAEL could be used to establish a reasonable range for
this UF.
Sub-chronic to chronic extrapolation
If only short-term (less than the majority of the lifetime of the animal) studies are
available, it is uncertain how much lower the NOAEL would be in a full chronic study.
A UF of 10 included to compensate for this uncertainty implies that the chronic NOAEL
is 1/10 of the sub-chronic LOAEL. This type of uncertainty is unidirectional. Other
information such as mechanistic information and a time-response curve for the class of
chemicals could be used to establish a reasonable range for this UF.
UF for inadequate database
If the database for a particular chemical does not include the results of studies
pertaining to a particular type of effect such as reproduction or immunotoxicology, an
uncertainty factor may be incorporated to compensate for this uncertainty. This type of
uncertainty is unidirectional. Information such as mechanistic information and a range of
ratios between chronic NOAELs and NOAELs for the missing type of study for other
members of the class of chemicals could be used to establish a reasonable range for this
UF.
Publications Relating to Probabilistic Analysis of Chemical Toxicity
Baird, S.J.S., J.T. Cohen, J.D. Graham, A.I. Shlyakhter and J.S. Evans. 1996. Noncancer
risk assessment: a probabilistic alternative to current practice. Human and Ecological
Risk Assessment 2(1):79-102.
EPA, 2000. Integrated Risk Information System (IRIS) file for Vinyl Chloride (CASRN
75-01-4). Last updated 8/07/2000. http://www.epa.gov/iris.
Evans, J.S., G.M. Gray, R.L. Sielken Jr., A.E. Smith, C. Valdez-Flores, and J.D. Graham.
1994. Use of probabilistic expert judgment in uncertainty analysis of carcinogenic
potency. Regulatory Toxicology and Pharmacology 20:15-36.
Evans, J.S., J.D. Graham, G.M. Gray, and R,L. Sielken, Jr. 1994. A distributional
approach to characterizing low-dose cancer risk. Risk Analysis 14(l):25-34.
Evans, J.S., L.R. Rhomberg, P.L. Williams, A.M. Wilson, and S.J.S. Baird. 2001.
Reproductive and developmental risks from ethylene oxide: A probabilistic
characterization of possible regulatory thresholds. Risk Analysis 21(4):697-717.
Ginsberg, G. 2003. Assessing cancer risks from short-term exposures in children. Risk
Analysis: 23(1): 19-34.
37
-------
Ginsberg, G-, W. Slikkcr, J. Bruckner, and B. Sonawane. Incorporating children's
toxicokinetics into a risk framework. 2004. Environmental Health Perspectives:
112(2): 272-283.
Landrigan, P., Kimmel, C., Correa, A., and B. Eskenazi. 2004. Children's health and the
environment: public health issues and challenges for risk assessment. Environmental
Health Perspectives: 112(2): 257-265.
Swartout, J.C. P.S. Price, M.L. Dourson, H.L. Carlson-Lynch, and R.E. Keenan. 1998.
A probabilistic framework for the reference dose (probabilistic RfD), Risk Analysis
18(3):271-282.
On March 17,2004 the EPA Office of the Science Advisor published a staff paper, An
Examination of EPA Risk Assessment Principles & Practices (EPA/100/B-04/001, March
2004). The 3MRA Panel's last meeting was March 18, 2004; therefore, the Panel did not
consider this staff paper in its review. The staff paper, which will serve as a vehicle for
opening up a broader dialog about the practice of risk assessment at EPA, does not
represent official EPA policy. However it does address uncertainty and variability in risk
assessment and may be of interest to the readers of this Appendix. __
Appendix 3b
3MRA Panel Review of the Generic Soil Column Model With Recommendations
for Improvement
Introduction
The 3MRA team developed the generic soil column model (GSCM) to describe
the dynamics of constituent fate and transport within non-wastewater waste management
units (WMUs) and surface soils in watershed areas. Because it was to be applied to all
the WMUs and watersheds the term GSCM was used. It has undergone a previous EPA
peer review process (Bartenfelder, 1999). Considering the fact that GSCM is not a
legacy model, and the key role it plays in mobilizing chemicals and providing the mass
inputs for all the "downstream" modules, the 3MRA panel considered elected to
contribute a further review. Selected panelists conducted this review and participated in
fact finding sessions with the EPA developers. In general this review examines aspects
of GSCM not covered by the original EPA review, however a few key issues raised by
the first panel are revisited. In addition the 3MRA panel review contains detailed
recommendations for redeveloping the GSCM and its associated modules.
This review consists of three parts: a) GSCM theory and process description, b)
status of module testing and validation and c) recommendations for GSCM and module
redevelopment.
GSCM Theory and Process Development
The governing equations used for the GSCM are similar to those proposed by
Jury et al. (1983, 1990) and Shan and Stevens (1995). These models were not truly
0
38
-------
I
I
multi-phase, multi-transport soil column models; rather, they were developed primarily
for pesticide evaporation from agricultural soils. Nevertheless the Jury et al. model has a
history of EPA sponsorship and was used as the starting point for developing the GSCM.
The 3MRA team fashioned an innovative solution technique that was computationally
efficient and sufficiently flexible for the unique design and operational aspects of each
WMU type. Details of GSCM theory and use are documented in Kroner and Cozzie
(1996). The following is a brief description of key sections in the document followed by
a critique.
In the Kroner and Cozzie document chemical fate in the soil column is controlled
by the three processes of diffusion, pore-water advection and reactive decay. Molecular
diffusion in the air-filled and the pore-water phases quantify the diffusion process. A
single chemical species mass balance is performed which combines the air, water and soil
phase to yield a second order partial differential equation (PDE) in terms of total
concentration CT in the soil mass. This dependent variable is a function of time (t) and
position (z) in the soil layer. As noted in the 3MRA documentation, an explicit finite
difference solution to the PDE exhibited high numerical diffusion; shorter time steps
were needed for thinner sections to reduce this problem, resulting in long computation
times. An analytical solution resulting from superimposing the three fate process was
adopted to overcome these problems to yield an innovative quasi-analytical approach.
Several versions of the GSCM with differing soil column physical structures were
adopted to accommodate the specific needs of the various WMUs. However, the basic
theory was consistent throughout.
In the simplest structural form the soil column is assumed to consist of one
homogeneous zone whose properties (i.e.; density, porosity, chemical composition, water
content, temperature, etc.) are initially uniform in space (z) and time (t). This column is
divided into horizontal chemical layer sources each of a depth dz. A standard error-
function analytical solution is used to solve the partial differential equation. Initially the
concentration profile is assumed uniform in each layer and zero elsewhere. With
application of the effective solute convective velocity, VE , the center of diffusive mass is
translated downward at a rate VE- Chemical disappearance within the layer is by an
assumed first order decay reaction.
Combined in a superposition fashion the three processes are quantified in the
individual layers. The adjoining layers do not interact; each is treated separately for its
diffusive, advective and reactive losses. The following two paragraphs describe how the
quasi-analytical model is used to estimate the chemical masses movement across the
upper and lower boundaries of the soil column.
Chemical mass is moved from both upper and lower ends of the soil column by
accounting for the movement from each individual soil layer through these interface
planes. In effect the upper (z=0) and lower soil column boundaries (z=zw) are planes
located in these z positions which are imbedded in an imaginary homogeneous soil
column that extends to infinity in each direction away from the interfaces. The process is
illustrated in Figure A3b-l for the top-most layer, it has one face at the air-soil interface.
39
-------
I
The interface is at the z =; 1 position. Initially (t=0), the concentration profile in the top
layer is the rectangle-shaped piece of highest CT° and width dz=2 as shown. After time t
for a diffusion-only process, a bell-shaped profile results. The fraction of the initial mass
that is attributed to diffusion from the first layer to the air is calculated from the dark red
shaded area. The adjacent layer deeper in the soil column, which operates independently,
Sol Column 2*0
Flgvrt-A3l>-1. GSCM Layer diffusion through (fee air-soli Interface.
has a smaller diffusion ail that goes beyond the air-soil interface. The third layer has a
smaller one still and on through the entire pile. However, this is not how the process of
"evaporation" works, is, not the one employed in the Jury et al. model, and has no
credence in inter-phase mass-transport theory (Thibodeaux, 1996). The substitution of a
solid phase on the atmospheric side of the interface slows down the natural transport
processes that would otherwise operate here. Rather than depicting a turbulent eddy
diffusion process, one of chemical molecular diffusion through a porous media is used.
The concentration profile structure depicted in Figure A3b-l on the air-side of the
interface is not theoretically possible and the procedure used in the GSCM will likely
result in underestimates of the mass transferred to the atmosphere by an unknown factor.
I
40
-------
t
Vadose-soii region
Soil column region
Figure A3b-2. GSCM Layer diffusion plus
acfrectfon through to soil-vadoie zone wferfaca.
A slightly more complex process that combines water advection can be used to
portray the wastepile-vadose interface. For advection with velocity VE superimposed
upon diffusion the concentration profile in the bottom most layer of the soil column
moves down the soil column (i.e., water flux driven chemical transport) as illustrated in
Figure A3b-2. The diffusion profile is not symmetrical about z'=0 as in Figure A3b-l
but is translated whole symmetrically around zl= -1. The interface between the bottom of
the waste pile at Z-ZK that abuts the vadose zone beneath is located at the zl = -1 position.
The fraction of the initial mass that is attributed to diffusion plus advection into the
vadose is calculated from the dark or red shaded area shown in Figure A3b-2.
According to the GSCM this mass moves into an imaginary, semi-infinite soil column
with waste-pile physical and chemical characteristics located below the bottom interface,
z = ZK;. The next waste-pile layer above the interface one contributes a smaller
"leaching" tail that crosses the bottom plane. Each successive layer behaves similarly
contributing "leached" fractions. Again, this is not how the chemical leaching process
occurs from the waste-pile to the vadose zone. Chemical movement into the vadose zone
does not reflect the physical or chemical properties and the reactive decay of the
receiving zone.
To account for chemical decay, the concentration CTO undergoes a first-order
(exp(-kt)) decay. The mass that crosses the interface boundaries is corrected using this
decay loss fraction. Due to the changes in the soil column properties across the interface,
going from waste to vadose, the profile structure depicted on the vadose side is
theoretically impossible. The dark or red shaded area depicted in Figure A3b-2 that
extends into the vadose zone is based on waste properties. The GSCM is theoretically
challenged to correctly quantify the interphase chemical transport to the vadose. The
following paragraph describes how the individual layer processes are combined to
transport mass from the soil column; the above just focused on individual layers.
41
-------
t
The soil column consist of a multi-layer stack, each layer dz in depth; the number
of layers is N=Zsc/dz. Figures A3b~l and A3b-2 are for the top and bottom layers only.
Each adjacent layer in the stack behaves identically to the two illustrated. However,
because they are one laytr thickness removed from the respective interfaces, less of their
initial mass moves across the two imaginary interface boundaries. Chemical movement
from below the layer into the surface layer at z^O + dz is similarly calculated. The N-
layer summed dark shaded equivalent mass fractions protruding through the air-soil
interface, corrected for the "water" out-diffusion, are used to obtain the volatilization
from the WP or LAU. That N-layer summed dark shaded equivalent mass fractions
protruding through the bottom interface is that used for computing the leachate from the
WMU and the reactive decay summed over each of the N-layer decreased that available
for movement from the soil column.
In Section 3.0 of the Kroner and Cozzie document, the GSCM is described as it is
used in the local watershed/soil column module. The developers did a particularly good
job including all the runoff processes and in connecting the GSCM to the runoff model.
A compartment model was used for the runoff and it was positioned atop the first layer in
the soil column. Positioned here the runoff compartment received chemical inputs from
the soil column and therefore passes the chemical composition needed for quantifying
resuspension/erosion losses in the hydraulic modeling component.
As described above the GSCM is used to commence chemical movement from the
non-wastewater WMUs; by all the release pathways. Although this is an innovative
solution to a complex environmental chemodynamic, process it is theoretically incorrect.
The faulty construct lies in assuming the soil column can be modeled by chemical release
into two semi-infinite imaginary soil zones and that a molecular diffusion process
simulates chemical mass movement across the top (air-soil) and bottom (soil-vadose)
interfaces of the soil column. In reality a phase change occurs at the upper interface and
a physical properties change occurs across the vadose zone interface. The concentration
profiles at the respective ends of the soil column will assume significantly different
shapes than those illustrated in Figures A3b-l and A3b-2 because of the constraints
imposed by the individual phase transport processes in the boundary layer adjoining these
surfaces.
The second major theoretical error with the GSCM involves its governing
equation (Equation 2-8 in Kroner and Cozzie, 1999). Generally three phases can exist
within the soil column. If it is saturated with water the soil phase is present but the air
(i.e., soil gas) phase is absent. In order to simplify the mathematics it is common to
employ a local equilibrium assumption (LEA) for chemical distribution between the three
places. This allows the expression of the concentrations in the individual phases to be
expressed in terms of the total contaminant concentration CT (see Equation 2-6 in the
Kroner and Cozzie document). Such a tactic is common in ground water contaminant
modeling such as is done in HYDRUS, however it is problematic for modeling a thin soil
column. For the WMU applications the soil column is positioned within a relatively thin
zone between an underlying vadose zone and an overlying air mass. Within these thin
zones the boundary process becomes very significant since large fractions of the chemical
42
-------
t
I
mass reside within and move across these planes. Typically in groundwater contaminant
plume modeling only a very small fraction of the chemical mass is located near the
boundaries.
A solution of Equation 2-8 at fixed time t is a quantitative relationship of total
concentration, CT, with distance z, with non-zero concentration gradients at both ends of
the soil column. The GSCM computed concentration profiles within the three soil phases
(i.e., pore-air, pore-water and solid particles) must have parallel, curvilinear shapes and
finite concentration gradients everywhere. Since the phases are assumed to be in
equilibrium the concentrations are related one to another by non-zero constants and all
the gradients are likewise related. For example, since dCr/dz is non-zero at the air-soil
interface the liquid concentration gradient dCi_/dz must be non-zero as well. However,
without liquid water on the air-side of the interface there can be no diffusion across
through this phase. Without diffusion the dCu/dz must be zero. But if it is zero, the
equilibrium non-zero gradient requirement is violated. Thus, performing only a single
mass balance that yields a solution giving the total chemical concentration and assuming
the phases are always in equilibrium leads to theoretically inconsistent concentration
patterns. In reality, due to the chemical depletion from and re-partitioning within the
mobile phases (i.e., air and water) of the layers near the interfaces, they cannot and will
not be so simply related implying that the GSCM computed concentration gradients and
profiles of CG, CL, and Cs are incorrect. The individual concentrations in pore air, pore
water and soil particles cannot be linearly related by constants as demanded by the LEA.
The EPA developers realized some of these theoretical difficulties and fashioned ad hoc
corrections (see p. 2-8, 2-9, 2-12 & 2-13 in Kroner and Cozzie, 1999).
In summary there are two theoretical problems with the GSCM. First, the quasi-
analytical solution requires two imaginary semi-infinite soil sections located top and
bottom the soil column to act as surrogate sinks to the air above and the vadose below for
the diffusive and advective mass transport. Secondly, the application of the LEA in the
relative short soil column forces the concentration profiles and chemical gradients in air,
water and solid particles near the two interfaces to have parallel profile shapes. Both
theoretical problems work together to fabricate a quantitative chemical release model that
is at odds with the known scientific fate and transport processes operative at the
interfaces and within this multi-phase system (Thibodeaux, 1996). The result is that the
GSCM computations will likely give erroneous chemical concentration and flux
predictions of unknown magnitudes.
Status of Module Testing
Prior to developing a new model it is common for the developer to give a
thorough review of existing similar models in the published literature. Only two such
literature citations were noted in this regard; they were Jury et al. (1990) and Shan and
Stephens (1995). Presumably these were the ones most applicable to the needs of the
3MRA model. Requested technical input received from EPA and its own literature
reviews revealed no appropriate model and this has convinced the panel that the Agency
was correct in commissioning the GSCM.
43
-------
I
Being a new model GSCM has a very brief history of use. Clearly it does not
have the legacy status of the two wastewater modules that constitute the other chemical
waste sources for the 3MRA model. Being innovative the model must pass a special
testing protocol that assures its conformity with the simpler, less innovative and
conventional ones presently in use. It is common practice to test a new model with
numerical calculations against existing similar models and against laboratory and/or pilot
scale experimental data. The 3MRA team responded to this comment and the panel
received a brief report of a one-to-one comparison of the GSCM and MODFLOW-
SURFACE (EPA, 2004).
The 3MRA team concluded that the GSCM performed well, producing
comparable infiltration results, performing very well for certain chemicals under defined
conditions and less well for others. This outcome is not unexpected since Table 1 in the
document shows that both models have basically the same theoretical key attributes. The
main difference is in the PDE solution technique. MODFLOW-SURFACE uses a
numerical Eulerian approach while that for the GSCM uses a semi-analytical Lagrangian
approach.
The numerical tpisting should include some level of heuristic sensitivity analysis
(SA) to test the realism of its computational response to obvious inputs. This approach
verifies some generally expected behavioral input versus output responses. For example,
an increase in infiltration rate should result in an increase in the leached chemical flux
from the bottom of the soil column and a decreased evaporation flux out the top. One
such test was performed and reported, but it was very simplistic and involved only the
single layer within the soil column (see Kroner and Cozzie, 1996 pages 2-6 and 2-7).
Nevertheless this demonstrates the type of numerical SA that should be performed on the
GSCM using several cnemicals that reflect the full range of waste substance properties.
The GSCM should be extensively tested using laboratory and/or pilot scale data.
Numerous sets of experimental data involving both vapor (alone) and aqueous (alone)
chemical breakthrough time profiles exist in the literature. These cover a wide range of
both chemical properties and soil types that provide real simulation challenges for the
GSCM, A more robust test that included the LAU module with both leachate and vapor
was performed with mixed results (Schmelling and Jewett, EPA 2002; Schmelling, Wang
and Liu, AWMA 2003). Although the results were encouraging, such continued
evaluation using only the GSCM is highly recommended by the 3MRA panel in order to
gain more confidence for its use in 3MRA. As it is presently doing, the EPA should
continue to publish the results in the peer review literature.
Five key modd assumptions are made in the operation of the GSCM. They are as
follows and are contained in the Kroner and Cozzie document for which the specific
citation locations are referenced:
1) The volatilization loss is assumed proportional to the total mass loss by the ratio
of gas-phase diffusivity to the total effective diffusivity (see Equation 2-23). Is
there experimental evidence for this assumption in a chemical three-phase system
at equilibrium?
44
-------
f
2) The developers assume that mass is not lost across the top boundary due to
diffusion in the aqueous phase in the soil. In making this assumption the
developers are tacitly acknowledging that the governing equation (Equation 2-8)
is theoretically incorrect. While it is true that no aqueous diffusion occurs, this
assumption is in effect a correction imposed on the computational algorithms. In
fairness to the developers, they do acknowledge that a more rigorous treatment
would be desirable.
3) An implied assumption is made that numerical diffusion can be avoided
completely by using Equation 2-26 forcomputing the integration time step. No
supportive arguments are offered to justify this assumption.
4) The model developers assume that a "reflective" soil column source below the
actual soil column is an appropriate procedure for transforming the zero
concentration boundary condition to a zero flux boundary condition (BC). The
parameter used to accomplish this is defined as "bcm"; the model user must
specify it over the range is 0 to 1. The "reflective" source concept is widely used
and accepted in simple air dispersion models for plumes that contact the ground
surface and when a zero flux BC is desired. By doing this, the developers are
introducing another correction into the chemical transport computational
algorithm. Based on the solution to the governing equation (see Equation 2-16) a
zero boundary is already applied to each layer in the soil column. It is not clear
how the modelers justify arbitrarily imposing a non-zero B.C. on the stack of
layers that form the waste/soil column. This bcm parameter plays a major role in
controlling the chemical diffusive rates emerging from the bottom of the column.
Based on what information does the user select a value of bcm to specify in the
algorithm? (see page 2-10). To the 3MRA reviewers it appears to be an
adjustable parameter. It is unclear to the reviewers whether the bottom layer
concentration, CTO, (see the sentence below Equation 2-26) which quantifies the
chemical mass convected out the lower boundary (i.e., leachate) is in anyway
adjusted by the choice of the bcm. It seems that it should. All these factors (both
diffusive and advective), taken together, affect the mass of chemical delivered to
the vadose zone below the waste soil column. This mass enters the ground water
pathway module, which in rum delivers a concentration to receptors using water
wells or surface waters for their water supplies.
5) In the solution to the governing equation the superposition solution requires a
sequential approach. The developers prioritize the processes with diffusion first,
followed by decay and then advection. They acknowledge that systemic error
could result from this choice and that the size of the error would be dependent on
the relative loss rates associated with the three processes. The ordering of
processes needs to be investigated numerically to resolve the. issue of the assumed
ordering. This GSCM limitation appears on page 2-12.
45
-------
Validation
In the previously section the specific issue of validating the GSCM as a separate
unit was covered. What follows is an evaluation of the validations performed on the
Land-Based Source Modular and Watershed Module of the 3MRA Modeling System; all
these modules contain the GSCM. The following comments are based on the two plus
pages in the section entitled "Summary of Validation" (4.2.4) in Volume 3 of the 3MRA
documentation. It is not clear what criteria are used to accept or reject the "data" versus
model predictions of a pEirticular validation test Examples of the "moving target" criteria
follow as each of the four validation activities are presented and discussed. The panel
agrees that using verified software components based on empirical data is a excellent
approach. However, the Land-based Source Modules and the Watershed Modules each
contain several of these empirical software components. The components are connected
by mass balances in the hydrology model, in the soil erosion model, and in the
constituent fate and transport model to produce the Local Watershed Model algorithm,
for example. In addition, performing the mass balances requires some assumptions to be
made about process structure, etc. The final result of this algorithm development
procedure includes the empirical data as imbedded elements. To claim that the final
modules are implicitly validated because they contain the imbedded empirical data is not
factual. In the opinion of the panel validation of the final overall module construct is
needed.
HELP model vs. LAU module. This was a model versus model comparison of run-off
and infiltration at six sites. Under the circumstances such model-to-model "bench
marking" is an appropriate validation activity. The following end-point comparisons
were listed: "...on EPA expected long-term averages to be in reasonable agreement. The
comparative results were mixed." "...predictions were quite similar...showed relative
large differences". "Wiih regard to differences in infiltration...there was no bias in the
3MRA." However for runoff the 3MRA predicted more at all sites. No numerical values
were given to quantify differences. In summary the bench marking results were ruled
adequate for the 3MRA national screening-level purposes.
Dioxin LAU half-life comparison. Soil half-lives in sewage sludge were compared.
Remaining concentrations at equivalent human health risks were calculated for the LAU
in order to estimate the half-lives. "The range of half-lives over the selected percentiles
was 20 to 48 years, which is in reasonable agreement with the observed half-lives at
several monitored sites." At a face-to-face panel meeting EPA provided data on the
observed half-lives to support the reasonable agreement assertion. EPA concluded that
the data vs. model was corroborated, at least in a broad sense.
Soil-column study data. The LAU Module is again compared to experimental data
obtained on organic chemicals during application of municipal wastewater onto soil.
Four elements of evaluation were tested: volatilization, first order chemical decay,
appropriateness of the quasi-analytical solution and whether LAU thickness and
temperature play significant roles in volatilization. The volatilization rate was reported to
be in the "right order of magnitude" for all categories of compounds, however for the
-------
s
highly volatiles the model was consistently lower than observed. The SA for layer
thickness showed none however the SA with temperature "showed certain sensitivity" on
volatilization. Although not specifically designed for 3MRA testing, the 1981 soil-
column study was the nearest thing to a pilot-scale validation performed on the GSCM.
At a face-to-face panel meeting EPA provided two manuscripts authored by Schmelling,
et al. (2002 and 2003) that detailed the soil-column experiment plus data and how it was
used in the LAU validation study. Based on the results, the combined LAU plus VZ
modules appear to be functioning properly as to leachate generation concentration and
quantity (the reader should appreciate that only the top 7.5cm of the 150 cm column was
the LAU portion). The other 142 cm section was the VZ. Data from three identical
columns was used in the validation study. The experiment design and the data generated
were more applicable to testing the VZ than testing the LAU module. In the opinion of
the 3MRA panel, based on this single experiment the LAU module (aka GSCM), the
validation is incomplete.
General observations on the validation of GSCM in comparison to the legacy
models. Although it is a key piece in the 3MRA model and has been incorporated into
several modules, the GSCM has in comparison undergone much less validation testing.
EXAMS, the surface water module, is compared to 8 data sets; EPACMPT, the vadose
zone and aquifer module, is compared to 4 data sets and ISCST, the air module, has been
validated extensively. The validation studies performed on these three modules suggest
that they are in substantial agreement with the available data. The EXAMS, EPACMPT
and ISCST are dependent on the LAU/Watershed modules for their inputs.
Understandably because they are new the GSCM and the Land-based Source/Watershed
modules have received more limited validation studies in comparison.
The panel recognizes the difficult challenges the 3MRA developers faced with the
absence of an appropriate chemical fate and transport legacy model for the waste-pile
source term and the watershed soil column characterization. The level-of-effort activities
at quickly developing an appropriate model and validating it are understandable and
consistent with the resource constraints of the 3MRA project. However, the GSCM is
hamstrung by some serious theoretical flaws that may frustrate the best intended efforts at
validation. The Agency may want to consider an alternative approach that is more
realistic from a chemical process perspective.
Recommendations for redevelopment of the GSCM and LAU Module
Based on its review the 3MRA panel finds that the GSCM to a high degree
contains features inconsistent with the science and practice in chemical fate and transport
processes for such multiphase systems. Most significantly the use of a single total
species mass balance based and the linear chemical equilibrium between phases
assumption limits the ability to incorporate mechanism-based boundary conditions at
either end of the modeled soil column. Mechanistically correct boundary fluxes are key
to launching the appropriate chemical masses to the air, surface water and groundwater
pathways from these non-wastewater WMUs. In the spirit of constructive criticism the
3MRA panel offers the following concepts and ideas for redeveloping the GSCM and its
47
-------
associated modules. The recommendations focus on correctly formulating the chemical,
physical and biological processes in order to realistically represent chemical behavior in
the soil column. The panel is well aware that there may be significant mathematical
constraints in the computational algorithms as to time and numerical error propagation to
contend with in affecting a solution of the redeveloped model. However, it seems logical
to commence the redevelopment task using the most realistic description possible.
The next section contains an overview of general concepts for developing an
alternative to the current GSCM. The proposed alternative retains the general approach
and many specific features of the original GSCM while adding some and changing other
features that will overcome its shortcomings. The overview contains the following
subsections: 1) soil column chemical processes, 2) column layer structure, 3) mass
balances and 4) boundary conditions.
Soil column chemical processes. Table A3b-l contains a list of all the processes
proposed for the alternative modeling approach. The original GSCM contained
molecular diffusion, water advection, and reactive decay. Recent advances in chemical
transport processes have focused on particle movement by biological macro-fauna and
similar particle movement mechanism in both surface soils (McLachlan et a!., 2002) and
surficial bed sediments (Thibodeaux and Bierman, 2003). These so called "vertical
direction sorbed phase" transport processes contribute significantly or dominate all other
diffusive ones for highly sorbed chemicals. McLachlan et al. report the volatilization rate
from the surface soil was up by a factor of 65 for chemicals with log KOA. range > 2 to <6.
This has been added in Table A3b-l as bioturbation. The original EPA peer review
(Bartenfelder, 1999) noted that rain-event driven rapid infiltration and vertical hydraulic
dispersion are very significant transport processes and should be included in the GSCM.
These are both related to water movement downward in the column. The air-side
resistance in soil columns becomes significant during short exposure times and for rapid
particle transport within the soil column surface layer. These three are also listed in the
table.
Process
Table A3b-l. Soil Column Processes vs. Unit Type
WMUs Watersheds
LAU \VP LF
molecular diffusion*
water advection
reactive decay
dispersion
bioturbation*
rapid infiltration
air-side resistance
? X
.? X
X
Local
AOI
* Includes air and water. * Also includes cryoturbation and surface cracking/erosion.
- Denotes process is active. ? Denotes process may be controlled.
X Denotes process is absent
48
-------
In Table A3b-l processes, the WMU modules and the watersheds modules that
utilize the GSCM are listed as well. Not all processes are active across all unit types. For
example, bioturbation is absent in the soil column of a landfill because of the daily cover
provided and the short duration the waste remains at the surface. In soils it may take
many months to years before the surface layer is colonized by the macro-fauna.
Depending on the way the WMU surfaces are managed, bioturbation may or may not be
present at LAUs, WPs and in the surface layers of the local watershed. Cultivation,
mowing and other grounds maintenance activities may hamper the development of
significant populations of macro invertebrates that contribute to the bioturbation of
particles. Rapid water infiltration is similarly affected by these maintenance operations.
Unlike molecular diffusion in air and water, which is ubiquitous, bioturbation and rapid
infiltration are site specific and highly seasonally variable. For the Monte Carlo
simulation these can be estimated by PDFs that capture their magnitudes and frequencies
of occurrence using a random number generator coupled to the seasons through Julian
Day realizations. Provisions should be made to include these process advancements into
the GSCM as they "come into practice" with increased knowledge and availability of
data.
Structure of the soil column layers. This structure should be consistent with the
processes to be modeled and the composition of the soil column. The WMUs are formed
of waste material "soil" and the watershed soil as a native, natural soil. Figure A3b-3
illustrates some general features of the layer structure of the soil column. Much here is
similar to that in the original GSCM. From the top down three stacks of layers are
shown. The top-column stack contains layers that are used within the upper region of the
soil column where bioturbation, cracking, rapid infiltration, etc. predominantly occur.
The first layer in this stack is the surface layer, which connects the soil column to the
atmosphere above. The mid-column stack contains layers where molecular diffusion,
water advection, and dispersion are the active processes. The bottom-column stack is
shown as a single layer. Its processes are identical to those in the mid-column stack but
its use is to connect the soil column to the vadose zone below.
49
-------
Air-Side Layer
Air-Sal Interlace
Top-column Stack
Vadose
Mid-column Stack
Bottom Cclumn Stack
Waste-Vadose Interface
Figure A3b-3. Generic Soil Column Structure
The number of '.ayers in the individual stacks is variable. Likely at a minimum
number, the top and middle stacks consist of three layers each and the bottom stack is one
layer. Using a larger numbers of layers may increased run time without any real
enhancement in computation accuracy. As will be presented in the next subsection, the
individual layers are compartments of uniform composition. In contrast, the soil column
layers in the original GSCM have concentration gradients within. Compartment model
constructs with uniform concentrations within appear elsewhere in the 3MRA model
system and its use here is appropriate as well.
Compartment mass balances. In this section the mathematical description that
incorporates the processes and the soil column structure will be presented. Due to the
similarities in process variables, the nomenclature used here is that of Kroner and Cozzie
(1999). The alternative governing mass balance equation in terms of the chemical
concentration in the water, CL, is presented in a similar layout as that of the original
GSCM which is Equation 2-8 and is as follows:
dt dz
dz
fife
dz
-/ --
dz
Eq. A3b-l
50
-------
+*«,.[C,(;)/ Kt - Ct (;)] - *«,.[Ct (/)//' - Cc(;)]
The dz variable is the height of the compartment and does not denote a
differential. However dt is a differential. The term to the left of the equal sign is the
chemical accumulation in the water phase (i.e., pore water). The first term on the RHS of
the equality is the net molecular diffusion of the chemical in the water from and two
adjacent compartments. Linear rate equations and finite differences in concentration
quantify the flux between compartments. The next line contains the net water advection
chemical infiltration from the j- compartment and the reactive decay disappearance of
the chemical in the water phase. The last line on the RHS of the equality is the input rate
from the adjoining solid phase and the output rate to the adjoining air/gas phase in the
porous structure. This line obviates using the LEA. The resulting governing equation is
a first order ordinary differential equation whereas that in the GSCM is a second order
partial differential equation. Inherently numerical solutions involving integration of
ordinary differential equations are simpler and involve more stable mathematical
procedures than PDEs.
However with these possible advantages are some disadvantages. Two additional
governing mass balances are required for each layer. These are for the air and soil phases
and are structured similar to Equation A3b-l. Thus for each soil layer/compartment there
are three governing equations. Three concentrations, CL, CG, and C§ appear in Equation
A3b-l; these represent the phase/compartment averages as illustrated in Figure A3b-4.
For the seven layer soil column structure shown in Figure A3b-4 a total of twenty one
(3x7) equations will be required. Procedures for integrating sets of linear, coupled ODES
are well developed.
The compartmental model approach proposed places greater emphasis on process
realism and less on mathematical exactness. For example, in the original GSCM the use
of Pick's first law for the diffusive fluxes results in the mathematically correct second
order PDE. The proposed alternative approach uses an integrated form of Pick's first law
that yields a first order ODE. The result is simpler mathematically without loss of
process rigor. Such constructs with their lesser mathematical burdens have been found to
make very good predictions and are used extensively in the environmental compartmental
modeling and this includes some 3MRA modules (see Appendix 2a-l for a description of
the types of models used in 3MRA.
Boundary conditions. The three equations for the surface layer and the three for the
bottom layer contain inter-phase transfer coefficients. These will result in flux equations
for the transfer of chemical mass to the air and vadose. Assumed zero concentrations in
the air and the vadose will uphold the feed-forward feature of the original GSCM.
Similarly to the original, the chemical concentrations in the surface layer will provide the
means for quantifying the concentrations needed for paniculate resuspension quantities
51
-------
Figure A3M. Soil Column Section
Three compartment stick with a single pore
Gas/Air Phase Q
Water Phase p
SoSd Phase ra
for both the air and surface water runoff pathways. Equation A3b-l contains only one of
the many processes listed in Table A3b-l. This was done as a simplification; obviously
some of the others listed will need to be included in the appropriate layer mass balances.
The upper and lower layer mass balances will need to include transport terms for
chemical movement to the air and vadose, respectively. The incorporation of inter-phase
transfer coefficient boundary conditions that capture the volatile, soluble, and particulate
phase chemical forms departing the upper surface add a dimension of realism not
possible with the original GSCM.
References
Bartenfelder, D.I999. Peer Review of EPA's Hazardous Waste Identification Rule-Risk
Assessment Model. EPAOSW, Arlington, VA.
Jury, W.A., W. F. Spencer and W. ]. Farmer. 1983. Journal Environmental Quality,
12(4): 558-564.
Jury, W. A., D. Russo, G. Streile and Hesham, Ed. Abd 1990. Water Resources Res.
26(1): 13-20.
Shan, G., and D. B. Stephens. 1995. Journal Contaminant Hydrology. 18: 259-277.
Kroner, S. M. and D.A. Cozzie. 1999. Source Modules for Nonwastewater WMUs, and
Watershed Module. USEPA Office Solid Waste, Washington, DC.
Thibodeaux, L. J. 1996. Environmental Chemodynamics, 2nd Edition, Chapter 6.1.
Wiley, N.Y.
Schmelling, S., D. Jewett. 2002. Evaluation of Vadose Zone and Source Modular of
3MRA Model. USEPA National Risk Management Research Laboratory, Ada, OK.
52
-------
Schmelling, S., M. Wang and K. Liu. 2003. Proceedings The Air and Waste
Management Association. National Mtg., Washington, D.C.
McLachlan, M. S.,G. Czub and F. Wania. 2002. Environmental Science Technology,
36,4860-4867.
Thibodeaux, L. J. and V.J. Bierman. 2003. Environmental Science and Technology, 37,
253A-258A.
HYDRUS. Http://www.ussl.ars.usda.gov/model/hydrus2d.HTM.
EPA. 2003. Additional verification exercises comparing the results from the GSCM in
3MRA to an alternative soil column model (MODFLOW-SURFACE). December
15. 6p.
Appendix 4-1
Comments Regarding 3MRA Documentation
Comments Not Specific to Particular 3MRA Volumes
The documentation indicates in numerous places that the earlier problems of mass
balance violations in the 1995 predecessor models have been corrected. The panel
recommends that the Agency should demonstrate using example 3MRA results where
mass balance is attained, and conversely, where a numerical verification is not possible
due to the nature of the model domain (this verification could perhaps be placed in
Volume III). Specifically, the 3MRA output files (and analysis tools such as the Site
Visualization Tool) should provide for the tabulation or visual plots of sufficient model
outputs (source terms, mass fluxes, concentration in environmental compartments, and,
where possible, chemical mass in environmental compartments) to allow an independent
user the ability track mass (and/or concentration) through the modules in a more readily
transparent manner. During panel and Agency public meetings, the Agency presented
several additional "visualization" summaries of the chemical mass and mass fluxes in
those environmental compartments where mass is readily tracked. The panel
recommends that the Agency complete the development of these "analysis" tools as a
component of the S VT to allow for a more transparent examination of model predictions.
The Agency presented the panel with very helpful information relating to earlier
Panel concerns about the possibility of mass "imbalance" when chemical burdens in
certain environmental compartments are based on simple biotransfer coefficients (e.g.,
plant uptake as one example). The panel agrees that this information addresses such
concerns and recommends that this information be included as an appendix to the 3MRA
documentation.
Table 4-1 in Volume IV provides a very nice summary of the 3MRA components.
The panel recommends using this table earlier (in Volume I) in its discussion of the many
interrelated components of 3MRA .
A glossary should be included. Many words in the documentation are not in
common use or are defined in this context differently from their everyday use. Perhaps
53
-------
EPA could consider creating a searchable electronic index of the entire documentation
where a modeling practitioner could query a topic and the "help" module would identify
a list of choices within the documentation from which an individual could Find details on
that topic.
Specific Comments Volume I
Volume I provides a useful overview of the purpose and structure of 3MRA, and
prepares the reader for the information provided in the subsequent volumes. The
repetition of fundamental information is generally summarized adequately in each
subsequent volume so that generally each volume can be read independently of the other
volumes. There remains a need to develop a more "digestible" summary, aimed at a non-
technical audience.
Although Volume I goes part of the way toward fulfilling this need, it needs an
expanded executive summary written in layman's terms. EPA could consider developing
a more graphical summary as one means of more effectively conveying the complex
topics addressed in 3MRA. Lacking a more understandable expanded summary, the only
people who are likely to understand the system truly will be the developers. If that is the
outcome, the decision-makers will not relegate their power to the creators of the model,
and the technical tools will sit on the shelf and collect dust.
Competing with the need for a more widely understandable summary, is the need
to add to the summary in Volume I to provide sufficient detail for the more technically
oriented reader. Thus, Volume I would benefit from the addition of some the additional
technical information in Volume II and Volume IV in order to provide sufficient
information on the intended strategy of application and the interpretation of modeling
results, especially the uncertainties.
A clearer description of what the exit concentrations refer to is required. Based
on discussions with the Agency during public meetings, the Panel understands that the
national exit level represents the concentration in the waste stream as it would enter a
WMU, and not the concentration of the contaminant within the WMU. A clearer
description of the distinction between the "exit concentration" and the concentration of
chemical as applied to a WMU is required in the document. In particular, Volume 1
should include explicit equations that indicate how the land-based source terms (e.g.,
Section 5) are calculated, including how the fraction waste (f*mu) term impacts the source
concentration. As noted in comments on Question 1, the Panel recommends that the
Agency include an analysis of the distribution of fwMU associated with national exit levels
in the final 3MRA documentation.
The document should be clear as to whether the national exit levels are being
calculated based on the percent population protected for a "site" as a whole (e.g.,
potentially accounting for multiple WMUs at a site), for specific WMU types, or for
specific waste types (e.g. liquid or solid. For example, on page 3-4 in Volume IV it is
suggested that EPA is considering developing WMU-specific exit levels (which the Panel
-------
believes has merit). Yet in Volume I, and elsewhere, the exit levels appear to be defined
only as the result of the sum total of all combined WMUs at a particular site. The
document should be clear on this point one way or another.
Another area where additional explanation of exit levels would help the reader, is
reiterating the fact that that the exit concentrations are chemical-specific. Even though
the Agency is developing exit levels for 40+ chemicals, chemical interactions are not
considered.
The concept of risk bins (intervals) is a new one and one that needs further
explanation. In addition, the notion of "percent of population protected" requires further
clarification. In particular, the documentation should provide more tangible examples of
how the spatial variation in modeled chemical concentration in environmental media are
combined with population density (census centroids) in order to calculate "population
risk" or "population protection" at a given site. Clear, specific examples, should be
provided.
The equation notation is oriented toward a modeler who specializes in one topic
area to see the clear linkage to related topic areas and algorithms, however there are
instances where different notation is used for the same equations depending on specific
sub-model considerations. These sometimes differing formulations of the same general
equation does create the possibility of injection confusion and ambiguity. One example
of this is the difference between Equation (5-6) used for the GSCM and Equation (5-16)
which describes the soil surface layer for the Watershed module. The later equation
contains a source/sink term not shown in the equation for the GSCM. It would appear
that the equation for the GSCM model should include a chemical source term added to
the soil column with waste applications. On a related note, it is potentially confusing to
the reader to use lower case symbols for the concentration terms (e.g., "cj", cz, etc.) for
Equation (5-14) then use upper case symbols in (5-16). Finally, using "C|" and "GZ" in
this context to represent the "total" concentration in runoff and soil, respectively, perhaps
unnecessarily introduces confusion between the definition of "c^" and "Ci" where that
later notation is used in the GSCM to define "total concentration."
Volume 1 (and elsewhere) could provide more context (in summary fashion) to
the reader on the nature of the 201 sites in the database (by region, size, industry, WMU
and waste types, etc.). Although there are indeed 201 sites in the database, exit levels for
many solid wastes will be'set based on specific land application units (LAU), of which
there are only 28, landfills (only 56 in the database), etc. Likewise, only 137 sites
managed liquid wastes, so exit levels for liquids will be based on only 137 sites. This
should be clarified in the documentation. As it stands, the indication that the database
included data for 201 Subtitle D sites can be erroneously interpreted to imply that the site
database is more robust than is in fact the case for specific WMU types. In addition,
Volume I should provide more details on what steps the Agency undertook to determine
that the 201 sites selected are representative of the universe of possible waste
management sites {including whether they are representative of current sites, given the
age of the WMU data in the database).
55
-------
According to Vol. 1, page 5-6 bullet 2: "concentration can be adjusted for other
wastes which do not contain the constituent." Again on page 9-7 of Volume 1 there is a
reference to incorporating a "fracture multiplier" for the aquifer module. It would be
helpful to add a table to tie documentation showing all the options and ad hoc
adjustments such as these that are contained within the 3MRA model and which option(s)
are selected for the purpc se of setting national exit criteria.
Inclusion of a bio-uptake factor in the human and ecological exposure modules
would enhance the versatility of the model. Any transfer factor or bio-uptake factor that
is omitted has an implick value of unity, but the omission of this implied value is not
apparent to the reader. Explicitly including such a factor improves transparency even if
the default value is one (1.0) in some cases where there is no evidence to support another
value. However, the inclusion of such a parameter would facilitate the use of chemical-
specific values when available and would also facilitate inclusion waste-specific bio-
uptake data in the future to support de-listing petitions.
The panel notes :hat Volume I generally uses graphics effectively to orient the
reader as to model structure and function, although improvements are needed. Some
example figures/graphics that the Panel found to be most in need of clarification are
identified below, although this list is not intended to be all inclusive. [Note that for some
of these, they appear in multiple places in the document and the Panel recommends that
the modifications would be consistent throughout the document.]
Vol. 1, Fig. 1-2. This figure (which also appears elsewhere) is very busy with a
multitude ofinterconnected compartments such that its value to the reader becomes lost.
Figure 2-3, which has similar elements, is much more intuitive. In addition, Figure 2-3
could possibly be enhanced with the addition of the model(s) that are associated with
each module (where appropriate and without adding undue clutter to the figure).
Vol. 1, Fig. 1-4. Both the Y-axis, and X-axis of this plot require better labeling to
clarify them (this applies to many of the plots depicting the risk outcomes in the form of
probability curves). The figure appears to imply that there is an increasing probability of
protecting a larger percentage of the population, whereas intuitively, the probability of
population protection should decrease as the percentage of the population protected
increases. (If "complimentary" cumulative distribution functions are to be used to
illustrate the results, then the graphs require better labeling, and the text should provide a
description of the reason a complimentary CDF is used in simple terms.)
Vol. 1, Fig. 1-5 isn't clearly labeled and also seems counterintuitive. What do the
individual curves labeled with different percentages depict'.' Also, for a given waste
concentration these percentages should move inversely, i.e. a high percentage of the
population might achieve 50% protection but a smaller percentage would achieve 95%
protection. Rather than attempt to fit all plots (5 Cw values) on a single figure (the
notation on the X-axis is quite confusing), the Panel recommends simplifying these
figures to show perhaps two values of Cw on a plot such as Figure 1-4.
56
-------
Figure 2-9 is not clearly labeled and could be improved. This figure is used to
illustrate the notion of "risk binning" of the MCA results and seems counterintuitive,
suggesting that an increasingly large number of people are subject to increasing risk.
Specific Comments Volume II
Although the data sets used are generally identified, it would be helpful to provide
a concise summary (perhaps by module) of the date, size and scope of the data set, and
other important contextual information that identify the major data sets used to support
the models. While this information may exist in the voluminous documentation, a
concise summary in one location would be helpful.
There are many data distributions that are indicated as being selected using "best
professional judgment." Again, it would be useful to summarize in a more "global"
manner, the types of important model parameters that are based on empirical data, and
those that are based on professional judgment. As suggested earlier, a global matrix of
data inputs categorized by "empirical/site, "empirical/national or regional," "professional
judgment," and "operational" would be helpful.
Specific Comments - Volume HI
Volume III provides a reasonably straightforward indication of the
verification/validation efforts conducted to date by the Agency. However, as noted in
responses to Question 3, the Panel has concerns about the completeness and extent of
model validation and important components of the validation have yet to be completed.
The ongoing validation efforts and results will require updating the documentation.
Specific Comments -~ Volume IV
To characterize and bound the uncertainties for the policy marker it is essential
for them to understand the potential impact that their decision will have. It is equally
important for them to understand how to delineate that uncertainty and comprehend how
sensitive the 3MRA system is in its yielding exit concentrations. Therefore, it is
important that the material in Volume 4, Uncertainty and Sensitivity Analysis, be either
re-written at a more understandable level or take the majority of the material and place it
in an appendix for the reader/user to pursue at a his or her leisure. The chapter is dense,
even for those whose vocation is risk assessment and those familiar with probabilistic
methods used for risk characterization. Eliminating redundant information could in many
instances condense the material in Volume IV.
Section 2 in particular reads like a textbook in some places. The panel suggests
that the discussion be more focused on the actual methods used in the 3MRA and how the
results thereof should be interpreted for decision-makers and stakeholders. There simply
is too much tutorial information that gets in the way of learning what uncertainty,
variability and sensitivity analysis is all about in 3MRA. The need for clarity and
57
-------
simplicity of explaining how 3MRA addresses uncertainty versus variability (if indeed it
does this explicitly) takes precedence over completeness in describing the "taxonomy" of
sensitivity, uncertainty, and the like. -The document should target the model user as the
principal reader, not the academic scholar or statistician. As it stands, the document is
guilty of swamping the reader in a "sea of linguistic ambiguity," (e.g., p. 2-13) rather than
providing clarity. In particular, the document relies on 3M.RA jargon such as ELPI,
ELPII, RSOF, etc. which are perhaps useful, shorthand for describing some of the
mechanics of how 3MRA stores and processes information, but this type of information
is more geared toward modelers, rather than describing the conceptual model
formulation.
There is a rich and often confusing lexicon of terms describing uncertainty,
variability, and sensitivity in the literature, and the report devotes considerable space to
reviewing this literature. However, it is not the Agency's primary job to sort all that out
for the benefit of the 3MRA user. Rather their responsibility is to define clearly and
unambiguously how the terms uncertainty, variability, and sensitivity (and their
derivatives) are used and addressed in the 3MRA context. Specific examples should be
provided, rather than speaking in vague generalities using statistical jargon. How are
they estimated, examined, analyzed, and interpreted in 3MRA? Only then should the
authors elaborate on how the use of these terms/concepts/analyses, etc. as implemented in
3MRA relate to others in the literature, and only as is necessary to clarify for the
reader/user what 3MRA is doing. Furthermore these elaborations can be relegated to an
appendix.
The documentation must be consistent in its treatment of variability and
uncertainty. Although the documentation (e.g. Section 2.6) spells out the various kinds
of uncertainty and identifies those that the model addresses and those that it does not
address, other places might give the impression that variability and uncertainty are
separately quantified. Volume 1, Section 1.2.1 states, "Quantifying variability and
uncertainty in exposure and risk estimates is an important capability of any modeling
system. The 3MRA modeling system was designed with a two-stage Monte Carlo
analysis capability, which enables users to distinguish between variability and uncertainty
in input variables". Section 2.1.1 (page 2-4, paragraph 1,) states "the distilled output
prediction can, for example, be represented as predicting 90% receptor population
protection at 95% of sites with a 98% probability (or confidence or belief) of meeting this
'dual criteria' population protection level." The panel does not believe that 3MRA
presents a quantitative separation of variability and uncertainty in the "traditional" sense
of evaluating input parameter uncertainty/variability (and the documentation recognizes
this also, albeit the ambiguity in the documentation gives rise to confusion on this issue),
nor does the panel believe that a more rigorous 2-Stage Monte Carlo analysis that would
separate and quantify uncertainty is realistic in the near future. The documentation
should be clear and consistent on this point, and the panel recommends that the "pseudo
2-D" terminology be avoided. Although the panel believes that a true 2-D uncertainty
analysis may not be possible with current data and computing resources, as noted in
Question 2c, we recommend that a second dimensional "input uncertainty" analysis be
included in the uncertainty/sensitivity analysis for 3MRA.
58
-------
For some WMU types, the number of disposal sites in the Agency database are
relatively small (e.g., Landfills = 56, LAU = 28, Waste Pile = 61). The documentation
should discuss the implications this small number of sites has on the level of confidence
that can be placed on statements about the "percent of sites protected."
The sequence of Tables 8-9a-t and Tables 8-10a-n ("dictionary files" for SSF and
GRF files) are quite useful, but do not provide sufficient information to allow a user to
interpret the results in the SSF and GRF files. The Panel recommends that the Agency
provide further documentation regarding the information found in the SSF and GRF files.
Figures illustrating the MCA iteration/looping scheme could be simplified to
illustrate the approach for a single chemical and single WMU.
Specific Comments Volume V
While the 3MRA model may be intended primarily as a tool for establishing
regulatory exit levels, the Agency has indicated it may have other uses as well. In
addition, the Public must be provided sufficient and transparent documentation to be able
to run and evaluate the 3MRA model results. With this in mind, the Panel feels that the
existing user's guide elements of the documentation could and should be improved.
In attempting to run the model and its initial example cases, some Panel members
found that information from both Volumes IV and V contained needed model,summary
material and descriptions of application methods before the model could be run, but the
information could be improved by including it in a single volume. A set of several
sections seemed to contain sufficient information for someone with a general knowledge
of the purpose of the model and its constituent elements, but who wanted to run the
model with minimum time devoted to "refresher" reading. A candidate outline of the
material that would go into such a 3MRA User's Manual is found in Appendix 4-2. The
outline draws information from Volumes IV and V, and leaves Volumes I, II and III for a
separate reading exercise.
A diagram showing stepwise requirements for installing, and running the model
would be helpful. Such a figure could include necessary steps to initialize any header
files, run "cleanup" routines between runs, and provide information on model
input/output file locations.
Because one of the major requirements of the system is to implement on IBM-
compatible personal computers (thereby making it an accessible PC-based system), it
would be useful to present the minimum requirements up front not only in the User's
Manual, but also in the very beginning (maybe as a separate stand alone box). This
would be especially useful for those who either are not technically adept, or lack up-to-
date systems. Furthermore, the minimum requirements as stated (64 megabytes of RAM)
appear to be incorrect. Some panel members systems could run a portion of the program
and then simply could not continue because it didn't have enough "horsepower."
-------
Statements in Volume 4 mention that the reader might get conflicting impressions
as to whether 3MRA version 1.0 actually distinguishes between uncertainty and
variability. As noted above, the Panel agrees that discrepancies between statements in
Volume IV and Volume I do indeed confuse this issue, and these discrepancies should be
resolved.
In addition, the document creates confusion in the reader regarding the various
versions of 3MRA (e.g.. ver. 1.0, ver. l.X, and ver. 2.0). The additional functional
capabilities of 1 .X and 2.0 over 1.0 are outlined in the report, but what about problem
solving? What kinds of problems can be investigated with the PC version 1.0? On the
one hand it appears that the Monte Carlo Analysis requires the SuperMUSE, yet the
reason(s) for this remain unclear in the current documentation. The document should be
very clear on the distinctions between the versions, and which version is being used to
develop exit levels.
Software issues/Initial Conditions. Some concentration ranges need to be
expanded to spread out the probabilities. It does little good to have 100% protection at
all concentrations. The ongoing plans for further Monte Carlo model sensitivity and
verification testing could be improved or clarified. The discussion as it stands is
described in abstract terms. The panel recommends that the documentation provide
specific examples (using real 3MRA parameters) of just how the uncertainty/sensitivity
analysis results will be presented in tabular and graphical form. As it stands, the
document discusses some of the mechanics of the proposed analysis using 3MRA
"jargon," but does not provide the reader with a clear understanding of the kinds of
output uncertainty/sensitivity analysis will generate, and how this analysis might
enlighten the regulators who formulate the exit levels, and the public who must interpret
the results. Furthermore, the discussion in the document included many development
program details, such as budget estimates and schedule timing that, while they may be of
interest to some readers, seemed peripheral to the mission of the main document. They
may just be an indication of a "work in progress", but those facts relevant to the more
permanent readership could be included as an Appendix or Addendum.
The summary of model parameters in tables in Section 8 (e.g., Tables 8-9a, b,
etc.) should include the 2nd moment (e.g., variance or standard deviation) where
appropriate when describing probably distributions. Currently, only the first moment is
provided, with a range. The text and tables in Section 8 should offer more details and
clarification to the reader in order to interpret the distinction between the site-specific
"empirical" data/distributions used in the modeling versus the regional/national
distributions. Only after several discussions with the Agency was it clear how the
"constant" distribution types for the site data actually was not meant to imply a constant
value across all sites, but instead an empirical sample from the Westat Subtitle D survey
database, that was "constant" at a given site but varied from site to site as a. site-specific
value.
59
-------
(2)]
Appendix 4-2
Candidate Outline for Improved "3MRA User's Manual"
1.0 INTRODUCTION [Combination of present IV (1.0) and V (1.0)]
2.0 OVERVIEW OF SCIENCE [Current IV (3)]
3.0 OVERVIEW OF 3MRA VERSION 1.0 [Current IV (4)]
4.0 MODEL METHODOLOGY SUMMARY [Current V (2)]
5.0 INSTALLATION AND USE OF 3MRA [Current V (4)]
6.0 CASE EXAMPLES
6.1 Single Site, Single Realization [Current IV (3.2)]
6.2 Example Benzene Case [Current IV (7)]
6.3 Example Mercury Case [New Example from model validation
experience]
7.0 INTERPRETATION OF RESULTS AND UNCERTAINTIES [Current IV
(1.3,7.2)]
8.0 REFERENCES
9.0 TECHNICAL SUPPORT FOR CURRENT AND FUTURE 3MRA
APPLICATIONS
Appendix A - 3MRA Technology [Current V (3)]
Appendix B - 3MRA Inputs & Outputs [Current IV (8)]
Appendix C - Probability Models and UASA Applications for 3MRA [current IV
Appendix D - 3MRA Version 1 .X Enhancements [Current IV (6)
Appendix E -The SuperMUSE System for Testing 3MRA [Current IV (5)]
Appendix F - UASA Plan [Current IV (9)]
Much of the inspiration for this approach came from trying to run the model the
first two times. Because the initial attempt immediately followed a reading of all of Vol.
IV, including Section 3, as well as Volume V, the logic seemed relatively clear.
However on subsequent return, it seemed difficult to remember where some of the key
instruction material was located: Volume IV or Volume V?.
Appendix 4-3
3MRA Editorial Comments
Throughout the document, reference is made to "soil concentration," "air
concentration," etc. While it may seem cumbersome, it is more appropriate and correct
to refer to "chemical concentration in soil," and "chemical concentration in air," etc.
The word "data" is plural. There is not a consistent treatment of the verb form that
follows "data."
There were occasions when tables and figures referenced in the text were either
not present, or incorrectly referenced (see Volume V, p. 2-3 and p. 2-15 as examples).
62
-------
The documentation in Volume V, Section 4.3, devoted to Post Simulation
Analysis is a candidate f jr further improvement. The authors may have assumed that the
typical reader of Section 3, particularly Section 3.3.9 would have a reliable memory of
how the model output wis organized and how all of the postprocessors use those files.
The current documentation was a bit abbreviated and could lead to new model-user
frustration, but with modest user training, could not be greatly faulted. Although with
user training, it might not be unreasonable to have Agency personnel continue to use the
present documentation, the potential for frustration of new users due to the abbreviation
of several key topics in Volume V, especially that section devoted to post-processing
analysis.
The topics under the sub-heading of "Consolidation of Risk Time Output Data" in
Section 2.0 (p. 2-11) an; sufficiently important that it deserves its own sub-section (e.g.,
Section 2.1,4). Furthermore, this section would really be enhanced with a graphic
displaying how consolidation of data occurs, using an example of how the risk bins get
filled as the simulations proceed.
Special emphasis should be given to the "clean-up" procedure. Several panelists
have been puzzled by results that do not make sense, only to learn later that the processor
was analyzing results cf an earlier run.
The discussion of 3MRA Monte Carlo Scheme (Section 2.3) again introduces
confusion regarding whether input parameter uncertainty is included in the 3MRA
analysis. While the matrix provided in Figure 2.3 is a useful means to illustrate a
traditional "2-D" analysis, it does not reflect the actual analysis conducted in 3MRA. In
addition, the examples in Figure 2.3 require improved annotation and labeling. They are
unclear and use non-standard abbreviations/shorthand that is not explained in the text.
Additional examples and model scenarios. The panel suggests that additional
simulation exercises (some example problems) be included in Volume 5. These
examples would provide more context for the types of different questions that can be
addressed by 3MRA, and also provide an independent user of the mode! "benchmark"
examples of model output (to provide "confirmation" that the user can correctly run the
model under different conditions). This model-use training exercise should include
multiple runs to allow the user to be sure that he or she has used the clean-up procedure
correctly.
61
-------
Biosketches for SAB
Multimedia, Multipathway, and Multireceptor Risk Assessment Modeling System
Review Panel Members
(in alphabetical order)
Andrea Boissevain
Ms, Andrea Boissevain is the Principal and Senior Scientist with Health Risk
Consultants, Inc., a woman-owned environmental consulting firm in Fairfield, CT. Ms.
Boissevain has extensive experience as a risk assessor with skills that range from
designing exposure models to managing multi-media quantitative human health
assessments for state and federal Superfund sites across the nation. After receiving her
Masters in Public Health (Environmental Health Concentration) from Yale University
Department of Epidemiology and Public Health in 1984, she worked with a large
environmental engineering concern before starting her own firm in 1989.
Ms. Boissevain is currently developing exposure assessment methodologies to
evaluate individual exposures to a variety of indoor pollutants, including volatile organic
compounds. Several of the sites she is working on are grappling with exposure to soil
gas vapors associated with impacted groundwater. Knowing the science, assessing the
health risks, and developing outreach strategies to inform the public are daily challenges
she addresses. Risk communication and making science understandable to myriad
audiences now comprise a large component of her work. Her basic science background
(A.B. Vassar College, Biology) and her pursuit of toxicology (graduate school and
beyond) coupled with her love of writing has shaped her firms commitment to
communicating with people (clients and the public alike) about the health implications of
exposures (both acute and chronic) to hazardous substances.
With respect to funding sources and contract support, HRC serves a variety of
private (Fortune 100 firms, engineering and law firms) and public sector clients, most
notably the Department of the Navy, U.S. Environmental Protection Agency, the
Connecticut Department of Public Health and the Town of Stratford. Ms. Boissevain is a
long standing member of the Society for Risk Analysis, American Public Health
Association, and the New England Society for Risk Analysis. She also served on panel
of experts that-employed risk-based principles to screen and prioritize over 2000 state-
classified abandoned hazardous waste sites for the Virginia Department of Environmental
Quality (VDEQ). A subset of sites were sampled, information collected, and a hazardous
ranking scheme developed. The expert panel assembled provided professional judgment
in the final priority assignments of the sites to enable VDEQ to assess state [financial]
liability for cleaning up abandoned sites.
Lintleld Brown
Linfield C. Brown is Professor and former Chairman of the Civil and
Environmental Engineering Department at Tufts. Professor Brown earned his BSCE and
MS from Tufts and his Ph.D. in Sanitary Engineering at the University of Wisconsin-
Madison. His research has covered a broad range of topics in sampling strategies, flow
-------
There are occasions where the notation used in figures differs from the notation
used for variables in the text (capitalization, acronyms, etc.). In addition, there are
occasions where the figures introduce acronyms that are not explained in the text.
The document has a tendency to introduce many acronyms that are in some cases not
needed, and detract frorr the readability. This is particularly so in the discussion of
uncertainty and variability and the discussion of the "mechanics" of the "exit level
processors." A more judicious use of acronyms would enhance the documentation from a
readability standpoint.
Vol. 1, p. 5-14. The boundary condition in the second bullet appears to be
inconsistent with the statement in the bullet on the bottom of p. 5-24.
Each volume is i stand-alone document; therefore it would be helpful for either a
header or footer that contains a reference to what volume it is.
A more judicious use of commas would enhance the overall reading, especially
for those chapters whose writers prefer to use long sentences.
63
-------
In addition, Dr. Carbone has extensive experience with a variety of both U.S. and
European fate and transport models. He also closely monitors endocrine disrupter issues
and is a key advisor for the Rohm and Haas Co. regarding the European Chemicals
Policy and the Water Framework Directive. Dr. Carbone is a member of the Society for
Environmental Toxicology and Chemistry and also serves on the editorial board of
Environmental Toxicology and Chemistry where his expertise is in fate and transport
modeling and environmental risk assessment. Dr. Carbone also works with the
Alkylphenol Ethoxylates Research Council where he is an active member of the
environmental subcommittee. Dr. Carbone's work is fully supported by the Rohm and
Haas Co.
James Carlisle
Dr. Carlisle is Senior Toxicologist, Office of Environmental Health Hazard
Assessment, California Environmental Protection Agency. He also holds the following
degrees: Doctor of Veterinary Medicine, University of California, Davis, and Master of
Science in Aquatic Pathobiology, University of Stirling, Scotland.
His professional responsibilities include oversight of the Emerging Environmental
Challenges Program; the Environmental Indicators Program; the OEHHA Califomia/Baja
California Border Environmental Program; the Development of Guidelines and Health
Criteria for the Cal EPA; the Schools Risk Assessment Program; Oversight of contract
research to develop transfer factors for contaminants at school sites; and Risk Assessment
review and oversight for the State Water Resources Control Board, the Integrated Waste
Management Board, and local agencies in California.
He previously served on the Governor's Panel of Experts in Carcinogen
Identification. His professional activities and responsibilities do not involve external
grant or contract support.
Peter L. deFur
Dr. Peter L. deFur is president of Environmental Stewardship Concepts,- an
independent private consultant, serving as a technical advisor to citizen organizations and
government agencies. He is an Affiliate Associate Professor in the Center for
Environmental Studies at Virginia Commonwealth University where he conducts
research on environmental health and ecological risk assessment. Dr. deFur is President
of the Association for Science in the Public Interest (ASIPI) and on the board of the
Science and Environmental Health Network (SEHN). Dr. deFur was previously a senior
scientist at the Environmental Defense Fund (now ED) in Washington, DC and held
faculty positions at two universities before that. He has extensive experience in risk
assessment and ecological risk assessment regulations, guidance and policy. He served
on the NAS/NRC various study committees, including the Risk Characterization
Committee that released its report entitled, "Understanding Risk," in June 1996. Dr.
deFur has served on numerous scientific reviews of EPA ecological and human health
risk assessments, including the assessment for the WTI incinerator in Ohio and EPA's
-------
equalization, oxygen transfer, and most recently, uncertainty analysis in water quality
modeling, multi-response; parameter estimation, and the use of genetic algorithms for
modei calibration.
Dr. Brown has served as consultant to both industry and government. As a
research engineer with the National Council for Air and Stream Improvement (NCASI),
he developed their national program in mathematical water quality modeling. While on
sabbatical leave at the U3EPA Center for Exposure Assessment Modeling (CEAM), he
designed and implemented a computational framework for incorporating uncertainty
analysis into the water quality model, QUAL2E. He is the author of over 50 technical
papers and reports covering the fields of environmental engineering and statistics and has
offered over two dozen workshops in the U.S., Spain, Poland, England, and Hungary on
water quality modeling and control. He is co-author of the book Statistics for
Environmental Engineers, which describes the practical application of statistics to a
variety of environmental engineering problems. He founded and was academic director
of an innovative multi-disciplinary Masters program in Hazardous Materials
Management, and initiated a similar program in Environmental Science and Management
for mid-career professionals, targeted specifically for women and minorities. He
received from Tufts, the prestigious Lillian Liebner Award for excellence in teaching and
advising. Dr. Brown currently serves as consultant to the Environmental Models Sub-
committee of the U.S. EPA Science Advisory Board and is director of the Tufts ABET
accredited BSEvE program. In addition to his university support, Dr. Brown receives
funding from the New England Water Pollution Control Commission, which, in rum
receives that funding from EPA Region I.
John P. Carbone
Dr. Carbone is currently a senior scientist within the Toxicology Department of
the Rohm and Haas Co., one of the world's largest manufacturers of specialty chemicals.
Dr. Carbone received his Ph.D. in endocrine physiology in 1-932, his graduate research
focused on PCB and PBB effects on thyroid and adrenal function. After a postdoctoral
fellowship at Thomas Jefferson University Hospital, Dr. Carbone joined the faculty of
Thomas Jefferson University Medical school where here participated in teaching,
research and grant writing. In 1991, Dr. Carbone joined the Toxicology Department at
the Rohm and Haas Co. His initial responsibilities included sub-chronic study director.
Dr. Carbone migrated toward environmental risk assessment where during the past 11
years he has developed expertise in environmental exposure analysis, specifically fate
and transport modeling of chemicals in the environment.
Dr. Carbone participated in the FIFRA Environmental Modeling Task Force as
chair of the statistics subcommittee. In that committee, Dr. Carbone led the development
and implementation o f an uncertainty analysis approach for a multiparametric fate and
transport model, PRZM. PRZM models chemical movement via runoff and movement
through the vadose zone. In the approach that was developed, uncertainty associated
with model parameterization was accounted for by using a sensitivity analysis coupled
with a Monte Carlo approach to account for the variability associated with these inputs.
-------
Ecological Risk Assessment Guidelines. Dr. deFur also served on EPA's Endocrine
Disruptor Screening and Testing Advisory Committee and is now on EDMVS.
Dr. deFur received B.S. and M.A. degrees in Biology from the College of William
and Mary, in Virginia and a Ph.D. in Biology from the University of Calgary, Alberta.
He was a postdoctoral fellow in neurophysiology in the Department of Medicine at the
University of Calgary. Dr. deFur conducts research on the identification of and effects of
endocrine disrupting chemicals, particularly in aquatic crustaceans. He is also interested
in the effects of low oxygen conditions on aquatic animals and systems in estuaries and
coastal environments. He also conducts research on precautionary approaches to
environmental regulations and on citizen involvement in environmental programs,
policies and regulations.
Dr. deFur was appointed to BEST of the National Academy of Sciences/National
Research Council in 1996. He is on the Advisory Committee to the Board of the
Coalition to Restore Coastal Louisiana, and a peer reviewer for professional journals. He
has published numerous peer reviewed articles, invited perspectives and review articles
for the public on subjects ranging from habitat quality to wetlands, toxic chemical and
risk assessment. During the past ten years, Dr. deFur has been extensively involved in
scientific research, regulation and policy concerning the generation, release and discharge
of dioxin and related compounds. He has published a number of papers on regulation and
policy aspects of these compounds, considered in many ways prototype endocrine
disrupters. Dr. deFur has been extensively involved in the EPA reassessment of dioxin
since 1991. He was a technical advisor to the EPA Superfund Ombudsman office, and is
presently technical advisor for the Port Angeles cleanup of the Rayonier mill site, the
water quality program in the state of Indiana, and to citizens groups for the Rocky
Mountain Arsenal superfund site. Dr. deFur serves as a technical consultant to citizen
organizations that are involved in cleanup actions at contaminated sites around the
country.
Joseph DePinto
Dr. DePinto is currently a Senior Scientist at Limno-Tech, Inc. (LTI) an
environmental consulting company specializing in the development and application of
water quality and ecosystem models for addressing a myriad of problems in aquatic
ecosystems. He joined LTI in June, 2000 after spending 27 years in academia, including
10 years as Director of the Great Lakes Program at the University at Buffalo. During that
time, Dr. DePinto was an active part of the Great Lakes research community and he is
continuing in that role at LTI. During his professional career, Dr. DePinto has directed
projects on such topics as nutrient-eutrophication, toxic chemical exposure analysis,
contaminated sediment analysis and remediation, aquatic ecosystem trophic structure and
functioning, and watershed, river, and lake modeling. Dr. DePinto received his Ph.D. in
Environmental Engineering in 1975 from the University of Notre Dame. His studies
have led to over 100 publications and the direction of more than 45 Master's theses and
12 Ph.D. dissertations.
-------
Recent projects, both prior to and subsequent to joining LTI, that are relevant to
the subject SAB panel include (funding source in parentheses) development and
application of an integrated exposure model for PCBs in Green Bay, Lake Michigan
(EPA-ORD); development and application of sediment and contaminant fate and
transport models to assess and evaluate remediation of contaminated sediments in several
river systems, including the Buffalo River (EPA-Great Lakes National Program Office
(GLNPO)), St. Clair River (Ontario Ministry of Environment), Lower Fox River (Fox
River Group), Kalamazoo River (Kalamazoo River Study Group), Niagara River, and
Hudson River (EPA-Reg 2 through TAMS); assisted the Delaware River Basin
Commission in development of a PCB fate and transport model for application to a
TMDL analysis for the Delaware River/Estuary (DRBC); led a team of scientists and
engineers at the University at Buffalo in the development of a Geographically-based
Watershed Analysis and Modeling System (GEO-WAMS), a Modeling Support System
that coupled a Geographic Information System (ARC-INFO) with existing and newly
developed watershed and water quality models (EPA-ORD); development and
application of a contaminant fate, transport and bioaccumulation model for Lake Ontario
in support of the development of a lake wide management plan (LaMP) and TMDL for
that system (EPA-Region 2); and development of an aquatic ecosystem model for
Saginaw Bay, Lake Huron to investigate the ecological impacts of zebra mussels on
nutrient cycling and primary production and on PCB cycling and bioaccumulation (EPA,
ORD and GLNPO).
Three relevant ongoing projects being conducted by LTI with Dr. DePinto as the
Principal Investigator are: "Developing a Model Framework for Assessing Ecological
Impacts of Water Withdrawals in the Great Lakes Basin" (Great Lakes Protection Fund);
"Development of an integrated ecological response model for the International Joint
Commission Lake Ontario - St. Lawrence River water levels/flows study" (USACE-
IWR); and "Linking a fine scale hydrodynamic model (POM) for Lake Ontario with a
course grid toxic chemical exposure model (LOTOX2)'1 (EPA-GLNPO through
University at Buffalo).
Dr. DePinto has also participated in several workshops and advisory panels
relevant to the topic. He participated in the SET AC Pellston Conference on "Criteria for
Persistence and Long-Range Transport of Chemicals in the Environment," in!998; was a
Peer Reviewer for EPA, ERL-Duluth, on the Dioxin Aquatic Risk Assessment Report,
(July 1993 - October, 1993); invited expert review panel member, "Workshop on
Application of 2,3,7,8-TCDD Toxicity Equivalence Factors to Fish and Wildlife," EPA-
sponsored workshop, Chicago, IL (January 20-22, 1998); invited member of Model
Evaluation Group (MEG) for the Contamination Assessment and Reduction Project
(CARP) of the New York/New Jersey Harbor Estuary Program (Oct. 2000 - present);
commissioned reviewer, "Florida Pilot Mercury Total Maximum Daily Load (TMDL)
Study" report preparec. by Tetra Tech, Inc. for Florida Dept. of Environmental Protection
documenting modeling work with E-MCM (April, 2000); is a member of the
International Joint Commission, Council of Great Lakes Research Managers; and is an
Associate Editor of the Journal of Great Lakes Research and Chair of the Publications
Committee of IAGLR.
-------
Alan Eschenroeder
Dr. Eschenroeder serves on the faculty of Harvard School of Public Health and
operates an independent consulting firm. He received both his BME and PhD degrees in
engineering at Cornell University. He has performed numerous risk assessments and has
developed novel multimedia modeling techniques both for health and climate change
investigations. His current area of research focuses on exposure analyses for
contaminants emitted during military actions in the Middle East conflicts. In addition to
serving EPA as a peer reviewer over recent decades, he has served and chaired various
National Academy of Science special committees and subcommittees. His most recent
grant support has come from the U.S. Agency for International Development, the China
Project at Harvard, and the United Nations fund for reparations. Current support for
consulting work derives from the law firm of Broiles and Timms, LLP on behalf of a
private industrial client involved in litigation.
During the decade following his education and military service, he implemented
computer-based tools in the field of hypersonic fluid dynamics to provide design inputs
for space and defense applications. Using some of these same techniques he began the
development of simulation models tracing the evolution of photochemical smog. This
modeling work subsequently evolved into multimedia descriptions of contaminant fate
and transport in air, water, soil and biota, as applied to exposure and health risk
assessment. Examples of his recent research interests include: greenhouse gas tradeoffs
in waste management, comparative health risks of rural burning versus controlled
combustion of domestic waste in Slovakia, health impacts of mobile sources in China and
the addition of socioeconomic influences to health risk assessments and life cycle
analyses.
Jeffrey Foran
Dr. Foran is a broadly trained environmental scientist with expertise in toxicology,
human and ecological risk assessment, and science-policy. He holds a Ph.D. in
Environmental Sciences from the University of Florida, an M.S. in Biology from Central
Michigan University, and a B.S. in Biology from the University of Michigan. Dr. Foran
has served as a Scientist with the National Wildlife Federation, as Associate Professor at
the George Washington University School of Medicine and Health Sciences, as
Executive Director of the ILSI Risk Science Institute in Washington, D.C., and as
Director of the UW-Milwaukee WATER Institute. Currently, he is President of Citizens
for a Better Environment (CBE), is a private consultant for foundations
and non-profit NGOs, and provides litigation support. He also holds an adjunct faculty
position at the University of Michigan School of Natural Resources and Environment,
Dr. Foran is a member of both Tau Beta Pi (Engineering Honorary) and Sigma Xi
(Scientific Research Honorary), he is a member of the Board of Directors of the Einstein
Institute for Science, Health, and the Courts, and is President of the World Council of the
Society of Environmental Toxicology and Chemistry (SETAC). He has served as an
-------
advisor and consultant to numerous organizations including the IJ.S./Canadian
International Joint Commission, the Organization for Economic Cooperation and
Development (OECD), the World Health Organization, the International Program on
Chemical Safety (IPCS), the U.S. Environmental Protection Agency, Centers for Disease
Control and Prevention, the U.S. General Accounting Office, and the U.S. Department of
Defense.
Randy Maddalena
Randy Maddalena, Ph.D., is a Scientist in the Exposure and Risk Analysis Group
within the Environmental Energy Technologies Division at Lawrence Berkeley National
Laboratory. He received his BS in Environmental Toxicology (1992) and his Ph.D. in
Agricultural and Environmental Chemistry (1998) from the University of California,
Davis.
The primary focus of his research-is development, evaluation and application of
models that predict chemical fate in multiple environmental media (air, water, soil,
vegetation, sediment) and chemical exposures through multiple pathways (drinking
water, food, feed, indoor air) for both human and ecological receptors. He also develops
tools and methods for performing probabilistic risk assessment and sensitivity analysis
applied to complex regulatory models. His most recent work combines the use of models
and experimental data to investigate how vegetation influences the environmental fate
and transport of semivolatile organic pollutants and how the uptake of these pollutants
into ecological or agricultural food chains might contribute to dietary exposures.
Dr. Maddalena is a Co-chair of the Society of Environmental Toxicology and
Chemistry (SETAC) Advisory Group on Fate and Exposure Modeling where he serves as
an Editor of the Fate and Exposure Modeling column in the SET AC Globe. He is also a
member of the International Society of Exposure Analysis and a member of the SAB's
Integrated Human Exposure Committee. He receives funding from the EPA's National
Exposure Research Lab for research on fate and exposure models; the DOE's Fossil
Energy Program for experimental work on plant uptake of petroleum related
hydrocarbons; and from the EPA's Office of Air Quality Planning and Standards for his
work on the TRIM.FaTE model. Dr. Maddalena also recently completed a project funded
by the EPA's Office of Emergency and Remedial Response where he developed a
standardized approach for constructing inputs to probabilistic risk assessment models.
David Merrill
Mr. Merrill, a Principal at Gradient Corporation, has 15 years of experience in
negotiating technically sound and cost effective solutions to environmental contamination
problems. His expertise includes directing large-scale, multi-disciplinary risk
assessments, multimedia chemical fate and transport modeling, complex data analysis,
and database design for systems such as landfills, lagoons, chemical plants, MGPs, river
systems, and groundwater contaminant plumes. With his extensive risk assessment
experience and strong engineering background, he has negotiated risk-based cleanup
levels and remedial strategies, interpreted complex site investigation data into effective
conceptual site models, and evaluated many types of contaminant transport conditions,
-------
including multimedia transport in water, sediments, and air. He has worked extensively
with PCBs, solvents, metals and NAPLs and has served as an expert on cases involving
PR? cost allocation disputes. Mr. Merrill has prepared technical comments on behalf of
industry and trade organizations on Agency regulations including the PCB Megarule and
multimedia modeling and risk assessment aspects of the LDR and the HWIR Rules.
All of Mr. Merrill's professional work is performed for Gradient. Gradient's
client base includes Fortune 500 companies, law firms, trade associations, and to a lesser
extent state and local municipalities and regulatory agencies. Over the last two years Mr.
Merrill's clients have included law firms representing individual companies and PRP
groups, trade associations, chemical companies, natural gas pipeline and oil companies,
energy generation companies, and the U.S. EPA. Mr. Merrill received his B.S. in Soil
and Water Science from the University of California at Davis, and his M.S. in
Agricultural Engineering (Civil/Environmental Engineering focus) from Cornell
University where he also completed the coursework and qualifying exams toward a
doctorate degree.
Ishwar Murarka
Dr. Murarka is Chief Scientist and President of Ish Inc., a minority owned
environmental consulting business. He also serves as visiting research associate at the
University of Illinois in Chicago. Dr. Murarka holds a Ph.D. in Soil Science and
Statistics (1971), and an MBA of Management Science (1974).
His areas of expertise include: Environmental Science and Technology topics
pertaining to: management of solid and liquid wastes; characterization and assessment of
contaminated sites; in-situ treatment technologies (e.g. chemical oxidation); and
remediation/restoration of impacted land, groundwater, and sediments. His research
activities cover transport, transformation, and fate of metals and organic compounds in
the land and water environments.
Dr. Murarka has served on the External Advisory Committee of the Institute for
Environmental Science & Policy for University of Illinois in Chicago; served as Peer
Reviewer on Mercury Studies for EPA; served as a consultant for the EPA Science
Advisory Board. He is involved in U.S. Experts Panel for an USAID project in India
He receives research grants/funding from USDOE/CBRC, EPRI, GTI, and
NYGAS, as well as contract support on projects involving characterization and
remediation of contaminated sites from various utility companies (e.g., Duke Energy,
NYSEG, RG&E, Consumers Energy, Georgia Power, We Energy, First Energy,
NISOURCE, SCANA, etc.
Douglas G. Smith
Douglas G. Smith, Sc.D. is a Principal Scientist in ENSR's Risk Assessment
group with degrees in Environmental Health Sciences (specializing in Air Pollution and
Industrial Hygiene) and Physics. He has 28 years of experience in risk assessment of
toxic airborne materials, including atmospheric transport and diffusion modeling, with
applications to environmental siting and permitting.
-------
Most recently, Dr. Smith has led more than a dozen multi-pathway risk
assessment projects in support of RCRA permitting and strategic planning for chemical
industry members who u;ie incinerators, or boilers and industrial furnaces (BIFs) for
waste disposal and energy recovery. These projects are active in U.S. EPA Regions 2, 3,
4, 5, and 6 and have included supporting applications or updates for permits in New
York, New Jersey, Ohio, Pennsylvania, Illinois, Georgia, Kentucky, Tennessee, W.
Virginia, Louisiana, and Texas. In early 2000, Dr. Smith presented ENSR's team
findings in response to a:i EPA request for an independent external peer review of their
"Human Health Risk Assessment Protocol for Hazardous Waste Combustion Facilities."
Dr. Smith has also provided expert testimony on several other occasions for chemical
industry clients in toxic tort proceedings and has authored more than 25 publications and
technical presentations en hazardous air pollutants, modeling issues and accidental
releases. His Sc.D. and M.S. degrees in Environmental Health Sciences are from
Harvard University School of Public Health, and his A.B. in Physics is from Franklin and
Marshall College.
Dr. Smith has also provided expert advice and support to clients in the chemical
and pharmaceutical industries on exposure and risk analysis, as well as emergency
response planning, preparedness and communication requirements for effective risk
management programs. This support has included overall program design, as well as
training and auditing fo:r OSHA's Process Safety Management (PSM) rule, and U.S.
EPA's Risk Management Planning (RMP) rule.
William Stubblefield
Dr. William Stubblefield is a senior environmental toxicologist with Parametrix,
Inc. in Corvallis, Oregon. He also holds a courtesy faculty appointment in the
Department Molecular and Environmental Toxicology at Oregon State University. Dr.
Stubblefield has a Ph.D. in Environmental Toxicology from the University of Wyoming,
a M.S. degree in Toxicology/Toxicodynamics from the University of Kentucky, and a
B.S. in Biology from Eastern Kentucky University.
Dr. Stubblefield has more than 15 years of experience in environmental
toxicology, ecological risk assessment, water quality criteria derivation, and aquatic and
wildlife toxicology studies. He has authored more than 50 peer-reviewed publications
and technical presentations in the areas of aquatic and wildlife toxicology and
environmental risk assessment. He is a co-editor of a recently published book entitled,
"Re-evaluation of the State of the Science for Water Quality Criteria," that specifically
examines the issues and approaches to be used in the evaluation of environmental
impacts associated with contaminants in multiple media. Dr. Stubblefield's research
efforts have looked at the fate and effects of metal and hydrocarbon contaminants in the
environment and the relationships between these contaminants in the water/sediment/soil
compartments.
He has also investigated food chain concerns through research efforts such as the
. investigation of metals transfer in resident aquatic and terrestrial organisms on Alaska's
North Slope. His most recent research uses a combination of laboratory and field
-------
methods to investigate the effects of storm water-associated short-term pulse exposures
of metals to aquatic organisms and examines the fate and disposition of storm water-
associated metals in natural systems.
About 70% of Parametrix projects are funded by municipal and other government
agencies the remainder are industrial clients. Funding for the majority of Dr.
Stubblefield's metal related work comes from industrial trade associations or not-for-
profit research organizations working in cooperation with U.S. EPA. Dr. Stubblefield is
an active member of the Society of Environmental Toxicology and Chemistry, where he
serves as the Society's vice-president, member of the Board of Directors, chairman of the
Publications Advisory Council, chairman of the Metals Advisory Group, past member of
the Editorial Board for Environmental Toxicology and Chemistry, and 2002 annual
meeting co-chair. He has been an invited participant at a number of scientific and
regulatory conferences, served on U.S. EPA peer-review panels, and frequently acts as a
technical reviewer for a number of scientific publications.
Thomas L. Theis
Professor Theis is Professor of Civil and Materials Engineering and Director of
the Institute for Environmental Science and Policy at University of Illinois at Chicago, a
center that focuses on the development of new cross-disciplinary research initiatives in
the environmental area. He was most recently at Clarkson University, where he was the
Bayard D. Clarkson Professor and Director of the Center for Environmental
Management.
Professor Theis received his doctoral degree in environmental engineering, with a
specialization in environmental chemistry, from the University of Notre Dame. His areas
of expertise include the mathematical modeling and systems analysis of environmental
processes, the environmental chemistry of trace organic and inorganic substances,
interfacial reactions, subsurface contaminant transport, hazardous waste management,
industrial pollution prevention, and industrial ecology. He has been principal or co-
principal investigator on over forty funded research projects totaling in excess of eight
million dollars, and has authored or co-authored over one hundred papers in peer
reviewed research journals, books, and reports.
He is a member of the U.S. EPA Science Advisory Board (Environmental
Engineering Committee), is past editor of the Journal of Environmental Engineering, and
serves on the editorial boards of The Journal of Contaminant Transport, and Issues in
Environmental Science and Technology. From 1980-1985 he was the co-director of the
Industrial Waste Elimination Research Center (a collaboration of Illinois Institute of
Technology and University of Notre Dame), one of the first Centers of Excellence
established by the U.S. EPA. In 1989 he was an invited participant on the United
Nations' Scientific Committee on Problems in the Environment (SCOPE) Workshop on
Groundwater Contamination, and in 1998 he was invited to by the World Bank to assist
in the development of the first environmental engineering program in Argentina. Among
his current projects is the Environmental Manufacturing Management Program, one of
10
-------
the Integrative Graduate Education Research and Training (IGERT) grants of the
National Science Foundation, which involves research on industrial pollution prevention
problems emphasizing a systems approach.
Louis Thibodeaux
Louis Joseph Thibodeaux is currently the Jesse Coates Professor in the Gordon A.
and Mary Cain Department of Chemical Engineering, College of Engineering, Louisiana
State University, Baton Rouge, LA. His terminal degree is a Ph.D. in chemical
engineering and presently his teaching, research and service is dominated by the field of
environmental chemodynamics, or chemical fate and transport in multimedia
compartments of the natural environment. Current areas of research expertise include
chemical release processes to water from sediment beds and to air from soil-like dredged
materials as well as chemical releases to water and air from environmental dredging
activities. The key area of educational expertise is the textbook entitled:
ENVIRONMENTAL CHEMODYNAMICS in its 2nd Edition, published by J.
Wiley(NY) in 1996. It i> used by practitioners worldwide and by numerous universities
in engineering, environmental chemistry, geosciences and other environment oriented
academic departments. He is the Emeritus Director of the U.S. EPA funded South and
Southwest Hazardous Substance Research Center, headquartered at LSU and Directed by
Danny D. Reible.
Professor Thibodeaux has served on advisory committees for the U.S. EPA, U.S.
ACE, DOD, DOE, NRC and the private sector; all being related to environmental
chemodynamic issues. He is a member of the Environmental Division of the American
Chemical Society, the Society of Environmental Toxicology and Chemistry, and the
Environmental Division of the American Institute of Chemical Engineers.
Professor Thibodeaux is fully employed by LSU doing research and teaching
both graduate and undergraduate students. He also serves on the editorial board of
several environmental journals and is presently receiving grant and/or contract support on
four research projects from the U.S. EPA and the U.S. ACE. He receives research project
funds through the cooperative agreement U.S. EPA/LSU in the S/SW Haz. Res. Ctr.,
ORD, Wash, DC. He also receives research funds from the U.S. Army Corps of
Engineers for the ERDC or Waterway Experiment Station, Vicksburg, MS.
Curtis Travis
Dr. Curtis Travis has more than 25 years experience in the energy and
environmental business sector and has published widely in the areas of environmental
policy, molecular biology, and risk analysis. He holds a B.S. and M.S. in Mathematics
from California State University (Fresno) and earned a Ph.D. in Applied Mathematics
from the University of California (Davis). He is an internationally recognized expert in
the field of risk analysis, and was the founding Director of the Center for Risk
Management at Oak Ridge National Laboratory, where he was employed for 18 years.
-------
He has worked in many areas of risk analysis including multimedia modeling,
food chain uptake, pharmacokinetics, interspecies extrapolation, dose-response, and risk
policy. Recently, he has worked on the cleanup of DOE hazardous waste sites, risk
assessment for antimicrobial drug use in animals, and security issues related to food
infrastructure in the United States.
Dr. Travis has authored over 270 publications, 8 books, and is on the editorial
board of seven international journals. He has served on numerous National Academy of
Science panels and governmental and private advisory boards. He is a past President and
Fellow of the International Society of Risk Analysis and served as Editor-in-Chief of
Risk Analysis: An International Journal for 17 years. Dr. Travis is a private consultant
with his own firm, Quest Technologies. Almost all his work is for government agencies:
the Department of Energy, the Food and Drug Administration, and the Department of
Agriculture. He has received no financial support from EPA in the past 10 years, other
than in a review capacity.
12
-------
------- |