United States EPA Science Advisory EPA-SAB-RAC-ADV-01-002
Environmental Board (1400A) une 2001
Wachin/^+nn
&EPA GENII VERSION 2
ENVIRONMENTAL
RADIATION DOSIMETRY
SYSTEM: AN SAB
ADVISORY
REVIEW OF THE ORIA'S USE AND
ADAPTATION OF THE GENII
VERSION 2 ENVIRONMENTAL
RADIATION DOSIMETRY SYSTEM
BY THE EPA SCIENCE ADVISORY
BOARD (SAB)
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
June 26, 2001
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
EPA-SAB-RAC-ADV-01-002
The Honorable Christine Todd Whitman
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Re: A Review (Advisory) of ORIA' s Use and Adaptation of The GENTJ Version 2
Environmental Radiation Dosimetry System
Dear Governor Whitman:
At the request of EPA's Office of Radiation and Indoor Air (ORIA), the Radiation Advisory
Committee (RAC) of the Science Advisory Board (SAB) reviewed ORIA's use and adaptation of
version 2 of GENII (GENeration U computer programs) developed by Pacific Northwest National
Laboratory. GENII v.2 is a software package (GENII v.2 code) for use as a tool to conduct generic
or site-specific environmental radiation dose or risk estimates. The GENII v.2 code incorporates a
suite of computer modules and is a update and modification of the earlier GENII Environmental
Radiation Dosimetry System. The RAC convened in public meeting in Washington, DC on April
25-27, 2000 to receive briefings from ORIA staff, take public comment, and discuss the relevant
issues. The resulting report addresses the specific Charge questions as well as other issues beyond the
Charge identified during the public meetings. The RAC's report is designated an "Advisory," since the
ORIA's document is considered to still be a "work in progress," rather than a final document. The
RAC expects that ORIA will seek additional peer review before their document is finalized.
In general, the RAC found the GENII v.2 code to include the appropriate modules and was
especially pleased that the code has the capability of providing stochastic estimates of risk rather than
simply deterministic point estimates. The Committee considers the GENII v.2 code to be a useful
addition to the dose and risk assessment toolbox. Since this is a "work in progress," future additions
and changes can be made to the code to add to its applicability and improve its flexibility, accuracy,
and transparency.
The complete Charge for this review is provided in section 2.2 of the enclosed report. The
RAC's findings on each element of the Charge are summarized below.
-------
The first element of the Charge asked if FRAMES is a reasonable platform for supporting an
integrated system of tools for meeting the diverse environmental modeling needs of ORIA. The RAC
concluded that the FRAMES platform is a powerful and flexible tool and that GENII v.2 will be very
useful to EPA. However, ORIA must develop a clear vision and attendant mission statement for
FRAMES and GENII v.2 that details the "diverse environmental modeling needs" of EPA as a basis
for determining whether FRAMES is the best tool to meet these needs. In other words, ORIA must
have a good idea of who will use the code as well as how, and for what purposes, it will be used. The
RAC found the SUM3 module to be a particularly useful addition to the GENII v.2 code. However,
the Committee notes that the platform is untested and needs validation and verification.
Although the RAC was not provided with many details about the structure of FRAMES, it
appears that it has only been used, to date, for modules that do not need to exchange information with
high frequency in time or at high spatial resolution. The Committee is concerned that linking modules at
numerous points in space or with high frequency might prove cumbersome in FRAMES. We
recommend that ORIA consult with other EPA offices developing similar type modeling systems (e.g.,
the Office of Research and Development (ORD) and the Office of Air Quality Planning and Standards
(OAQPS)). ORIA also needs to consider incorporation of other models and generalizing the
FRAMES interface to accept other types of models and to allow feedback among compartments. The
RAC provided a number of detailed recommendations for additions and improvements to enhance the
usefulness of the code.
The second Charge question addressed the adequacy of the GENII v.2 codes in addressing
environmental transport of radionuclides; the need for additional features (or modules); and approaches
for modeling exposures to radon, tritium, and carbon-14.
The RAC found the environmental transport modeling capabilities for air and surface water
releases of radionuclides to be adequate for screening purposes but not necessarily appropriate for
detailed analysis or emergency situations. The Committee came to much the same conclusions
concerning the modeling of exposures to radon, tritium, and carbon-14. The enclosed reports details
specific recommendations to address these issues.
The third Charge element sought the Committee's advice on the adequacy of the examples and
documentation provided with the GENII v.2 software, and on ways of presenting the output and
uncertainty results.
The RAC commends the efforts of ORIA to make FRAMES and GENII v.2 "user friendly."
However, the documentation and the presentation of issues relating to uncertainty/variability are both
"works in progress" and not adequate at this time. ORIA is continuing its efforts in this direction.
Again, the RAC recommended several actions and enhancements to the documentation to improve the
usefulness of GENII v.2.
In addition to the formal Charge, the Committee commented on several other issues. These
include:
-------
a) The need for the dose and risk estimates to be as unbiased as possible. The high level
of conservatism apparently built in to the GENII v.2 code is not sufficiently transparent
to the user, who must be able to decide explicitly on the level of conservatism
appropriate for the particular application.
b) The conservative nature of the code may lead to excessively conservative dose
estimates (i.e., higher than more realistic assumptions might produce), resulting
unnecessarily costly controls and unnecessary expenditures in site cleanup operations.
The RAC strongly encourages ORIA to provide more realistic bounds on their dose
and risk estimates.
We appreciate the opportunity to review these issues, and look forward to your response.
Sincerely,
/S/
Dr. William Glaze, Chair
EPA Science Advisory Board
/S/
Dr. Janet Johnson, Chair
Radiation Advisory Committee
EPA Science Advisory Board
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board, a
public advisory group providing extramural scientific information and advice to the Administrator and
other officials of the Environmental Protection Agency. The Board is structured to provide balanced,
expert assessment of scientific matters related to problems facing the Agency. This report has not been
reviewed for approval by the Agency and, hence, the contents of this report do not necessarily
represent the views and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names or commercial products
constitute a recommendation for use.
Distribution and Availability: This EPA Science Advisory Board report is provided to the EPA
Administrator, senior Agency management, appropriate program staff, interested members of the
public, and is posted on the SAB website (www.epa.gov/sab). Information on its availability is also
provided in the SAB's monthly newsletter (Happenings at the Science Advisory Board). Additional
copies and further information are available from the SAB Staff [US EPA Science Advisory Board
(1400A), 1200 Pennsylvania Avenue, NW, Washington, DC 20460-0001; 202-564-4546].
-------
ABSTRACT
At the request of the EPA's Office of Radiation and Indoor Air (ORIA), the Radiation
Advisory Committee (RAC) of the Science Advisory Board (SAB) reviewed the GENII v.2 computer
code developed by Pacific Northwest National Laboratory (PNNL) to perform dose and risk
assessments of environmental releases of radionuclides. The code builds a conceptual site model
linking modules through the FRAMES platform. The RAC found the GENII v.2 code to include
appropriate modules and concluded that FRAMES provides a reasonable and flexible platform.
However, the RAC recommended adding newer models to the GENII v.2 code, specifically for air
dispersion and ground and surface water transport of radionuclides as well as models capable of
handling emergency conditions. The RAC was concerned about the potential for non-transparent and
unrealistically conservative (i.e., higher than more realistic assumptions might produce) risk estimates.
The RAC commended ORIA for including the capability of providing stochastic estimates of
risk through the Sensitivity/Uncertainty Multimedia Modeling Module (SUM3) driver but questioned its
ability to investigate the degree of conservatism in the code, identify the importance of input parameters,
and provide useful measures of uncertainty.
In general, the RAC found the GENII v.2 code to be a useful addition to the dose and risk
assessment toolbox. The RAC suggested several strategies for making the code more user friendly,
including improvement in the documentation and User's Guide as well as providing training for potential
users. The RAC encouraged ORIA to develop a vision and an attendant mission statement for
FRAMES and GENII v.2 as a basis for evaluating these tools.
KEYWORDS: Radionuclide risk assessment; radionuclide dose assessment; dose assessment model;
GENII; stochastic model.
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
RADIATION ADVISORY COMMITTEE
April 25-27, 2000 GENII v.2 Review
CHAIR
Dr. Janet A. Johnson, Shepherd Miller, Inc., Ft. Collins, CO
MEMBERS
Dr. Lynn R. Anspaugh, University of Utah, Salt Lake City, UT
Dr. Vicki M. Bier, University of Wisconsin, Madison, WI
Dr. Bruce B. Boecker, Lovelace Respiratory Research Institute, Albuquerque, NM
Dr. Stephen L. Brown, R2C2 Risks of Radiation & Chemical Compounds, Oakland, CA
Dr. Gilles Bussod, Los Alamos National Laboratory, Los Alamos, NM
Dr. Thomas F. Gesell, Idaho State University, Pocatello, ID
Dr. Jill Lipoti, New Jersey Dept. Of Environmental Protection, Trenton, NJ
Dr. Ellen Mangione, Colorado Department of Public Health and Environment, Denver, CO
Dr. Genevieve S. Roessler, Radiation Consultant, Elysian, MN
CONSULTANTS
Dr. Richard W. Hornung, Institute for Health Policy and Health Services Research, University of
Cincinnati, Cincinnati, OH
Dr. Wu- Seng Lung, Department of Civil Engineering, University of Virginia, Charlottesville, VA
Dr. Jana Milford, Department of Mechanical Engineering, University of Colorado, Boulder, CO
Dr. Bobby R. Scott, Lovelace Respiratory Research Institute, Albuquerque, NM1
Dr. James E. Watson, Jr., Professor, Environmental Science & Engineering Department, University
of North Carolina, Chapel Hill, NC
Did not attend the public meeting of April 25-27, 2000, but participated in the review.
Ill
-------
SCIENCE ADVISORY BOARD STAFF
Ms. Melanie Medina-Metzger, Designated Federal Officer, US EPA Science Advisory Board
(1400A), US EPA, 1200 Pennsylvania Avenue, NW, Washington, DC 20460
Mr. Samuel Rondberg, Designated Federal Officer, US EPA Science Advisory Board (1400A), US
EPA, 1200 Pennsylvania Avenue, NW, Washington, DC 20460
Ms. Diana L. Pozun, Management Assistant, US EPA Science Advisory Board (1400A), US EPA,
1200 Pennsylvania Avenue, NW, Washington, DC 20460
Ms. Dorothy Clark, Management Assistant, US EPA Science Advisory Board (1400A), US EPA,
1200 Pennsylvania Avenue, NW, Washington, DC 20460
IV
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 6
2.1 Background 6
2.2 Charge 6
3. RESPONSE TO THE CHARGE 8
3.1 (Charge Question 1) Is FRAMES a Reasonable Platform for Supporting an Integrated
System of Tools for Meeting the Diverse Environmental Modeling Needs of ORIA? . 8
3.2 (Charge Question 2) Are the GENII v.2 Environmental Transport Models Adequate?; (b)
What Additional Features (Or Modules) Should Be Added?;
and (c) What Approaches Should Be Used to Model Exposures to Radon,
Tritium, and Carbon-14? 10
3.2.1 Are the models adequate? 10
3.2.2 What additional features should be added? 12
3.2.2.1 Recommendations for "Major" Additions 12
3.2.2.2 Recommendations for "Minor" Additions 13
3.2.3 What approaches should be used to model exposures to radon, tritium,
and carbon-14? 14
3.2.3.1 Tritium and 14C 14
3.2.3.2 Radon 15
3.3 Charge Question 3: (a) Are the examples and documentation provided with the software
adequate and helpful? (b) How should the output and uncertainty
results be presented? 15
3.3.1 Are the Examples and Documentation Helpful? 16
3.3.1.1 General Comments 16
3.3.1.2 SUM3 Documentation 17
3.3.1.3 User Training 18
3.3.2 How should the uncertainty results be presented? 19
3.3.2.1 Statistical Interpretation of the Uncertainty Results 19
3.3.2.2 Conservatism in the Uncertainty Analysis 20
3.3.2.3 Variability versus Uncertainty 21
3.3.2.4 Suggested Extensions to SUM3 21
4. COMMENTS BEYOND THE CHARGE 23
4.1 Potential Use of FRAMES 23
4.1.1 Generic Assessment of Source Categories in Various Settings 23
4.1.2 Site Specific Use 23
4.1.3 Emergency Response - Plume Phase 23
4.1.4 Emergency Response - Ingestion Phase 24
-------
4.1.5 Emergency Response - Recovery Phase 24
4.1.6 Unusual Situations 24
4.1.7 Other Uses 25
4.2 Other Comments Beyond the Charge 25
APPENDIX A EDITORIAL COMMENTS A-l
GLOSSARY OF ACRONYMS AND TERMS G-l
REFERENCES R-l
VI
-------
1. EXECUTIVE SUMMARY
The Environmental Protection Agency's (EPA) Office of Radiation and Indoor Air (ORIA) has
requested advice from the Radiation Advisory Committee (RAC) of the Science Advisory Board
(SAB) with regard to the strengths and limitations of the GENII v.2 (GENeration n computer
programs) radiation dose and risk assessment software package (GENII v.2 code) ORIA is adapting
the GENII code so that it may replace other computer models currently in use. The code is intended to
be used for prospective analyses. The RAC's report is designated an "Advisory," since the ORIA's
document is considered to still be a "work in progress," rather than a final document. The RAC
expects that ORIA will seek additional peer review before their document is finalized.
The GENII v.2 code is a one of a series of computer software packages developed by Pacific
Northwest National Laboratory (PNNL). The GENII v.2 code user builds a conceptual site model,
linking modules through the Framework for Risk Analysis in Multimedia Environmental Systems
(FRAMES) platform, also developed by PNNL. In contrast to previous versions, the GENII v.2 code
is completely stochastic through the use of the FRAMES SUM3 (Sensitivity/Uncertainty Multimedia
Modeling Module) driver.
In general, the RAC found the GENII v.2 code to include the appropriate modules and was
especially pleased that the code has the capability of providing stochastic estimates of risk rather than
simply deterministic point estimates. However, the RAC is concerned with the conservative nature of
the code and warns that excessively conservative dose estimates (i.e., higher than more realistic
assumptions might produce) may lead to unnecessarily costly controls and unnecessary expenditures in
site cleanup operations. The RAC strongly encourages ORIA to provide more realistic bounds on their
dose and risk estimates.
The complete Charge for this review is provided in section 2.2, following. The RAC's findings
on each element of the Charge are summarized below.
The first element of the Charge asked if FRAMES is a reasonable platform for supporting an
integrated system of tools for meeting the diverse environmental modeling needs of ORIA. The RAC
concluded that the FRAMES platform is a powerful and flexible tool and that GENII v.2 will be very
useful to EPA. However, ORIA must develop a vision and an attendant mission statement for
FRAMES and GENII v.2 that details the "diverse environmental modeling needs" of EPA as a basis
for determining whether FRAMES is the best tool to meet these needs. ORIA must have a good idea
of who will use the code as well as how and for what purposes it will be used. The RAC found the
SUM3 module to be a particularly useful addition to the GENII v.2 code.. However, the Committee
notes that the platform is untested and needs validation and verification.
Although the RAC was not provided with many details about the structure of FRAMES, it
appears that it has only been used, to date, for modules that do not need to exchange information with
high frequency in time or at high spatial resolution. The Committee is concerned that linking modules at
1
-------
numerous points in space or with high frequency might prove cumbersome in FRAMES. We
recommend that ORIA consult with other EPA offices developing similar type modeling systems (e.g.,
the Office of Research and Development (ORD) and the Office of Air Quality Planning and Standards
(OAQPS)). ORIA also needs to consider incorporation of other models and generalizing the
FRAMES interface to accept other types of models and to allow feedback among compartments. The
RAC recommends the following additions and improvements to enhance the usefulness of the code:
a) Modify the interface with FRAMES to allow access to more complex, site specific
hydro geologic flow and transport process models in order to better deal with
groundwater issues.
b) Divide the Atmospheric transport interface into "near-field" and "far-field" components
The second Charge question addressed the adequacy of the GENII v.2 environmental transport
models for radionuclides; the need for additional features (or modules); and approaches for modeling
exposures to radon, tritium, and carbon-14.
The RAC found the environmental transport modeling capabilities for air and surface water
releases to be adequate for screening purposes but not necessarily appropriate for detailed analysis or
emergency situations. The RAC recommends the following additions and improvements to enhance the
usefulness of the code:
a) Evaluate newer air dispersion models being developed by the Agency, such as
AERMOD and AERMET for inclusion in GENII v.2.
b) Augment the air dispersion module in GENII v.2 to accommodate log-normal particle
size distributions with different Aerodynamic Median Activity Diameters (AMAD).
c) Evaluate the use of more complex environmental radionuclides transport modeling
inputs required for catastrophic events (e.g., fires, explosions, accidents and terrorist
acts) which involve "near-field" physics not captured by the generalized GENII v.2
Atmospheric Transport module. The highly simplified surface water transport module
should be augmented to take into account more than a single input point and dilution by
confluences. In addition, ORIA should consider incorporating sedimentation and
resuspension into the surface water transport models.
d) Add estuary or tidal effect model.
e) Evaluate the use of existing, more detailed groundwater (GW) flow and transport
process models which may be required to describe site-specific conditions. (The
Committee recommends against building a new GW module for GENII v.2.)
Other, lower priority, recommendations are included in Section 3.
-------
The RAC found that the approach in GENII v.2 for modeling exposures to 3H and 14C are
adequate for screening level analyses but would most likely result in gross overestimates of dose and
risk in many situations. The GENII v.2 models for 3H are appropriate only for tritiated water (HTO)
when, in fact, most of the past major releases have be hydrogen gas (HT) that would present very
different doses and risks. In addition, the model assume instantaneous equilibrium between 14CO2 in air
and 14C in plants. The GENTI v.2 user should be warned that the true doses may be very different from
the calculated doses as a result of these unrealistic assumptions.
While the GENII v.2 code includes a module for exposure to radon indoors due to release of
radon from domestic water supplies, it should also address the dose due to radon emanation from soils
contaminated with 226Ra and 228Ra. It is not clear that it does so in its present form.
The third Charge element sought the Committee's advice on the adequacy of the examples and
documentation provided with the GENII v.2 software, and on ways of presenting the output and
uncertainty results.
The RAC commends the efforts of ORIA to make FRAMES and GENH v.2 "user friendly."
However, the documentation is a "work in progress" and is not adequate at this time. We understand
that ORIA is continuing its efforts in this direction. The RAC recommends several actions and
enhancements to the documentation to improve the usefulness of GENII v.2.
a) Involve the end-users in development of the documentation and expand the text using
readily understandable terms.
b) Use more example cases to document the capabilities of the code and expand on the
block-diagraming of module components and their results.
c) Include some basic information on the model formulation, assumptions, and limitations
in the User's Guide.
d) Work through "word problems" as examples, showing the data input and output of
each screen.
e) Check carefully the information in the User's Guide for accuracy and for applicability to
different types of hardware. (When one member of the RAC tried to use the code he
discovered that the instructions did not apply to his machine.) If the instructions are
machine-dependent, the user should be so warned.
f) Detail the format and specifications necessary for coupling FRAMES to other codes.
The SUM3 documentation does not adequately explain a number of issues. In particular, the
RAC is concerned with the clarity of the documentation on Monte Carlo and Latin Hypercube
sampling. If Latin Hypercube sampling is employed in the uncertainty analysis, the user should be
-------
warned that the calculation of confidence limits for the means and percentiles of distributions may not be
straightforward.
The RAC discussed the proper use of the software in considerable detail. User training will be
an important part of assuring that the el is GENII v.2 code is used appropriately. Misuse of the code
could result in wrong or misleading dose estimates. The RAC suggests the following actions as
possibilities for user education and troubleshooting:
a) Conduct formal classes
b) Establish a "User's Group"
c) Develop a phased approach to learning and using the software, allowing increased
flexibility in parameter choices as the user becomes more experienced.
d) Post a list of "Frequently Asked Questions" with the answers on the web site.
The RAC is concerned with certain aspects of the uncertainty analysis as treated by the SUM3
module in FRAMES:
a) The SUM3 module is designed to be a tool for investigating the degree of conservatism
in the code but it will do so only if the user identifies the most conservative parameters
in the models and replaces them with reasonable distributions. If the choice of
parameters and assignment of parameter distributions are not transparent or are buried
within the basic philosophy of the modeling, the conservatism in the code may not be
adequately defined by the use of SUM3.
b) It is important in the discussion of uncertainty to distinguish between uncertainty and
variability. Uncertainties can, in principle, be reduced by further investigation; variability
is inherent in the system.
c) The SUM3 documentation states that uncertainty analysis can be used to understand the
importance of input parameters; however, the software does not provide the capability
for such calculations. However, there are numerous ways to approach this problem,
ranging from sensitivity analysis to the use of quantitative importance measures.
Uncertainty analysis by itself does not provide information on the importance of input
parameters. The RAC suggests the use of sensitivity analyses for the determination of
the sensitivity of the code's' results to input parameter variations.
In addition to the formal Charge, the Committee commented on several other issues.
-------
As noted above, the ORIA did not provide the RAC with a clear mission statement for the
GENII v.2 code so the RAC, in its deliberations, considered several situations in which the code might
be employed:
a) The code was designed to evaluate doses prospectively but could be adapted for use in
retrospective analyses.
b) A component to model the impact of various human interventions, such as construction
of a cap, that would allow the user to compare different reclamation strategies would be
a useful addition to the code.
c) The emergency response capabilities of the code could be enhanced by the ability to
use real-time meteorological data.
d) It would be useful for the code to have various levels of FRAMES that vary in
complexity as the user becomes more familiar with it. The FRAMES platform could
also be adapted to help the user decide whether the suite of models selected is
appropriate for the situation by including some front end prompts.
The RAC is concerned that a high level of conservatism may be built in to the GENII v.2 code.
The dose and risk estimates should be as unbiased as possible. The user could then decide on the level
of conservatism appropriate for the particular application.
In general, the RAC found the GENII v.2 code to be a useful addition to the dose and risk
assessment toolbox. Since this is a "work in progress," additions and changes can be made to the code
to add to its applicability and improve its flexibility, accuracy, and transparency.
-------
2. INTRODUCTION
2.1 Background
The GENII computer code was developed by PNNL to provide a state-of-the-art set of
programs for calculating radiation dose and risk from radionuclides released to the environment.
Although it was initially designed to be applied at the Hanford Reservation, flexibility to accommodate a
wide variety of types of sites was built into the code. The Environmental Protection Agency (EPA)
Office of Radiation and Indoor Air (ORIA) is in the process of updating and adapting the GENII family
of codes so that it may replace the other computer codes currently being used to evaluate radiation
dose and risk from environmental radioactivity. GENII v.2 includes the most recent computer models
for estimating terrestrial, atmospheric, and surface water transport of radionuclides as well as
International Commission on Radiological Protection (ICRP)-based dose coefficients and Federal
Guidance Report No. 13 risk coefficients. The User's Guide (EPA 2000) states that GENII v.2 is
"completely stochastic using the FRAMES (Framework for Risk Analysis in Multimedia Environmental
Systems) SUM3 (Sensitivity/Uncertainty Multimedia Modeling Module) driver."
As noted, GENII v.2 uses FRAMES, developed by PNNL as a platform for interfacing and
linking various modules from one or more models. The user builds a Conceptual Site Model, linking
media icons, to represent the flow of radionuclides through the environment. Simulation modules in
GENII v.2 allow the user to estimate doses and risks to specific individuals and populations. The user
may also add other computer modules to evaluate radionuclides transport and pathways not included in
the GENII v.2 modules.
EPA has requested advice from the Radiation Advisory Committee (RAC) of the Science
Advisory Board (SAB) with regard to the strengths and limitations of the proposed code (GENII v.2)
for conducting generic and site-specific radiation dose and risk assessments. In addition, EPA
requested advice as to whether FRAMES is an appropriate platform for linking the transport and
dose/risk estimation modules.
The Committee met on April 25 and 26, 2000 to receive a briefing from EPA on the code, hear
public comment, and develop appropriate advice .
2.2 Charge
The Charge included three elements:
a) Question #1 :Is FRAMES a reasonable platform for supporting an integrated system of
tools for meeting the diverse environmental modeling needs of ORIA?
b) Charge Question #2: (1) Are the GENII v.2 environmental transport models adequate?
(2) What additional features (or modules) should be added to GENII v.2? (3) What
-------
approaches should be used in GENII v.2 to model exposures to radon, tritium, and
carbon-14?
c) Charge Question #3: (1) Are the examples and documentation provided with the
software adequate and helpful? (2) How should the output and uncertainty results be
presented?
-------
3. RESPONSE TO THE CHARGE
3.1 (Charge Question 1) Is FRAMES a Reasonable Platform for Supporting an Integrated
System of Tools for Meeting the Diverse Environmental Modeling Needs of ORIA?
In general, the GENII v.2 code, as implemented within the FRAMES modeling framework,
appears to be a powerful and flexible tool for analyzing human dose and risk from many scenarios for
releases of radionuclides to the environment. The inclusion of a stochastic add-on, SUM3, for analyzing
the effects of variability and uncertainty is a particularly attractive feature in comparison to older,
deterministic-only models. GENII v.2 appears to be a very innovative approach which will be useful to
EPA and perhaps other agencies.
The RAC believes that it is important for ORIA to develop a clear vision and an attendant
mission statement for FRAMES that enumerates their diverse environmental modeling needs so that a
determination can be made as to whether those needs are best met through a FRAMES platform.
Lacking such a clear statement regarding potential uses of the code, the RAC assumed several general
types of situations in which FRAMES might be valuable. Specific situations, particularly those involving
emergency response, are discussed in detail in Section 3.4 (Issues Beyond the Charge).
Use of an object-oriented, open-architecture system that accommodates plug-ins of various
models and provides compatibility with a variety of platforms is a good approach. Inclusion of the
SUM3 uncertainty analysis module is extremely valuable. Although the RAC was not provided with
many details about the structure of FRAMES, it appears that it has only been used, to date, for
modules that do not need to exchange information with high frequency in time or at high spatial
resolution. Linking modules at numerous points in space or with high frequency might prove
cumbersome in FRAMES. ORIA should track and exchange ideas with other organizations in the
Agency that are undertaking similar efforts such as the TRIM modeling system in the Office of Air
Quality Planning and Standards (OAQPS) and the MODELS-3 system in the Office of Research and
Development (ORD).
Based on the information provided to the RAC, FRAMES appears to be a versatile platform
used to integrate several models pertaining to atmospheric, hydrologic, biotic and radionuclide transport
problems related to specific dose/risk assessments. FRAMES seems to facilitate integration by
allowing:
a) The merging of information gained by several models.
b) The introduction of a versatile platform with the potential to accommodate different
model components each with diverse complexities.
c) The display of input and output parameters, so as to impart transparency to the
realizations.
-------
d) Reasonable computational times, which allow prompt feedback to end user(s).
e) Access to a built-in Monte Carlo module.
f) Access to multiple realizations for sensitivity analyses.
g Access to a multi-disciplinary approach to the problem under investigation.
h) The use of a block diagram to characterize each module.
The RAC has some cautionary comments with regard to the limitations of the FRAMES
platform and some recommendations for extending its usefulness:
a) The platform is untested and needs some form of thorough verification and testing
b) "Data specification" may limit its versatility by forcing the use of simple (inadequate)
models.
c) The coupling of all model results is unidirectional even though feedback between
compartments may exist.
d) FRAMES is limited in its present treatment of the groundwater component (i.e., not
specified).
e) FRAMES may be limited to using parameterized models.
f) FRAMES does not allow use of real-time meteorology for emergency response.
g) FRAMES may be limited to prospective analysis.
h) FRAMES is limited to a WINDOWS platform.
i) The linkages and choice of modules for FRAMES should be kept versatile to attract
multiple users.
j) The FRAMES platform input needs should be specified to favor the creation of
FRAMES-compatible interfaces in other groundwater models.
ORIA needs to consider the incorporation of other physically-based models and generalization
of the interface for use with particle-track, analytical, full dynamic and complex multi-dimensional
realizations. ORD is developing MODELS-3 to perform these functions.
-------
3.2 (Charge Question 2) Are the GENII v.2 Environmental Transport Models Adequate?; (b)
What Additional Features (Or Modules) Should Be Added?; and (c) What Approaches Should
Be Used to Model Exposures to Radon, Tritium, and Carbon-14?
Charge Question 2 contains three parts. Each part is considered separately in the following
sections. However, the response to Charge Question 2b is, to some degree, related to the response to
Charge Question 2a.
3.2.1 Are the models adequate?
The structure of the GENII v.2 code is so complex that no individual is likely to be able to
comment meaningfully on its overall scientific merits. Undoubtedly, there are elements of the modeling
that might be controversial among modeling specialists in those particular areas. However, the built-in
flexibility of GENII v.2 with respect to model choices and the ability of FRAMES to accept different
modules for different aspects of the overall health risk modeling obviate much of the concern over
model details.
GENII v.2 appears to provide adequate environmental radionuclides transport modeling
capabilities for screening analyses of releases in air and surface water. The air dispersion model
included in GENII v.2 is generally consistent with the level of sophistication found in EPA's Industrial
Source Code (ISC) dispersion model, which is widely used for permitting applications. However,
ORIA may want to evaluate the new AERMOD dispersion model that has recently been developed by
EPA in collaboration with the American Meteorological Society to determine whether some of the
more up-to-date algorithms included in AERMOD could be used in GENII v.2. In particular,
AERMOD provides improved treatment of dispersion in convective conditions, as well as dispersion in
complex terrain. The AERMET system provides the capability of interpolating data from multiple
meteorological stations that is missing from GENII v.2. It should be noted in the GENII v.2
documentation that the current dispersion modeling capabilities are limited to open, flat terrain. One
notable advantage of the GENII v.2 treatment relative to some other air dispersion models is
conservation of mass associated with deposition or scavenging losses. The source term adjustment for
these processes is a good approach.
The architectural framework of the GENII v.2 and FRAMES software products are well
designed for use by EPA and other end-users for preliminary assessments and screening purposes.
This design has the advantage of being computationally efficient. However, the Agency may want to
access more physically realistic process models in some cases, depending on the degree of complexity
of the problem. As an example, the basic diffusion model from the Atmospheric Transport module
used in the GENII v.2 code is well accepted and commonly used in the atmospheric sciences
community for simple atmospheric transport problems. However, the straight-line Gaussian and
Lagrangian-puff models were designed for "well-behaved" pollution transport from chimney "stacks"
and do not apply to more critical scenarios involving fires, explosions and accidental or terrorist aerial
releases of contaminants, which the EPA may be called on to evaluate. Under such conditions, the
physics and chemistry of the problem require the use of more sophisticated, physically based models
10
-------
(e.g., RAMS, FIRETEC, and FflGRAD). While the insertion of these or similar computationally
complex models onto the GENII-FRAMES platform is not recommended at this time, it may be
necessary to provide a link between the results from complex process models and the EPA software
products. In order to facilitate this, one might suggest that "near-field" and "far-field" inputs to the
Platform be envisaged. In complex catastrophic events that the EPA may be required to assess, the
"near-field" physics of the problem can be drastically different from the "far-field" phenomena and
result in different exposures and risks to the public. If "near-field" results from complex process models
could be input as source terms for a GENII v.2 "far-field" atmospheric module, it would allow the EPA
to better assess the consequences of the catastrophic event.
For similar reasons, it is proposed that the FRAMES platform be modified to allow access to
more complex hydrogeologic flow and transport process models. This is because groundwater (GW)
models are very site-specific and therefore the complexity and assumptions vary greatly. It would
therefore be computationally inefficient to build an all-encompassing GW module for GENII v.2.
However, the EPA could identify several existing flow and transport models (e.g., the PRESTO family
of codes; MODFLOW, FEHM, TOUGH, FLOTRAN, TRACR3D, etc.) for which the output files
could be modified to be compatible with the EPA platform proposed (i.e., GENII-FRAMES).
Alternatively, with distributed computing becoming more common in the future, the computer
resource issue associated with running process models as part of such an analysis should become more
tractable. If the GENII v.2 and FRAMES software are used as the "driver" programs that execute
process models, then the data can be passed to and from the process-model codes through shared data
arrays or through subroutine calls in which information is passed through the arguments of the
subroutine. A similar applications system is used in GOLDSEVI-V6.01, a dynamic, probabilistic
simulation program (Golder Associates, 1999). In either case, this solution allows continued
development of the GENII-FRAMES computational software system in parallel with its use. The main
challenge to such an approach then becomes the issue of "transparency" in the overall model that occurs
when physically based, complex models are used. In the near future however, the EPA will increasingly
depend on complex software systems for addressing critical environmental issues. As a result, failure to
capture the proper physical reality in risk assessments and policy management will become increasingly
serious. The solutions proposed above should help mitigate this problem and with proper Verification
and Validation (V&V) protocols, result in an increasing use of a rigorous EPA screening process.
The number of particle size classes allowed in the system needs to be extended, as particle size
is a very important parameter in governing deposition patterns during transport as well as deposition in
the human respiratory tract. The sites of deposition in the respiratory tract can, in turn, influence both
the subsequent disposition of the inhaled material in the body and the doses received by various body
organs and tissues. At the present time, the atmospheric transport module in GENII v.2 only
accommodates one particle size. Provision is made in GENII v.2 to use the most recent ICRP
guidance for describing the dosimetry of internally deposited radionuclides. These reports use, for the
most part, the new ICRP lung model that provides extensive information on the effects of aerosol
particle size on the deposition and disposition of inhaled materials (ICRP 1989, 1994, 1995a, 1995b,
11
-------
and 1996). Thus, the ability to use only one particle size in the atmospheric transport module handicaps
the subsequent analyses that can accommodate differences in particle size.
To improve the credibility and flexibility of the GENII v.2 code, the RAC recommends that a
provision be added to the GENII v.2 air transport module to accommodate log-normal particle-size
distributions with different activity median aerodynamic diameters (AMAD). At a minimum, it would be
desirable to study the transport of a particle-size distribution with an AMAD of 1 um, the default
parameter for environmental exposures given by the ICRP. If dealing with an aerosol size distribution
characterized by a median value is not possible, provision should be made to study several different
individual particles sizes.
The models used in GENII v.2 should be reviewed on an ongoing basis as improvements are
made and new models become available. Additional capability should be added to GENII v.2, as
recommended in section 3.2.2 of this advisory.
3.2.2 What additional features should be added?
The answer depends, in part, on the intended use of the code. As noted previously, ORIA
should provide a clear mission statement for GENII v.2. Given the limitations of its understanding of
the purpose for which ORIA intends to use GENII v.2, the RAC has several suggestions for
enhancement of the code.
In all cases the parameterized model should be tested through verification, calibration and if
possible validation of some or all of its components. The User Guide states that all steps of the user
code development have been documented and tested and the code's implementation of major transport
and exposure pathways have been verified by manual calculations for a subset of radionuclides.
However, the RAC believes that GENII v.2 will require further peer review and bench marking of the
full model if it is used by Agency. The RAC recommends that bench marking involve comparison to
more complex, physically based models and analytical solutions.
The RAC considered where there are missing pieces in the GENII v.2 modules and where
GENII v.2 could be expanded to enhance its usefulness for a variety of situations. The additional
features recommended by the RAC have been subjectively classified as "major" or "minor" additions in
order to assist ORIA in setting priorities.
3.2.2.1 Recommendations for "Major" Additions
A major deficiency in GENII v.2 appears to be in the water transport models. The surface
water treatment is highly simplified, accommodating only a single channel with no spatial variations in
channel geometry or flow. It appears that only a single input point was accommodated, with no
tributaries or branching of the stream. It was not obvious that the model has the ability to take into
account dilution by merging streams and, if it can, where in the model it would occur. Consideration
12
-------
should also be given to incorporating sedimentation and re-suspension from the stream bed into the
transport models.
GENII v.2 does not have a groundwater module. This may not be crucial if FRAMES can
accept more complex site-specific inputs for that module. The deficiency can be remedied by
converting any one of several available groundwater models to be compatible with FRAMES.
Based on Committee suggestions, it appears that an estuary or tidal effect model needs to be added to
GENII v.2. Reticulation and recycling options may be desirable. These models have been developed
previously through the auspices of other organizations (e.g. Procter and Gamble) and may be integrated
into GENII v.2 or made compatible with the FRAMES platform. Access to dispersion coefficients
associated with different modules would be beneficial and allow end-users to perform custom sensitivity
analyses on these important parameters.
The terrestrial transport model is also very simple, with three completely mixed boxes used to
represent surface soils, deep soils, and the source region. Infiltration through the surface soil
compartment results in loss of contaminant from the system, which may not be a good assumption if, in
fact, infiltration moves contaminants from the surface compartment to the deep soil compartment. The
soil modules did not predict concentration gradients as a function of horizontal position. It is not clear
whether the coupling between the air model and soil deposition automatically provides some spatial dis-
aggregation for the soil models if the source is a release to air.
As noted in Section 3.1, GENTI v.2 also may be inadequate in terms of some aspects of the
atmospheric transport module. It may require further development of "near-field" and "far-field"
components with different scaling and grid resolution conditions as required by the different physical
phenomena in these domains. "Near-field" and "far-field" components of the GENII v.2 code, would
allow site-specific treatment of model simulations and associated uncertainties in the results at both site-
and regional-scales, where the physics may be drastically different. The site-scale model results could
be input into the regional-scale model as an initial condition or source term.
As noted in Section 3.1, the atmospheric dispersion model could also be expanded to allow
inclusion of multiple meteorological data to allow for improved wind fields and extrapolation of those
fields. Similarly, the inclusion of real-time meteorological input and parameters for emergency response
problems could be considered (if required). Models which address issues of topography and re-
entrainment of particles could be added to enhance the usefulness of the code.
3.2.2.2 Recommendations for "Minor" Additions
The following recommendations are of lower priority than the issues addressed in the previous
section.
a) ORIA should consider whether the code can be customized to take into account rather
unusual conditions or exposure scenarios such as use of impacted water in saunas or
13
-------
Native American sweat lodges. Consideration of these pathways could become
important in certain situations, particularly for volatile radionuclides in water.
b) The GENE v.2 system lacks multimedia capabilities in that there is no way to track and
maintain mass balance on transfers between air, surface water, soils, and vegetation.
ORIA may want to look at the TKDVI.Fate system being developed by OAQPS for air
toxics applications.
c) All of the simplifying assumptions used in GENE v.2 may be justifiable for use in a
screening tool that is not intended for detailed, site-specific applications. However,
they should all be clearly noted in both printed and on-line documentation.
3.2.3 What approaches should be used to model exposures to radon, tritium, and carbon-
14?
3.2.3.1 Tritium and 14C
Tritium(3H) and 14C are important radio nuclides that probably should be included in a general
model system, but the present treatment in GENII v.2 is limited. The approach in GENII v.2 might be
adequate for screening purposes, but the present model formulation would provide gross overestimates
under many circumstances. 3H and 14C behave rather differently in the environment, and the usual
modeling approach related to radionuclides attached to particles is not appropriate.
Of the two radionuclides, 3H is probably the more important because major releases of 3H have
occurred in the past at the nuclear weapons laboratories, at reactors, at Rocky Flats, and at
commercial facilities. Large scale releases of 14C are rare, but have occurred from the explosions of
large nuclear weapons, from nuclear reactors, and from nuclear fuel reprocessing plants (which are no
longer in operation in the U.S.). Under such conditions, the traditional approach has been to calculate
the "global" dose from 14C, rather than the local or regional dose indicated in GENII v.2. Small scale
releases of 14C undoubtedly occur more frequently from university and other laboratories that use 14C
as a tracer in studies of metabolism, etc., although anomalous release may occur associated with
contaminant remobilization during forest fires (e.g., Cerro Grande fire, Los Alamos, New Mexico,
2000).
One major problem with the GENII v.2 formulation for 3H is that it is appropriate only for
titrated water (HTO) and not for hydrogen gas (HT), although most of the major releases of 3H in the
past have been in the form of HT. The local and regional doses from the release of HT are remarkably
different from releases of HTO. Yet, the user is not warned of this problem. Even for the releases of
HTO the formulation in GENII v.2 is quite conservative, as it is assumed that instantaneous equilibrium
occurs in terms of the specific activity of 3H between moisture in air and water in vegetation. In reality
this has never been observed, as the specific activity of 3H in vegetation is usually about a factor of two
less than the specific activity in atmospheric water (Anspaugh et al, 1971).
14
-------
For 14C, GENII v.2 assumes that all 14C released is in the form of 14CO2. This is probably a
reasonable assumption, but the user should be warned of this and advised that the release of other
forms would lead to different results. The assumption of instantaneous equilibration for 14C is even less
justified than for 3H. The release of 14C at night, for example, when plants are not taking up CO2,
would lead to remarkably different results than would be predicted by the GENII v.2 code.
Most of the models for 3H and 14C have been more concerned with the global doses, rather
than with local and regional doses. This is because in reality it takes a very long time for equilibration to
occur. Again, the user should be warned of this problem, or the GENII v.2 code should be modified to
include the global aspects of doses from the releases of 3H and 14C.
The GENII v.2 code may need to incorporate simple coupling of air-water and surface-vadose
zone processes as they apply to tritium, particularly to capture precipitation and evapo-transpiration,
and dilution and percolation processes that can strongly affect the 3H budget. This could be tested with
other models used at environmental remediation sites where sources are well defined as well as
concentrations vs. time downstream of contaminated sites.
3.2.3.2 Radon
The GENII v.2 modules for water pathways included an indoor exposure pathway due to
release of radon from domestic water supplies during household use. The code uses the conversion
factor of 0.1 pCi per cubic meter of air per pCi per liter in water. This is consistent with the
relationship between water and air concentrations reported in the National Academy of
Sciences/National Research Council (NAS/NRC) Risk Assessment of Radon in Drinking Water (NAS
1999).
It was not apparent in the information provided in the User's Guide that the code considers
radon emanation from soils contaminated with radium (^Ra and 228Ra). Inhalation of the short-lived
decay products of 222Rn and 220Rn emanating from contaminated soils is a significant exposure pathway.
The emanation rate of 222Rn is a function of the thickness of the contaminated layer of soil and its depth
beneath the surface. There are several codes, including the RADON code (NRC, 1989) that can be
used to estimate 222Rn emanation. The atmospheric dispersion models already incorporated into
GENII v.2 can be used to estimate off-site radon decay product concentrations.
3.3 Charge Question 3: (a) Are the examples and documentation provided with the software
adequate and helpful? (b) How should the output and uncertainty results be presented?
As with Charge Question #2, Charge Question #3 had multiple related parts which the RAC
considered separately.
15
-------
3.3.1 Are the Examples and Documentation Helpful?
The RAC commends the ongoing efforts of ORIA to make FRAMES and GEMI.v.2 "user
friendly" and "user intelligent." The consensus of the authors of this effort and the RAC is that the
documentation is inadequate at this time. Efforts are underway to improve this aspect.
The examples and written and on-line documentation represent a good start on providing
adequate user support. However, these materials need to be tested in some depth through trials with
novice users. Several RAC members attempted to use the code and were able to load and look at
some elements of the example problems but even some experienced code users had problems
negotiating through all of the screens or completing the example runs. The software error messages that
showed up did not provide adequate instructions to understand or fix the problems.
3.3.1.1 General Comments
The RAC offers the following general suggestions to improve the usefulness of GENII v.2:
a) Involve end-users in the preparation of the documentation of the model applications.
b) Use more example cases to document capabilities of model, including any
verification/validation tests and calibration exercises. These could be placed in a
separate code verification document or within the code user manual.
c) Expand on the block-diagraming of module components and their results.
c) Enhance the capabilities for visualization of uncertainty analysis results either by
providing further visuals within the code or by facilitating the use of commercially
available visualization software for statistical data.
d) Although this may be a matter of preference, the RAC would like to see some basic
information on model formulation, assumptions, and limitations included in the User's
Guide. This is likely to be the first source of information for most users. The User's
Guide needs a table of contents and page numbers for easier reference.
e) Several limitations in the documentation have been noted by the RAC, including use of
jargon that is not adequately explained (i.e., the discussion of "joint frequency data" in
Section 3.1 of the Guide) and difficulties in understanding Appendices D and F of the
Software Design Document.
f) The documentation should include some comments about the quality of data input and
its impact on uncertainty of results.
g) The document should have page numbers and a table of contents.
16
-------
h) The manual would be easier to understand if "word problems" were worked through as
examples, showing the data input and the output of each screen as the problem was
solved to reach an estimate.
i) Although much technical information had been provided in and about the software, the
supporting text and illustrations need substantial attention to make GENII v.2 "user-
friendly" to the broad range of users that is envisioned. Several Members of the
Committee, using the software and instructions provided by EPA, were successful in
installing GENII v.2 on their personal computers and were able to follow and execute
the sample calculations provided by the Agency. However, none of the RAC members
could successfully execute analyses of their own devising. Moving beyond installing
and starting the program, it is important that the user have a good understanding, in a
general way, of what steps the program is going through to obtain the desired dose and
risk information. Reproduction and explanation of all the input screens and possible
default values is a critical ingredient. Another desirable addition would be the use of
block diagrams such as those presented by the ORIA Staff to illustrate what activities
go on in each of the modules plugged into FRAMES.
j) Expansion of the text using readily understandable terms would particularly help the
inexperienced user. As tests of the degree to which the authors have been successful in
the quest for user friendliness, the RAC recommends that software be tested and used
by a group of people with a range of scientific (or non-scientific) backgrounds similar to
that envisioned for the end users.
k) Clear descriptions of the input format requirements for FRAMES should be provided
to facilitate the development of compatible groundwater and atmospheric transport
process models to be used in the near-future.
3.3.1.2 SUM3 Documentation
A number of issues are not adequately explained in the SUM3 documentation. In addition,
some of the information that is given is not clear. For example: the explanation of the exponential
distribution on page 4.9 is garbled. Sentences 3 and 4 of that section should be replaced by a
statement such as the following: "For example, if the time until some event (e.g., radioactive decay of an
atom) is exponentially distributed, then the remaining time until that event will always have the same
exponential distribution, regardless of how much time has elapsed." Delete "for radioactive decay of
strontium-90." The last sentence on page 4-11 of the SUM3 document is incomplete. Figure 4.13
provides an explanation but it is inconsistent with the text.
The RAC has specific concerns with regard to the clarity of the documentation on Monte Carlo
and Latin Hypercube sampling. The Introduction, page 1.1, states that "statistical methods used in
SUM3 are based on Monte Carlo sampling using Latin Hypercube random numbers." The discussion
of sampling on pages 6.3 and 6.4 is inconsistent. Page 6.3 suggests that straight Monte Carlo sampling
17
-------
is used for correlated variables, with Latin Hypercube sampling used only for uncorrelated variables.
Page 6.4 then describes a method to induce dependence among variables in Latin Hypercube sampling.
There is no discussion of Monte Carlo sampling on page 6.3, and the discussion of Latin Hypercube
sampling on page 6.4 is not sufficiently detailed and may not be comprehensible to readers who are not
already familiar with the method. Also, the last two sentences in the paragraph seem misplaced.
Clarity would be improved by rearranging the paragraph such that the first of those sentences comes
right after sentence 3, and the second one, after sentence 9. Finally, the document does not discuss
how the user can set the number of realizations in the simulation. This critical parameter appears only in
Figure 7.1.
A built-in capability to perform regression analysis on Monte Carlo simulation results to identify
influential sources of uncertainty would be a very useful addition to SUM3. Also, the system should
provide diagnostic statistical information on the samples that are generated (e.g., sample mean,
variance, and correlation matrices), so that the users know how closely the Latin Hypercube samples
that the system produced approached their specifications.
3.3.1.3 User Training
A considerable amount of RAC discussion was focused on the issue of the proper use of this
software. Because of the many modules and parameters that can be chosen by the operator, a broad
range of answers can be obtained, many of which might be wrong or misleading due to the improper
choice or use of various parameters and pathways. Proper training is a vital ingredient and it is
important that ORIA and its contractor consider the best ways to improve the operator's skills in
applying and using the various features of the software. Classes are one possibility but not everyone
can make the necessary arrangements to attend. A Users' Group was also mentioned as a way for
users to interact and help each other work out important problems and issues. Tutorials with clearly
defined word problems are another facet that will be particularly important for users who cannot
participate in face-to-face training sessions. The RAC also discussed the possibility of a phased
approach to learning and using the software. In the beginning phase(s), the number of parameter
choices would be limited and more defaults used. This segment would be particularly useful for various
analysts comparing their results for the same site. As more experience is gained with GENII v.2, the
operator can move onto the more flexible uses of GENII v.2 with a higher degree of confidence and
understanding about the various parameters and their choices.
Although FRAMES and the GENII v.2 code and suite of modules will be widely available, the
stakeholders must be able to comprehend its value in order for the code to be universally accepted.
Training needs to be provided, not just in the "how to" but in the philosophy of the use of models and
the importance of understanding uncertainty in the answers that result from their use. As the cadre of
people that use GENII v.2 grows, users need to interact with each other to find and fix problems,
expand the code's capabilities, explore new uses, and help each other to understand the results.
It would be easy to use some unrealistic parameter values and have the code conclude all kinds
of strange things. The question of whether it is EPA's job to prevent misuse of the code and its
18
-------
modules was discussed. The RAC concluded that EPA should at least provide some warnings to the
user. By making it easy for anyone to use the code and arrive at conclusions about risk, the EPA is
taking on the responsibility to at least caution the user about the credibility of the results.
In general, the difficulties encountered in using the GENII v.2 code could be alleviated by a
training course complemented by better documentation and a dynamically updated file of "Frequently
Asked Questions."
3.3.2 How should the uncertainty results be presented?
The RAC has several suggestions and concerns with regard to the presentation of uncertainty
analysis results. The RAC's principal concern is with the statistical interpretation of uncertainty results
and the built-in conservatism of the code.
The RAC also questions why there are so few distributions available in the uncertainty analysis.
This lack of flexibility is especially problematic given the unavailability of a user- defined distribution.
3.3.2.1 Statistical Interpretation of the Uncertainty Results
As noted in the previous section, it is not clear whether Latin Hypercube sampling is always
used, or whether the user has the option of selecting straight Monte Carlo sampling. While Latin
Hypercube sampling gives more precise estimates of mean values, its performance in estimating the
shape and degree of spread of the distributions of percentiles was not well understood .
In particular, when straight Monte Carlo sampling is used, confidence limits can be computed in
a fairly straightforward manner both for the mean value resulting from the simulation, and also for the 5th
and 95th percentiles (or other user-specified percentiles), as discussed in Section 4.3 of the GENII
Version 2 Software Design Document. Care must be taken, however, in computing confidence limits
for results obtained using Latin Hypercube simulation. In particular, the standard confidence limit
calculations assume independence and random sampling, and hence cannot be directly applied to
results obtained using Latin Hypercube sampling.
With some care, confidence limits for the mean value can be obtained by applying Equation
4.34 to the batch means resulting from the various Latin Hypercube runs, rather than to the individual
realizations themselves. This is valid because the batch means are independent of each other, while the
realizations within a batch are not. However, the RAC is not aware of methods for computing
confidence limits for quantiles obtained using Latin Hypercube sampling. Unless ORIA is able to
identify such methods in the recent literature, the unavailability of confidence limits for quantiles should
be indicated as a limitation of Latin Hypercube sampling. ORIA may also want to investigate whether
the standard confidence limit calculations for this case (e.g., Equation. 4.43) can be used as
approximations or bounds on the confidence limits for Latin Hypercube sampling.
19
-------
It is also unclear what simulation diagnostics, if any, are available even for straight Monte Carlo
sampling (e.g., confidence limits). It is inappropriate to present simulation results without such
diagnostics, since the results can be essentially meaningless if the confidence limits are too broad. In
particular, the lack of diagnostics makes it very difficult to distinguish model bugs and input errors from
unanticipated or spurious results due to an inadequate number of realizations in the simulation. The
automatic inclusion of basic simulation diagnostics such as confidence limits is critical. This should not
be an option left to the discretion of the user. Some users may not be sophisticated enough to request
or compute such confidence limits on their own.
The summary in the Guide states that uncertainty analysis can be used to understand the
importance of the input parameters. It was not clear to the RAC that measures for doing this, such as
capabilities for sensitivity analysis or uncertainty importance calculations, are actually available in SUM3.
Also, the current approach to presentation of results in SUM33 is simply to list the sample input
values and the associated outputs of interest. The justification appears to be that this provides adequate
information for a user to compute any desired output information in a post-processor. However, this
approach is not acceptable. Even some reasonably expert transport modelers may not be comparably
expert in Monte Carlo and uncertainty analysis techniques. As noted above, it is incumbent on the
software designer to provide the appropriate output information (including simulation diagnostics such
as confidence limits), rather than relying on users to request or compute this information.
Uncertainty results that should be presented include at a minimum the mean, median, standard
deviation, and 5th and 95th percentiles, along with the option for other user-specified quantiles.
Graphical presentation of results would also be desirable. Histograms or estimated probability density
functions are likely to be of more value to many users than cumulative distribution functions, so should
be routinely provided. (Cumulative distribution functions are often difficult to interpret, since people are
generally better at judging heights than slopes.)
3.3.2.2 Conservatism in the Uncertainty Analysis
The documentation suggests that the selection of default parameter values and other similar
choices were generally made with a conservative bias (i.e., tending to overstate rather than understate
risk). Although the SUM3 model is advertized as a tool for investigating the effects of differing degrees
of conservatism, it will do so only to the extent that the user identifies the most conservative choices in
the models and replaces them with reasonable distributions. Conservative choices that are not
transparent, or are buried in the basic philosophy of the modeling, may not be discovered by using
SUM3.
For example, it is not clear that changing from a maximum exposed individual (MET) focus to a
population risk approach actually provides a realistic set of risk estimates for the latter. Although
maximum exposure assumptions may be replaced by average exposure assumptions, it is not clear that
the geographic determinants of risk (e.g., distance of residence and workplace from source of
radionuclides) can be treated realistically in the current GENII v.2 structure.
20
-------
In general, the RAC would recommend avoiding deliberately conservative models or parameter
values where possible. Cases in which it is not possible to avoid such conservative assumptions should
be clearly documented. In some cases, it may also be appropriate to provide users with a choice
between "conservative" and "best estimate" models. In such cases, there should be a warning to
indicate that the results may not reflect the full range of uncertainty if a user elects to perform an
uncertainty analysis using biased parameters and/or models. For example, propagating the uncertainties
associated with input parameters through a conservative model will not yield a realistic statement of
uncertainties but rather a hybrid of conservative and realistic assumptions which will be difficult to
interpret or even meaningless.
3.3.2.3 Variability versus Uncertainty
The RAC understands that SUM3 is not intended to perform two-dimensional Monte Carlo
analysis to quantify variability and uncertainty separately from each other. However, the documentation
should at least explain the distinction between variability and uncertainty, particularly with respect to the
different influences they can have on risk management decisions. At a minimum, the user should be
warned about the difference, so that the results of an uncertainty analysis will not be interpreted
incorrectly.
For example, uncertainty can, at least in principle, be reduced by further investigation, but
variability cannot. Hence, a clear understanding of the difference can be important in determining
whether further research on a particular subject might be worthwhile. In addition, variability can create
concerns about equity (e.g., among different individuals or communities). By contrast, uncertainty can
create concerns about the level of overall societal risk (e.g., if the risks at all sites of a certain type have
been systematically underestimated).
It is also important to point out that what constitutes uncertainty in one application may well be
variability for the purposes of a different application. For example, consider a distribution that reflects
the differences among sites of a given type across the country. For purposes of setting national policy,
this distribution would reflect inherent variability that could not be reduced by further investigation.
However, in a site-specific analysis, the same distribution might be used only as an initial starting point,
with the option of reducing the uncertainties by collecting site-specific data.
3.3.2.4 Suggested Extensions to SUM3
The discussion above suggests areas where further extensions to SUM3 may be desirable in the
future. One such extension might be to provide capabilities for two-dimensional Monte Carlo analysis,
thereby allowing users to quantitatively and explicitly distinguish between variability and uncertainty.
Another extension that may be desirable is to allow for a quantitative treatment of model
uncertainty. In particular, one would generally expect a relatively coarse screening model to yield wider
confidence limits than a more detailed model, but this might not be revealed by Monte Carlo simulation
based solely on uncertainty in the input parameters to the model. Similarly, the existence of competing
21
-------
models with differing degrees of conservatism would tend to suggest substantial uncertainty about how
best to model a particular phenomenon. This uncertainty again might not be captured by Monte Carlo
simulation on the inputs to a single model.
Finally, the summary in the SUM3 documentation states that uncertainty analysis can be used to
understand the importance of the input parameters. However, at present the software does not appear
to have the capability to perform such uncertainty analysis or uncertainty importance calculations. This
would be another option to consider for inclusion in a future version of the software. For example,
regression analysis could be used to indicate which input parameters contributed the most to the
uncertainty about a particular output of interest.
22
-------
4. COMMENTS BEYOND THE CHARGE
The following comments go beyond the Charge from the Agency but the RAC believes that
some expansion on the issues addressed in the charge is necessary. Given the RAC's overall favorable
impression of GENII v.2 and FRAMES, these comments would often apply equally or even more to
competing modeling systems currently available, and should be viewed as contributing to potential
improvements rather than as criticisms of the current structure.
4.1 Potential Use of FRAMES
In the absence of a specific statement from ORIA on the intended use of the GENII v.2 code,
the RAC discussed several situations in which the FRAMES platform, in general, and the code, in
particular, might be employed.
4.1.1 Generic Assessment of Source Categories in Various Settings
This appears to be what GENII v.2was designed to do. FRAMES could provide a good
platform for assessing environmental impacts and human health risks. The calculational models within
GENII v.2 and available on FRAMES become the factor that would limit its use. The code is designed
for prospective use, but it could be adapted to retrospective use.
4.1.2 Site Specific Use
GENII v.2 may be useful for some relatively simple sites, but not for complex sites. As noted in
Section 3.2, FRAMES needs a good groundwater module. The FRAMES platform provides a great
potential for expansion and adaptation. As more calculational models are developed that are
compatible with the FRAMES platform, additional capabilities will be discovered and used.
FRAMES has no specific component to model various human interventions that would be
options for a site cleanup. To the extent that various site remedial techniques can be approximated with
a model, FRAMES would provide a good platform for trying different remediation schemes and
comparing and contrasting the results. For example, FRAMES could include an icon that could be
pulled down to model the effects of a cap.
4.1.3 Emergency Response - Plume Phase
A platform to be used for dose and risk assessment in emergency situations must be able to
incorporate real time meteorological data. That capability is currently not available in FRAMES or
GENII v.2 but could be readily added.
The platform must allow for simple data inputs. During an emergency, the data available are
limited. The code must provide information quickly to guide decision-makers. That information must
23
-------
be relatively easy to interpret and without a great deal of complexity. The module used in GENII
V.2/FRAMES is RASCAL, a code used by emergency planners during exercises.
The ability to use real time meteorological data, field team readings with location identifiers
(northing and easting), source location, and time of release, and have the model calculate a source term
would be a very useful feature for emergency response. That capability is not yet provided by GENII
v.2.
Emergencies can occur at all levels. The FRAMES platform needs to be able to provide
advice to the user about model selection based on the scale of the emergency. Currently, only small
emergencies could be handled by GENII v.2. The consequences of emergencies with great motive
force, such as explosions or fires, would not be addressed adequately by GENII v.2. In addition,
GENII v.2, as it is currently constructed, would not be an appropriate vehicle for predicting the impacts
of severe accidents where deterministic effects may be involved. Other models are available to deal
with such situations. These limitations are not necessarily inherent in the FRAMES platform, but are a
function of the models currently contained in the GENII v.2 software package.
4.1.4 Emergency Response - Ingestion Phase
After radioactivity has been deposited in an area, it becomes necessary to make decisions
about crop interdiction and other restrictions on agricultural, industrial, and residential use. The
FRAMES platform with appropriate modules could be useful in assisting decision-makers in evaluating
the risk associated with various activities.
4.1.5 Emergency Response - Recovery Phase
It would be useful to be able to model the various activities that would normally occur in the
impacted area to determine whether it would be "safe" to bring people back in and to allow them to
resume normal activities such as working, shopping, walking the dog, playing in the yard, etc. The
FRAMES platform has the flexibility to be expanded for this type of decision-making, but is limited by
the models that are available in GENII v.2.
4.1.6 Unusual Situations
While not quite reaching the level of an emergency, there are many times when a quick model is
needed to see if some rare activity could be a problem. For instance, when a patient with a palladium
seed implant for prostate cancer treatment died and his family wanted to cremate him, a screening air
dispersion model was needed. This seems to be a good use of FRAMES and the air dispersion model
(if you can properly partition the palladium that would volatilize versus remaining behind as ash.)
24
-------
4.1.7 Other Uses
It would be useful for any of these environmental modeling needs to have levels of FRAMES
that vary in complexity for users who are new to modeling as well as users who are experienced and
wish to employ all the complex functions. The FRAMES platform could also help the inexperienced
user decide whether the model selected is appropriate for a particular situation. Some front end
prompts such as the following would be useful:
a) Do you want to use real time meteorology or a 30 year wind distribution?
b) Do you want to do a retrospective or prospective run?
c) Do you want a "best" estimate or "conservative" estimate of risk?
d) Do you want the risk to a person in a certain location or a population in a certain area?
e) Do you want default values from a certain section of the country?
f) For a certain time of the year?
g) For a certain sub-population?
h) What kind of data quality do you need?
i) What kind of scale (in space and in time) do you need?
4.2 Other Comments Beyond the Charge
The Committee had the following additional comments:
a) Based on the experience with using GENII v.2, the RAC suspects that different naive
users might obtain vastly different results if simply given a problem (e.g., sampling
results from a Superfund Site) and the FRAMES/GENII v.2 tools. Although training
might reduce the spread, it would be interesting to know how much user-dependence is
possible with the existing documentation.
b) Essentially no guidance is given for the selection of parameter values for site-specific
analysis. For example, how to select a river flow rate for the surface water module is
not described. The RAC believes that harmonic mean flow, not the annual average, is
the appropriate parameter. In fact, it is not even clear what the domain of applicability
for the models is supposed to be. Perhaps the modeling
system is intended only for broad programmatic analysis; if so, that should be stated.
25
-------
c) ORIA should clarify whether the GENII v.2 model includes special treatment of
children. Categories for dose and risk coefficients for "adult" and "general" categories
are mentioned.
d) Conservatism should not be built in to the GENII v.2 code. The dose and risk point
estimates should be as unbiased as possible. Conservative estimates could be
generated by the user by inputting conservative parameters.
e) ORIA should make an effort to test the code by stringing together several modules at a
time (as would be done in using GENII v.2) in addition to the verification and validation
of individual modules.
f) Default values should be clearly explained to the users so that they know what the
model is doing when data are not available for some parameters. It would also be
useful to supply uncertainty distributions for the default values.
g) The "acute" scenarios appear to refer to acute releases (accidents, etc.) rather than to
acute exposure periods. Nowhere do short-term concentration values appear to be
used to predict risks of acute health effects. With radionuclides, where the principal
concern is with cancer, this focus is probably appropriate. However, a user with a
chemical risk perspective may expect prediction of acute doses to be compared with
criteria for acute health effects. Some further explanation is in order.
h) One of the options for risk calculations is the use of the Health Effects Assessment
Summary Tables (HEAST) slope factors. Given the lack of easy availability and
confidence in the HEAST document as a general matter, and given the fact that the
HEAST values are basically derived from Federal Guidance Report No. 13, with a few
assumptions, it may be better to describe this risk system in some other way.
26
-------
APPENDIX A EDITORIAL COMMENTS
a) It would be very helpful to the user (or casual reader) who is not conversant with
computer jargon to include a glossary in any user's manual or other documents issued
with reference to the GENII v.2 code In addition, for potential users who are not
experienced in the use of "platforms" it would be helpful to include a graphic showing
how the modules link together under the FRAMES "umbrella." Specific instructions in
the user's guide on how to install the code would also be helpful.
b) The types of scenarios, i.e., "near field", "acute", and "chronic" should be defined in the
introduction in the User's Guide.
c) Connecting the modules did not work as stated in the documentation. The user needed
to shift left click and drag rather than right click and drag. Conversely, the module
options (General Info, User Input, Run Model) could be accessed with a simple right
click, not a shift click as described in the documentation. If these procedures are
machine-dependent, the user should be warned.
d) A Glossary should be included in the User's Guide as well as the Software Design
Document.
e) SUM3 Document, p. 6-4: In the 2nd to last sentence on the page, "illiterates" should
probably be "illustrates".
A-l
-------
GLOSSARY OF ACRONYMS AND TERMS
AERMET
AERMOD
AIRMIC
AERMOD Meteorological Preprocessor
AIRMIC Dispersion Model
American Meteorological Society/Environmental Protection Agency
Regulat
ory
Model
Improv
ement
Commi
ttee
AMAD
Benchmarking
Aerodynamic Median Activity Diameters
Calibration
Code
Code Verification:
EPA
FEHM
FIRETEC
Part of the software verification process that involves comparing results of two
or more codes against each other, or to an analytical solution. It entails the use
of a standardized problem or test that serves as a basis for evaluation or
comparison of software system performance. This mathematical analysis
assures that the behavior of the code to be benchmarked is predictable and
performs as intended.
With reference to models, refers to the use of experimental and/or field data to
constrain the value of the variables and parameters used in a model to satisfy its
use for a specific application.
Software package consisting of calculational models
Refers to software development. Verification is a form of code control, which
involves establishing that the software is mathematically sound, accurate, and
numerically stable. Verification results in the implementation of specified
Software Certification goals. This is a reiterative process, comparable to the
use of "blanks" and "standards" in experimental protocols. Verification implies
reaching a certain level of confidence in the correctness of the software system.
A common verification technique involves running the code with specified
boundary conditions and parameters and comparing the results to other codes
under the same conditions (e.g., benchmarking).
Environmental Protection Agency
Los Alamos National Laboratory (LANL) Finite Element Heat and Mass
transfer model
Joint LANL & Lawrence Livermore National Laboratory (LLNL) system to
predict wildfire behavior. A computer model that incorporates basic physical
G-l
-------
FLOTRAN
FRAMES
FRAMES platform
GENII
FflGRAD
HT
HTO
IAEA
ICRP
ISC
MEI
Model
MODELS-3
and chemical properties of fire. By investigating, understanding and modeling
the fundamental principles of fire, the researchers can build models that more
accurately predict wildfire behavior
FLOTRAN is a finite element analysis program for solving fluid flow and
conjugate heat transfer problems, developed by? It assists with the analysis of
Computational Fluid Dynamics (CFD) phenomena such as flow through ducts,
channels, or over airfoils.
Framework for Risk Analysis in Multimedia Environmental Systems
Software package with a user interface that allows the user to select specific
calculational components included in the analysis, select radionuclides, view
intermediate and result files, prepare result charts and perform uncertainty and
sensitivity analyses
GENeration n computer programs which include calculational components that
can be exercised under the control of FRAMES
A High-resolution and strong GRADient application model that simulates
weather variables across a fire line, LANL National Center for Atmospheric
Research (NCAR)
3HH - tritiated hydrogen gas
3HHO - tritiated water
International Atomic Energy Agency
International Commission on Radiological Protection
EPA Industrial Source Code Model
Maximally Exposed Individual
A mathematical representation of possibly complex physical, chemical, and/or
biological processes. A model may be phenomenological in that it tries to
represent in very fine detail (usually by coupled differential equations) the
fundamental processes occurring, or it may be entirely empirical.
EPA Third Generation Air Quality Modeling System
G-2
-------
MODFLOW
OAQPS
ORD
ORIA
Platform Validation
PNNL
PRESTO
RAC
RAMS
SAB
SUM3
TOUGH
TRACR3D
TREVLFate
Validation
VAMP
Verification
U.S. Geological Survey (USGS) Modular Three-Dimensional Ground-Water
Flow Model
Office of Air Quality Planning and Standards
Office of Research and Development (EPA)
Office of Radiation and Indoor Air (EPA)
A process whereby model(s) are run independently of the platform and the
model results are compared to the results when the same model(s) is run under
control of the computing platform. Complete verification is very difficult, and
several different problems should be run to test as thoroughly as possible the
extreme conditions of the model(s).
Pacific Northwest National Laboratory
Prediction of Radiological Effects due to Shallow Trench Operations family of
codes
Radiation Advisory Committee
Regional Atmospheric Modeling System, originally developed at Colorado
State University for the National Oceanic and Atmospheric Administration's
(NOAA) Air Resources Laboratory (ARL)
Science Advisory Board
Sensitivity/Uncertainty Multimedia Modeling Module
Lawrence Berkeley National Laboratory (LBL) Transport of Unsaturated
Groundwater and Heat
A Model of Flow and Transport in Porous Fractured Media, Los Alamos
National Laboratory (LANL)
Total Risk Integrated Methodology (TRIM) Environmental Fate, Transport, &
Ecological Exposure Module
see Platform Validation
VAlidation of Model Predictions
see Platform Validation
G-3
-------
REFERENCES
Anspaugh, L.R., Koranda, J.J., Robison, W.L., and J.R. Martin. 1971. The dose to man via
food-chain transfer resulting from exposure to tritiated water vapor: in Tritium, A. A. Moghissi,
M.W. Carter, ed. Las Vegas. Messenger Graphics.
ICRP (International Commission on Radiological Protection). 1996. Age-Dependent Doses to
Members of the Public from Intake of Radionuclides: Part 5. Compilation of Ingestion and
Inhalation Dose Coefficients. ICRP Publication 72. Ann. ICRP 26, 1.
ICRP (International Commission on Radiological Protection). 1995a. Age-Dependent Doses to
Members of the Public from Intake of Radionuclides, Part 3: Ingestion Dose Coefficients. ICRP
Publication 69. Ann. ICRP 25, 1.
ICRP (International Commission on Radiological Protection). 1995b. Age-Dependent Doses to
Members of the Public from Intake of Radionuclides, Part 4: Inhalation Dose Coefficients.
ICRP Publication 71. Ann. ICRP 25, 3/4.
ICRP (International Commission on Radiological Protection). 1994. Age-Dependent Doses to
Members of the Publicfrom Intake of Radionuclides, Part 2: Ingestion Dose Coefficients. ICRP
Publication 67. Ann. ICRP 23, 3/4.
ICRP (International Commission on Radiological Protection). 1989. Age-dependent doses to members
of the public from intake of radionuclides, Part 1. ICRP Publication 56. Ann. ICRP 20, 2.
NAS (National Academy of Sciences). 1999. Risk Assessment of Radon in Drinking Water. National
Academy Press. Washington, DC.
NRC (U.S. Nuclear Regulatory Commission). 1989. Regulatory Guide 3.64, Calculation of Radon
Flux Attenuation by Earthen Uranium Mill Tailings Covers. Office of Nuclear Regulatory
Research.
R-l
------- |