Risk Assessment Guidance
for Superfund:
Volume III - Part A,
Process for Conducting
Probabilistic Risk Assessment
-------
c/EPA
EPA 540-R-02-002
OSWER 9285.7-45
PB2002 963302
vwwv.epa.gov/superfund/RAGS3A/index.htm
December 2001
Superfund
Risk Assessment Guidance for Superfund:
Volume III - Part A, Process for Conducting
Probabilistic Risk Assessment
Office of Emergency and Remedial Response
U.S. Environmental Protection Agency
Washington, DC 20460
Printed on Recycled Paper
-------
DISCLAIMER
This document provides guidance to EPA Regions concerning how the
Agency intends to exercise its discretion in implementing one aspect of the
CERCLA remedy selection process. The guidance is designed to implement
national policy on these issues.
Some of the statutory provisions described in this document contain
legally binding requirements. However, this document does not substitute for
those provisions or regulations, nor is it a regulation itself. Thus, it cannot
impose legally-binding requirements on EPA, States, or the regulated
community, and may not apply to a particular situation based upon the
circumstances. Any decisions regarding a particular remedy selection
decision will be made based on the statute and regulations, and EPA decision
makers retain the discretion to adopt approaches on a case-by-case basis that
differ from this guidance where appropriate.
Interested parties are free to raise questions and objection about the
substance of this guidance and the appropriateness of the application of this
guidance to a particular situation, and the Agency welcomes public input on
this document at any time. EPA may change this guidance in the future.
-------
ABOUT THE REVISION
WHAT IT is
WHO IT'S FOR
WHAT'S NEW
EPA's Processfor Conducting Probabilistic Risk Assessment'^ an update of
the 1989 Risk Assessment Guidance for Superfund (RAGS). It is Volume III,
an update to the existing two-volume set of RAGS. Volume III: Part A
provides policy and guidance on conducting probabilistic risk assessment for
both human and ecological receptors.
RAGS Volume III: Part A is written primarily for risk assessors. Risk
assessment reviewers, remedial project managers, and risk managers
involved in Superfund site cleanup activities will also benefit from this
addition to RAGS.
RAGS Volume III: Part A provides guidance on applying probabilistic
analysis to both human health and ecological risk assessment. New
information and techniques are presented that reflect the views of EPA
Superfund program. A tiered approach is described for determining the
extent and scope of the modeling effort that is consistent with the risk
assessment objectives, the data available, and the information that may be
used to support remedial action decisions at Superfund hazardous waste sites.
RAGS Volume III: Part A contains the following information:
For the risk assessor—updated policies and guidance; discussion and
examples of Monte Carlo modeling techniques for estimating
exposure and risk.
• For the risk manager and the remedial project manager—an
introduction to PRA, a chapter on communicating methods and
results of PRA with the public, and a chapter on the role of PRA in
decision making.
-------
TABLE OF CONTENTS
Table of Contents [[[ iv
Acronyms and Abbreviations [[[ xvi
Preface [[[ i
1.0 What is the Purpose of RAGS Volume 3 Part A? .............................. ii
2.0 What is Probabilistic Risk Assessment and how is it used in Risk Characterization? . . . ii
3.0 What are the Advantages and Disadvantages of PRA for Remedial Decisions? ....... iii
4.0 How is RAGS Volume 3, Part A Organized? .................................. iii
5.0 What are the Key Guiding Concepts in RAGS Volume 3: Part A! ................. iii
References for Preface [[[ v
Chapter 1 Overview of Probabilistic Approach to Risk Assessment ..................... 1-1
1.0 Introduction
1.1 The Role of Risk Assessment in Superfund
1.1.1 Risk Assessment in the United States
1.1.2 Risk Assessment at EPA
1.1.3 Risk Assessment in Superfund
1.1.4 Probabilistic Risk Assessment and Its Role in Superfund
1 .2 Basic Concepts of Probabilistic Risk Assessment
-1
-4
-4
-5
-5
-7
-9
1.2.1 What is PRA? ................................................. 1-10
1.2.2 What is a Monte Carlo Simulation? ................................ 1-13
1 .2.3 Why is Variability Important in Risk Assessment? How is it Addressed
by the Point Estimate and Probabilistic Approaches? .................. 1-15
1.2.4 Why is Uncertainty Important in Risk Assessment? How is Uncertainty
Addressed by the Point Estimate and Probabilistic Approaches? ......... 1-17
1.2.5 Reasonable Maximum Exposure at the High-end ..................... 1-21
1.3 Advantages and Disadvantages of Point Estimate and Probabilistic Approaches .... 1-21
1.4 Conducting an Acceptable PRA ......................................... 1-24
1.4.1 Key Policies for Applying PRA at Superfund Sites ................... 1-26
1.5 Organization of the Guidance ........................................... 1-27
1.6 Next Steps for PRA Implementation ...................................... 1-30
References for Chapter 1 [[[ 1-31
Exhibit 1-1 Definitions for Chapter 1 ................................................ 1-2
Exhibit 1-2 Nine Criteria for Evaluation of Cleanup Alternatives .......................... 1-6
Exhibit 1-3 Cancer and Noncancer Risk Models ...................................... 1-11
-------
Figure 1-1 Example of a normal distribution that characterizes variability in adult body weight 1-12
Figure 1-2 Conceptual model of Monte Carlo analysis 1-14
Figure 1-3 Example of a probability distribution for risk illustrating the 95th percentile and two
different risk levels of concern (A and B) 1-16
Figure 1-4 Illustration of "Vertical" and "Horizontal" Confidence Intervals (or limits) on a risk
estimate 1-19
Chapter 2 Workplan and The Tiered Approach 2-1
2.0 Introduction 2-1
2.1 Workplan 2-1
2.2 Special Administrative Considerations in PRA 2-4
2.2.1 Scoping of PRA 2-4
2.2.1.1 PRA Scope of Work for Fund-lead Sites 2-4
2.2.1.2 PRP Scope of Work for PRP-Lead Sites 2-5
2.2.2 Development of Probability Distributions 2-5
2.2.3 EPA Review of PRA Documents 2-6
2.2.4 Peer-Review 2-6
2.2.5 Response to Comments on PRA 2-6
2.2.6 Administrative Record 2-6
2.2.7 Communication with Stakeholders 2-6
2.2.8 Communication with EPA Management 2-7
2.3 Overview of the Tiered Approach 2-7
2.3.1 Getting Started 2-11
2.3.2 Tier 1 2-11
2.3.3 Tier 2 2-14
2.3.4 Tier 3 2-17
2.3.5 Flexibility in Defining Tiers 2-18
References for Chapter 2 2-19
Exhibit 2-1 Definitions for Chapter 2 2-2
Exhibit 2-2 Examples of Important Contents of A PRA Workplan 2-4
Exhibit 2-3 Stakeholders Potentially Involved in EPA's Decision-Making Process for PRA 2-8
Exhibit 2-4 Typical Elements of Tier 1 Risk Assessment 2-11
Exhibit 2-5 Typical Elements of Tier 2 Risk Assessment 2-15
Exhibit 2-6 Typical Elements of Tier 3 Risk Assessment 2-17
Figure 2-1 Schematic Diagram of Tiered Approach 2-9
Figure 2-2 Schematic diagram of deliberation/decision cycle in the tiered process for PRA .... 2-10
Chapter 3 Using Probabilistic Analysis in Human Health Assessment 3-1
3.0 Introduction 3-1
3.1 Characterizing Variability In Exposure Variables 3-1
3.1.1 Developing Distributions For Exposure Variables 3-5
3.1.2 Characterizing Risk Using PRA 3-6
3.2 Role of the Sensitivity Analysis 3-9
3.3 Exposure Point Concentration Term 3-10
3.4 Characterizing Uncertainty in Exposure Variables 3-11
-------
3.4.1 Parameter Uncertainty 3-11
3.4.2 Scenario and Model Uncertainty 3-17
3.5 Example of PRA for Human Health 3-17
References for Chapter 3 3-27
Exhibit 3-1 General Equation for Exposure 3-1
Exhibit 3-2 Definitions for Chapter 3 3-2
Exhibit 3-3 Equation for Cancer Risk 3-7
Exhibit 3-4 Equation for Noncancer Hazard Quotient 3-7
Exhibit 3-5 Using the Tiered Process for PRA Hypothetical Case Study for Human Health Risk
Assessment 3-18
Exhibit 3-6 Risk Equations 3-23
Figure 3-1 Example of a frequency distribution for adult drinking water ingestion rates 3-4
Figure 3-2 Hypothetical PRA results showing a PDF and CDF 3-8
Figure 3-3 CDFs of risk based on Monte Carlo simulations described in Table 3-2 3-16
Figure 3-4 CDFs of risk based on Monte Carlo simulations described in Table 3-2 3-16
Figure 3-5 Site map for future wildlife refuge 3-22
Figure 3-6 Results of sensitivity analysis for preliminary 1-D MCA (Tier 2) 3-26
Table 3-1 Methods for characterizing parameter uncertainty with Monte Carlo
simulations 3-12
Table 3-2 Example of 1-D MCA and 2-D MCA 3-14
Table 3-3 Concentrations in Surface Soil (mg/kg) 3-22
Table 3-4 Exposure Parameters used in Point Estimate Analysis 3-24
Table 3-5 Point Estimate Risks and Exposure Pathway Contributions 3-24
Table 3-6 Input Distributions for Exposure Variables used in 1-D MCA for Variability 3-25
Table 3-7 1-D MCA Risk Estimates using Preliminary Inputs 3-25
Table 3-8 Exposure Duration Survey Results 3-26
Table 3-9 Refined Point Estimate and 1-D MCA Risk Estimates 3-26
Chapter 4 Probabilistic Analysis in Ecological Risk Assessment 4-1
4.1 Introduction 4-1
4.1.1 Basic Approach for Performing Ecological Risk Assessments 4-1
4.1.2 Predictive vs Observational Approaches 4-6
4.1.3 Potential Advantages and Limitations of Probabilistic Methods in ERA 4-7
4.1.4 Focus of This Chapter 4-8
4.2 Deciding If and When to Use PRA in Ecological Risk Assessment 4-8
4.2.1 Technical Considerations 4-9
4.2.2 Cost and Schedule Considerations 4-11
4.3 Problem Formulation 4-11
4.4 Modeling Variability in Exposure 4-12
4.4.1 Characterizing Variability in Dose 4-12
4.4.2 Characterizing Variability in Exposure Concentration 4-15
4.5 Modeling Variability in Toxicity 4-15
4.5.1 Variability in Response Among Members of a Population 4-15
4.5.2 Variability in Response Among Species 4-20
4.6 Modeling Variability in Risk 4-22
4.6.1 Variability in Hazard Quotient 4-22
-------
4.6.2 Variability in Response 4-26
4.6.3 Joint Probability Curves 4-30
4.7 Modeling Uncertainty in Ecological Risk Assessments 4-31
4.7.1 Uncertainty in Exposure 4-31
4.7.2 Uncertainty in Toxicity 4-32
4.7.4 Uncertainty in Response 4-34
4.7.3 Uncertainty in Hazard Quotient 4-35
4.8 Interpreting Results of an Ecological PRA 4-37
4.9 Guidelines For Planning And Performing a Probabilistic ERA 4-39
4.9.1 Planning an Ecological PRA 4-39
4.9.2 Evaluating an Ecological PRA 4-41
4.10 Example of the Tiered Process in ERA 4-41
References for Chapter 4 4-49
Exhibit 4-1 Definitions for Chapter 4 4-3
Exhibit 4-2 Ecological Risk Assessment Guidance and Policy Directives 4-4
Exhibit 4-3 Modeling Variability in Response for a Dichotomous Endpoint 4-17
Exhibit 4-4 Modeling Variability in Response for a Continuous Endpoint 4-19
Exhibit 4-5 Hypothetical Species Sensitivity Distribution 4-21
Exhibit 4-6 Modeling Variability in a Dichotomous Response 4-27
Exhibit 4-7 Modeling Variability in a Continuous Response 4-29
Exhibit 4-8 Example Elements of a Workplan for Ecological PRA 4-40
Exhibit 4-9 Checklist for Including a PRA as Part of the ERA 4-41
Exhibit 4-10 Refined Screening Point Estimate Inputs and Results 4-43
Exhibit 4-11 Screening Level PRA Calculations of HQ Distribution 4-45
Exhibit 4-12 Simulated Distribution of Responses 4-47
Figure 4-1 Ecological Risk Assessment Framework (U.S. EPA, 1992a) 4-1
Figure 4-2 Eight-step Ecological Risk Assessment Process for Superfund 4-5
Figure 4-3 Example of cases where use of PRA may be helpful 4-10
Figure 4-4 Example Graphical Presentations of Dose Distributions 4-14
Figure 4-5 Example Comparison of Exposure Distribution to TRV 4-22
Figure 4-6 Example Distribution of HQ Values 4-23
Figure 4-7 Example Presentation of Species Sensitivity Distribution 4-25
Figure 4-8 Example Joint Probability Curve 4-30
Figure 4-9 Example Presentation of Uncertainty in Exposure 4-31
Figure 4-10 Example Presentation of Uncertainty in Response 4-35
Figure 4-11 Example Presentation of Uncertainty in Exposure and TRV 4-36
Figure 4-12 Example Presentation of Uncertainty in HQ Estimates 4-37
Chapter 5 Probabilistic Risk Assessment and Preliminary Remediation Goals 5-1
5.0 Introduction 5-1
5.1 General Concepts Regarding EPCs and PRGs 5-4
5.1.1 Sources of Uncertainty in the EPC 5-5
5.1.2 Pre- and Post-Remediation Exposure Point Concentrations 5-6
5.1.3 Remediation Action Levels and 95% UCL Calculation
Methods 5-6
5.1.4 Consideration of Risk from Acute Toxicity 5-7
VI
-------
5.1.5 Characterization of Uncertainty in the EPC: Point Estimates
and Distributions 5-8
5.1.6 Multiple Chemicals 5-8
5.2 When to Use PRA for Developing PRGs 5-9
5.3 Methods for Developing PRGs 5-19
5.4 Backcalculation 5-10
5.4.1 Difficulties with Backcalculation 5-11
5.5 Iterative Methods 5-11
5.5.1 Iterative Reduction 5-12
5.5.2 Iterative Truncation 5-13
5.5.3 Example of Iterative Methods 5-14
5.5.4 Multiple Exposure units and Iterative Methods 5-17
5.6 PRGs for Groundwater 5-18
5.7 PRGs for Other Contaminated Media 5-19
5.8 Measurement of Attainment 5-21
5.9 Summary of Recommended Methods 5-23
References for Chapter 5 5-24
Exhibit 5-1 Summaries of Some Key Terms 5-1
Exhibit 5-2 Definitions for Chapter 5 5-2
Exhibit 5-3 Criteria for Iterative Truncation 5-14
Exhibit 5-4 Example of Iterative Methods 5-16
Exhibit 5-5 Evaluation of Alternative RALs Using Iterative Truncation 5-20
Figure 5-1 A hypothetical example of the use of iterative methods 5-12
Figure 5-2 Lognormal probability plot of soil concentrations, including 4 nondetects 5-16
Figure 5-3 Hypothetical example of a mixed, bimodal distribution 5-18
Table 5-1 Soil sample 5-16
Table 5-2 Pre- and Post-Remediation EPCs (95% UCLs) for Chemical X in Surface
Soil Samples 5-17
Table 5-3 Summary of Potential Methods for PRG Development by Environmental
Medium 5-23
Chapter 6 Communicating Risks and Uncertainties in Probabilistic Risk Assessments 6-1
6.0 Introduction 6-1
6.1 Stakeholder Involvement 6-4
6.2 Communication and Presentation 6-5
6.2.1 Communication of PRA With Concerned Citizens, Other Stakeholders, and
Managers: An Overview 6-6
6.2.2 Steps for Communication of the Results of the PRA 6-7
6.3 Communicating Differences Between Point Estimate and PRA 6-10
6.4 Graphical Presentation of PRA Results to Various Audiences 6-11
6.4.1 Public Meeting 6-11
6.4.2 EPA Senior Staff 6-17
6.4.3 Press Releases 6-19
6.5 Perception of Risk And Uncertainty 6-19
6.6 Trust and Credibility 6-21
Vll
-------
6.7 Communication Issues for RPMs 6-21
References for Chapter 6 6-23
Supplemental References 6-24
Exhibit 6-1 Definitions for Chapter 6 6-2
Exhibit 6-2 Stakeholders Potentially Involved in the Decision-Making Process for PRA 6-4
Exhibit 6-3 Important Steps for Communicating PRA Results 6-7
Exhibit 6-4 Key Considerations in Developing Understandable Material 6-8
Figure 6-1 Hypothetical PRA results showing a PDF and CDF 6-12
Figure 6-2 Results of a sensitivity analysis shown as a pie chart and tornado plot 6-16
Figure 6-3 The results of a 2-D MCA 6-17
Table 6-1 Examples of Graphics for Communicating PRA Concepts in this
Guidance Document 6-14
Chapter 7 Role of the PRA in Decision Making 7-1
7.0 Introduction 7-1
7.1 General Principles of Risk-Based Decision Making In Superfund 7-1
7.2 Interpreting A Risk Distribution 7-3
7.2.1 What Is A Distribution Of Risk And What Does It Look Like? 7-3
7.2.2 What Is the RME Range? 7-4
7.2.3 Relating the Risk Distribution to the Risk Management Goal for
Human Health 7-4
7.2.4 Relating the Risk Distribution to the Risk Management Goal for
Ecological Risk Assessment 7-6
7.3 Factors to Consider in Choosing the Percentile for the RME 7-6
7.4 Uncertainty Associated with the Use of the 99.9th Percentile 7-11
7.5 Moving From A PRO To A Remedial Goal 7-11
References for Chapter 7 7-15
Exhibit 7-1 Definitions for Chapter 7 7-2
Exhibit 7-2 Examples of Demographic, Cultural, and Behavioral Factors that Can
Affect Exposure 7-7
Exhibit 7-3 Examples of Physical or Geographical Factors that Can Affect Exposure 7-7
Exhibit 7-4 Examples of Toxicity Considerations 7-9
. 7-3
. 7-5
7-10
Figure 7-1 Hypothetical PRA results showing a CDF for lifetime excess cancer risk
Figure 7-2 Example of a probability distribution for risk illustrating the 95th percentile
Figure 7-3 Box and whisker plots characterizing uncertainty in the RME
Figure 7-4 Example of graphic showing variability in risk (i.e., RME range, or 90th to 99.9th
percentiles) associated with different choices of PRG for plutonium in soil (pCi/g). . 7-14
Figure 7-5 Example of graphic showing uncertainty in a 95th percentile of the risk distribution
associated with the same choices of PRG as Figure 7-4 7-14
Appendix A Sensitivity Analysis: How Do We Know What's Important? A-l
-------
A.O
Introduction A-l
A. 1.0 Utility of Sensitivity Analysis A-3
A.2.0 Common Methods of Sensitivity Analysis A-10
A.2.1 Tier 1 Approaches A-l 1
A.2.1.1 Percentage Contribution of Exposure Pathways to Total Risk A-12
A.2.1.2 Inspection of Risk Equation A-13
A.2.1.3 Sensitivity Ratio (SR) A-13
A.2.1.4 Sensitivity Score A-19
A.2.2 Tier 2 Approaches A-21
A.2.2.1 Graphical Techniques A-21
A.2.2.2 Correlation Coefficients A-21
A.2.2.3 Focusing on the RME Range of the Risk Distribution A-27
A.2.2.4 Inspection A-27
A.3.0 Advanced Concepts in Sensitivity Analysis A-28
A.3.1 Relating the Change in Risk to the Change in Input Variable X A-28
A.3.2 Normalized Partial Derivative A-31
A.3.3 Regression Analysis: R2, Pearson r, and Partial Correlation Coefficients A-32
A.3.3.1 Calculations of R2 and Adjusted R2 A-33
A.3.3.2 Relative Partial Sum of Squares (RPSS) A-35
A.3.3.3 Spearman's Rank Correlation Coefficient (Rho) A-36
References for Appendix A A-37
Exhibit A-l Definitions for Appendix A A-2
Exhibit A-2 Utility of Sensitivity Analysis A-3
Exhibit A-3 Some Key Indices of Sensitivity Analysis A-10
Exhibit A-4 Categories of Solutions for Sensitivity Ratios of
Multipicative or Additive Equations A-17
Exhibit A-5 Simplifying Assumptions in Regression Analysis A-32
Figure A-l Results of 2-D MCA in which parameters of input distributions describing variability are
assumed to be random values A-9
Figure A-2 Scatterplots of simulated random values from a 1-D MCA of variability. The output
from the model is a contaminant concentration in soil (C) that corresponds with a
prescribed (fixed) level of risk for a hypothetical population A-23
Figure A-3 Scatterplots of simulated random values from a 1-D MCA of variability for example in
Section A.2.0 A-24
Figure A-4 Top panel - bar graph showing the r2 values (square of Spearman rank correlation
coefficient), a metric for the dependence of HI on exposure factors based on 1-D MCA
for variability. Bottom panel - bar graph, sometimes referred to as "tornado plot",
showing rank correlation coefficient A-25
Figure A-5a Hypothetical 2-D response surface for 7given one input variable: Y=F(X) A-29
Figure A-5b Hypothetical 3-D response surface for 7given two input
variables: Y =f(X1 X2) A-30
Figure A-5c Hypothetical 3-D response surface when 7 is a linear function of two input variables:
Y^Xj X2) A-30
Table A-l Overview of Sensitivity Analysis Methods Applicable in
Tiers 1, 2, and 3 of a PRA A-4
Table A-2 Point estimates and probability distributions for input variables used in the hypothetical
example of HI associated with occupational exposure via water and soil ingestion. A-l 1
Table A-3 Percent contribution of exposure pathways to HI for the example
in Section A.2 A-12
IX
-------
Table A-4 Results of the Sensitivity Ratio (SR) approach applied to the hypothetical example of
RME HI given in Section A.2.0. Includes both soil ingestion and tap water ingestion
pathways A-14
Table A-5 Results of the Sensitivity Ratio (SR) approach applied to the hypothetical example of
RME HI given in Section A.2.0. Includes only tap water
ingestion pathway A-15
Table A-6 Examples of algebraic solutions to Sensitivity Ratio calculations for additive and
multiplicative forms of risk equations A-17
Table A-7 Calculation of coefficient of variation (CV = SD / Mean) for the hypothetical
example of RME HI given in Section A.2.0 A-19
Table A-8 Results of the Sensitivity Score (Score) approach applied to the hypothetical
example of RME HI given in Section A.2.0 A-20
Table A-9 Results of Tier 2 sensitivity analyses applied to hypothetical example in
Section A.2.0: Pearson product moment correlations and Spearman
rank correlations A-22
Appendix B Selection and Fitting of Distributions B-l
B.O Introduction B-l
B.1.0 Conceptual Approach for Incorporating a Probability Distribution in a PRA B-3
B.2.0 Preliminary Sensitivity Analysis B-4
B.3.0 What Does The Distribution Represent? B-5
B.3.1 Concepts of Population and Sampling B-6
B.3.2 Considering Variability and Uncertainty in Selecting and Fitting Distributions .... B-12
B.4.0 Do Data Exist To Select Distributions? B-13
B.4.1 What are Representative Data? B-14
B.4.2 The Role of Expert Judgment B-15
B.5.0 Fitting Distributions to Data B-16
B.5.1 Considering the Underlying Mechanism B-17
B.5.2 Empirical Distribution Functions (EDFs) B-22
B.5.3 Graphical Methods for Selecting Probability Distributions B-22
B.5.4 Parameter Estimation Methods B-24
B.5.5 Dealing with Correlations among Variables or Parameters B-26
B.5.6 Censored Data B-28
B.5.7 Truncation B-30
B.6.0 Assessing Quality of the Fit B-31
B.6.1 What is a Goodness-of-Fit Test? B-31
B.6.2 What are some common Goodness-of-Fit Techniques? B-33
B.6.3 Cautions Regarding Goodness-of-Fit Tests B-34
B.6.4 Accuracy of the Tails of the Distribution B-34
B.7.0 Selecting Probability Distributions Based on State of Knowledge B-35
References for Appendix B B-49
Exhibit B-l Definitions for Appendix B B-2
Exhibit B-2 General Strategy for Selecting and Fitting Distributions B-3
Exhibit B-3 Factors to Consider in Selecting a Probability Distribution B-16
Exhibit B-4 Variations of the EOF B-22
Exhibit B-5 Estimating the area of a hypothetical exposure unit B-24
Exhibit B-6 Criteria for Evaluating Parameter Estimation Methods B-25
Exhibit B-7 Parameter Estimation Methods B-25
-------
Exhibit B-8 Correlation of Input Variables for 1-D MCA of Variability B-27
Exhibit B-9 Steps for Simulating Uncertainty in Linear Regression Equation Using a Bivariate
Normal Distribution to Correlate Parameters (P0, P^ B-47
Figure B-l (page 1 of 2). Conceptual approach for incorporating probability distributions
for variability in PRA B-7
Figure B-l (page 2 of 2). Conceptual approach for incorporating probability distributions
for variability in PRA B-8
Figure B-2a (page 1 of 3). Conceptual approach for quantifying model and parameter
uncertainty in PRA B-9
Figure B-2a (page 2 of 3). Conceptual approach for quantifying model and parameter
uncertainty in PRA B-10
Figure B-2a (page 3 of 3). Conceptual approach for quantifying model and parameter
uncertainty in PRA B-l 1
Figure B-3 Comparison of step-wise EDF and linearized EDF for ingestion rate B-38
Figure B-4 Graphical assessment of beta and lognormal distributions fit to the cumulative
distribution reported in the literature (circles) B-39
Figure B-5 Histograms of lead concentrations in quail breast muscle B-41
Figure B-6 Lognormal probability plots of lead in mourning dove breast tissue B-43
Figure B-7 Histograms of meal size B-44
Figure B-8 Probability plot of meal size data B-45
Figure B-9 Simple linear regression of zinc concentrations in soil and dust B-48
Figure B-10 Results of Monte Carlo simulation B-49
Table B-l Examples of Preliminary Distributions Based on Information Available B-5
Table B-2 Examples of Selected Probability Distributions for PRA B-l8
Table B-3 Theoretical bounds and parameter values for selected distributions B-30
Table B-4 Strategies for conducting PRA based on available information B-36
Table B-5 Selected statistics for reported and fitted distributions for ingestion rate (mg/day). . B-38
Table B-6 Sample values of lead concentration (ppm) in quail breast muscle B-41
Table B-7 Parameter estimates for lognormal distribution of lead concentrations (ppm) B-42
Table B-8 Meal size (g/meal) B-44
Table B-9 Zinc concentrations in paired (i.e., co-located) soil and dust samples (ppm) for n=21
locations B-48
Appendix C Characterizing Variability and Uncertainty in the Concentration Term C-l
C.O The Concentration Term and the Exposure Unit C-l
C.1.0 Variability in PRA C-l
C. 1.1 Temporal Variability C-2
C.1.2 Spatial Variability C-3
C.I.3 Example of Temporal and Spatial Variability C-4
C.I.4 Spatial and Temporal Variability for Different Exposure Media C-5
C.I.4.1 Variability of Concentrations in Soil C-5
C.I.4.2 Variability of Concentrations in Groundwater C-5
C.I.4.3 Variability of Concentrations in Surface Water C-5
C.I.4.4 Variability of Concentrations in Sediment C-5
C.I.4.5 Variability of Concentrations in Fish C-5
C.I.4.6 Examples of Temporal and Spatial Variability in the Concentration Term
for Selected Exposure Media C-6
C.2.0 Nonrandom Exposures C-7
XI
-------
C.3.0 Sources of Uncertainty in the Concentration Term C-8
C.3.1 Quantification of Uncertainty Based on the Size of the Exposure Unit C-8
C.3.1.1 When the Exposure Unit Is Smallerthan the Site C-8
C.3.1.2 When the Exposure Unit is the Same Size as the Site C-9
C.3.1.3 When the Exposure Unit is Larger than the Site C-9
C.4.0 Summary of Recommendations for the Concentration Term C-10
C.5.0 Methods for Estimating Uncertainty in the Mean Concentration C-10
C.5.1 Quantifying Uncertainty without Information About Locations of
Samples and Receptors C-12
C.5.2 Quantifying Uncertainty with Information About Locations of
Samples and Receptors C-12
References for Appendix C C-14
Figure C-l Spatial and temporal variability in contaminant concentrations in groundwater C-7
Table C-l Examples of temporal and spatial variability in selected media for the
concentration term in common exposure scenarios C-6
Table C-2 Summary of factors that may be considered in developing an EPC C-10
Appendix D Advanced Modeling Approaches for Characterizing Variability and Uncertainty D-l
D.O Introduction D-l
D.1.0 Expressing Variability and Uncertainty Simultaneously D-l
D.2.0 Two-Dimensional Monte Carlo Analysis (2-D MCA) D-3
D.3.0 Microexposure Event Analysis D-6
D.4.0 Geospatial Statistics D-10
D.4.1 Correlation and Spatial Autocorrelation D-l 1
D.4.2 Effective Sample Size (n*) and Degrees of Freedom D-12
D.4.3 Assessment of Additional Site Sampling D-13
D.4.4 Map Generalization D-15
0.4.5 Implementation Issues Related to Georeferenced Data D-l6
D.5.0 Expert Judgment and Bayesian Analysis D-16
References for Appendix D D-25
Exhibit D-l Definitions for Appendix D D-3
Exhibit D-2 Positive Spatial Autocorrelation D-10
Exhibit D-3 Examples of Risk Assessment Issues Linked to Geospatial Statistics D-10
Exhibit D-4 Effect of Spatial Autocorrelation (r) on Effective Sample Size («*) D-13
Exhibit D-5 Components of Bayes Theorem in PRA D-17
Figure D-l Panel A shows a family of 20 CDFs for a hypothetical random variable. Panel B shows
the "90% credible interval" for the CDF based on 2500 simulations D-2
Figure D-2 Diagram showing of a 2-D Monte Carlo model D-4
Figure D-3 Output from a 2-D MCA showing the estimated mean Hazard Quotient (HQ) and the
90% confidence interval D-5
Figure D-4 Time Step for MEE D-7
Figure D-5 Flowchart showing general approach for Microexposure Event (MEE) analysis. . . . D-8
Figure D-6 Hypothetical example showing the effect of model time step on the probability
distribution for soil and dust ingestion rate in children over a 1-year period D-9
Figure D-7 Effect of an outlier on measured correlation D-12
Figure D-8 Conceptual model of Bayesian Monte Carlo analysis D-l 8
xii
-------
Figure D-9 Expected Loss associated with various types of information incorporated into a generic
uncertainty analysis D-21
Figure D-10 Conceptual model for evaluating the expected value of including uncertainty in a
Bayesian Monte Carlo analysis D-23
Appendix E Definitions of Terms Relevant to PRA and References for Further Reading ... E-l
E.O Definitions of Terms E-l
E. 1 Additional Information E-14
References for Appendix E E-15
References for Further Reading E-16
Appendix F Workplan and Checklist for PRA F-l
F.O Introduction F-l
F.1.0 Workplan F-l
F.2.0 Focal Points for PRA Review F-2
F.3.0 Checklist for Reviewers F-2
F.4.0 Internal and External Review F-3
References for Appendix F F-6
Exhibit F-l Examples of Elements of the Workplan for PRA F-l
Exhibit F-2 Key Focal Points for PRA Review F-2
Table F-1 Example of a Generic Checklist for Reviewers F-4
Appendix G Frequently Asked Questions for PRA G-l
References for Appendix G G-6
Appendix H Index H-l
-------
ACRONYMS AND ABBREVIATIONS
l-D MCA
2-D MCA
95% UCL
AM
ARARs
AT
AWQC
BCa
BMD
BMDS
BMR
BTAG
BW
C
CAG
CDF
CI
CIC
CIP
CLT
COC
CQR
CSF
CTE
CV
DI
DQO
EC0
-EC20
ECDF
ED
ED10
EOF
EF
EPA
EPC
ERA
ERAF
ERAGS
EU
EVIU
EVOI
EVPI
EVSI
GIS
GM
GoF
GSD
HEAST
HHEM
HI
One-dimensional Monte Carlo analysis
Two-dimensional Monte Carlo analysis
95% upper confidence limit
Arithmetic mean
Applicable or relevant and appropriate requirements
Averaging time
Ambient water quality criterion
Bias correction acceleration method
Benchmark dose
Benchmark dose software
Benchmark Response
Biological Technical Assistance Group
Body weight
Concentration
Community advisory group
Cumulative distribution function
Confidence interval
Community involvement coordinator
Community involvement plan
Central limit theorem
Chemical of concern
Continuous quadratic regression
Cancer slope factor
Central tendency exposure
Coefficient of variation
Daily intake
Data quality objectives
Exposure concentration that produces zero effect
Concentration that causes a 20% effect
Empirical cumulative distribution function
Exposure duration
Dose that causes a 10% effect
Empirical distribution function
Exposure frequency
U.S. Environmental Protection Agency
Exposure point concentration
Ecological risk assessment
Risk Assessment Forum
Ecological Risk Assessment Guidance for Superfund
Exposure unit
Expected value of including uncertainty
Expected value of information
Expected value of perfect information
Expected value of sample information
Geographical Information Systems
Geometric mean
Goodness-of-Fit
Geometric standard deviation
Health effects assessment summary table
Human Health Evaluation Manual
Hazard Index
-------
HQ
IR
Irsd
IRIS
LADD
LCL
LED10
LHS
LOAEL
LOD
LOEC
MCA
MCL
MDC
MEE
MLE
MoMM
NCP
NOAEL
NOEC
OLS
PBPK
PCBs
pCi/g
PDF
PDFu
PDFv
PMF
PPT
PRA
PRO
PRP
QAPP
RAGS
RAL
RBC
RCRA
RfC
RfD
RG
RI/FS
RME
RMSE
ROD
ROS
RPSS
RPM
RSS
SCM
SD
Hazard Quotient
Iterative reduction
Soil and dust ingestion rate
Integrated Risk Information System
Life-time average daily intake
Lower confidence limit
Lowest effect dose - lower confidence bound for dose that causes a 10% effect
Latin hypercube sampling
Lowest-observed-adverse-effect level
Limit of detection
Lowest-observed-effect-concentration
Monte Carlo analysis
Maximum contaminant levels
Maximum detected concentration
Microexposure Event Analysis
Maximum Likelihood Estimation
Method of Matching Moments
National Oil and Hazardous Substances Pollution Contingency Plan
No-observed-adverse-effect level
No-observed-effect-concentration
Ordinary least squares
Physiologically-based pharmacokinetic
Polychlorinated biphenyls
Picocuries/gram
Probability density function
Probability distribution for variability
Probability distribution for uncertainty
Probability mass function
Parts per trillion
Probabilistic risk assessment
Preliminary remediation goal
Potentially responsible party
Quality Assurance Project Plan
Risk Assessment guidance for Superfund
Remedial action level
Risk based concentration
Resource Conservation and Recovery Act
Reference concentration
Reference dose
Remediation goal
Remedial Investigation/Feasibility Study
Reasonable maximum exposure
Root mean squared error
Record of decision
Rank order statistic
Relative partial sum of squares
Remedial project manager
Regression sum of squares
Site conceptual model
Standard deviation
-------
SE Standard error
SMDP Scientific/Management Decision Point
SOW Statement of Work
SR Sensitivity ratio
SSD Species sensitivity distribution
SSE Sum of squares due to error
SSR Sum of squares due to regression
SST Sum of squares for total (regression plus error)
TAB Technical Assistance to Brownfields Community
TAG Technical assistance grant
TOSC Technical outreach services for communities
TRV Toxicity reference value
TSS Total sum of squares
UCL Upper confidence limit
VOI Value of information
-------
AUTHORS, CONTRIBUTORS, AND REVIEWERS
This manual was developed by EPA's Office of Emergency and Remedial Response. A
number of individuals have reviewed and/or have been contributing authors of this document.
Members of the EPA RAGS Volume III Workgroup, which was responsible for developing this
document, included the following EPA headquarters and regional office staff.
RAGS VOLUME III WORKGROUP PARTICIPANTS
EPA HEADQUARTERS
Office of Emergency and Remedial Response
David A. Bennett
S. Steven Chang
David E. Cooper
Janine Dinan
Elizabeth Lee Hofmann
Office of Policy Economics and Innovation Timothy M. Barry
EPA REGIONAL OFFICES
Region 1 Ann-Marie Burke
Region 2 Audrey Galizia
Marian Olsen
Region 3 Nancy Rios Jafolla
Region 4 Ted W. Simon
Sharon R. Thorns
Region 5
Region 6
Region 8
Amy Mucha
James Chapman
Maria L. Martinez
Susan Griffin
Gerry Henningsen
Dale Hoff
Region 10 Joe Goulet
Technical assistance and production support was provided to EPA in the development of
this guidance under Contract Numbers GS-10F-0137K and GS-35F-0555K.
An earlier draft of this document was peer reviewed by a panel of experts at a peer-
review workshop held in November 2000. In addition, individuals in EPA and from the public
provided valuable comments on earlier drafts of this guidance during the peer review process.
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Preface ~ December 31, 2001
PREFACE
Risk Assessment Guidance for Superfund (RAGS) Volume III: Part A (hereafter referred to as
RAGS Volume 3: Part A) provides technical guidance on the application of probabilistic risk assessment
(PRA) methods to human health and ecological risk assessment in the U.S. Environmental Protection
Agency (EPA) Superfund program. RAGS Volume 3: Part A supplements existing human health and
ecological assessment guidance provided in the RAGS series. This guidance focuses on Monte Carlo
analysis (MCA) as a method of quantifying variability and uncertainty in risk. Primarily geared toward
the risk assessor, it is intended, both in content and format, to be most accessible to those readers who are
familiar with risk assessment and basic statistical concepts. Chapters 1, 2, 6, and 7 are also directed
towards risk managers. The term risk manager is used in this guidance to refer to individuals or entities
that serve as the decision makers at hazardous waste sites. The term is used to emphasize the separation
between risk assessment and risk management activities. Risk managers may include individual remedial
project mangers (RPMs), site partnering teams, senior EPA managers (Section Chiefs, Branch Chiefs, or
Division Directors), or other decision makers.
An attempt has been made in this document to define all relevant technical terms using plain
language and to illustrate concepts with examples. An exhibit at the beginning of each chapter provides
definitions of terms used in that chapter. In addition, a comprehensive definition of terms is provided in
Appendix E. Other useful information has been presented in exhibits placed throughout each chapter.
Bullets are used throughout the text to emphasize important concepts and policy statements related to the
use of PRA. References are listed at the end of each chapter.
RAGS Volume 3: Part A was developed by the Superfund Probabilistic Risk Assessment
Workgroup and the Ecological Risk Assessment Forum (ERAF); both are intra-Agency workgroups that
have focused on improving the Risk Assessment Guidance for Superfund and implementing Superfund
Reform activities. The guidance has undergone extensive review by Superfund and other programs
within the Agency. In February 2000, a draft of the guidance was announced in the Federal Register to
provide an opportunity for public comment (U.S. EPA, 2000a). In August 2000, a notice of peer review
was announced in the Federal Register (U.S. EPA, 2000b), and in November 2000, RAGS Volume 3: Part
A received a formal peer review from panelists outside the Agency.
The Agency may incorporate PRA under fund-lead and Potentially Responsible Party (PRP)-lead
risk assessments. Implementation of successful PRAs requires careful planning. EPA strongly
recommends that PRPs involve the Agency in all decisions regarding the planning, submittal, and
technical details of any PRA. Coordinating with EPA early in the process will help ensure that PRAs
conform to the recommended guidelines as part of the Superfund risk assessment process for protecting
human and ecological health. PRPs should submit workplans for Agency review before initiating any
PRA. Similarly, when EPA chooses to use PRA for an EPA-lead risk assessment, a PRA workplan will
assist in directing site investigation and risk assessment activities, whether conducted by EPA or an EPA
contractor. A workplan specifies contractor activities in the risk assessment and provides risk assessors
and risk managers with an opportunity to obtain internal feedback from knowledgeable EPA staff, prior to
initiating work on the assessment.
A tiered approach to PRA is advocated, which begins with a point estimate risk assessment.
Important considerations include the time required to perform the PRA, the additional resources involved
in developing the PRA, the quality and extent of data on exposure that will be used in the assessment, and
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Preface ~ December 31, 2001
the value added by conducting the PRA. Project scoping is an essential component of all risk assessments
and is especially important in PRA.
Implementation of a PRA usually requires special computer software that may be commercially
available or that may need to be custom-designed for a specific application. Although commercial
software packages are noted in this guidance, any mention or use of a particular product in RAGS
Volume 3: Part A does not constitute an endorsement of that product by the Agency.
1.0 WHAT is THE PURPOSE OF RAGS VOLUME 3 PART A?
RAGS Volume 3: Part A addresses the technical and policy issues associated with the use of PRA
in EPA Superfund program. This guidance builds upon basic concepts of risk assessment outlined in
RAGS Volume I (U.S. EPA, 1989a; 2001), recent guidance for ecological risk assessment (U.S. EPA,
1992, 1994, 1997a, 1998a; 1999), and the Agency Probabilistic Analysis Policy document (U.S. EPA,
1997b). RAGS Volume 3: Part A addresses the use of PRA for both human health and ecological risk
assessments. RAGS Volume 3: Part A was developed to provide risk assessors and risk managers with
basic guidelines for incorporating PRA into Superfund site-specific risk assessments. It is not intended to
be a detailed technical reference on PRA methods, however, it does direct the reader to appropriate
literature on important technical subjects. A primary purpose of RAGS Volume 3: Part A is to help
prevent misuse and misinterpretation of PRA.
2.0 WHAT is PROBABILISTIC RISK ASSESSMENT AND HOW is IT USED IN RISK
CHARACTERIZATION?
PRA is a risk assessment that uses probability distributions to characterize variability or
uncertainty in risk estimates. In a PRA, one or more variables in the risk equation is defined as a
probability distribution rather than a single number. Similarly, the output of a PRA is a range or
probability distribution of risks experienced by the receptors. The evaluation of variability and
uncertainty is an important component of the risk characterization of all risk assessments. As stated in the
1995 Risk Characterization memorandum from Administrator Carol Browner (U.S. EPA, 1995),
... we must fully, openly, and clearly characterize risks. In doing so, we will disclose the
scientific analyses, uncertainties, assumptions, and science policies which underlie our
decisions... There is value in sharing with others the complexities and challenges we face
in making decisions in the face of uncertainty.
In addition, the 1997 EPA Policy for Use of Probabilistic Analysis in Risk Assessment (U.S.
EPA, 1997b) states:
is" It is the policy of the U.S. Environmental Protection Agency that such
probabilistic analysis techniques as Monte Carlo analysis, given adequate
supporting data and credible assumptions, can be viable statistical tools for
analyzing variability and uncertainty in risk assessments.
A more extensive general discussion of PRA can be found in Chapter 1 of the guidance. The use
of PRA in Superfund remedial decision making is presented in Chapter 7 of the guidance.
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Preface ~ December 31, 2001
3.0 WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF PRA FOR REMEDIAL
DECISIONS?
The primary advantage of PRA within the Superfund program is that it can provide a quantitative
description of the degree of variability or uncertainty (or both) in risk estimates for both cancer and
non-cancer health effects and ecological hazards. The quantitative analysis of uncertainty and variability
can provide a more comprehensive characterization of risk than is possible in the point estimate approach.
Another significant advantage of PRA is the additional information and potential flexibility it
affords the risk manager. Superfund remedy decisions are often based on an evaluation of the risk to the
individual at the reasonable maximum exposure (RME) level (U.S. EPA, 1990). The RME represents the
highest exposure reasonably likely to occur (U.S. EPA, 1989a). When using PRA, the risk manager can
select the RME from the high-end range of percentiles of risk, generally between the 90th and
99.9th percentiles, referred to in this guidance as the RME range.
However, PRA may not be appropriate for every site. Disadvantages of PRA are that it generally
requires more time, resources, and expertise on the part of the assessor, reviewer, and risk manager than a
point estimate approach.
4.0 How is RAGS VOLUME 3, PART A ORGANIZED?
Although the primary audience of this guidance is the risk assessor, Chapter 1 provides a basic
overview of PRA for risk assessors and risk managers. The centerpiece of RAGS Volume 3: Part A is the
tiered approach described in Chapter 2. The tiered approach is a framework that enables the risk manager
to decide if and when to undertake a PRA and to determine the appropriate level of complexity for the
PRA. Chapter 3 provides a description of using PRA for human health risk assessment. Chapter 4
discusses the issues of using PRA for ecological risk assessment. Chapter 5 presents a discussion of using
PRA to determine preliminary remediation goals. Chapter 6 details issues associated with communicating
risk estimates developed with PRA. Chapter 7 provides information for risk managers choosing to base
remedial decisions on the results of a PRA.
Eight appendices to this guidance expand on technical aspects of topics important to PRA, such
as sensitivity analysis and selecting and fitting probability distributions.
5.0 WHAT ARE THE KEY GUIDING CONCEPTS IN RAGS VOLUME 3: PART A?
(1) Use a tiered approach to incorporating PRA into site risk assessments.
(2) Submit a workplanfor Agency review prior to initiating work on a PRA.
(3) Perform a point estimate assessment prior to considering a PRA.
(4) While PRA can provide a useful tool to characterize and quantify variability and uncertainty
in risk assessments, it is not appropriate for every site.
(5) PRA generally requires more time, resources, and expertise on the part of the assessor,
reviewer, and risk manager than a point estimate risk assessment.
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Preface ~ December 31, 2001
(6) The decision to use PRA is site-specific and is based on the complexity of the problems at the
site, the quality and extent of site-specific data, and the likely utility of the result.
(7) If the additional information provided from a PRA is unlikely to affect the risk management
decision, then it may not be prudent to proceed with a PRA. However, if there is a clear
value added from performing a PRA, then the use of PRA as a risk assessment tool generally
should be considered despite the additional resources that may be needed.
(8) Communicating the results of a PRA will be more challenging than communicating the
results of a point estimate risk assessment because PRA and its perspective will be new to
most participants.
(9) If the decision is made to conduct a PRA, it is important to include community in the
planning process. Communication on PRA may involve: providing the community with a
basic understanding of the principles of PRA, discussing the proposed workplan and inviting
comments on the proposed approach, discussing site-specific data, and communicating the
final results and how they impact decisions for the site.
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Preface ~ December 31, 2001
REFERENCES FOR PREFACE
U.S. EPA. 1989a. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health
Evaluation Manual (HHEM) (Part A, Baseline Risk Assessment). Interim Final. Office of
Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40
CFR 300: 55 Federal Register, 8666-8865, March 8.
U.S. EPA. 1992. Final Guidelines for Exposure Assessment. EPA/600/Z-92/001. Federal Register,
22888-22938. May 29.
U.S. EPA. 1994. Role of Ecological Risk Assessment in the Baseline Risk Assessment. Office of Solid
Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-17.
U.S. EPA. 1995. Memorandum from Carol Browner on Risk Characterization. Office of the
Administrator, Washington, DC. February 22.
U.S. EPA. 1997a. Ecological Risk Assessment Guidance for Superfund: Process for Designing and
Conducting Ecological Risk Assessments. Interim Final. Environmental Response Team, Edison,
NJ. EPA/540/R-97/006, OSWER Directive No. 9285.7-25. June.
U.S. EPA. 1997b. Memorandum from Deputy Administrator Fred Hansen on the Use of Probabilistic
Techniques (including Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for
Monte Carlo Analysis. Office of Research and Development, Washington, DC.
EPA/630/R-97/001. May 15.
U.S. EPA. 1998a. Guidelines for Ecological Risk Assessment. Risk Assessment Forum.
Environmental Protection Agency, Washington DC. EPA/630/R-95/002F. April. Federal
Register 63(93): 26846-26924. May 14.
U.S. EPA. 1999. Ecological Risk Assessment and Risk Management Principles for Superfund Sites. Final.
Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive
No. 9285.7-28P.
U.S. EPA. 2000a. Superfund Probabilistic Risk Assessment to Characterize Uncertainty and Variability.
Washington, DC. Federal Register [FRDoc. 06-3492] 65(31): 7550-7552. February 15.
U.S. EPA. 2000b. Peer Review for Superfund Probabilistic Risk Guidance. Washington, DC. Federal
Register [FRDoc. 00-21197] 65(162): 50694. August21.
U.S. EPA. 2001. Risk Assessment Guidance for Superfund: Volume I. Human Health Evaluation Manual
(Part D, Standardized Planning, Reporting, and Review of Superfiind Risk Assessments). Office
of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9285.7-47.
December.
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
CHAPTER 1
OVERVIEW OF PROBABILISTIC APPROACH TO RISK ASSESSMENT
1.0 INTRODUCTION
This chapter is intended for risk managers and risk assessors as an overview of the probabilistic
approach to risk assessment in the context of the Superfund program at the U.S. Environmental Protection
Agency (EPA). The goals of this chapter are to provide the reader with information about (1) the role of
risk assessment in the Superfund program; (2) the basic concepts of probabilistic risk assessment (PRA);
(3) important policies and guiding principles for PRA, as outlined throughout this guidance; and (4) the
next steps that will be undertaken in the Superfund program to provide guidance on PRA.
Section 1.1 (1.1.1-1.1.3) describes the role of risk assessment from three perspectives, including
the role of risk assessment in areas external to EPA, Agency-wide, and within Superfund. Section 1.1
(1.1.4) also introduces PRA and identifies its place in the Superfund program. Section 1.2 introduces the
basic concepts of PRA, including the key terms of variability, uncertainty, Monte Carlo analysis (MCA),
and reasonable maximum exposure (RME). PRA concepts are presented using a comparison between
PRA and the traditional point estimate approach. Sections 1.2.4 and 1.3 summarize the advantages and
disadvantages of PRA and point estimate risk assessment. Section 1.4 provides a summary of policies
and guiding principles for using PRA in the Superfund program. EPA's policies on conducting PRA are
highlighted throughout the guidance using pointers and are linked to more detailed policy discussions in
other chapters in the guidance. Section 1.5 outlines the organization of this document and provides a
brief summary of the content of each subsequent chapter and appendix. Section 1.6 presents EPA's next
steps for PRA implementation in the Superfund program.
Key terms used throughout this guidance include: Probabilistic Risk Assessment (PRA), Monte
Carlo Analysis (MCA), Probability Density Function (PDF), Cumulative Distribution Function (CDF),
Reasonable Maximum Exposure (RME), Sensitivity Analysis, Tiered Approach, Variability, Uncertainty,
and Preliminary Remediation Goal (PRO). Terms and their definitions are identified in an exhibit at the
beginning of each chapter. Terms and definitions relevant to Chapter 1 are presented in Exhibit 1-1. In
addition, a glossary of terms used throughout the guidance is given in Appendix E.
Page 1-1
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
EXHIBIT 1-1
DEFINITIONS FOR CHAPTER 1
Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in a population,
usually considered to be the mean or median of the distribution.
Confidence Interval - A range of values that are likely to include a population parameter. Confidence intervals may
describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95th percentile
risk). When used to characterize uncertainty in a risk estimate, it is assumed that methods used to quantify
uncertainty in the model inputs are based on statistical principles such as sampling distributions or Bayesian
approaches. For example, given a randomly sampled data set, a 95% confidence interval for the mean can be
estimated by deriving a sampling distribution from a Student's t distribution.
Confidence Limit - The upper or lower value of a confidence interval.
Countablv Infinite - Used to describe some discrete random variables, this term refers to a set of numbers that can be
counted with integers (e.g., one, two, three) and that has no upper limit. Examples include the number of tosses
required for a coin to show a head—we can count each toss, but it is possible that at least one more toss is
needed. The number of dust particles in a volume of air is another example. Countably finite implies there is
an upper limit (e.g., days of work per year).
Credible Interval - A range of values that represent plausible bounds on a population parameter. Credible intervals
may describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95th percentile
risk). The term is introduced as an alternative to the term confidence interval when the methods used to
quantify uncertainty are not based entirely on statistical principles such as sampling distributions or Bayesian
approaches. For example, multiple estimates of an arithmetic mean may be available from different studies
reported in the literature - using professional judgment, these estimates may support a decision to describe a
range of possible values for the arithmetic mean.
CTE Risk - The estimated risk corresponding to the central tendency exposure.
Cumulative Distribution Function (CDF) - Obtained by integrating the PDF, gives the cumulative probability of
occurrence for a random independent variable. Each value c of the function is the probability that a random
observation x will be less than or equal to c.
Expected Value of Information (EVOI) - The expected increase in the value (or decrease in the loss) associated with
obtaining more information about quantities relevant to the decision process. EVOI is a measure of the
importance of uncertainty in risk and the potential for changing a risk management decision if uncertainty is
reduced (see Appendix D).
Frequency Distribution or Histogram - A graphic (plot) summarizing the frequency of the values observed or
measured from a population. It conveys the range of values and the count (or proportion of the sample) that
was observed across that range.
Monte Carlo Analysis (MCA) or Monte Carlo Simulation - A technique for characterizing the uncertainty and
variability in risk estimates by repeatedly sampling the probability distributions of the risk equation inputs and
using these inputs to calculate a range of risk values.
Numeric Stability - Stochastic variability, or "wobble" associated with random sampling, calculated as the average
percent change in the model output after rerunning Monte Carlo simulations with the same set of input
assumptions. Used as a metric for evaluating the adequacy of the number of iterations in a MCA.
Parameter - A value that characterizes the distribution of a random variable. Parameters commonly characterize the
location, scale, shape, or bounds of the distribution. For example, a truncated normal probability distribution
may be defined by four parameters: arithmetic mean [location], standard deviation [scale], and min and max
[bounds]. It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithmetic
mean ingestion rate).
Point Estimate - In statistical theory, a quantity calculated from values in a sample to estimate a fixed but unknown
population parameter. Point estimates typically represent a central tendency or upper bound estimate of
variability.
Page 1-2
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
EXHIBIT 1-1
DEFINITIONS FOR CHAPTER 1—Continued
Point Estimate Risk Assessment - A risk assessment in which a point estimate of risk is calculated from a set
of point estimates for exposure and toxicity. Such point estimates of risk can reflect the CTE, RME, or
bounding risk estimate depending on the choice of inputs.
Probabilistic Risk Assessment (PRA) - A risk assessment that yields a probability distribution for risk,
generally by assigning a probability distribution to represent variability or uncertainty in one or more
inputs to the risk equation.
Probability Density Function (PDF) - A function representing the probability distribution of a continuous
random variable. The density at a point refers to the probability that the variable will have a value in a
narrow range about that point.
Probability Distribution - A mathematical representation of the function that relates probabilities with
specified intervals of values for a random variable. Also called ^.probability model.
Probability Mass Function (PMF) - A function representing the probability distribution for a discrete random
variable. The mass at a point refers to the probability that the variable will have a value at that point.
Random Variable - A variable that may assume any value from a set of values according to chance. Discrete
random variables can assume only a finite or countably infinite number of values (e.g., number of
rainfall events per year). A random value is continuous if its set of possible values is an entire interval
of numbers (e.g., quantity of rain in a year).
Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site
(U.S. EPA, 1989a). The intent of the RME is to estimate a conservative exposure case (i.e., well above
the average case) that is still within the range of possible exposures.
Remedial Investigation/Feasibility Study (RI/FS) - Studies undertaken by EPA to delineate the nature and
extent of contamination, to evaluate potential risk, and to develop alternatives for cleanup.
RME Risk - The estimated risk corresponding to the reasonable maximum exposure.
Sensitivity Analysis - Sensitivity generally refers to the variation in output of a model with respect to changes
in the values of the model's input(s). Sensitivity analysis can provide a quantitative ranking of the
model inputs based on their relative contributions to model output variability and uncertainty. Common
metrics of sensitivity include:
*• Pearson Correlation Coefficient - A statistic r that measures the strength and direction of linear
association between the values of two quantitative variables. The square of the coefficient (r2)
is the fraction of the variance of one variable that is explained by the variance of the second
variable.
>• Sensitivity Ratio - Ratio of the change in model output per unit change in an input variable;
also called elasticity.
*• Spearman Rank Order Correlation Coefficient - A "distribution free" or nonparametric statistic
r that measures the strength and direction of association between the ranks of the values (not
the values themselves) of two quantitative variables. See Pearson (above) for r2.
Stochastic Dominance - Implies no intersection between two or more CDFs. For example, if the CDF for A
and B do not overlap and the CDF for A is greater than the CDF for B, then at every cumulative percentile,
the value of A is greater than that of B. Therefore, it can be stated that distribution A stochastically
dominates distribution B. It should be noted that even when the CDFs for A and B do not overlap, the
PDFs for A and B can overlap.
Uncertainty -Lack of knowledge about specific variables, parameters, models, or other factors. Examples
include limited data regarding the concentration of a contaminant in an environmental medium and lack of
information on local fish consumption practices. Uncertainty may be reduced through further study.
Variability - True heterogeneity or diversity that characterizes an exposure variable or response in a
population. Further study (e.g., increasing sample size, ri) will not reduce variability, but it can provide
greater confidence (e.g., lower uncertainty) in quantitative characterizations of variability).
Page 1-3
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
1.1 THE ROLE OF RISK ASSESSMENT IN SUPERFUND
The role of risk assessment in the Superfund program today is built upon a foundation of
scientific and management principles, policies, and laws that have been established over the past two
decades. Since the enactment of the Comprehensive Environmental Response, Compensation, and
Liability Act (CERCLA) in 1980 the risk assessment policies and guidance documents have evolved to
reflect advances in science and changes in federal regulations.
1.1.1 RISK ASSESSMENT IN THE UNITED STATES
Risk assessment has a long history beginning in 1940. In 1983, the National Research Council
published Risk Assessment in the Federal Government: Managing the Process (NRC, 1983) which
outlines the four steps of risk assessment (hazard identification, dose-response, exposure assessment, and
risk characterization) that are used today.
The NRC addressed three main objectives in risk assessment: (1) assessment of the benefits of
separating the analytical process of risk assessment from the regulatory process of risk management;
(2) consideration of the feasibility of creating a single regulatory agency for the purpose of conducting all
government risk assessments; and (3) consideration of the feasibility of creating uniform guidelines for
risk assessment (NRC, 1983).
The Committee concluded that regulatory agencies should maintain a conceptual distinction
between risk assessment and risk management, and develop uniform inference guidelines in risk
assessment for use by all federal regulatory agencies. The Committee also recommended that Congress
establish a Board on Risk Assessment Methods in order to ensure that risk assessment procedures be
continuously reviewed and modified as the science advances. The Committee rejected the proposal for a
single federal risk assessment agency based on inadequate evidence to show that one administrative
structure would be more advantageous (NRC, 1983).
Since 1983, there have been ongoing advancements in the field of risk assessment. These
include: (1) a continued increasing role for risk assessment in the decision-making process of many
regulatory agencies, as exemplified by several bills introduced by the 103rd and 104th Congresses in
1994-1995; (2) an increased awareness of the need for uncertainty analysis and for quantifying and
communicating uncertainties in risk estimates (Science and Judgement in Risk Assessment, NRC, 1994);
(3) guidance about more inclusive approaches to risk assessment, as exemplified by environmental health
legislation such as the Food Quality Protection Act (FQPA) of 1996 and the Presidential/Congressional
Commission on Risk Assessment and Risk Management (1997); and (4) setting the stage for a more open
decision-making process through stakeholder involvement in the risk management process, as outlined in
Improving Risk Communication (NRC, 1989).
1.1.2 RISK ASSESSMENT AT EPA
EPA has refined the risk paradigm through deliberations of the Risk Assessment Forum, Science
Policy Council, and other Agency-wide bodies. Such deliberations have led to consensus in guidance,
policies, and memoranda that respond to the requirements set out by various environmental statutes.
Individual offices have also developed regulations, guidance, and other supporting documents to aid in
the implementation of particular environmental statutes.
Page 1-4
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
In 1986, EPA issued final guidelines relating to risk assessment for cancer, mutagenic effects,
developmental effects, exposure assessment, and chemical mixtures. Since 1986, EPA has updated or
issued revised final guidelines for developmental toxicity, exposure assessment, reproductive toxicity,
neurotoxicity, and ecological risk assessment; and is now revising carcinogen risk assessment guidelines.
(See http://www.epa.gov/ncea/raf/rafguid.htm for details on guidelines)
Other notable documents that guide risk assessment at EPA include:
• Framework for Ecological Risk Assessment (U.S. EPA, 1992b)
• Guidelines for Ecological Risk Assessment (U.S. EPA, 1998)
• Guidance for Risk Characterization (U.S. EPA, 1995a)
• Policy for Risk Characterization (U.S. EPA, 1995c)
• Policy on Evaluating Health Risks to Children (U.S. EPA, 1995d)
• Policy for Use of Probabilistic Analysis in Risk Assessment (U.S. EPA, 1997g)
• Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment
(U.S. EPA, 1997g)
• Guidance on Cumulative Risk Assessment. Part 1. Planning and Scoping
(U.S. EPA, 1997e)
• Risk Characterization Handbook (U.S. EPA, 2000)
1.1.3 RISK ASSESSMENT IN SUPERFUND
The activities and publications described above have provided a strong foundation for the
development of risk assessment guidance on conducting human health—and ecological risk assessments
in the Superfund program. EPA uses risk assessment (NRC, 1983, 1994) to carry out CERCLA, as
amended by the Superfund Amendments and Reauthorization Act of 1986 (SARA). Under
CERCLA/SARA, EPA's Superfund program is authorized to protect human health and the environment
from current and potential threats posed by releases of hazardous substances, pollutants, or contaminants.
The blueprint for the Superfund program is the National Oil and Hazardous Substances Pollution
Contingency Plan (NCP) (U.S. EPA, 1990). Among other things, the NCP calls for the identification and
mitigation of environmental impacts at hazardous waste sites, and for the selection of remedial actions to
protect human health and the environment. An important part of the NCP is the implementation of a
Remedial Investigation and Feasibility Study (RI/FS), which is designed to support risk management
decisions within the Superfund program. A risk assessment is an integral part of the RI/FS, and is
Page 1-5
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
EXHIBIT 1-2
NINE CRITERIA FOR EVALUATION OF
CLEANUP ALTERNATIVES (U.S. EPA, 1990)
Threshold Criteria
1. Overall protection of human health and the
environment
2. Compliance with ARARs
Balancing Criteria
3. Long-term effectiveness and permanence
4. Reduction in toxicity, mobility, or volume
through treatment
5. Short-term effectiveness
6. Implementability
7. Cost
Modifying Criteria
8. State acceptance
9. Community acceptance
generally conducted at a site to determine the
need for action and to ensure that a selected
remedy will be protective. The NCP also
establishes some benchmarks for protectiveness
and lays out nine criteria (some risk-based)
against which each cleanup option should be
evaluated (see Exhibit 1-2).
Guidance for risk assessment in the
Superfund program has been developed to
facilitate consistent site-specific responses.
Early major guidance documents developed by
EPA included: Risk Assessment Guidance for
Superfund (RAGS): Volume I. Human Health
Evaluation Manual (HHEM) (Part A, Baseline
Risk Assessment) (U.S. EPA, 1989a) and Risk
Assessment Guidance for Superfund. (RAGS):
Volume II. Environmental Evaluation Manual
(U.S. EPA, 1989b). RAGS Volume I: Part A
provides an approach for conducting
site-specific baseline (i.e., without remediation
or institutional controls) human health risk
assessments. RAGS Volume II, aimed at site
managers, provides a framework for considering
environmental effects at sites. More recently,
EPA developed guidance for conducting ecological risk assessments within the Superfund program. This
guidance, Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting
Ecological Risk Assessments (U.S. EPA, 1997a), discusses scientific methods and stakeholder input.
Over the years, the Superfund program has expanded RAGS to include the following documents
relating to human health:
• RAGS Volume I, Part B: Development of Risk-based Preliminary Remediation Goals (Risk
Equations and Parameters) (U.S. EPA, 1991b)
• RAGS Volume I, Part C: Risk Evaluation of Remedial Alternatives (U.S. EPA, 1991c)
• RAGS Volume I, Part D: Standardized Planning, Reporting, and Review of Superfund Risk
Assessments (U.S. EPA, 2001a)
• RAGS Volume I, Part E: Supplemental Guidance for Dermal Risk Assessment (U.S. EPA,
200 Ibj
Additional ecological guidance documents include:
• Role of the Ecological Risk Assessment in the Baseline Risk Assessment. OSWER Directive
No. 9285.7-17 (U.S. EPA, 1994a)
• Issuance of Final Guidance: Ecological Risk Assessment and Risk Management Principles
for Superfund Sites. OSWER Directive 9285.7-28 P (U.S. EPA, 1999)
• The Role of Screening-Level Risk Assessments and Refining Contaminants of Concern in
Baseline Risk Assessments. 12th Intermittent Bulletin, ECO Update Series. (U.S. EPA, 2001d)
Page 1-6
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
This document (RAGS Volume 3: Part A) provides guidance for probabilistic approaches for both
human health and ecological risk assessment.
The Superfund program has also issued supplementary documents, including:
• Human Health Evaluation Manual, Supplemental Guidance: "Standard Default Exposure
Factors "(U.S. EPA, 1991a)
• Supplemental Guidance to RAGS: Calculating the Concentration Term (U.S. EPA, 1992d)
• Role of the Baseline Risk Assessment in Superfund Remedy Selection Decisions
(U.S. EPA, 1991d)
• Use of IRIS (Integrated Risk Information System) Values in Superfund Risk Assessment
(U.S. EPA, 1993)
• Final Soil Screening Guidance, May 17, 1996. Soil Screening User's Guide (U.S. EPA,
1996)
• Supplemental Guidance for Developing Soil Screening Levels for Superfund Sites
(U.S. EPA, 2001c).
EPA will continue to develop Superfund guidance and tools to improve the practice of risk
assessment. Superfund guidance documents are available from EPA's Superfund publications web site
(http://www.epa. gov/superfund/pubs .htm).
The role of risk assessment in Superfund, described above, can be summarized by a number of
principles that are followed and developed in RAGS Volume 3: Part A, including:
• The Superfund risk assessment process should rely on early problem formulation, planning,
and scoping for improved remedial investigations and feasibility studies, risk assessments,
and risk management decisions.
• The use of a tiered process in Superfund risk assessment and management is beneficial in that
it promotes an efficient allocation of resources and improved decision-making.
Early and continuing involvement of stakeholders throughout the Superfund risk assessment
process provides an opportunity to build stakeholder trust and meet stakeholder needs, which
can result in improved risk assessments and faster, more-informed risk management
decisions.
1.1.4 PROBABILISTIC RISK ASSESSMENT AND ITS ROLE IN SUPERFUND
RAGS Volume I (U.S. EPA, 1989aj and supporting guidance describe a point estimate approach
to risk assessments in the Superfund program. Point estimate risk assessments use single values (point
estimates) to represent variables in a risk equation. The output of the risk equation in a point estimate risk
assessment is, therefore, a point estimate of risk, which can be a central tendency exposure (CTE)
estimate of risk (e.g., the average expected risk) or reasonable maximum exposure (RME) estimate of risk
(e.g., the risk expected if the RME was to occur), depending on the input values used in the risk equation.
RAGS Volume 3: Part A describes a probabilistic approach to risk assessment. Probabilistic risk
assessment uses probability distributions for one or more variables in a risk equation in order to
quantitatively characterize variability and/or uncertainty. The output of a PRA is a probability
distribution of risks that reflects the combination of the input probability distributions. If the input
Page 1-7
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
distributions represent variability, then the output risk distribution can provide information on variability
in risk in the population of concern. If the input distributions reflect uncertainty, then the output risk
distribution can provide information about uncertainty in the risk estimate. Information from a PRA can
be used to make statements about the likelihood of exceeding a risk level of concern, given the estimated
variability in elements of the risk equation. Since the results of point estimate methods generally do not
lend themselves to this level of risk characterization (e.g., quantitative uncertainty assessment), PRA can
provide unique and important supplemental information that can be used in making Superfund risk
management decisions at Superfund sites.
Monte Carlo Analysis (MCA) is perhaps the most widely used probabilistic method in PRA.
MCA is a specific probabilistic method that uses computer simulation to combine multiple probability
distributions in a risk equation (see Section 1.2.2 for further discussion of Monte Carlo simulation).
Monte Carlo methods have been in used in modeling since 1946 when Stanislaw Ulam used MCA to
conduct uncertainty analysis at Los Alamos during the conceptual stage of the hydrogen bomb project.
The history of the use of MCA (from the 1940s to the present) can be found in Rugen and Callahan, 1996.
The application of probabilistic analysis to human health and ecological risk assessment is a
relatively recent development that was facilitated by development of statistical sampling techniques to
obtain a probabilistic approximation to the solution of a mathematical equation and/or model, and
increased speed and capacity of modern computers which can support the intensive computational
requirements of MCA. Desktop computers and commercial software are currently available which enable
risk assessors to make, in minutes, PRA calculations that only a few years ago would have required days.
The potential value of PRA to support risk-based decisions has become increasingly apparent
over the last several years. This has prompted the need for appropriate policy and guidance documents
that define the role of PRA in the Superfund program and that promote and facilitate the highest quality
and consistent application of PRA in the Program where appropriate. EPA previously issued guidance
that addresses the use of quantitative uncertainty analysis in risk assessment. RAGS Volume I (U.S. EPA,
1989a) and the Final Guidelines for Exposure Assessment Guidelines (U.S. EPA, 1992a) emphasize the
importance of assessing variability and uncertainty in risk estimates conducted in the Superfund program.
Guidance is also available for characterizing the 95% upper confidence limit (UCL) for the mean
exposure concentration (U.S. EPA, 1992d, 1997f). At the regional level, EPA Regions 3 and 8 issued
guidance on the appropriate use of probabilistic methods in risk assessment (U.S. EPA, 1994b, 1995e).
The importance of adequately characterizing variability and uncertainty is addressed in the 1995
memorandum on Risk Characterization Policy and Guidance (U.S. EPA, 1995b). In the spring of 1997,
EPA re leased the memorandum, Use of Probabilistic Techniques (including Monte Carlo Analysis) in
Risk Assessment (U.S. EPA, 1997g). According to the Policy Statement of the memorandum,
probabilistic analysis techniques, "given adequate supporting data and credible assumptions, can be
viable statistical tools for analyzing variability and uncertainty in risk assessments." As such, a PRA,
"will be evaluated and utilized in a manner that is consistent with other risk assessments submitted to the
Agency." Along with this Policy Statement, the Agency released a set of guiding principles for use and
review of probabilistic analyses (U.S. EPA, 1997g). Hence, both RAGS and Agency-wide guidance
emphasize the importance of review of the scientific and technical merit of a probabilistic analysis to
determine whether or not the assessment is of sufficient quality to support a remedial decision.
Currently, EPA's Office of Emergency and Remedial Response (OERR) is implementing PRA as
part of its Superfund reform activities. This guidance, RAGS Volume 3: Part A, provides risk assessors
with comprehensive guidance on when and how it may be appropriate to conduct PRAs using Monte
Page 1-8
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
Carlo analysis within the Superfund program. It describes basic concepts in PRA, an approach for
conducting MCA, and EPA's policy for implementing PRA in the Superfund program. The Agency also
intends to supplement this guidance with additional examples and case studies in PRA (see Section 1.6).
1.2 BASIC CONCEPTS OF PROBABILISTIC RISK ASSESSMENT
This section describes what a PRA is and compares and contrasts it to the more familiar point
estimate methods for human health risk assessment (U.S. EPA, 1989a) and ecological risk assessment
(U.S. EPA, 1997a). A risk assessment performed using probabilistic methods is very similar in concept
and approach to the point estimate method, with the main difference being the methods used to
incorporate variability and uncertainty into the risk estimate. A variety of modeling techniques can be
used to characterize variability and uncertainty in risk. This guidance focuses on MCA, perhaps the most
common probabilistic method that risk assessors will encounter. Basic concepts on how to use MCA to
propagate variability and uncertainty in exposure through a risk model are presented. Many of the
concepts presented in this guidance are applicable to other probabilistic approaches to risk assessment.
At some sites, probabilistic analysis can provide a more complete and transparent characterization
of the risks and uncertainties in risk estimates than would otherwise be possible with a point estimate
approach. However, a PRA is not necessary or desirable for every site. The tiered approach presented in
Chapter 2 highlights important scientific and management decisions for determining if PRA is appropriate
at a specific site. The decision to perform PRA is appropriate only after the risk assessor and the remedial
project manager (RPM) at the site determine whether a PRA will enhance decision making at the site. If a
PRA is conducted, the assumptions and inputs to the probabilistic model should be sufficiently
documented so that the results can be independently reproduced.
An essential concept in PRA that will be important throughout this section and the rest of the
guidance is the distinction between "variability" and "uncertainty". Variability refers to true
heterogeneity or diversity. For example, among a population that drinks water from the same source and
with the same contaminant concentration, the risks from consuming the water may vary. This may be due
to differences in exposure (i.e., different people drinking different amounts of water, having different
body weights, exposure frequencies, and exposure durations) as well as differences in response (e.g.,
genetic differences in resistance to a chemical dose). Differences among individuals in a population are
referred to as inter-individual variability, while differences for one individual over time are referred to as
intra-individual variability.
Uncertainty occurs because of a lack of knowledge. For example, we can be very certain that
different people drink different amounts of water, but we may be uncertain about how much variability
there is in water intakes among the population. Uncertainty can often be reduced by collecting more and
better data, while variability is an inherent property of the population being evaluated. Variability can be
better characterized with more data, but it cannot be reduced or eliminated.
Sometimes there can be confusion about whether data are representative of variability or
uncertainty, especially when the distinction depends on how the problem is framed. For example, one of
the exposure variables that may be considered in a risk assessment of workers exposed via inhalation to
an indoor air contaminant is the fraction of time spent indoors on site. Assume that time-activity
information is available from surveys of a representative population of workers. This data set may be
used to define a probability distribution (e.g., empirical, normal) that characterizes inter-individual
Page 1-9
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
variability in exposure times among workers. Sources of uncertainty would include the choice of the
probability distribution used to characterize variability, as well as the parameter estimates that are based
on a finite data set. Using the same data set, uncertainty in a parameter, such as the arithmetic mean
exposure time, may also be defined by a probability distribution. Efforts to clearly distinguish between
variability and uncertainty are important for both risk assessment and risk communication. Section 1.2.4
and Chapter 3, Section 3.4 present an overview of the different sources of uncertainty. Guidance on
selecting and fitting probability distributions is given in Appendices B and C, and advanced methods for
characterizing both variability and uncertainty are discussed in Appendix D.
1.2.1 WHATISPRA?
Probabilistic risk assessment is a general term for risk assessments that use probability models to
represent the likelihood of different risk levels in a population (i.e., variability) or to characterize
uncertainty in risk estimates.
A risk assessment performed using probabilistic methods would rely on the same fundamental
exposure and risk equations as do point estimate approaches. U.S. EPA guidance, including RAGS
Volume I: Part A (U.S. EPA, 1989a), the Standard Default Exposure Factors Guidance (U.S. EPA,
1991a), Supplemental Guidance for Developing Soil Screening Levels (U.S. EPA, 2001c), and Ecological
Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk
Assessments (U.S. EPA, 1997a) present methods for estimating risk using standardized exposure and risk
models. Examples of typical exposure and risk equations that would be used in risk calculations, in this
case, for a drinking water exposure scenario, are provided in Exhibit 1-3:
Page 1-10
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
EXHIBIT 1-3
CANCER AND NONCANCER RISK MODELS
Exposure Model: .-, Jn rr- rn
F L x 1R x hr x hD
CDI= -
Cancer Risk Model: B W x A T
Risk = GDI x CSF
Noncancer Risk Model:
GDI
GDI = chronic daily intake
of the chemical (mg/kg-day)
C = concentration of the chemical in an exposure medium (e.g., mg/L)
IR = ingestion rate (e.g., L/day for water, mg/day for soil, etc.)
EF = exposure frequency (days/year)
ED = exposure duration (years)
BW = body weight (kg)
HQ = hazard quotient
AT = averaging time (equal to ED x 365 days/year for noncarcinogens and 70 years
x 365 days/year for carcinogens)
CSF = cancer slope factor (linear low-dose cancer potency factor) for the chemical (mg/kg-day)"1
RfD = reference dose for the chemical for assessing noncancer health effects (mg/kg-day)
In the point estimate approach, a single numerical value (i.e., point estimate) is chosen for each
variable shown in Exhibit 1-3. For example, point estimates may include a drinking water ingestion rate
of 2 L/day and a body weight of 70 kg for an adult. Based on the choices that are made for each
individual variable, a single estimate of risk is calculated. In the probabilistic approach, inputs to the risk
equation are described as random variables (i.e., variables that can assume different values for different
receptors in the population) that can be defined mathematically by a probability distribution. For
continuous random variables, such as those in Figure 1-1 (body weight), the distribution may be
described by a PDF, whereas for discrete random variables (e.g., number offish meals per month), the
distribution may be described by a probability mass function (PMF). The key feature of PDFs and PMFs
is that they describe the range of values that a variable may assume, and indicate the relative likelihood
(i.e., probability) of each value occurring within that range for the exposed population. For example, the
distribution of tap water ingestion (mL/day) among the general U.S. population might be characterized by
a lognormal distribution with a log-mean of 6.86 and a log-standard deviation of 0.575 (Table 3-11 of
U.S. EPA 1997b). One might use a PDF to show how approximately half the population drinks more
than 1 L/day of tap water, but only 10% of the population drinks more than 2 L/day. After determining
appropriate PDF types and parameter values for selected variables, the set of PDFs is combined with the
toxicity value in the exposure and risk equations given in Exhibit 1-3 to estimate a distribution of risks.
Guidance on selecting and fitting distributions for variables in risk equations is provided in Appendix B.
In human health risk assessments, probability distributions for risk should reflect variability or
uncertainty in exposure. In ecological risk assessments, risk distributions may reflect variability or
uncertainty in exposure and/or toxicity (see Sections 1.4 and 1.4.1, Item 3).
Page 1-11
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
A continuous probability distribution can
be displayed in a graph in the form of either a
PDF or corresponding CDF; however, for clarity,
it is recommended that both representations be
presented in adjacent (rather than overlaid) plots.
Figure 1-1 illustrates a PDF and CDF for a
normal probability distribution for adult body
weight. Both displays represent the same
distribution, but are useful for conveying different
information. Note that it is helpful to include a
text box with summary statistics relevant to the
distribution (e.g., mean, standard deviation). The
types of information that PDFs and CDFs are
most useful for displaying are presented in
Exhibit 1-4.
EXHIBIT 1-4
USE A PDF AND CDF To DISPLAY:
PDF
• The relative probability of values
• The most likely values (e.g., modes)
• The shape of the distribution (e.g., skewness,
kurtosis, multimodality)
• Small changes in probability density
CDF
• Percentiles, including the median
• High-end risk range (e.g., 90th to 99th percentiles)
• Confidence intervals for selected percentiles
• Stochastic dominance (i.e., for any percentile,
the value for one variable exceeds that of any
other variable)
Source: U.S. EPA, 1997g
0.030
0.020 -
Q
.£•
•5 0.010 -
o
0_
0.000
PDF
1 oo 200
Body Weight (kg)
300
1.00
t 0.75
0.50 -
O 0.25 -
0.00
CDF
100 200
Body Weight (kg)
300
Figure 1-1. Example of a normal distribution that characterizes variability in adult body weight (males
and females combined). Arithmetic mean=71.7 kg, standard deviation=15.9 kg (Finley and Paustenbach,
1994). Body weight may be considered a continuous random variable. The left panel shows a
bell-shaped curve and represents the PDF, while the right panel shows an S-shaped curve and represents
the CDF. Both displays represent the same distribution (including summary statistics), but are useful for
conveying different information.
Page 1-12
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
The CDF for risk can be especially informative for illustrating the percentile corresponding to a particular
risk level of concern (e.g., 95th percentile=lE-06). A text box may also be included on the graph to
highlight important summary statistics, such as the parameters of the input distribution, or selected
percentiles of the output distribution for risk. For example, a clear description of the parameters for the
probability distribution should be given, as well as an indication of whether the distribution represents
variability or uncertainty.
1.2.2 WHAT is A MONTE CARLO SIMULATION?
Perhaps the most common numerical technique for PRA is Monte Carlo simulation. Monte Carlo
simulation has been widely used to explore problems in many disciplines of science as well as
engineering, finance, and insurance (Rugen and Callahan, 1996). The process for a Monte Carlo
simulation is illustrated in Figure 1-2. In its general form, the risk equation can be expressed as a
function of multiple exposure variables (V;) and atoxicity term: Risk=f(V1, V2, ...Vn) x Toxicity.
Solutions for equations with PDFs are typically too complex for even an expert mathematician to
calculate the risk distribution analytically. However, numerical techniques applied with the aid of
computers can provide very close approximations of the solution. This is illustrated here for the
simplified case in which the assessment variables are statistically independent, that is, the value of one
variable has no relationship to the value of any other variable. In this case, the computer selects a value
for each variable (V;) at random from a specified PDF and calculates the corresponding risk. This process
is repeated many times (e.g., 10,000), each time saving the set of input values and corresponding estimate
of risk. For example, the first risk estimate might represent a hypothetical individual who drinks 2 L/day
of water and weighs 65 kg, the second estimate might represent someone who drinks 1 L/day and weighs
72 kg, and so forth. Each calculation is referred to as an iteration, and a set of iterations is called a
simulation.
US' A convenient aid to understanding the Monte Carlo approach for quantifying
variability is to visualize each iteration as representing a single individual
and the collection of all iterations as representing a population.
Each iteration of a Monte Carlo simulation should represent a plausible combination of input values (i.e.,
exposure and toxicity variables), which may require using bounded or truncated probability distributions
(see Appendix B). However, risk estimates are not intended to correspond to any one person. The
"individuals" represented by Monte Carlo iterations are virtual and the risk distributions derived from a
PRA allow for inferences to be made about the likelihood or probability of risks occurring within a
specified range for an exposed human or ecological population. A simulation yields a set of risk
estimates that can be summarized with selected statistics (e.g., arithmetic mean, percentiles) and displayed
graphically using the PDF and CDF for the estimated risk distribution. Often the input distributions are
assumed to be independent, as shown in Figure 1-2. More complex Monte Carlo simulations can be
developed that quantify a dependence between one or more input distributions by using conditional
distributions or correlation coefficients (see Appendix B, Section B.5.5 for a discussion of correlated
input distributions).
Page 1-13
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
Probability Distribution for Random Variables
N
n
Risk =/(V1? V2, «* Vn) x Toxicity
i
O.OE+00 l.OE-06 2.0E-06
Risk
3.0E-06
Figure 1-2. Conceptual model of Monte Carlo analysis. Random variables (Vb V2, ...Vn) refer to exposure
variables (e.g., body weight, exposure frequency, ingestion rage) that are characterized by probability
distributions. A unique risk estimate is calculated for each set of random values. Repeatedly sampling (V;)
results in a frequency distribution of risk, which can be described by a PDF. In human health risk assessments,
the toxicity term should be expressed as a point estimate. In ecological risk assessment (see Sections 1.4
and 1.4.1) the toxicity term may be expressed as a point estimate or as a probability distribution.
Page 1-14
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
The rapid evolution in computing power has greatly reduced concerns among regulators
regarding the number iterations needed in MCA.
US' While this guidance does not prescribe specific criteria or set an arbitrary
"minimum " number of iterations needed for PRA, a general rule of thumb is
that a sufficient number of iterations should be run to obtain numerical
stability in percentiles of the output (e.g., risk distribution) that are important
for decision making.
Numerical stability refers to the stochastic variability, or "wobble" associated with random sampling, and
can be evaluated by running multiple simulations with the same set of input assumptions and calculating
the average percent change in a specified percentile of the output (e.g., Maddalena et al., 2001). For
example, it may be determined that 5,000 iterations are sufficient to achieve numerical stability in the
50th percentile, but insufficient for the 95th percentile risk estimate when a criteria of ± 1% is applied for
multiple simulations. As discussed in Section 1.4, one of the eight conditions specified by EPA for the
acceptance of PRA is that the numerical stability of the output be presented and discussed, since it will
vary depending on what percentile of the risk distribution is evaluated. While some commercial software
now have a feature to automatically stop simulations after a specified criterion for numerical stability is
achieved (Burmaster and Udell, 1990), care should be taken to understand how this criterion is
implemented across the entire range of the output distribution.
1.2.3 WHY is VARIABILITY IMPORTANT IN RISK ASSESSMENT? How is IT ADDRESSED BY THE
POINT ESTIMATE AND PROBABILISTIC APPROACHES?
As noted previously, variability refers to true heterogeneity or diversity that occurs within a
population or sample. Factors that lead to variability in exposure and risk include variability in
contaminant concentrations in a medium (air, water, soil, etc.), differences in ingestion rates or exposure
frequencies, or in the case of ecological assessments, inter- and intra-species variability in dose-response
relationships. Risk Assessment Guidance for Superfund Volume I (Section 6.1.2 of U.S. EPA, 1989a) and
the NCP Preamble (U.S. EPA, 1990) state that human health risk management decisions at Superfund
sites will generally be based on an individual that has RME. Likewise, RME estimates of risk are the
most appropriate basis for decision making using an ecological risk assessment. Use of the RME and
CTE risk descriptors in ecological risk assessment are discussed in Chapter 4. The intent of the RME is
to estimate a conservative exposure case (i.e., well above the average case) that is still within the range of
possible exposures based on both quantitative information and professional judgment (Sections 6.1.2
and 6.4.1 of U.S. EPA, 1989a). In addition, the Agency released guidance in 1992 (U.S. EPA, 1992c)
recommending the inclusion of a "central tendency" exposure estimate to an individual, as well as a
high-end exposure estimate, in the risk assessment. Generally, the CTE is considered to be a measure of
the mean or median exposure. The difference between the CTE and the RME gives an initial impression
of the degree of variability in exposure or risk between individuals in an exposed population.
Depending on assessment needs at a site, a range of point estimates of risk can be developed to
represent variability in exposures. To support the evaluation of RME risk estimates using the point
estimate approach described in Section 1.3, the Superfund program developed guidance with
recommended default values for exposure variables as inputs to the risk equations (U.S. EPA, 1992a,
1996, 1997a, 2001d). These standardized values are a combination of average (e.g., body weight, skin
surface area) and high-end exposure assumptions (e.g., drinking water intake, exposure duration). A CTE
risk estimate is based on central estimates (e.g., mean, 50th percentile) for each of the exposure variables.
Page 1-15
-------
RAGS Volume 3 Part A
Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
Available site-specific data on plausible mean and upper range values for exposure variables should be
used to support CTE and RME risk estimates. The point estimate approach to risk assessment does not
determine where the CTE or RME risk estimates lie within the risk distribution. For example, the RME
risk estimated with the point estimate approach could be the 90th percentile, the 99.9th percentile, or some
other percentile of the risk distribution. Without knowing what percentile is represented by the RME risk
estimate, the risk manager might be unsure about the likelihood of the RME risk occurring or being
exceeded in the receptor population and about what level of remedial action is justified or necessary to
achieve the protective objectives of CERCLA.
In a PRA, distributions used as inputs to the risk equations can characterize the inter-individual
variability inherent in each of the exposure assumptions. By characterizing variability with one or more
input distributions, the output from the Monte Carlo simulation is a distribution of risks that could occur
in that population (Figure 1-3). The central tendency of the risk distribution (e.g., arithmetic mean,
geometric mean, 50th percentile) may be characterized as the CTE risk estimate. Similarly, the high-end
of the risk distribution (e.g., 90th to 99.9th percentiles) is representative of exposures to the RME
individual. In addition to providing a better understanding of where the CTE and RME risks occur in the
distribution, a PRA can also provide an estimate of the probability of occurrence associated with a
particular risk level of concern (e.g., cancer risk of 1E-05). A PRA that quantifies variability can be used
to address the question, "What is the likelihood (i.e., probability) that risks to an exposed individual will
exceed 1E-05?" Based on the best available information regarding exposure and toxicity, a risk assessor
might conclude, "The estimated distribution for variability in risk across the target population indicates
that 10% of the individuals exposed under these circumstances have a risk exceeding 1E-05." This type
of evaluation can be achieved using a technique known as one-dimensional Monte Carlo Analysis
(1-D MCA). Guidelines for interpreting the high-end of the risk distribution in terms of the RME risk
estimate are discussed further in Section 1.4.1 and Chapter 7.
0.06
V)
c
level of concern), remedial action may be warranted. For Case B
(RME < level of concern), remedial action may be unnecessary.
Page 1-16
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
The agreement (or lack of agreement) between the results of the point estimate calculations and
the PRA calculations is expected to vary as a function of the form of the exposure or risk model and the
attributes of the input variables. In general, if the terms in the denominator of the exposure or risk
equation have low variability and do not approach zero, then the CTE point estimate is likely to agree
quite well with the arithmetic mean from the PRA simulation, and the RME point estimate is likely to
correspond to the high-end of the risk distribution (see discussion of RME range in Section 1.2.5).
However, if the exposure or risk model has terms in the denominator that are a significant source of
variability, or if the terms approach zero, then the agreement between the point estimate values and the
PRA values may be more substantial. In addition, since the RME point estimate of risk reflects a
combination of central tendency and high-end input values, it is difficult to anticipate what percentile of a
distribution of variability it represents.
f If results of PRA calculations differ substantially from point estimate
calculations, a risk manager may benefit from understanding the reasons for
the differences and the relative strengths of the different approaches.
Since point estimate and PRA approaches may yield different estimates of CTE and RME risks, the two
approaches also may support different risk management decisions. This does not imply that either
approach is invalid. Likewise, a correspondence between the point estimate and PRA results does not
imply a greater accuracy or certainty in the modeling assumptions and inputs. Simply stated, PRA, based
on the same risk equations and data as the point estimate approach, provides a different means of
characterizing variability and uncertainty. Potential sources of variability and uncertainty in risk
estimates should be identified, discussed, and to the extent practicable, quantified. Advantages and
disadvantages of PRA and point estimate risk assessment are discussed in Section 1.2.4 and 1.3.
1.2.4 WHY is UNCERTAINTY IMPORTANT IN RISK ASSESSMENT? How is UNCERTAINTY ADDRESSED
BY THE POINT ESTIMATE AND PROBABILISTIC APPROACHES?
Uncertainty derives from a lack of knowledge. Various taxonomies of uncertainty relevant to risk
assessment have been presented (Finkel, 1990; Morgan and Henrion, 1990; Cullen and Frey, 1999). U.S.
EPA guidance, including the Final Guidelines Exposure Assessment Guidelines (U.S. EPA, 1992a),
Exposure Factors Handbook (U.S. EPA, 1997b,c,d), and Guiding Principles for Monte Carlo Analysis
(U.S. EPA, 1997g) describe a variety of different types of uncertainty in risk assessment as well as
modeling strategies for quantifying uncertainties. Potential sources of uncertainty in risk assessment can
be divided into one of three broad categories:
(1) Parameter uncertainty - uncertainty in an estimate of an input variable in a model. In PRA,
this may refer specifically to a statistical concept of uncertainty in estimates of population
parameters (e.g., arithmetic mean, standard deviation) from random samples, due to the
quality, quantity, and representativeness of available data as well as the statistical estimation
method.
(2) Model uncertainty - uncertainty about a model structure (e.g., exposure equation) or intended
use, including the relevance of simplifying assumptions to the endpoint of the risk
assessment, the choice of probability distribution to characterize variability, and interpolation
or extrapolation beyond the scale used to calibrate a model from empirical data.
Page 1-17
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
(3) Scenario uncertainty - uncertainty regarding missing or incomplete information to fully
define exposure. This may include descriptive errors regarding the magnitude and extent of
chemical exposure or toxicity, temporal and spatial aggregation errors, incomplete analysis
(i.e., missing exposure pathways), and potential mis-specification of the exposed population
or exposure unit.
Sources of uncertainty described by these categories are important because they can influence
risk management decisions in both point estimate and probabilistic risk assessment. As additional sources
of uncertainty are quantified and included in the risk assessment, uncertainty in risk estimates may appear
to increase, suggesting there may be little confidence in a risk management decision. This situation may
appear to be counterintuitive for those managers who expect confidence to increase as uncertainty is
quantified. However, as discussed below and in Chapter 6 (see Section 6.4.2), uncovering and
quantifying these sources of uncertainty may help to provide perspective, and make the decisions using
the tiered process more transparent. In PRA, there are a variety of methods that can be used to effectively
quantify uncertainty as well as communicate confidence in risk estimates (see Chapter 3, Section 3.4;
Chapter 6, Section 6.4, and Section 6.5).
Parameter uncertainty may be the most readily recognized source of uncertainty that is quantified
in site-specific risk assessments at hazardous waste sites. Parameter uncertainty can occur in each step of
the risk assessment process from data collection and evaluation, to the assessment of exposure and
toxicity. Sources of parameter uncertainty may include systematic errors or bias in the data collection
process, imprecision in the analytical measurements, inferences made from a limited database when that
database may or may not be representative of the variable under study, and extrapolation or the use of
surrogate measures to represent the parameter of interest.
In the point estimate approach, parameter uncertainty is addressed in a qualitative manner for
most variables. For example, the uncertainty section of a point estimate risk assessment document might
note that a soil sampling plan yielded a small sample size that may not be representative of overall
contaminant concentrations and, as a result, the risk estimate may over- or under-estimate actual risk.
Uncertainty in the concentration term is addressed quantitatively to a limited extent in a point estimate
approach by using the 95% UCL for the arithmetic mean concentration in both CTE and RME risk
estimates; this accounts for uncertainty associated with environmental sampling and site characterization
(U.S. EPA, 1992d, 1997f). The 95% UCL is combined in the same risk calculation with various central
tendency and high-end point estimates for other exposure factors.
Some examples of the models that EPA uses in the risk assessment process are the equations used
to calculate exposure and risk, the linearized multistage model used to estimate cancer dose-response
relationships, and media-specific models to estimate contaminant concentrations. All models are
simplified, idealized representations of complicated physical or biological processes. Models can be very
useful from a regulatory standpoint, as it is generally not possible to adequately monitor long term
exposure for populations at contaminated sites. However, models that are too simplified may not
adequately represent all aspects of the phenomena they were intended to approximate or may not capture
important relationships among input variables. Other sources of model uncertainty can occur when
important variables are excluded, interactions between inputs are ignored, or surrogate variables that are
different from the variable under study are used.
Page 1-18
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
0.9 --
0.8 --
0.7 --
0.6 --
Veritical Confidence
Limit
Horizontal Confidence
Limit
C 0.5 -
Lower confidence
bound on risk
Best estimate of risk
In most probabilistic assessments, the first step of analysis is usually an analysis of variability in
exposure or risk. However, PRA methods may also be used to characterize uncertainty around the best
estimate of the exposure or risk distribution. This is done using "2-dimensional" MCA (2-D MCA) (see
Appendix D). One convention that has been used to distinguish between probability distribution
functions for variability
1.0
and uncertainty is to use
subscripts "v" and "u" to
indicate PDFs that
characterize variability
(PDFv) or uncertainty
(PDFu). Figure 1-4
shows an example of the
results of this type of 2-D
MCA. This analysis can
provide a quantitative
measure of the confidence
in the fraction of the
population with a risk
exceeding a particular
level; which is sometimes
referred to as a vertical
confidence interval
(Figure 1-4). For
example, a conclusion
based on this type of
output might be, "While
the best estimate for the
£ 0.4 --
0.3 --
0.2 --
0.1 --
Upper confidence
bound on risk
1.E-09
1.E-08
1.E-07 1.E-06
Risk
1.E-05
1.E-04
1 .E-03
Figure 1-4. Illustration of "Vertical" and "Horizontal" Confidence Intervals (or
limits) on a risk estimate. This type of output can be produced from a 2-D MCA in
which probability distributions of uncertainty are introduced into the risk equation.
See Chapter 3 and Appendix D for further discussion of 2-D MCA in quantitative
variability distribution for uncertainty analysis.
risk across the target
population indicates that 10% of the individuals exposed under these circumstances have a risk exceeding
1E-06, the uncertainty is such that we can only be reasonably certain (e.g., 95% sure) that no more than
20% of the exposed population has a risk that exceeds 1E-06." Additionally, the output from a 2-D MCA
can provide a quantitative measure of the confidence in the risk estimate for a particular fraction of the
population; which is sometimes referred to as a horizontal confidence interval. This type of output might
support the following type of conclusion, "While the best estimate for the variability distribution for risk
across the target population indicates that 10% of the individuals exposed under these circumstances have
a risk exceeding 1E-06, the uncertainty is such that we can only be reasonably certain (e.g., 95% sure)
that the risk for this group of individuals does not exceed 2E-06." The term "confidence interval" is used
loosely in this context to convey information about uncertainty; however, it is not the same as a statistical
confidence interval that one might obtain by estimating a population parameter from a sample. The
vertical and horizontal bars shown in Figure 1-4 represent a range of possible estimates for the percentile
given one or more sources of uncertainty that were included in the simulation. If the target audience for
this graphic has a greater understanding of statistics, it may be less confusing if alternative phrases are
used to describe the results, such as "credible interval" or "probability band".
In general, one should avoid developing input distributions to a PRA model that yield a single
risk distribution that intermingles, or represents both variability and uncertainty. By separately
characterizing variability and uncertainty, the output from a PRA will be easier to understand and
communicate. A number of tools can aid in evaluating the uncertainty in estimated distributions for
variability. Both simple and very complex approaches have been applied to this problem. Two basic
Page 1-19
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
methods for quantifying variability and parameter uncertainty simultaneously are described in
Exhibit 1-5. PRAs that use these approaches can provide quantitative estimates of uncertainty in
percentiles of the risk distribution based on confidence intervals or credible intervals for one or more
parameter estimates. Techniques for characterizing both variability and uncertainty in PRA are discussed
in more detail in Chapters 3, 4, 5, and 7, and Appendices A, C, and D.
A common apprehension
concerning the utility of PRA is that it may
require more information and data than are
available to generate credible PDFs. Risk
assessors may feel that they can't specify a
PDF because they don't have enough
information to choose a distribution type,
estimate parameters, or evaluate the
representativeness to the site population of
concern. However, if sufficient
information exists to support a meaningful
point estimate evaluation (i.e., if some sort
of central tendency and upper bound values
are available for each input variable), then
it is usually possible to perform a screening
level, or preliminary 1-D MCA that may
provide additional useful information
regarding variability. Likewise, an initial
two-dimensional analysis may be
performed that does not require collection
of any new data, but simply characterizes
uncertainty in the existing data. The
results of such a 2-D MCA can help to
identify the main sources of uncertainty in
the risk results, and can support decisions
to collect more data and/or proceed with
additional tiers of analysis in order to
improve the assessment. As with a
preliminary 1-D MCA, the decision to
conduct a more advanced probabilistic
analysis does not always result in added
data requirements.
EXHIBIT 1-5
QUANTIFYING VARIABILITY AND UNCERTAINTY
1. Single source of uncertainty
Run multiple one-dimensional Monte Carlo
simulations (1-D MCA) in which each simulation
uses a different point estimate for a parameter
selected from an uncertainty distribution, combined
with PDFv's for one or more variables. For example,
separate simulations can be run in which the mean of
the exposure concentration variability distribution is
represented by either the 95% lower or upper
confidence limit on the mean. A comparison of the
output of these simulations would provide a partial
characterization of the quantitative impact of
uncertainty in the mean exposure concentration on
the risk estimate (provided that certain conditions
hold; i.e., risk increases with increasing exposure
concentration) (see Chapter 3, Section 3.3.1).
2. Multiple sources of uncertainty
Run a single two-dimensional Monte Carlo
simulation (2-D MCA), in which separate probability
distributions are specified for variability and
parameter uncertainty and values from these
distributions are randomly selected and used in each
iteration of the Monte Carlo simulation (see
Appendix D).
Use of probabilistic methods (e.g., MCA) to propagate variability and uncertainty through risk
models offers five key advantages over point estimate approaches in addressing uncertainty in risk
estimates:
(1) Probabilistic methods may often provide a more complete and informative characterization of
variability in exposure or risk than is usually achievable using point estimate techniques.
(2) Probabilistic methods can provide a more quantitative expression of the confidence in risk
estimates than the point estimate approach.
(3) Sensitivity analysis methods using PRA may help risk assessors to better identify influential
exposure factors.
Page 1-20
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
(4) Probabilistic methods can account for dependencies between input variables (e.g., body
weight and skin surface area).
(5) Probabilistic methods provide quantitative estimates of the expected value of additional
information that might be obtained from data collection efforts (Morgan and Henrion, 1990).
The importance of quantifying uncertainty in an expected value of information (EVOI)
framework is discussed in Appendix D.
Since both point estimate and probabilistic approaches in risk assessment are applied to the same
conceptual models (i.e., the same exposure and risk models), uncertainties in the conceptual model are
generally addressed in the same manner. If other models are available to explain or characterize a given
phenomenon, the risk estimates associated with each of those conceptual models could be compared to
determine the sensitivity of the risk to the uncertainty in the choice of a model (see Chapter 2 and
Appendix A). For example, when deciding on a contaminant concentration term for tetrachloroethylene
in groundwater for a residential exposure assessment 10 years in the future, it would be appropriate to
compare and contrast several fate and transport models and their results before deciding on a
concentration term.
1.2.5 REASONABLE MAXIMUM EXPOSURE AT THE HIGH-END
Risk management decisions at Superfund sites should be based on an estimate of the risk to a
reasonably maximum exposed receptor, considering both current and future land-use conditions. The
RME is defined as the highest exposure that is reasonably expected to occur at a site. In general, risks
corresponding to the 90th to 99.9th percentiles of the risk distribution estimated from a PRA are considered
plausible high-end risks, and the RME risk should be selected within this range (see Section 1.2.4,
Section 1.4.1, and Chapter 7 for further discussion). In comparison with point estimate risk assessments,
PRA can provide the entire range of estimated risks as well as the likelihood of values within the range
(i.e., the frequency distribution)
As noted in Chapter 7, estimates of risk become more uncertain at very high percentiles (e.g., the
99.9th), so results of PRA calculations at these extreme values should be used with caution. Risk
frequency distributions toward the 99.9th percentile may be numerically unstable due to the uncertainties
embedded in the input exposure assumptions. This guidance recommends that a risk manager select the
RME in consultation with a risk assessor. One item for discussion should be the numerical stability of the
high-end RME risk value (i.e., a stable value on the frequency distribution within the high-end range that
could be reproduced in successive Monte Carlo simulations.)
1.3 ADVANTAGES AND DISADVANTAGES OF POINT ESTIMATE AND PROBABILISTIC
APPROACHES
As discussed in Chapter 2, a PRA should not be conducted until adequate point estimate
calculations have been completed. Once this has been done, the potential benefits of proceeding to a PRA
evaluation should be based on an understanding of the potential advantages and limitations in each
approach. Potential advantages and disadvantages of point estimate calculations are summarized in
Exhibit 1-6 and potential advantages and disadvantages of PRA are listed in Exhibit 1-7.
Page 1-21
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
In general, compared to a point estimate risk assessment, a PRA based on the same state of
knowledge may offer a more complete characterization of variability in risk, can provide a quantitative
evaluation of uncertainty, and may provide a number of advantages in assessing if and how to proceed to
higher levels of analysis. However, there are also some real and perceived disadvantages regarding
additional effort on the part of both the risk assessor and the risk manager, and the potential to cause
confusion if the effort is not clearly presented.
In general, the key question to consider in deciding whether a PRA should be performed is
whether or not the PRA analysis is likely to provide information that will help in the risk management
decision making. For some sites, the additional information provided by a PRA will not affect the
decision that would have been made with a point estimate approach alone, and a PRA will not be useful.
However, when the decision whether or not to take action is not completely clear, PRA may be a valuable
tool. The tiered process for PRA (Chapter 2) introduces the concept of scientific management decision
points (SMDPs) to guide the complexity of analysis that may be needed for decision making. An SMDP
marks a point in the process in which the potential that another analysis may influence the risk
management decision is evaluated based on the problem formulation, the information available to define
input variables, the results of previous analyses, and the feasibility of a subsequent analysis.
«s" A point estimate approach is conducted for every risk assessment; a
probabilistic analysis may not always be needed.
EXHIBIT 1-6
ADVANTAGES AND DISADVANTAGES OF POINT ESTIMATE APPROACH
Advantages
Calculations are simple and do not require any advanced software.
EPA has established default inputs and methods to help standardize point estimate
calculations between sites.
Useful as a screening method—may allow risk management decisions with no
additional work.
Central tendency and RME estimates of risk provide a semi-quantitative measure of
variability.
Method is easily described and communicated.
Requires less time to complete; not as resource intensive.
Disadvantages
Computational simplifications may result in deviations from target values.
Results are often viewed as "the answer"; importance of uncertainty is sometimes
lost.
Information from sensitivity analysis is generally limited to dominant exposure
pathways and chemicals of concern; may not highlight the key exposure variables and
uncertain parameters.
Does not provide a measure of the probability that risk exceeds a regulatory level of
concern, or the level of confidence in a risk estimate.
• Provides fewer incentives for collecting better or more complete information.
May not utilize all available data for characterizing variability and uncertainty in risk
estimates.
Page 1-22
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
EXHIBIT 1-7
ADVANTAGES AND DISADVANTAGES OF PROBABILISTIC RISK ASSESSMENT
Advantages
Can make more complete use of available data when defining inputs to the risk equation.
Can provide a more comprehensive characterization of variability in risk estimates.
Can provide a more comprehensive characterization of uncertainty in inputs, which may
support statements regarding confidence in risk estimates. Communication of uncertainty in
the risk assessment can help to build trust among stakeholders.
Sensitivity analysis can identify the exposure variables, probability models, and model
parameters that influence the estimates of risk.
Puts the risk assessment in a Value-of-Information framework (see Appendix D). Can identify
data gaps for further evaluation/data collection and can use wider variety of site-specific
information.
Allows available site-specific information to inform the choice of high-end percentile from the
risk distribution that corresponds with RME risk.
Disadvantages
Concepts and approaches may be unfamiliar; there is often apprehension regarding added costs
and potential for inadvertent error and/or intentional misrepresentation.
Places more burden on risk assessors to ensure the PPvA is done correctly and on managers to
understand and make decisions within a range of alternatives.
May require more time and resources to select and fit probability distributions, and may require
greater effort to communicate methodology and results.
May convey false sense of accuracy when data are sparse.
Complexities of the PPvA approaches may obscure important assumptions or errors in basic
exposure or risk models.
If communication of the more complex PRA is unsuccessful, then it may generate mistrust of
the assessment and risk management decisions.
Page 1-23
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
1.4 CONDUCTING AN ACCEPTABLE PRA
In 1997, EPA issued a memorandum which contained its policy statement on PRA (U.S. EPA,
1997g). The 1997 EPA Policy Statement is as follows:
It is the policy of the U.S. Environmental Protection Agency that such probabilistic analysis
techniques as Monte Carlo analysis, given adequate supporting data and credible
assumptions, can be viable statistical tools for analyzing variability and uncertainty in risk
assessments. As such, and provided that the conditions described below are met, risk
assessments using Monte Carlo analysis or other probabilistic techniques will be evaluated
and utilized in a manner that is consistent with other risk assessments submitted to the
Agency for review or consideration. It is not the intent of this policy to recommend that
probabilistic analysis be conducted for all risk assessments supporting risk management
decisions. Such analysis should be a part of a tiered approach to risk assessment that
progresses from simpler (e.g., deterministic) to more complex (e.g., probabilistic) analyses
as the risk management situation requires. Use of Monte Carlo or other such techniques in
risk assessments shall not be cause, per se, for rejection of the risk assessment by the
Agency. For human health risk assessments, the application of Monte Carlo and other
probabilistic techniques has been limited to exposure assessments in the majority of cases.
The current policy, Conditions for Acceptance and associated guiding principles are not
intended to apply to dose response evaluations for human health risk assessment until this
application of probabilistic analysis has been studied further. In the case of ecological risk
assessment, however, this policy applies to all aspects including stressor and dose-response
assessment.
In support of this policy statement, EPA has outlined eight conditions for acceptance (in italics
below), and good scientific practice of PRA. A PRA that is submitted to the Agency for review and
evaluation should generally comply with each condition in order to ensure that adequate supporting data
and credible assumptions are used in the assessment. These conditions are as follows:
(1) The purpose and scope of the assessment should be clearly articulated in a "problem
formulation " section that includes a full discussion of any highly exposed or highly
susceptible subpopulations evaluated (e.g., children, the elderly). The questions the
assessment attempts to answer are to be discussed and the assessment endpoints are to be
well defined.
(2) The methods used for the analysis (including all models used, all data upon which the
assessment is based, and all assumptions that have a significant impact upon the results) are
to be documented and easily located in the report. This documentation is to include a
discussion of the degree to which the data used are representative of the population under
study. Also, this documentation is to include the names of the models and software used to
generate the analysis. Sufficient information is to be provided to allow the results of the
analysis to be independently reproduced.
Possible sources of bias inherent in the input distributions should be discussed along with the
expected impacts on the resulting risk estimates. For example, if a site-specific study offish consumption
indicated consumption rates are five to ten times higher than other studies from similar populations, this
possible bias or inaccuracy should be discussed in the document. Computer programs should generally
Page 1-24
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
be described in sufficient detail to allow the reviewer to understand all aspects of the analysis. Computer
code/spreadsheets should provide adequate documentation and annotation.
(3) The results of sensitivity analyses are to be presented and discussed in the report.
Probabilistic techniques should be applied to the compounds, pathways, and factors of
importance to the assessment, as determined by sensitivity analyses or other basic
requirements of the assessment.
Sensitivity analysis is a valuable tool in any tier of a PRA.
(4) The presence or absence of moderate to strong correlations or dependencies between the
input variables is to be discussed and accounted for in the analysis, along with the effects
these have on the output distribution.
(5) Information for each input and output distribution is to be provided in the report. This
includes tabular and graphical representations of the distributions (e.g., probability density
function and cumulative distribution function plots) that indicate the location of any point
estimates of interest (e.g., mean, median, 95th percentile). The selection of distributions is to
be explained and justified. For both the input and output distributions, variability and
uncertainty are to be differentiated where possible.
(6) The numerical stability of the central tendency and the higher end (i.e., tail) of the output
distributions are to be presented and discussed.
As discussed in Section 1.2.5, numerical stability refers to the observed numerical changes in
parameters of the output distribution (e.g., median, 95th percentile) from a Monte Carlo simulation as the
number of iterations increases. Because most risk equations are linear and multiplicative, distributions of
risk will generally be right-skewed, and approximate a lognormal distribution. Values in the tails of the
distribution typically are less stable than the central tendency, and the rate of convergence for the tails
will depend on the form of the risk model, the skewness of the probability distributions selected for input
variables and the numerical methods used to simulate probability distributions. Provided that appropriate
numerical methods are employed, numerical stability is generally not a concern for most 1-D MCA
models, which can be run with a sufficient number iterations in minutes with modern high speed
computers; however, it can be an important consideration for more complex simulations, such as with
2-D MCA models.
(7) Calculations of exposures and risks using deterministic (e.g., point estimate) methods are to
be reported if possible. Providing these values will allow comparisons between the
probabilistic analysis and past or screening level risk assessments. Further, deterministic
estimates may be used to answer scenario specific questions and to facilitate risk
communication. When comparisons are made, it is important to explain the similarities and
differences in the underlying data, assumptions, and models.
If results of PRA calculations differ substantially from point estimate calculations, a risk manager
may benefit from understanding the reasons for the differences and the relative strengths of the different
approaches. Sometimes, a closer look at uncertainties in the underlying data, assumptions, and models
will lead a risk assessor to revisit parts of the assessment in order to provide a more consistent basis for
comparison.
Page 1-25
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
(8) Since fixed exposure assumptions (e.g., exposure duration, body weight) are sometimes
embedded in the toxicity metrics (e.g., Reference Doses, Reference Concentrations, Cancer
risk factors), the exposure estimates from the probabilistic output distribution are to be
aligned with the toxicity metric.
1.4.1 KEY POLICIES FOR APPLYING PRA AT SUPERFUND SITES
EPA's recommended process for conducting an acceptable PRA generally follows the policy and
guiding principles presented above. In addition, this section highlights four key policies for conducting
acceptable PRAs at hazardous waste sites.
(1) Follow the Tiered Approach to PRA
In accordance with the 7997 EPA Policy Statement (U.S. EPA, 1997g), this guidance
recommends using a tiered approach when considering PRA to help with risk management decisions. A
tiered approach begins with a relatively simple analysis and progresses stepwise to more complex
analyses. The level of complexity should match the site-specific risk assessment objectives and the risk
management goals. The tiered approach, with helpful suggestions on risk communication, is presented in
Chapter 2. A brief introduction is given below.
The premise for recommending a tiered approach is that there is a balance between the benefits
of conducting a more complex analysis, and the cost in terms of additional time, resources, and challenges
for risk communication. PRA may require additional resources compared with the point estimate
approach, and may not be used routinely for screening level assessment. At more complex hazardous
waste sites, PRA may not be warranted if the investment of time and resources is unlikely to provide
information on variability and uncertainty in risk that will affect the risk management decision.
This guidance recommends that a point estimate risk assessment be conducted in the first tier
after completing the remedial investigation (RI) planning, site scoping, problem formulation, data
collection, and the development of a site conceptual model. In general, when site decision making would
benefit from additional analysis beyond the point estimate risk assessment, and when the risk manager
needs more information to complete the RI/FS process, the risk manager would proceed to higher tiers.
Sensitivity analysis should be conducted in each tier to guide decisions regarding data collection and the
complexity of the analysis needed to characterize variability and/or uncertainty in risk. Sensitivity
analysis can also play an important role in risk communication by supporting decisions to continue
characterizing less influential variables with point estimates in higher tiers.
(2) Select the RME Risk from the RME Risk Range (9ffk to 99.9th percentile)
The RME is defined as the highest exposure that is reasonably expected to occur at a site. Final
Guidelines for Exposure Assessment (EPA, 1992a) states that the "high-end" of exposure for a population
occurs between the 90th and 99.9th percentiles, with the 99.9th percentile considered a bounding estimate.
Using a point estimate approach, the calculation of the RME risk would be based on high-end input
values in combination with average input values. For example, for estimation of risks from the ingestion
of groundwater, default exposure is based on a high-end water intake rate (2 L/day), a high-end exposure
frequency and duration (350 days/year for 30 years), and an average body weight (70 kg).
Page 1-26
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
With the probabilistic approach, the calculation of the RME risk would be based on a range of
input values, or frequency distributions, including low, average, and high-end values for each of the input
exposure factors. For example, for estimation of risks from ingestion of groundwater, exposure would be
based on the combination of lognormal distributions for water intake rate, body weight, and exposure
duration (each using a specified mean and standard deviation) and a triangular distribution for exposure
frequency (using a specified minimum, most likely value, and maximum). As a result, the RME risk
would become a probability distribution ranging from low- to high-end values based on varying a
combination of input values. In PRA, a recommended starting point for risk management decisions
regarding the RME is the 95th percentile of the risk distribution (see Chapter 7).
(3) Use PRA for Dose-Response in Ecological Assessment, not in Human Health Assessment
Approaches to characterizing variability and uncertainty in toxicological information should
reflect both the latest developments in the science of hazard and dose-response evaluation and consistent
application of EPA science policy. This statement is consistent with the 1997 EPA Policy Statement
presented in Section 1.4 above (U.S. EPA, 1997g). Probabilistic approaches to ecological dose-response
assessment may be explored, as discussed and demonstrated in Chapter 4. This guidance does not
develop or evaluate probabilistic approaches for dose-response in human health assessment and, further,
discourages undertaking such activities on a site-by-site basis. Such activities require contaminant-
specific national consensus development and national policy development. Parties wishing to undertake
such activities should contact the OERR to explore ways in which they might contribute to a national
process for the contaminant of interest to them.
(4) Prepare a Workplanfor EPA Review and Approval
A workplan should be developed and submitted for review before commencement of a PRA. The
workplan should document the combined decisions of the RPM and risk assessor involved in the risk
assessment, and positions of the stakeholders. The workplan should address conditions and policies
presented in this section of RAGS Volume 3: Part A, the software to be used, the exposure routes and
models, and the input probability distributions and their basis, including appropriate literature references.
The workplan is discussed in more detail in Chapter 2.
A checklist of some of the key considerations to assist in the review of a PRA is provided in
Appendix F.
1.5 ORGANIZATION OF THE GUIDANCE
Subsequent chapters of RAGS Volume 3: Part A focus on the following topics:
Chapter 2 - The Tiered Approach to PRA
Chapter 2 includes information regarding organizational issues that may need to be considered by
the RPM in developing a PRA. Examples, include: workplans, involvement of the Community
Involvement Coordinator (CIC), additional meetings with communities, and review of PRA documents.
Chapter 2 also presents the tiered approach in full detail. The approach begins with RI planning,
scoping, problem formulation, and data collection. Tier 1 entails a point estimate risk assessment and
sensitivity analysis. Tier 2 proceeds with additional data collection, a MCA to characterize variability
Page 1-27
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
and/or uncertainty, and a more in-depth sensitivity analysis. More advanced techniques are used in Tier 3
to simultaneously characterize variability and uncertainty. The endpoint of the tiered approach is to
provide information that helps risk managers complete the RI/FS process.
Chapter 3 - Probabilistic Human Health Risk Assessment
Chapter 3 provides a discussion of how PRA approaches may be utilized in human health risk
assessment. Probabilistic approaches focus on the exposure assessment, and an example is included to
illustrate the application of the tiered approach to a human health risk assessment.
Chapter 4 - Probabilistic Ecological Risk Assessment
Chapter 4 provides a discussion of how PRA approaches may be utilized in ecological risk
assessment. This includes a discussion of basic tactics, such as how to decide if, and when, a PRA is
needed, along with technical discussions and examples of how to model variability and/or uncertainty in
exposure, toxicity, and risk (characterized both as hazard quotients and responses) for different types of
ecological receptors, both within and between species. The chapter also provides a discussion of how the
results of an ecological PRA can be used in risk management decision making, and provides guidelines
for planning and performing an ecological PRA.
Chapter 5 - PRA and Preliminary Remediation Goals (PRGs)
This chapter provides a discussion about issues associated with deriving PRGs from both point
estimate risk assessment and PRA. Issues and limitations associated with back calculation are
highlighted, along with an explanation and recommendation regarding the iterative forward calculations.
Chapter 6 - Communicating Risks and Uncertainties in PRA
Chapter 6 provides a basic overview of the current Superfund guidance on communicating with
the public. With this as a basis, the chapter provides specific information regarding continuous
involvement of stakeholders in the PRA process, various tools that may be useful in communicating the
principles of PRA, organizational issues regarding planning of communication strategies, and examples of
procedures that may be helpful at individual sites. This chapter also provides references to various
documents on current approaches for communicating risk to the public.
Chapter 7 - Role of PRA in Decision Making
This chapter provides guidance on how to interpret the results of a PRA to determine if an
unacceptable risk is present, and criteria to consider when moving from a risk-based PRO to a remedial
goal.
Page 1-28
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
Appendix A - Sensitivity Analysis
Important information from PRA includes the results of sensitivity analysis. This appendix
outlines the methodology and interpretation of statistical methods used to conduct sensitivity analysis
with point estimate and probabilistic models.
Appendix B - Selecting and Fitting Distributions
One of the more challenging aspects of PRA is choosing appropriate probability distributions to
represent variability and uncertainty in the input variables. This appendix presents a process for selecting
and fitting distributions to data, including hypothesizing families of distributions, parameter estimation
techniques, and goodness-of-fit tests.
Appendix C - Exposure Point Concentration (EPC)
An important variable in most risk assessments is the concentration term. This appendix presents
the basic principles of the EPC, and different methods for quantifying both variability and parameter
uncertainty in the EPC.
Appendix D - Advanced PRA Models
Sometimes a more complex modeling approach can be used to improve the representativeness of
the probabilistic risk estimates. These approaches are generally anticipated to be applied in Tier 3 of the
tiered approach. Examples include the use of Microexposure Event modeling, geostatisics, and Bayesian
Monte Carlo analysis.
Appendix E - Definitions
A list of definitions is provided at the beginning of each chapter. This appendix provides a
compilation of all definitions presented in the guidance.
Appendix F' - Generic Checklist
After a PRA has been submitted to the Agency, an efficient process is needed to evaluate the
accuracy and clarity of the results. This appendix suggests a series of elements of the review process that
can be adopted to structure the review of PRAs for both human health and ecological risk assessment.
Appendix G - Frequently Asked Questions (FAQ) about PRA
Risk assessors and risk managers who read RAGS Volume 3: Part A will find that probabilistic risk
assessment covers a wide variety of topics ranging from statistical theory to practical applications and
policy decisions. U.S. EPA OERR plans to maintain and periodically update a list of frequently asked
questions and responses on an EPA Superfund web page at http://www.epa.gov/superfund/index.htm.
This appendix provides a preliminary list of anticipated questions.
Page 1-29
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
Appendix H - Index
This index includes keywords and concepts used throughout this guidance document. They are
listed alphabetically with numbers indicating the appropriate chapter and page number(s) within each
chapter. Commas separate page numbers within a chapter or appendix, while semi-colons separate
chapters and appendices. For example: probability density function, 1-5, 6-8; 4-3, 10-12; C-l, 8-10. This
would indicate Chapter 1, page 5, and pages 6-8; Chapter 4, page 3, and pages 10-12; Appendix C, page 1
and pages 8-10.
1.6 NEXT STEPS FOR PRA IMPLEMENTATION
This guidance has presented the current principles, including the tiered approach, and examples to
aid in conducting acceptable PRAs at Superfund sites. Policies and practices will change over time as
scientific advances continue in the future. The PRA Workgroup intends to keep current and provide new
information on EPA Superfund web page at http://www.epa.gov/superfund/index.htm. EPA expects to
make the following PRA support items available on-line in the near future:
• RAGS Volume 3: Part B: A workbook that serves as a companion to RAGS Volume 3: Part A;
it will include case studies and examples in PRA.
• Guidance on Probability Distributions: Documents and/or spreadsheets to aid in selecting
and fitting probability distributions for input variables.
• Guidance on Data Representativeness: A ranking methodology to evaluate data
representativeness for various exposure scenarios.
• Hands-On Training: Basic MCA training materials, and limited computer hands-on training
sessions available to Regional EPA and State staff.
• Access to PRA Workgroup: A workgroup to provide support on PRA to EPA regional risk
assessors.
• FAQs: A list of Frequently Asked Questions (FAQs) about PRA and responses from the PRA
Workgroup, maintained and periodically updated on-line.
Page 1-30
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
REFERENCES FOR CHAPTER 1
Burmaster, D.E. and B.C. Udell. 1990. A Review of Crystal Ball®. Software Review 10: 343-345.
Cullen, A.C. and H.C. Frey. 1999. Probabilistic Techniques in Exposure Assessment: A Handbook for
Dealing with Variability and Uncertainty in Models and Inputs. Plenum Press, NY.
Finley, B.L. and D.J. Paustenbach. 1994. The Benefits of Probabilistic Exposure Assessment: Three Case
Studies Involving Contaminated Air, Water and Soil. Risk Anal. 14(l):53-73.
Finkel, A.M. 1990. Confronting Uncertainty in Risk Management: A Guide for Decision Makers. Center
for Risk Management, Resources for the Future. Washington, DC.
Maddalena, R.L., T.E. McKone, D.P.H. Hsieh, and S. Geng. 2001. Influential Input Classification in
Probabilistic Multimedia Models. Stochastic Environmental Research and Risk Assessment
Morgan, G.M. and M. Henrion. 1990. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative
Risk and Policy Analysis . Cambridge University Press, NY.
National Research Council (NRC). 1983. Risk Assessment in the Federal Government: Managing the
Process. National Academy Press. Washington, DC.
National Research Council (NRC). 1989. Improving Risk Communication. National Academy Press.
Washington, DC.
National Research Council (NRC). 1994. Science and Judgement in Risk Assessment. National
Academy Press. Washington, DC.
Presidential/Congressional Commission on Risk Assessment and Risk Management. 1997. Risk
Assessment and Risk Management in Regulatory Decision Making. Final Report, Volume 2.
Rugen, P. and B. Callahan. 1996. An Overview of Monte Carlo, A Fifty Year Perspective. Hum Ecol
Risk Assess. 2(4):671-680.
U.S. EPA. 1989a. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health
Evaluation Manual (HHEM) (Part A, Baseline Risk Assessment). Interim Final. Office of
Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1989b. Risk Assessment Guidance for Superfund. (RAGS): Volume 11. Environmental
Evaluation Manual . Interim Final. Office of Emergency and Remedial Response, Washington,
DC. EPA/540/1-89/001.
U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40
CFR 300: 55 Federal Register, 8666-8865, March 8.
Page 1-31
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
U.S. EPA. 1991a. Risk Assessment Guidance for Superfund (RAGS): Volume I-Human Health Evaluation
Manual Supplemental Guidance: "Standard Default Exposure Factors. " Interim Final. Office of
Solid and Emergency Response, Washington, DC. OSWER Directive No. 9285.6-03.
U.S. EPA. 1991b. RAGS Volume I, Human Health Evaluation Manual (Part B: Development of
Risk-based Preliminary Remediation Goals). Office of Emergency and Remedial Response.
Washington, DC. EPA/540/R-92/003. December.
U.S. EPA. 1991c. RAGS Volume I, Human Health Evaluation Manual (Part C: Risk Evaluation of
Remedial Alternatives). Office of Emergency and Remedial Response. Washington, DC.
OSWER Directive No. 9285.7-01C. October.
U.S. EPA. 1991d. Role of the Baseline Risk Assessment in Superfund Remedy Selection Decisions.
Office of Solid Waste and Emergency Response, Washington, DC. OSWER
Directive No. 9355.0-30.
U.S. EPA. 1992a. Final Guidelines for Exposure Assessment. EPA/600/Z-92/001. 57 Federal Register,
22888-22938, May 29.
U.S. EPA. 1992b. Framework for Ecological Risk Assessment. EPA 630/R-92/001. February.
U.S. EPA. 1992c. Guidance on Risk Characterization for Risk Managers and Risk Assessors.
Memorandum from F. Henry Habicht II, Deputy Administrator. Office of Solid Waste and
Emergency Response, Washington, DC.
U.S. EPA. 1992d. Supplemental Guidance to RAGS: Calculating the Concentration Term. Office of
Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-081.
U.S. EPA. 1993. Use of IRIS (Integrated Risk Information System) Values in Superfund Risk
Assessment. Memorandum from William H. Farland and Henry L. Longest II. Office of Solid
Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7.16, December
21.
U.S. EPA. 1994a. Role of Ecological Risk Assessment in the Baseline Risk Assessment. Office of Solid
Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-17.
U.S. EPA. 1994b. Use of Monte Carlo Simulation in Risk Assessments. Region 3, Hazardous Waste
Management Division. Office of Superfund programs, Philadelphia, PA. EPA/903/F-94/001.
U.S. EPA. 1995a. Guidance for Risk Characterization. Office of Research and Development.
Washington, DC. http://www.epa.gov/ORD/spc/rcpolicv.htm.
U.S. EPA. 1995b. Memorandum from Carol Browner on Risk Characterization. Office of the
Administrator, Washington, DC. February 22.
U.S. EPA. 1995c. Policy for Risk Characterization. Office of Research and Development. Washington,
DC. http: //www .epa. gov/ORD/spc/rcpolicv .htm.
Page 1-32
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
U.S. EPA. 1995d. Policy on Evaluating Health Risks to Children. Office of Children's Health
Protection. Washington, DC. http://www.epa.gov/children/whatwe/rrguide.pdf.
U.S. EPA. 1995e. Use of Monte Carlo Simulation in Performing Risk Assessments (Technical Section).
Region 8, Hazardous Waste Management Division, Superfund Management Branch Technical
Guidance, Denver, CO, RA-10.
U.S. EPA. 1996. Final Soil Screening Guidance, May 17, 1996. Soil Screening User's Guide. Office
of Solid Waste and Emergency Response, Washington, DC. EPA 540/R-96/018.
U.S. EPA. 1997a. Ecological Risk Assessment Guidance for Superfund: Process for Designing and
Conducting Ecological Risk Assessments. Interim Final. Environmental Response Team, Edison,
NJ. EPA/540/R-97/006, OSWER Directive No. 9285.7-25, June.
U.S. EPA. 1997b. Exposure Factors Handbook, Volume 1. Office of Research and
Development,Washington, DC. EPA/600/P-95/002Fa.
U.S. EPA. \991c.ExposureFactorsHandbook, Volume 2. Office of Research and Development,
Washington, DC. EPA/600/P-95/002Fb.
U.S. EPA. 1997d. Exposure Factors Handbook, Volume 3. Office of Research and Development,
Washington, DC. EPA/600/P-95/002Fc.
U.S. EPA. 1997e. Guidance on Cumulative Risk Assessment. Phase 1. Planning and Scoping.
Washington, DC.
U.S. EPA. 1997f Lognormal Distribution in Environmental Applications. Office of Research and
Development, and Office of Solid Waste and Emergency Response, Washington, DC.
EPA/600/R-97/006. December.
U.S. EPA. 1997g. Memorandum from Deputy Administrator Fred Hansen on the Use of Probabilistic
Techniques (including Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for
Monte Carlo Analysis. Office of Research and Development, Washington, DC.
EPA/630/R-97/001.May.
U.S. EPA. 1998. Guidelines for Ecological Risk Assessment. Final. National Center for Environmental
Assessment, Washington, DC. EPA/630/R-95/002F.
U.S. EPA. 1999. Ecological Risk Assessment and Risk Management Principles for Superfund Sites. Final.
Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive
No. 9285.7-28P.
U.S. EPA. 2000. Risk Characterization Handbook. Office of Science Policy. Office of Research and
Development. EPA 100-B-00-002. December.
Page 1-33
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 1 ~ December 31, 2001
U.S. EPA. 2001a. Risk Assessment Guidance for Superfimd: Volume I. Human Health Evaluation Manual
(Part D, Standardized Planning, Reporting, and Review of Superfund Risk Assessments). Office
of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9285.7-47.
December.
U.S. EPA. 2001b. Risk Assessment Guidance for Superfund: Volume 1, Human Health Evaluation
Manual (Part E, Supplemental Guidance for Dermal Risk Assessment). Interim. Review
Draft-For Public Comment. Office of Emergency and Remedial Response. Washington, DC.
OSWER Directive No. 9285.7-02E-P. September.
U.S. EPA. 2001c. Supplemental Guidance for Developing Soil Screening Levels for Superfund Sites.
Office of Emergency and Remedial Response. Washington, DC. OSWER Directive
No. 9355.4-24. December.
U.S. EPA. 2001d. The Role of Screening-Level Risk Assessments and Refining Contaminants of Concern
Baseline Risk Assessments. Office of Solid Waste and Emergency Response. 12th Intermittent
Bulletin, ECO Update Series. EPA 540/F-01/014. June.
Page 1-34
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
CHAPTER 2
WORKPLAN AND THE TIERED APPROACH
2.0 INTRODUCTION
While probabilistic risk assessment (PRA) can provide useful information for risk management,
not all site decisions will benefit from probabilistic approaches. Similarly, not all PRAs need involve
complex models and quantitative uncertainty analysis methods; often, very useful information can be
obtained by taking the point estimate approach one step further to explore variability in selected input
variables. The level of effort and complexity of the risk assessment should match site-specific needs.
The use of a tiered approach for moving from a point estimate risk assessment to PRAs of varying levels
of complexity is recommended (Figure 2-1 and 2-2). This chapter outlines the basic steps of a tiered
approach for including PRA in a site risk assessment. The major feature of the tiered approach is an
iterative evaluation of the risk estimates developed at each tier to determine if they are sufficient for risk
management decisions. Built into the tiered approach are opportunities for communication with
stakeholders with a view to saving time and costs, and facilitating a successful remedial process.
2.1 WORKPLAN
In practice, the potential value of PRA may be considered at various planning stages of a risk
assessment. For some sites, PRA and point estimate risk assessment approaches may be discussed in the
initial scoping of the risk assessment. For other sites, PRA may become a viable option only after the
point estimate risk assessment results are available. Ideally, PRA should be considered as early as
possible in the planning of risk assessment activities at a site so that sampling plans and data collection
efforts may be appropriately directed. Initial PRA discussions should be included as part of the risk
assessment workplan. If a PRA is being considered following completion of a point estimate risk
assessment, the original workplan for the point estimate assessment should be expanded to include needs
that are unique to PRA.
The methods and procedures used to prepare a workplan to gather additional information for a
baseline point estimate risk assessment are documented in RAGS Volume I: Part A (U.S. EPA, 1989).
This chapter of RAGS Volume 3: Part A describes the procedures that would be used to prepare a
workplan to gather additional information to conduct a PRA. Separate workplans may be warranted for
human health and ecological risk assessments.
Like the quality assurance project plan (QAPP), the workplan for a PRA should document the
combined decisions of the remedial project manager (RPM) and the risk assessor. Meaningful
involvement of stakeholders early in the decision-making process also will save time and effort.
Page 2-1
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
EXHIBIT 2-1
DEFINITIONS FOR CHAPTER 2
Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in a population,
usually considered to be the mean or median of the distribution.
Countably Infinite - Used to describe some discrete random variables, this term refers to a set of numbers that can be
counted with integers (e.g., one, two, three) and that has no upper limit. Examples include the number of tosses
required for a coin to show a head—we can count each toss, but it is possible that at least one more toss is needed.
The number of dust particles in a volume of air is another example. Countably finite implies there is an upper
limit (e.g., days of work per year).
CTE Risk - The estimated risk corresponding to the central tendency exposure.
Monte Carlo Analysis (MCA) or Monte Carlo Simulation - A technique for characterizing the uncertainty and
variability in risk estimates by repeatedly sampling the probability distributions of the risk equation inputs and
using these inputs to calculate a range of risk values.
Parameter - A value that characterizes the distribution of a random variable. Parameters commonly characterize the
location, scale, shape, or bounds of the distribution. For example, a truncated normal probability distribution may
be defined by four parameters: arithmetic mean [location], standard deviation [scale], and min and max [bounds].
It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithmetic mean
ingestion rate).
Point Estimate - In statistical theory, a quantity calculated from values in a sample to estimate a fixed but unknown
population parameter. Point estimates typically represent a central tendency or upper bound estimate of
variability.
Point Estimate Risk Assessment - A risk assessment in which a point estimate of risk is calculated from a set of point
estimates for exposure and toxicity. Such point estimates of risk can reflect the CTE, RME, or bounding risk
estimate depending on the choice of inputs.
Potentially Responsible Party (PRP) - PRPs are individuals, companies, or any other party that are potentially liable for
payment of Superfund cleanup costs.
Preliminary Remediation Goal (PRG) - Initially developed chemical concentration for an environmental medium that is
expected to be protective of human health and ecosystems. PRGs may be developed based on applicable or
relevant and appropriate requirements (ARARs), or exposure scenarios evaluated prior to or as a result of the
baseline risk assessment. (U.S. EPA, 1991a, 1991b).
Probabilistic Risk Assessment (PRA) - A risk assessment that yields a probability distribution for risk, generally by
assigning a probability distribution to represent variability or uncertainty in one or more inputs to the risk
equation
Probability Density Function (PDF) - A graph that shows the probability of occurrence of an unknown or variable
quantity. A PDF is used to characterize a continuous random variable, X. PDFs can be used to display the shape
of the distribution for an input variable or output variable of a Monte Carlo simulation. The term density comes
from the concept that a probability at a point, x, for a continuous distribution is equal to the area under the curve
of the PDF associated with a narrow range of values around x.
Probability Distribution - A mathematical representation of the function that relates probabilities with specified
intervals of values for a random variable. Also called a probability model.
Probability Mass Function (PMF) - A function representing the probability distribution for a discrete random variable.
The mass at a point refers to the probability that the variable will have a value at that point.
Random Variable - A variable that may assume any value from a set of values according to chance. Discrete random
variables can assume only a finite or countably infinite number of values (e.g., number of rainfall events per year).
A random value is continuous if its set of possible values is an entire interval of numbers (e.g., quantity of rain in
a year).
Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site (U.S.
EPA, 1989). The intent of the RME is to estimate a conservative exposure case (i.e., well above the average case)
that is still within the range of possible exposures.
Remedial Investigation/Feasibility Study (RI/FS) - Studies undertaken by EPA to delineate the nature and extent of
contamination, to evaluate potential risk, and to develop alternatives for cleanup.
Page 2-2
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
EXHIBIT 2-1
DEFINITIONS FOR CHAPTER 2—Continued
RME Risk - The estimated risk corresponding to the reasonable maximum exposure.
Scientific/Management Decision Point (SMDP) - A point during the tiered process in PRA when the risk assessor
communicates results of the assessment to the risk manager. At this point, the risk manager determines whether the
information is sufficient to arrive at a decision or if additional data collection or analysis is needed. SMDPs provide a
tool for transitioning to a subsequent tier or for exiting the tiered process.
Sensitivity Analysis - Sensitivity generally refers to the variation in output of a model with respect to changes in the values
of the model's input(s). Sensitivity analysis can provide a quantitative ranking of the model inputs based on their
relative contributions to model output variability and uncertainty. Common metrics of sensitivity include:
>• Pearson Correlation Coefficient - A statistic r that measures the strength and direction of linear association
between the values of two quantitative variables. The square of the coefficient (r2) is the fraction of the variance
of one variable that is explained by the variance of the second variable.
>• Sensitivity Ratio - Ratio of the change in model output per unit change in an input variable; also called elasticity.
*• Spearman Rank Order Correlation Coefficient - A "distribution free" or nonparametric statistic r that measures
the strength and direction of association between the ranks of the values (not the values themselves) of two
quantitative variables. See Pearson (above) for r2.
Uncertainty - Lack of knowledge about specific variables, parameters, models, or other factors. Examples include limited
data regarding the concentration of a contaminant in an environmental medium and lack of information on local fish
consumption practices. Uncertainty may be reduced through further study.
A PRA workplan should be developed early in the risk assessment planning process for the site,
regardless of who will actually develop the PRA (e.g., Environmental Protection Agency (EPA), EPA
contractor, or potentially responsible party (PRP)). If a PRP performs the PRA, the workplan should be
submitted to EPA for review and approval prior to commencing the PRA. It should describe the intended
PRA in sufficient detail so that EPA can determine if the work products will adequately address risk
assessment and management needs (see Exhibit 2-2 for contents of a typical workplan). It is important
that the risk assessor and RPM discuss the scope of the probabilistic analysis and the potential impact it
may have on the remedial investigation/feasibility study (RI/FS).
is" Given the time and effort that can be expected to be invested in conducting a
PRA, it is important that a workplan undergo review and approval by EPA,
prior to proceeding with the assessment.
In general, regions should not accept probabilistic analysis when a workplan for the analysis has
not been submitted to the Agency, and approved by the regional risk assessor and RPM.
The tiered process for PRA, described in Section 2.3, is an iterative process. As new information
becomes available, it should be used to evaluate the need to move to a higher tier. The decision to move
an assessment to a higher tier of complexity should result in a revised workplan reflecting the greater
complexity and demands of the higher tier. The proposed probabilistic sensitivity analysis developed at
the lower tier should be included in the revised workplan, along with a point estimate risk assessment
based on any data collected as part of a lower tier. The probabilistic methods used in a PRA can often be
restricted to the chemicals and pathways of concern that contribute the greatest risk. The less sensitive
chemicals and exposure pathways should still remain in the PRA using point estimates, unless there is a
compelling reason to exclude them from the assessment altogether. As stated in Appendix A (Section
A. 1, Risk Communication), the decision to represent an input variable with a point estimate, rather than a
probability distribution, will generally be made on a case-by-case basis. The decision will reflect an
Page 2-3
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
attempt to balance the benefits of simplifying the
analysis (e.g., easier to communicate; focuses
discussion on more critical areas) with the potential
for arbitrarily reducing the variance in the output
distribution (e.g., discounting variability in multiple
variables with negligible contributions to risk may
end up having a non-negligible effect on the RME
percentile).
Throughout the process of developing the
PRA, EPA risk assessor and other contributors to the
assessment should have a continuing dialogue to
discuss the elements of the workplan and their
potential impacts on the assessment. This dialogue,
along with interim deliverables, will help to ensure
that the risk assessment report will meet the needs of
the Agency and that any problems are identified and
corrected early in the process.
2.2 SPECIAL ADMINISTRATIVE
CONSIDERATIONS IN PRA
Inclusion of a PRA in the RI/FS will
generate certain administrative activities for the
RPM. The scope of these activities will depend on
whether the PRA is conducted by EPA and its
contractors or by the PRP. The following sections
provide practical advice for the RPM who is
considering applications of PRA at a site.
2.2.1 SCOPING OF PRA
EXHIBIT 2-2
EXAMPLES OF IMPORTANT
CONTENTS OF A PRA WORKPLAN
1. Statement of the ecological assessment
endpoints and/or human risk
2. Summary of the point estimate risk
assessment
3. Potential value added by conducting a PRA
and proceeding to the subsequent tiers
4. Discussion of adequacy of environmental
sampling for PRA or moving to a
successive tier (e.g., data quality issues)
5. Description of the methods and models to
be used (e.g., model and parameter
selection criteria)
6. Proposal for obtaining and basis for using
exposure factor distributions or ecological
toxicity distributions
7. Methods for deriving the concentration
term
8. Proposal for probabilistic sensitivity
analysis
9. Software (e.g., date and version of product,
random number generator)
10. Bibliography of relevant literature
11. Proposed schedule, discussion points, and
expertise needed
The RPM will generally be involved in the discussions among EPA project team, as well as PRPs
and other stakeholders, regarding the level of PRA that is appropriate for the site. As outlined in the
tiered approach (see Section 2.3), the scope and complexity of the PRA should satisfy the risk assessment
and management decision making needs of the site. Team members should meet to discuss the scope of
the PRA, the anticipated community outreach, and the required level of review. These discussions can be
useful for ascertaining the level of contractor involvement, specific requirements for deliverables from
PRPs, and the anticipated number of responses to comments. These meetings should include
consideration of funding, resources, and availability of personnel to work on the PRA.
2.2.1.1
PRA SCOPE OF WORK FOR FUND-LEAD SITES
A Statement of Work (SOW) should be developed before any work is started on a PRA,
regardless of whether the PRA is to be submitted to the Agency or developed by the Agency. The SOW
should outline the general approach that EPA and its contractor will use in developing the PRA. The
SOW should include the general approaches for the following PRA items: selection of input probability
distributions, documentation of methods and results, selection of computer programs, submission of
Page 2-4
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
computer codes and outputs, comparison of the results from the point estimate and probabilistic
assessments, and the format for presenting the final PRA in the RI/FS document. The SOW should be
sufficiently detailed to support a milestone schedule, which should be submitted as part of the SOW.
Based on the complexity of the PRA, and consistent with the RAGS Volume I: Part D principles of
involving the risk assessor early and often in the risk assessment process (U.S. EPA, 2001), it may be
appropriate to obtain submission of interim deliverables to allow the risk assessor the opportunity to
identify potential problems early in the process.
Within the RI/FS workplan, additional resources may be required to hold additional meetings, to
respond to comments specific to the PRA, and to develop handouts describing PRA in terms accessible to
a wider audience than risk assessors. Where appropriate, these additional resource requirements should
be included in the SOW along with interim and final deliverable dates. Chapter 6 provides guidance on
communicating concepts and results of PRA to various audiences.
2.2.1.2 PRP SCOPE OF WORK FOR PRP-LEAD SITES
The SOW for PRP-lead sites should follow the same general outline as the SOW for fund-lead
sites (Section 2.2.1.1). Legal documents such as Unilateral Orders, Administrative Orders of Consent,
and Consent Decrees should contain language requiring the PRP to submit a workplan before any work
on the PRA is started. It is also important that interim deliverables, including computer code or
spreadsheet models, be submitted so that EPA can review and verify the results of the PRA. A
comparison of the results of the PRA and the point estimate assessment should be included in the final
RI/FS.
Depending on the complexity of the site and the anticipated PRA, the RPM may be involved in
more extensive negotiations with the PRPs. These negotiations may involve both EPA staff and
contractor support. These activities may need to be included in the appropriate SOWs.
If warranted by the complexity of the PRA, the RPM may consider the need to expand oversight
contracts to include additional resources for the contractor to review and comment on the interim
deliverables and finalize the PRA. This may require a specialized level of expertise that will need to be
discussed with the contractor. Further, the contract section regarding community involvement may also
need to be expanded to include additional resources for developing handouts describing PRA in terms
accessible to a wider audience than risk assessors and for holding additional community meetings.
2.2.2 DEVELOPMENT OF PROBABILITY DISTRIBUTIONS
A key component of any PRA is the selection of representative probability distributions. The
information available to support the characterization of variability or uncertainty with probability
distributions may be an important factor in the decision to conduct a PRA. In some cases, this may
require resources to conduct exploratory data analysis or to collect site-specific information. As part of
this process, a PRA using preliminary distributions based on the available information may be considered
to identify the variables and exposure pathways that may have the strongest effect on the risk estimates.
Appendix B (Section B.2.0) provides a more detailed description of preliminary distributions and their
potential role in the tiered process. All of these activities may require extensive discussions with the
PRPs and the community. In addition, for PRP-lead sites, they may require additional resources to
critically review the proposed distributions. The RPM should consider these potential activities in
developing the SOW and legal documents to assure adequate resources are available to address them.
Page 2-5
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
2.2.3 EPA REVIEW OF PRA DOCUMENTS
The review of PRA documents may require more time than is usually allocated for point estimate
risk assessments. In part, the additional time is needed for reviewing and discussing input distributions,
for developing and running computer simulations, and for discussing outcomes of the assessment with the
PRP or EPA contractor. The early involvement of an EPA risk assessor may reduce the time needed for
review of the final risk assessment documents, although additional review time may still be required,
depending on the complexity of the PRA conducted.
In addition to EPA's review, it may also be important to include external reviewers with
specialized expertise in PRA to aid in the review. This additional support may involve resources and time
to review documents and verify simulation results, as well as additional contractual arrangements. As
stated in Chapter 1, Section 1.4 (Conducting an Acceptable PRA), it is important that negotiations with
the PRP address the assurance that adequate details will be included in the submission so that the methods
can be evaluated, and the results independently reproduced.
2.2.4 PEER-REVIEW
Depending on the level of complexity of the PRA, and whether new science is being used, it may
be necessary to conduct a peer review of the document. The Agency's guidance on peer review (U.S.
EPA, 2000b) should be consulted for information regarding the criteria for determining whether or not a
peer review is appropriate and, if it is, the process that should be followed.
2.2.5 RESPONSE TO COMMENTS ON PRA
The time and resources needed to respond to comments on a PRA may vary depending on the
complexity of the PRA. In developing the SOW, workplan, and schedule for the RI/FS, it is important
that the RPM include adequate resources and time for the thorough evaluation of the PRA. In developing
the response to comments, it may be necessary to consider alternative PRAs submitted by reviewers. The
RPM should plan for sufficient time and resources needed for such activities.
2.2.6 ADMINISTRATIVE RECORD
Criteria should be established for documentation to be included in the administrative record.
Examples may include documentation regarding the basis for selection of input distributions, a
description of the design of the PRA conducted, the computer codes used in simulations, how tiering
decisions are made, and the results of the PRA. The RPM should consider using technologies such as a
CD-ROM to document the appropriate information for the record.
2.2.7 COMMUNICATION WITH STAKEHOLDERS
Chapter 6 provides details regarding the goal of early involvement of the public in the PRA
process. For example, Section 6.1 of Chapter 6 provides additional topics for consideration in
development of community involvement plans (CIPs) where PRA is considered. In general, early
involvement of the community in the RI/FS process is important, but such involvement should meet the
site-specific needs. Important considerations include resources, funding, and the level of effort
appropriate for the site.
Page 2-6
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
2.2.8 COMMUNICATION WITH EPA MANAGEMENT
Communication with EPA managers regarding PRA is discussed in Chapter 6. The RPM may
need to consider allocating additional resources for prebriefings of appropriate management levels,
development of handouts, and follow-up to the management meetings. Coordination with appropriate
EPA staff and contractors may be necessary to assure the communication is effective.
2.3 OVERVIEW OF THE TIERED APPROACH
The tiered approach presented in this guidance is a process for a systematic, informed progression
to increasingly more complex risk assessment methods including PRA. A schematic presentation of the
tiered approach is shown in Figure 2-1 and Figure 2-2. Higher tiers reflect increasing complexity and, in
many cases, will require more time and resources. Higher tiers also reflect increasing characterization of
variability and/or uncertainty in the risk estimate, which may be important for making risk management
decisions. Central to the concept of a systematic, informed progression is an iterative process of
evaluation, deliberation, data collection, work planning, and communication (see Figure 2-2). All of
these steps should focus on deciding (1) whether or not the risk assessment, in its current state, is
sufficient to support risk management decisions (a clear path to exiting the tiered process is available at
each tier); and (2) if the assessment is determined to be insufficient, whether or not progression to a
higher tier of complexity (or refinement of the current tier) would provide a sufficient benefit to warrant
the additional effort.
The deliberation cycle provides an opportunity to evaluate the direction and goals of the
assessment as new information becomes available. It may include evaluations of both scientific and
policy information. The risk manager, in the decision-making process, is encouraged to seek input on a
regular basis from EPA staff and other stakeholders. Exhibit 2-3 lists some of the potential stakeholders
that may contribute to the deliberation process.
Page 2-7
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
Although PRA may involve technical dialogue between EPA and outside "experts", input from
members of the general public who may have an interest in the outcome of the remedial process should
also be sought at appropriate stages of the process. Frequent and productive communication between
EPA and stakeholders throughout the risk assessment process is important for enhancing the success of a
PRA.
EXHIBIT 2-3
STAKEHOLDERS POTENTIALLY INVOLVED IN
EPA's DECISION-MAKING PROCESS FOR PRA
EPA risk assessors and managers
Members of the public
Representatives from state or county
environmental or health agencies
Other federal agencies (e.g., health agencies,
Natural Resource Damage Assessment trustees,
etc.)
Tribal government representatives
Potentially responsible parties and their
representatives
Representatives from federal facilities (e.g.,
Department of Defense, Department of Energy,
etc.)
Page 2-8
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
a) -£
SO)
f -.
£ a
0) O
p* ^
>
O .«
C
a^H
O
O M3
(U
O
Tier 3 Advanced PRA
2-D MCA
Probabilistic Sensitivity Analysis
(Microexposure Modeling, Bayesian
Statistics, Geostatistics)1
t
0
Tier 2 PRA
1-DMCA
Probabilistic Sensitivity Analysis
t
k.
w
p
Tier 1 Point Estimate Risk Assessment
Point Estimate Sensitivity Analysis
i
^
w
0
o
5
-o_
n
C/5
O
VI
k
Problem Formulation/ Scoping/Work Planning/ Data Collection
= Decision Making Cycle: Evaluation, Deliberation, Data Collection,
Work Planning, Communication (See Figure 2-2)
^- At each tier, a decision may be to exit the tiered process
Figure 2-1. Schematic Diagram of Tiered Approach.
1 Examples of advanced methods for quantifying temporal variability, spatial variability, and
uncertainty (see Appendix D)
Page 2-9
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
Tier 1 or Tier 2
Analysis
Discuss
Sensitivity
Analysis
Identify Data
Gaps/ Needs
Communicate
with Stakeholders
Discuss
PRA
SMDP
Sufficient for
Risk Management
Decision?
No
1
1
Refine
Workplan
!
Collect
Additional
Data
Yes
^
r
Complete the
RI/FS Process
Only Refine the Risk
Assessment in the
Current Tier?
Tier 2 or Tier 3 Analysis
Complete RI/FS Process
Figure 2-2. Schematic diagram of deliberation/decision cycle in the tiered process for PRA. SMDP refers
to a scientific/management decision point, which implies that the decision involves consideration of not
only the risk assessment, but also Agency policy, stakeholder concerns, cost, schedule, feasibility and other
factors.
Page 2-10
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
2.3.1 GETTING STARTED
All risk assessments should begin with problem formulation, scoping, preparation of a workplan
(Section 2.1), and data collection. Problem formulation generally is an iterative process where
reevaluation may occur as new information and data become available. The RPM should convene a
scoping meeting prior to any risk assessment activities. Depending on the site-specific factors, discussion
of performing a PRA may be appropriate at this initial scoping meeting. Alternatively, this discussion
may be more productive at a later stage of the tiered process.
The risk manager should initiate discussions with EPA staff and other stakeholders early in the
process, well before planning a risk assessment. Early communication with risk assessors or other EPA
staff can help the risk manager evaluate the adequacy of the current information and plan additional
data-gathering activities. Early communication with communities and other stakeholders should establish
trust and facilitate a successful remedial process (see Chapter 6 on risk communication).
Generally, once the appropriate steps have been taken to adequately formulate and identify the
problem and complete a workplan (Section 2.1), data collection efforts towards the point estimate risk
assessment may begin. The process for conducting a point estimate risk assessment (Tier 1) is
documented elsewhere in various RAGS volumes and related Superfund risk assessment guidance
documents (e.g., U.S. EPA, 1989, 2001).
2.3.2 TIER 1
Tier 1 consists of the well-established
process for planning and conducting human
health and ecological point estimate risk
assessments. Typical elements of a Tier 1 risk
assessment, as they relate to higher tiers, are
presented in Exhibit 2-4. A more detailed
discussion of these elements can be found in
Chapters 3 and 4 and Appendix A (Sensitivity
Analysis).
A more detailed schematic presentation
of the tiered process, showing the various
elements of the deliberation/decision cycle and
their linkage to higher tiers is shown in
Figure 2-2. The two main factors to consider
when determining whether the results of a risk
assessment are sufficient for decision making
are: (1) the results of a comparison of the risk
estimate with the risk level of concern; and
(2) the level of confidence in the risk estimate.
EXHIBIT 2-4
TYPICAL ELEMENTS OF TIER 1 RISK ASSESSMENT
Analysis Tool - point estimate risk assessment
Variability Modeling - semi-quantitative, using
central tendency exposure (CTE) and reasonable
maximum exposure (RME) estimates as input
variables
Uncertainty Modeling - semi-quantitative using
confidence limits on certain point estimates (e.g.,
concentration term)
Sensitivity Analysis - point estimate calculation of
percentage contribution of exposure pathways, for
both CTE and RME risk. Systematically vary one
input variable at a time across a plausible range and
rank inputs based on sensitivity ratios or sensitivity
scores.
Risk-Based Decision-Making Output - point
estimate of risk—Does the point estimate exceed
the risk level of concern?
In Tier 1, comparison of risk estimates
with risk levels of concern is relatively straightforward, since the outcome of a point estimate risk
assessment is a single estimate of risk that either will exceed or not exceed the risk level of concern.
Evaluating confidence in the Tier 1 risk estimates is more difficult because quantitative measures of
uncertainty often are not easily obtained from a point estimate analysis. Uncertainty arises from two main
Page 2-11
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
sources: (1) uncertainty in the inputs to the risk equations that stems from lack of knowledge (data gaps),
and (2) uncertainty in the accuracy of the point estimate that stems from the mathematical simplifications
that are inherent in point estimate computations.
There are usually many sources of uncertainty in the values used to calculate risk. One of the
most familiar (but not always the most significant) is uncertainty in environmental concentration values of
contaminants. This source of uncertainty is usually accounted for by calculating a 95% upper confidence
limit (95% UCL) for the mean concentration in the exposure equation (U.S. EPA, 1992b). Chapter 5,
Appendix C, and Appendix D provide more complete discussions of policies and methods for quantifying
uncertainty in the exposure point concentration. Uncertainties in other variables in the risk equations
(intake rates, exposure frequency and duration, toxicity factors, etc.) may also be significant, and are
often addressed by choosing inputs that are more likely to yield an overestimate than an underestimate of
risk. These sources of uncertainty are usually addressed qualitatively, by providing a discussion of the
likely direction and magnitude of the error that may be associated with the use of the specific inputs (U.S.
EPA, 1989). Stakeholders can provide useful information about uncertain variables and sources for site-
specific data. This is an important reason to ensure that stakeholders are given the opportunity to review
the risk assessment and be involved in the process.
Decision Alternatives
The evaluation of the point estimate risk assessment will yield one of two outcomes: (1) sufficient
for risk management decisions; or (2) insufficient for risk management decisions. If the risk manager
views the results of the point estimate risk assessment as sufficient for risk management decision making,
the risk manager can exit the tiered approach and complete the RI/FS process (Figure 2-2). Depending on
site-specific information, the results may support a decision for "no further action" or for a "remedial
action." A "no further action" decision may result when the risk estimate is clearly below the level of
concern (e.g., the National Oil and Hazardous Substances Pollution Contingency Plan (NCP) risk range of
1E-04 to 1E-06) and confidence in the risk estimate is high. A decision for remedial action may result
when a national standard (e.g., maximum contaminant levels (MCLs) applied to groundwater) may be
exceeded, or when the risk is clearly above the level of concern (e.g., the NCP risk range of 1E-04 to
1E-06) and confidence in the risk estimate is high. The decision for a specific remedial action involves
consideration of the NCP's nine criteria for remedial decisions (U.S. EPA, 1990) and other site-specific
factors.
An alternative conclusion would be that the results of the point estimate risk assessment are not
sufficient for risk management decision making. For example, results may not be sufficient when the risk
estimate is within the NCP risk range of 1E-04 to 1E-06 and confidence in the risk estimate is low. In
this case, the risk manager should not exit the tiered approach. Instead, appropriate steps should be taken
to increase the confidence that a management decision is protective. These steps may include discussing
the point estimate sensitivity analysis, identifying data gaps, communicating with stakeholders (e.g., to
obtain site-specific information), discussing the potential value of conducting a PRA (or a more advanced
probabilistic analysis), work planning, and additional data collection (see Figure 2-2).
A sensitivity analysis can be a valuable component of the evaluation of a risk assessment.
Sensitivity analysis can identify important variables and pathways that may be targets for further analysis
and data collection. The type of information provided by a sensitivity analysis will vary with each tier of
a PRA. Several methods are available at each tier, and the results of the analysis can vary greatly
depending on the methods used. A comprehensive discussion of these methods is presented in
Appendix A and briefly summarized here. Sensitivity analysis in Tier 1 will usually involve relatively
Page 2-12
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
simple methods and will not involve Monte Carlo simulation. A typical approach would be to calculate
the relative contributions of individual exposure pathways to the point estimate of risk. A more complex
approach involves selecting values from a plausible range for a specific input variable to the exposure or
risk equation and to use these values (i.e., low-end estimate and high-end estimate) to calculate
corresponding point estimates of risk. The sensitivity of the risk estimate to each variable is then
evaluated by calculating a sensitivity ratio, which is simply the percentage change in the risk estimate
divided by the percentage change in the input variable value (see Appendix A, Section A.2.1.3,
Sensitivity Ratios).
The sensitivity ratio (SR) approach is typically applied to one variable at a time because jointly
varying point estimates for multiple variables can be cumbersome (see Chapter 3, Table 3-2 for an
example of two jointly varied inputs). Information provided by the SR approach is generally limited to
bounding estimates of risk based on small deviations and/or plausible ranges of point estimates for inputs.
Because the point estimate approach does not generate a distribution of risk, SRs cannot provide
quantitative information about the relative contributions of input variables to the variance in risk or the
uncertainty in selected percentile of the risk distribution. This limitation of the SR approach may be
particularly important if the ranking of input variables may change depending on the percentile range that
is evaluated. For example, in a probabilistic analysis, the soil ingestion rate variable may contribute most
to the variability in risk across the entire risk distribution, but the exposure duration may be the driver in
the high-end (> 90th percentile) of the risk distribution, where the RME risk is defined. In addition, for
standard product-quotient risk equations, the SR approach also has difficulty distinguishing the relative
importance of exposure variables in the risk equation. Appendix A presents a hypothetical example to
illustrate why this happens for the common risk equations. An improvement over the SR approach, called
Sensitivity Score, involves weighting each ratio by the variance or coefficient of variation of the input
variable when this information is available. In general, the most informative sensitivity analysis will
involve Monte Carlo techniques (see Appendix A, Table A-l). Potential strengths and weaknesses of
sensitivity analysis methods may be an important factor in deciding whether or not to conduct a
probabilistic analysis in Tier 2.
Once data gaps have been identified, steps may be taken to gather additional data and revise the
point estimates of risk based on these data. As with any data collection effort, the data quality objectives
(DQO) process should be followed to obtain samples appropriate for the risk assessment and sufficient to
support the remedial decision (U.S. EPA, 1992a, 1993, 1994, 2000a). The deliberation and decision cycle
(Figure 2-2) should then be reiterated to determine if the refined risk assessment is sufficient to support
risk management decisions. The collection of additional data may also provide a compelling reason to
consider moving to Tier 2 and conducting a PRA. If, during the PRA discussions, it is determined that
information from a PRA may influence the risk management decisions, PRA may be warranted. This
iterative process of collecting data, recalculating point estimates, and reconsidering the potential value of
PRA may continue until sufficient data are available to support risk management decisions, or data
collection efforts are not possible due to resource constraints. For example, soil ingestion rate data may
be limited to a few studies with small sample sizes, but a new soil ingestion study may be prohibitively
expensive, time consuming, or difficult to conduct in a manner that will reduce the uncertainty in the risk
estimate. Uncertainty due to data quantity is not necessarily a reason to exit the tiered process at Tier 1.
In cases where there is uncertainty in selecting a probability distribution because of small sample
sizes, it may be informative to develop a preliminary probability distribution such as a triangular or
uniform (see Appendix B, Section B.2.0). These preliminary distributions will contribute to the
variability in the risk estimate, and can therefore be included in the probabilistic sensitivity analysis.
Results of Monte Carlo simulations that include one or more preliminary distributions may lead to several
Page 2-13
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
alternative decisions. If the sensitivity analysis suggests that the risk estimate is relatively insensitive to
the variable described with the distribution, then the uncertainty associated with the choice of a
distribution should not affect the risk management decision process using the tiered approach (e.g., choice
of RME percentile, derivation of a PRO). In other words, the choice would be to continue with the tiered
process. If, however, the variables described by preliminary distribution are important sources of
variability or uncertainty in the risk estimate, then this information should be presented in the scientific
management decision point (see Figure 2-2). The uncertainty may be sufficiently important in the risk
management decision to warrant additional data collection efforts. Conversely, it may be necessary to
exit the tiered process if the uncertainty cannot be reduced. Although the tiered process may be stopped
at this point, it can still be informative to present the results from the PRA. For example, information
about uncertainty may affect the choice of the percentile used to characterize the RME risk. In addition,
it may be appropriate to weight the results of the point estimate analysis more heavily in the risk
management decision when uncertainty in the PRA is high. Further guidance on appropriate choices for
distributions based on the information available to characterize variability is given in Appendix B.
PRA also may be warranted if it would be beneficial to know where on the risk distribution the
point estimate lies. An example of this would be a risk estimate that is within the NCP risk range of
1E-04 to 1E-06. The assessment may be sufficient to support risk management decisions if it could be
shown that the point estimate of risk lies sufficiently high in the risk distribution. For example, a "no
further action" decision may be strengthened if the point estimate is at the 99th percentile of the risk
distribution, if risks in lower percentiles of the RME risk range are below the NCP risk range, and if there
is high confidence in the risk result. This type of evaluation can be conducted using PRA techniques.
Even if the RME point estimate of risk exceeds the risk level of concern, and PRA is not needed
to confirm this result, information from a PRA can be helpful in determining a strategy for achieving a
protective preliminary remediation goal (PRO). A detailed discussion of the use of PRA in setting
remediation action levels is given in Chapter 5. The advantages and disadvantages of the point estimate
approach and PRA are presented in Chapter 1 (Exhibits 1-5 and 1-6).
2.3.3 TIER 2
Tier 2 of the tiered approach to risk assessment will generally consist of a simple probabilistic
approach such as one-dimensional Monte Carlo analysis (1-D MCA). A 1-D MCA is a statistical
technique that may combine point estimates and probability distributions to yield a probability
distribution that characterizes variability or uncertainty in risks within a population (see Chapter 1).
Guidance for selecting and fitting distributions is presented in Appendix B. Typical elements of a Tier 2
risk assessment, as they relate to higher and lower tiers are presented in Exhibit 2-5. A more detailed
discussion of these elements can be found in Chapters 3 and 4, and Appendix A (Sensitivity Analysis).
Page 2-14
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
While most of the Tier 2 assessments
are expected to use 1-D MCA to characterize
variability in risk, sometimes a 1-D MCA of
uncertainty may be of interest. For example,
as suggested in Exhibit 2-5, a probability
distribution for uncertainty in the arithmetic
mean or median (i.e., 50th percentile) for
selected input variables may be specified in a
1-D MCA to yield a probability distribution
for uncertainty for the central tendency risk
estimate. However, as most Tier 2
assessments are expected to combine input
distributions for variability, this guidance
focuses on 1-D MCA for characterizing
variability in the risk estimate.
Decision Alternatives
EXHIBIT 2-5
TYPICAL ELEMENTS OF TIER 2 RISK ASSESSMENT
Analysis Tool - 1-D MCA
Variability Modeling - full characterization of
variability in risk using PDFs or PMFs for input
variables
Uncertainty Modeling - semi-quantitative estimate
of uncertainty using iterative 1-D MCA simulations
of variability, or a single 1-D MCA of uncertainty in
the CTE risk
Sensitivity Analysis - varying multiple variables
with probability distributions gives a quantitative
ranking (e.g., correlation coefficient) of the relative
contributions of exposure pathways and variables to
CTE or RME risk
Risk-Based Decision-Making Output - risk
distribution for variability: Does the risk level of
concern fall within an acceptable range on the risk
distribution (i.e., RME range)? Also, risk
distribution for uncertainty: What is the 90%
confidence interval for the CTE risk?
Generally, the three main questions to
consider when determining whether the results
of a 1-D MCA are sufficient for risk
management decisions are: (1) What is the
RME risk range and how does it compare to
the level of concern?; (2) Where does the point
estimate risk lie on the risk distribution?; and
(3) What is the level of confidence in the risk
estimate? In Tier 2, similar to the point
estimate approach, the level of confidence in a single 1-D MCA risk distribution is generally addressed in
a qualitative or semi-quantitative way. As discussed in Chapter 1 (Section 1.2.4) and Chapter 3
(Section 3.4.1), one should avoid developing input distributions to a PRA model that yield a single risk
distribution that intermingles, or represents both variability and uncertainty. In Tier 2, the preferred
approach for characterizing uncertainty in the risk estimate is to perform multiple 1-D MCA simulations
(of variability), which uses a different point estimate for uncertainty for one or more parameters,
combined with probability distributions for variability for one or more variables. Chapter 3 (see
Table 3-2 and Figures 3-3 and 3-4) presents an example of iterative 1-D MCA simulations using
combinations of point estimates characterizing uncertainty for two variables. More advanced PRA
techniques such as two-dimensional Monte Carlo analysis (2-D MCA), in which distributions for
variability and uncertainty are propagated separately through an exposure model, can be undertaken in
Tier 3 (Appendix D).
In order to use a PRA to determine if risks are unacceptable and to develop preliminary
remediation goals (PRGs) that are protective of the RME individual (see Chapter 5), a single point from
the RME risk range should be selected (e.g., 95th percentile). In general, this can be accomplished by
selecting an estimate within the RME risk range based on the level of confidence in the output of the
1-D MCA. Uncertainty in risk estimates may be quantified or reduced by considering site-specific
factors, biological data, and toxicity data. Stakeholders can provide useful information about uncertain
variables and sources for site-specific data. More detailed guidance for choosing a percentile value within
the RME range is provided in Chapter 7.
Page 2-15
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
The evaluation of the risk assessment in a 1-D MCA in Tier 2 will yield one of two outcomes:
(1) sufficient for risk management decisions; or (2) insufficient for risk management decisions. If
determined to be sufficient, the risk manager can exit the tiered approach and complete the RI/FS process.
The results of a 1-D MCA may support a decision for "no further action" or for a "remedial action." A
"no further action" decision may result when the RME risk range (or a specified point in the RME risk
range) is clearly below the level of concern (e.g., Hazard Index=l) and confidence in the risk distribution
is high. A decision for remedial action may result when a national standard (e.g., MCLs applied to
groundwater) may be exceeded, or when the RME risk range (or a specified point in the RME risk range)
is clearly above the level of concern and confidence in the risk distribution is high. The decision for a
specific remedial action involves consideration of the NCP's nine evaluation criteria for remedial
decisions (U.S. EPA, 1990; see Chapter 1) and other site-specific factors.
An alternative conclusion at the end of a Tier 2 analysis would be that the results of the 1-D MCA
are not sufficient for risk management decisions. There are several factors that might support this
conclusion:
(1) The RME risk range is close to the NCP risk range and confidence in the risk distribution is
low. In this case, the risk manager might decide to not exit the tiered approach, and instead
continue taking appropriate steps to increase the confidence in the risk estimate.
(2) Uncertainty is high and it is believed that more than one variable is a major contributor to the
uncertainty in the risk estimate. It can be difficult to explore uncertainty in more than one
variable using 1-D MCA simulations of variability, even using iterative approaches discussed
in Chapter 3 (Section 3.4.1).
(3) Results of the point estimate risk assessment differ significantly from the results of the
1-D MCA. While the RME risk estimates are not expected to be identical, typically the RME
point estimate will correspond with a percentile value within the RME range (i.e, 90th to
99.9th percentile) of the risk distribution. If the RME point estimates fall outside this range,
further steps may be warranted to evaluate the choices for input variables—both the RME
point estimates, and the probability distributions and parameters (including truncation limits)
for the 1-D MCA.
The deliberation/decision cycle (Figure 2-2) between Tier 2 and Tier 3 is similar to the cycle
between Tier 1 and 2 and includes discussing the Tier 2 probabilistic sensitivity analysis, identifying data
gaps, communicating with stakeholders (e.g., to obtain site-specific information), discussing the potential
value of further analysis with probabilistic methods, work planning, and additional data collection. As
with the Tier 1 assessment, additional data collection should follow the DQO process (U.S. EPA, 1992a,
1993, 1994, 2000a) and point estimates of risk should be revisited with the new data. The
deliberation/decision cycle is an iterative process in which the level and complexity of the analysis
increases until the scope of the analysis satisfies decision-making needs. This iterative process should
continue until sufficient data are available to support risk management decisions. As in all tiers,
stakeholder involvement should be encouraged. Once a 1-D MCA for variability or uncertainty is
completed and is available for review and interpretation, a stakeholder meeting should be convened.
Interested stakeholders should be given the opportunity to review the 1-D MCA and provide comments.
Communication issues specific to PRA are discussed in Chapter 6 (Risk Communication).
In addition to identifying data gaps, consideration for a refined 1-D MCA or more advanced PRA
techniques may begin as a means of determining what benefits they may confer to the decision-making
Page 2-16
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
process. If, during further discussions of PRA, it is determined that information from a more advanced
PRA may influence the risk management decision, the use of an advanced PRA may be warranted. If
additional data have been collected, the point estimate and 1-D MCA should be refined. Specifically, an
advanced PRA may be warranted if it would be beneficial to characterize uncertainty in more than one
variable at a time. A 2-D MCA can simultaneously characterize variability and uncertainty in multiple
variables and parameter estimates. The decision to employ such advanced methods should be balanced
with considerations of resource constraints and the feasibility of reducing uncertainty in a given variable.
A detailed discussion of advanced PRA methods, including 2-D MCA, is provided in Appendix D.
2.3.4 TIER 3
Tier 3 of the tiered approach to risk assessment consists of advanced PRA methods, such as
2-D MCA, Microexposure Event Analysis (MEE), geostatistical analysis of concentration data, and
Bayesian statistics. Typical elements of a Tier 3 risk assessment are presented in Exhibit 2-6. A more
detailed discussion of these elements is given in
Appendix D. As in other tiers, Tier 3 includes
an iterative process of deliberation and decision
making in which the level and complexity of the
analysis increases until the scope of the analysis
satisfies decision-making needs. As in all tiers,
stakeholder involvement is encouraged.
EXHIBIT 2-6
TYPICAL ELEMENTS OF TIER 3 RISK ASSESSMENT
Generally, the various elements of the
deliberation/decision cycle for Tier 3 are the
same as those for Tier 1 and 2 (Figure 2-2). An
advanced PRA would be conducted and made
available for review to the risk manager and
stakeholders. The risk manager must determine
if the results of the advanced PRA are sufficient
for risk management decision making. Issues to
consider when making this determination are
similar to those identified for evaluating point
estimate risk results and 1-D MCA results, and
focus on evaluating the sources and magnitude
of uncertainty in relation to the established risk
level of concern. If the results are sufficient for
risk management decisions, the risk manager
may exit the tiered approach and complete the
RI/FS process. If the results are not found to be
sufficient for risk management decisions, data
gaps should be identified and if additional data
are collected, all stages of the risk assessment, including the advanced PRA, the 1-D MCA, and the point
estimate risk assessment, should be refined. Alternatively, additional advanced PRA methods may be
explored. Refer to Appendix D for a discussion of more advanced PRA techniques. Overall, analysis
should continue within Tier 3 until sufficiently informed risk management decisions can be made.
Analysis Tool - 2-D MCA, MEE, geostatistics, and
Bayesian statistics
Variability Modeling - full characterization using
PDFs or PMFs for input variables
Uncertainty Modeling - quantitative, segregating
uncertainty from variability, and associated with
multiple variables simultaneously
Sensitivity Analysis - varying parameters of
probability distributions to identify and rank order
parameter uncertainty with the same sensitivity
analysis methods used for Tier 2 (see Appendix A).
Also, explore alternative choices of probability
distributions and sources of model uncertainty.
Risk-based Decision-Making Criteria - risk
distribution for variability with confidence
limits—Does the risk level of concern fall within an
acceptable range on the risk distribution (i.e., RME
range), and with an acceptable level of uncertainty?
Page 2-17
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
2.3.5 FLEXIBILITY IN DEFINING TIERS
The assignment of specific analytical tools to Tiers 1, 2, and 3 (Figure 2-1 and Exhibits 2-4
through 2-6) results in generalizations that may not be applicable to all site assessments. Upon
completion of the deliberation phase between Tier 1 and Tier 2, the conclusion may be that analytical
tools in Tier 3 would be applicable and beneficial for addressing decision making issues. For example,
geospatial modeling may be beneficial for improving estimates of uncertainty in the exposure point
concentration or in designing field sampling plans to further reduce uncertainty. An improved estimate of
the 95% UCL from geospatial analysis (shown in Exhibit 2-6 as a Tier 3 analytical tool) would then be
integrated into a Tier 2 assessment, or the complete distribution for uncertainty in the mean concentration
could be incorporated into a 2-D MCA in Tier 3. Flexibility in defining the level of complexity of the
analysis used in a given tier is essential to accommodating the wide range of risk assessment issues likely
to be encountered. An important benefit gained from use of the tiered approach is to ensure a deliberative
process in the advancement of the assessment to higher levels of complexity. It is far more important that
a deliberative process take place and be documented, than it is to constrain a set of analytical tools to a
specific tier.
Page 2-18
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 2 ~ December 31, 2001
REFERENCES FOR CHAPTER 2
U.S. EPA. 1989. Risk Assessment Guidance for Superfimd (RAGS): Volume I. Human Health
Evaluation Manual (HHEM) (Part A, Baseline Risk Assessment). Interim Final. Office of
Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40 CFR
300: 55 Federal Register, 8666-8865, March 8.
U.S. EPA. 1991a. Risk Assessment Guidance for Superfund (RAGS), Volume 1: Human Health
Evaluation Manual (FIHEM), Part B, Development of Risk-Based Preliminary Remediation
Goals. Office of Emergency and Remedial Response, Washington, DC. EPA/540/R-92/003.
NTIS PB92-963333
U.S. EPA. 1991b. Role of the Baseline Risk Assessment in Superfimd Remedy Selection Decisions.
Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive
No. 9355.0-30.
U.S. EPA. 1992a. Guidance on Data Usability in Risk Assessment. Part A. Final. Office of Solid Waste
and Emergency Response, Washington, DC. OSWER Directive No. 9285.7.09A. NTIS
PB92-96336.
U.S. EPA. 1992b. Supplemental Guidance to RAGS: Calculating the Concentration Term. Office of Solid
Waste and Emergency Response, Washington, DC. OSWER Directive 9285.7-081.
U.S. EPA. 1993. Data Quality Objectives Process for Superfund: Interim Final Guidance. Office of
Research and Development, Washington, DC. EPA/540/R-93/071.
U.S. EPA. 1994. Guidance for the Data Quality Objectives Process (EPA QA/G-4). Office of Research
and Development, Washington, DC. EPA/600/R-96/055. September.
U.S. EPA. 2000a. Data Quality Objectives Process for Hazardous Waste Site Investigations. Office of
Environmental Information, Washington, DC. EPA/600/R-00/007. January.
U.S. EPA. 2000b. Peer Review Handbook: 2ndEdition. Science Policy Council. Washington, DC.
EPA/1 OO/B-00/001. December.
U.S. EPA. 2001. Risk Assessment Guidance for Superfund: Volume I. Human Health
Evaluation Manual, Part D: Standardized Planning, Reporting, and Review of Superfimd Risk
Assessments. Office of Emergency and Remedial Response. Washington, DC. OSWER Directive
No. 9285-47. December.
Page 2-19
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
CHAPTER 3
USING PROBABILISTIC ANALYSIS IN HUMAN HEALTH ASSESSMENT
EXHIBIT 3-1
GENERAL EQUATION FOR EXPOSURE
3.0 INTRODUCTION
This chapter outlines how probabilistic analysis may be applied to human health risk assessments
in the Environmental Protection Agency's (EPA) Superfund program. The paradigm for human health
risk assessment as described in EPA's Risk Assessment Guidance for Superfund (U.S. EPA, 1989),
includes data collection/evaluation in addition to exposure and toxicity assessment and risk
characterization. Although the strategies and methods used in collecting and analyzing data can
significantly impact the uncertainty in a risk estimate, they are issues relevant to risk assessment in
general, and are addressed in other guidance documents, such as EPA's Guidance for Data Useability in
Risk Assessment (U.S. EPA, 1992b). RAGS Volume 3: Part A focuses on a tiered approach for
incorporating quantitative information on variability and uncertainty into risk management decisions.
3.1 CHARACTERIZING VARIABILITY IN EXPOSURE VARIABLES
Exhibit 3-1 gives the general equation
used for calculating exposure, often expressed as
an average daily intake. In a point estimate
approach, single values (typically a mixture of
average and high-end values) are input into the
equation. In probabilistic risk assessment (PRA),
the only difference is that a probability
distribution, rather than single value, is specified
for one or more variables. A Monte Carlo
simulation is executed by repeatedly selecting
random values from each of these distributions
and calculating the corresponding exposure and
risk. For the majority of PRAs, it is expected that
probability distributions will be used to
characterize inter-individual variability, which
refers to true heterogeneity or diversity in a
population. Thus, variability in daily intake, for
example, can be characterized by combining
multiple sources of variability in exposure, such
as ingestion rate, exposure frequency, exposure duration, and body weight. Variability in chemical
concentrations (Chapter 5 and Appendix C) and the toxicity term in ecological risk assessment
(Chapter 4) may also be considered in risk calculations.
C x CR x EF x ED
BWx AT
Eq. 3-1
where,
I
C
CR
EF
ED
BW
AT
daily intake
contaminant concentration
contact rate (ingestion, inhalation,
dermal contact)
exposure frequency
exposure duration
body weight
averaging time
Page 3-1
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
EXHIBIT 3-2
DEFINITIONS FOR CHAPTER 3
95% UCL for mean - The one-sided 95% upper confidence limit for a population mean; if a sample of size (w) was
repeatedly drawn from the population, the 95% UCL will equal or exceed the true population mean 95% of the
time. It is a measure of uncertainty in the mean, not to be confused with the 95th percentile (see below), which is a
measure of variability. As sample size increases, the difference between the UCL for the mean and the true mean
decreases, while the 95th percentile of the distribution remains relatively unchanged.
95th percentile - The number in a distribution that is greater than 95% of the other values of the distribution, and less
than 5% of the values. When estimated from a sample, this quantity may be equal to an observed value, or
interpolated from among two values.
Arithmetic Mean (AM) - A number equal to the average value of a population or sample. Usually obtained by
summing all the values in the sample and dividing by the number of values (i.e., sample size).
Assessment Endpoint - The specific expression of the population or ecosystem that is to be protected. It can be
characterized both qualitatively and quantitatively in the risk assessment.
Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in the population,
usually considered to be the arithmetic mean or median of the risk distribution.
Credible Interval - A range of values that represent plausible bounds on a population parameter. Credible intervals may
describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95th percentile risk).
The term is introduced as an alternative to the term confidence interval when the methods used to quantify
uncertainty are not based entirely on statistical principles such as sampling distributions or Bayesian approaches.
For example, multiple estimates of an arithmetic mean may be available from different studies reported in the
literature—using professional judgment, these estimates may support a decision to describe a range of possible
values for the arithmetic mean.
CTE Risk - The estimated risk corresponding to the central tendency exposure.
Cumulative Distribution Function (CDF) - Obtained by integrating the PDF or PMF, gives the cumulative probability
of occurrence for a random independent variable. Each value c of the function is the probability that a random
observation x will be less than or equal to c.
Exposure Point Concentration (EPC) - The average chemical concentration to which receptors are exposed within an
exposure unit. Estimates of the EPC represent the concentration term used in exposure assessment.
Frequency Distribution/Histogram - A graphic (plot) summarizing the frequency of the values observed or measured
from a population. It conveys the range of values and the count (or proportion of the sample) that was observed
across that range.
High-end Risk - A risk descriptor representing the high-end, or upper tail of the risk distribution, usually considered to
be equal to or greater than the 90th percentile.
Low-end Risk - A risk descriptor representing the low-end, or lower tail of the risk distribution, such as the 5th or 25th
percentile.
Parameter - A value that characterizes the distribution of a random variable. Parameters commonly characterize the
location, scale, shape, or bounds of the distribution. For example, a truncated normal probability distribution may
be defined by four parameters: arithmetic mean [location], standard deviation [scale], and min and max [bounds].
It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithmetic mean
ingestion rate).
Probability Density Function (PDF) - A function representing the probability distribution of a continuous random
variable. The density at a point refers to the probability that the variable will have a value in a narrow range about
that point.
Probability Mass Function (PMF) - A function representing the probability distribution for a discrete random variable.
The mass at a point refers to the probability that the variable will have a value at that point.
Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site (U.S.
EPA, 1989). The intent of the RME is to estimate a conservative exposure case (i.e., well above the average case)
that is still within the range of possible exposures.
Page 3-2
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
EXHIBIT 3-2
DEFINITIONS FOR CHAPTER 3—Continued
Sensitivity Analysis - Sensitivity generally refers to the variation in output of a model with respect to changes in the
values of the model's input(s). Sensitivity analysis can provide a quantitative ranking of the model inputs based
on their relative contributions to model output variability and uncertainty. Common metrics of sensitivity include:
>• Pearson Correlation Coefficient - A statistic r that measures the strength and direction of linear
association between the values of two quantitative variables. The square of the coefficient (r2) is the
fraction of the variance of one variable that is explained by the variance of the second variable.
>• Sensitivity Ratio - Ratio of the change in model output per unit change in an input variable; also called
elasticity.
*• Spearman Rank Order Correlation Coefficient - A "distribution free" or nonparametric statistic r that
measures the strength and direction of association between the ranks of the values (not the values
themselves) of two quantitative variables. See Pearson (above) for r2.
Target Population - The set of all receptors that are potentially at risk. Sometimes referred to as the "population of
concern". A sample population is selected for statistical sampling in order to make inferences regarding the target
population (see Appendix B, Section B.3.1, Concepts of Populations and Sampling).
Page 3-3
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Figure 3-1 shows a hypothetical example of an input distribution for drinking water ingestion
rate. Assume that survey data for drinking water ingestion rates were compiled in order to select and fit a
probability distribution. One of the first steps in exploring the data set may be to plot a frequency
distribution. In the graph, the height of the bars (the y-axis) represents the relative frequency of ingestion
rates in the population and the spread of the bars (the x-axis) is the varying amounts of water ingested
(L/day). Since ingestion rate is a continuous random variable, the probability distribution can also be
represented graphically with a probability density function (PDF). Assume that the following parameters
are estimated from the sample: arithmetic mean=1.36, standard deviation=0.36, geometric mean=1.31,
and geometric standard deviation=1.30. These parameter estimates may be used to define a variety of
probability distributions, including a 2-parameter lognormal distribution. The fit of the lognormal
distribution can be evaluated by visual inspection using the PDF given by Figure 3-1, or by a lognormal
probability plot (see Appendix B).
The y-axis for a PDF is referred to as the probability density, where the density at a point on the
x-axis represents the probability that a variable will have a value within a narrow range about the point.
This type of graph shows, for example, that there is a greater area under the curve (greater probability
density) in the 1-2 L/day range than 0-1 L/day or 2-3 L/day. That is, most people reported consuming
1-2 L/day of drinking water. By selecting a lognormal distribution to characterize inter-individual
variability, we can state more precisely that 1 L/day corresponds to the 15th percentile and 2 L/day
corresponds to the 95th percentile, so approximately 80% (i.e., 0.95-0.15=0.80) of the population is likely
to consume between 1 and 2 L/day of drinking water.
0.025
700
0.000
0.0
1.0 2.0 3.0
Ingestion Rate (L/day)
4.0
Figure 3-1. Example of a frequency distribution for adult drinking water ingestion rates, overlaid by
a graph of the probability density function (PDF) for a lognormal distribution defined by the sample
statistics. The distribution represents inter-individual variability in water intakes and is characterized
by two parameters. Typically, the geometric mean (GM) and geometric standard deviation (GSD), or
the arithmetic mean (AM) and arithmetic standard deviation (SD) are presented to characterize a
lognormal distribution.
Page 3-4
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
3.1.1 DEVELOPING DISTRIBUTIONS FOR EXPOSURE VARIABLES
When site-specific data or representative surrogate data are available, a probability distribution
can be fit to that data to characterize variability. Appendix B describes how to fit distributions to data,
how to assess the quality of the fit and discusses topics such as the sensitivity of the tails of the
distribution to various PDFs, and correlations among variables. Many of the issues discussed below
regarding the use of site-specific data or surrogate data are relevant to both point estimate risk assessment
andPRA.
For the majority of the exposure variables, such as exposure duration, water intake rates, and
body weight, site-specific data will not be available. The risk assessor will have to either select a
distribution from existing sources, or develop a distribution from published data sets and data summaries.
Examples of sources for these distributions and data sets are EPA's Exposure Factors Handbook (U.S.
EPA, 1997a,b,c), Oregon Department of Environmental Quality's Guidance for Use of Probabilistic
Analysis in Human Health Risk Assessment (Oregon DEQ, 1998), and the scientific literature. An
appropriate PDF should be determined in collaboration with the regional risk assessor. The process by
which PDFs are to be selected and evaluated should be described in the workplan. EPA's Superfund
program is in the process of developing a ranking methodology to evaluate data representativeness
relevant to various exposures scenarios. Following peer review and project completion, the results will be
posted on EPA Superfund web page.
os- At this time, EPA does not recommend generic or default probability
distributions for exposure variables.
Regardless of whether a PDF is derived from site-specific measurements or obtained from the
open literature, the risk assessor should carefully evaluate the applicability of the distribution to the target
population at the site. The distribution selected should be derived from the target population or from a
surrogate population that is representative of the target population at the site. For example, a distribution
based on homegrown vegetable consumption in an urban population would not be representative for a
farming population in the Midwest. If such a distribution were to be used, (and no other data were
available), the uncertainty and bias that this PDF would impart to the risk estimate should be
communicated to the risk decision makers.
For purposes of risk management decision making, the significance of not having site-specific
data should be evaluated in the context of representativeness and sensitivity analysis. If published data
are representative of the potentially exposed population, then site-specific data may be unnecessary. For
example, body weights of children and adults have been well studied from national surveys and can
generally be considered reasonable surrogates for use in site risk assessments. Furthermore, even if a
variable is likely to vary among different exposed populations, it may not contribute greatly to the
variance or uncertainty in risk estimates. In this case, surrogate data may also be used with confidence in
the risk estimate. In addition, the PRA may be simplified by using point estimates instead of probability
distributions for the "less sensitive" exposure variables. In part, the decision to use a point estimate in
lieu of a probability distribution must balance the benefit of simplifying the analysis and the
communication process (see Chapter 6), against the reduction (however small) in the variance of the risk
distribution. The utility of sensitivity analysis in identifying the important factors in a risk estimate is
discussed further below and in Appendix A.
It is also important to evaluate the sample design and sample size when deciding to apply a
distribution to a specific site. Depending on the situation, a very large data set derived from a national
Page 3-5
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
population may be more useful than a site-specific data set derived from a small, incomplete, or poorly
designed study. Appendix B provides additional discussion on how to evaluate the data and studies that
form the basis for a distribution. Often, the question arises regarding the appropriateness of combining
data sets to derive a PDF. Before combining data sets, a careful evaluation should be made of the
representativeness of the study populations, and the similarity in study designs and quality. In addition,
statistical tests may be used to determine whether or not data sets are compatible with a common
probability distribution (Hedges and Olkin, 1985; Stiteler et al., 1993). In general, risk assessors should
be reluctant to combine data sets for the purpose of developing a PDF that characterizes variability. Due
to the number of potential differences inherent in the study design, alternative data sets may provide a
better measure of uncertainty in the probability distribution and parameter estimates, rather than a means
of increasing the overall sample size for defining a single probability distribution. For example, if
multiple data sets are available, a more informative approach may be to incorporate each data set into the
PRA in a separate analysis, as a form of sensitivity analysis on the choice of alternative data sets.
Each probability distribution used in a Monte Carlo Analysis (MCA) should be presented with
sufficient detail that the analysis can be reproduced (see Chapter 1, Section 1.4, Condition #2). This
information may be presented in tabular and/or graphical summaries. Important information for a
summary table would include a description of the distribution type (e.g., lognormal, gamma, etc.), the
parameters that define the distribution (e.g., mean and standard deviation, and possibly upper and lower
truncation limits for a normal distribution), units, and appropriate references (see Table 3-6, for example).
The table should also indicate whether the distribution describes variability or uncertainty. The report
should discuss the representativeness of the data and why a particular data set was selected if alternatives
were available. Graphical summaries of the distributions may include both PDFs and cumulative
distribution functions (CDFs), and should generally be used to document distributions that characterize
site-specific data.
3.1.2 CHARACTERIZING RISK USING PRA
Quantitative risk characterization involves evaluating exposure (or intake) estimates against a
benchmark of toxicity, such as a cancer slope factor or a noncancer hazard quotient. The general equation
used for quantifying cancer risk from ingestion of contaminated soil is shown in Exhibit 3-3, and the
equation for noncarcinogenic hazard is shown in Exhibit 3-4. A Hazard Index is equal to the sum of
chemical-specific Hazard Quotients.
At this time, this guidance does not propose probabilistic approaches for dose-response in human
health assessment and, further, discourages undertaking such activities on a site-by-site basis. Such
activities require contaminant-specific national consensus development and national policy development
(see Chapter 1, Section 1.4.1). Chapter 4 discusses methods for applying probabilistic approaches to
ecological dose-response assessment.
The probabilistic calculation of risk involves random sampling from each of the exposure
variable distributions. The output of this process is a distribution of risk estimates. When the calculation
of risk (or any other model endpoint) is repeated many times using Monte Carlo techniques to sample the
variables at random, the resulting distribution of risk estimates can be displayed in a similar fashion. The
type of summary graph used to convey the results of a MCA depends on the risk management needs. For
example, Chapter 1, Figure 1-3 shows how a PDF for risk might be used to compare the probabilistic
estimate of the RME risk (e.g., 95th percentile) with a risk level of concern. This type of summary can
also be used to effectively illustrate the relationship between the RME risk determined from point
estimate and probabilistic approaches.
Page 3-6
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Example for Soil Ingestion
where,
C
IR
CF
EF
Risk =
EXHIBIT 3-3
EQUATION FOR CANCER RISK
Risk = Dose x CSF
CxIRxCFxEFxED
BWxAT
CSF,
oral
concentration in soil (mg/kg) ED
soil ingestion rate (mg/day) BW
conversion factor (1E-06 kg/mg) AT
exposure frequency (days/year) CSF
exposure duration (years)
body weight (kg)
averaging time (days)
oral cancer slope factor (mg/kg-day)"1
where,
EXHIBIT 3-4
EQUATION FOR NONCANCER HAZARD QUOTIENT
7 _ Dose Concentration
Hazard Quotient = or
RfD
RfC
RfD
reference dose, oral or dermally adjusted (mg/kg-day)
reference concentration, inhalation (|J.g/m3)
RfC
In addition, the CDF can be especially informative for illustrating the percentile corresponding to
a particular risk level of concern (e.g., cancer risk of 1E-04 or Hazard Index of 1). Figure 3-2 illustrates
both the PDF and CDF for risk for a hypothetical scenario. Factors to consider when applying the PDF or
CDF are discussed in Chapter 1, Exhibit 1-3. When in doubt about the appropriate type of summary to
use, both the PDF and CDF should be provided for all risk distributions. At a minimum, each summary
output for risk should highlight the risk descriptors of concern (e.g., 50th, 90th, 95th, and 99.9th percentiles).
It can also be informative to include the results of the point estimate analysis—the risks corresponding to
the central tendency exposure (CTE) and the reasonable maximum exposure (RME).
Page 3-7
-------
RAGS Volume 3 Part A
Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
0.06
99th %ile = 1.8E-06
95th %ile = 1.2E-06
90th %ile = 9.2E-07
50th%ile = 4.1E-07
0.00
O.OE+OO
2.5E-06
3.0E-06
CD
£2
O
CD
'
E
^
O
1.00
0.80 -
0.60 -
0.40 -
0.20 -
0.00
99th%ile = 1.8E-06
95th%ile = 1.2E-06
90th %ile = 9.2E-07
50th%ile = 4.1E-07
O.OE+OO
5.0E-07
1.0E-06
1.5E-06
Risk
2.0E-06
2.5E-06
3.0E-06
Figure 3-2. Hypothetical PRA results showing a PDF (top panel) and CDF (bottom panel) for
cancer risk with selected summary statistics. The CDF rises to a maximum cumulative
probability of 1.0. The CDF clearly shows that the level of regulatory concern chosen for this
example (1E-06) falls between the 90th and 95th percentiles of the risk distribution.
Page 3-8
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
3.2 ROLE OF THE SENSITIVITY ANALYSIS
Prior to conducting a PRA, it is worthwhile to review several points pertaining to the sensitivity
analysis. As shown in Chapter 2 (Figures 2-1 and 2-2), sensitivity analysis can play an important role in
decision making at each tier of the tiered process. Beginning with Tier 1, a point estimate for risk should
be calculated prior to conducting a PRA. Based on the results of the point estimate, the risk assessor and
risk decision makers should determine whether a probabilistic analysis will offer additional benefit. One
factor in this decision may be the results of a sensitivity analysis. A primary objective of the sensitivity
analysis is to determine which variables and pathways most strongly influence the risk estimate. At many
Superfund sites, an estimate of cumulative risk considers contamination in multiple media, moving
through multiple pathways and interacting with a number of receptors. Depending on the complexity of
the site, and the modeling approaches, a risk assessment may involve one exposure pathway and few
variables, or multiple pathways with many variables (e.g., multimedia fate and transport models).
However, resources and time are often limited. The sensitivity analysis is invaluable in focusing these
limited resources on the most influential variables and pathways.
Several methods for conducting sensitivity analysis are described in Appendix A. It is important
to note that when a sensitivity analysis is performed and the major variables are identified, this does not
mean that the less influential pathways and variables should be eliminated from the risk assessment. It
means that because they are not major contributors to the variability or uncertainty in risk, they can be
described with point estimates without affecting the risk management decision. If distributions are
readily available for these less influential variables, one may use distributions. The key goal is to provide
a comprehensive risk characterization that is scientifically credible and sufficient for risk decision
making. The time and effort required to achieve various levels of complexity should be weighed against
the value of the information provided to the risk managers.
Additionally, if a variable is specified as influential in the sensitivity analysis, this does not
automatically mean that a distribution has to be developed for this variable. If the risk assessor feels that
data are simply not sufficient from which to develop a distribution, then a plausible point estimate can be
used. The risk assessor should be aware of a possible problem arising from using point estimates in the
absence of data adequate to support a distribution. If a variable has the potential to significantly impact
the risk outcome, and a very high-end or low-end point estimate is used in the PRA, this has the potential
to right-shift or left-shift the final distribution of risk. Even though there might not be enough data to
develop a distribution of variability for an influential variable, it would be prudent to communicate the
importance of this data gap to the risk decision makers, and perhaps run multiple simulations with several
plausible input distributions for that variable. Communication of this uncertainty may persuade the risk
decision makers to collect additional data to better define the variable.
Page 3-9
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
3.3 EXPOSURE POINT CONCENTRATION TERM
A brief discussion of the concentration term is provided below. A more complete discussion of
the concentration term in PRA is provided in Appendix C. The reader is also referred to Chapter 5 on
development of PRGs.
The major source of uncertainty in Superfund risk assessments is often incomplete knowledge of
the concentration of one or more chemicals in various exposure media. In any risk assessment, the
derivation of the concentration term will reflect assumptions about: (1) properties of the contaminant,
(2) the spatial and temporal variability in contamination, (3) the behavior of the receptor, and (4) the time
scale of the toxicity of the chemical(s).
Contaminant concentrations contacted by a receptor are likely to vary depending on the spatial
variability of contamination and the movements of the receptor. Different individuals may be exposed to
different concentrations based on inter-individual variability in activity patterns. If information regarding
activity patterns is unavailable, receptors are typically assumed to exhibit random movement such that
there is an equal probability of contacting any area within the exposure unit (EU). An EU is defined as
the geographical area in which a receptor moves and contacts contaminated medium during the period of
the exposure duration. In addition, in Superfund risk assessments, the toxicity criteria are often based on
health effects associated with chronic exposure (e.g., lifetime risk of cancer following chronic daily intake
over a period of 30 years). Hence, the most appropriate expression for the concentration term, for the
majority of risk assessments, is one that characterizes the long-term average exposure point concentration
within the EU.
us- The most appropriate expression of the exposure point concentration term
for chronic exposure will characterize the long-term average concentration
experienced by a receptor within the exposure unit.
In point estimate risk assessments, the exposure point concentration term is usually calculated as
the 95% upper confidence limit (95% UCL) of the arithmetic mean because of the uncertainty associated
with estimating the true (i.e., population) mean concentration at a site. If the sampling density is sparse
relative to the size of the EU, the uncertainty may be high due to the relatively small number of
measurements available to estimate the mean concentration within the EU. The decision to use the upper
confidence limit to define the concentration term introduces a measure of protectiveness by reducing the
chance of underestimating the mean. Although there will be situations in which modeling variability in
concentration will be the appropriate choice (e.g., non-random movement within an EU, acute exposure
events, migration of groundwater contaminant plume, migration offish, etc.), in most cases,
characterization of the concentration term will focus on uncertainty. Appendix C provides a more
complete discussion on characterizing both variability and uncertainty in the concentration term.
Table 3-1 summarizes a number of appropriate methods for characterizing uncertainty in the parameter of
an exposure variable, such as the arithmetic mean of the concentration term.
Page 3-10
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
3.4 CHARACTERIZING UNCERTAINTY IN EXPOSURE VARIABLES
Uncertainty is described as a lack of knowledge about factors affecting exposure or risk. To
evaluate regulatory options, risk assessors are expected to translate the available evidence, however
tentative, into a probability of occurrence of an adverse health effect. Data from a sample or surrogate
population are used to develop estimates of exposure and risk in a specific target population (see
Section 3.1.4 and Appendix B, Section B.3.1). This extrapolation requires assumptions and inferences
that have inherent strengths and limitations, and may bias the outcome of the risk estimate. For example,
a common assumption in risk assessments for carcinogens is that a contaminant concentration within the
boundaries of a hazardous waste site represents the concentration that a receptor is exposed to throughout
the period of exposure, with the corresponding dose averaged over the course of a lifetime. This
assumption may be conservative (i.e., result in overestimation of exposure) if it is unlikely that receptors
will be exposed at the hazardous waste site for the entire exposure duration. It is incumbent on the risk
assessor to clearly present the rationale for the assumptions used in a risk assessment, as well as their
implications and limitations.
U.S. EPA guidance, including the Exposure Assessment Guidelines (U.S. EPA, 1992a), Exposure
Factors Handbook (U.S. EPA, 1997a,b,c), and Guiding Principles for Monte Carlo Analysis (U.S. EPA,
1997d) have classified uncertainty in exposure assessment into three broad categories:
(1) Parameter uncertainty - uncertainty in values used to estimate variables of a model;
(2) Model uncertainty - uncertainty about a model structure (e.g., exposure equation) or intended
use; and
(3) Scenario uncertainty - uncertainty regarding missing or incomplete information to fully
define exposure.
Each source of uncertainty is described in detail below, along with strategies for addressing them in PRA.
3.4.1 PARAMETER UNCERTAINTY
Parameter uncertainty may be the most readily recognized source of uncertainty that is quantified
in site-specific risk assessments at hazardous waste sites. Parameter uncertainty can occur in each step of
the risk assessment process from data collection and evaluation, to the assessment of exposure and
toxicity. Sources of parameter uncertainty may include systematic errors or bias in the data collection
process, imprecision in the analytical measurements, and extrapolation from surrogate measures to
represent the parameter of interest. For example, soil data collected only from the areas of highest
contamination, rather than the entire area that a receptor is expected to come into contact, will result in a
biased estimate of exposure.
In general, parameter uncertainty can be quantified at any stage of the tiered process, including
point estimate analysis (Tier 1), one-dimensional Monte Carlo analysis (1-D MCA) (Tier 2), and two-
dimensional Monte Carlo analysis (2-D MCA) (Tier 3). In the point estimate approach, parameter
uncertainty may be addressed in a qualitative manner for most variables. For example, the uncertainty
section of a point estimate risk assessment document might state that an absorption fraction of 100% was
used to represent the amount of contaminant in soil absorbed from the gastrointestinal (GI) tract, and as a
result, the risk estimate may overestimate actual risk. In addition, a sensitivity analysis may be
performed, wherein one input variable at a time is changed, while leaving the others constant, to examine
the effect on the outcome. In the case of absorption from the GI tract, different plausible estimates of the
Page 3-11
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
high-end, or RME absorption fraction might be used as inputs to the risk equation. The differences in the
risk estimates would reflect uncertainty in the RME absorption fraction.
Quantitative approaches for characterizing parameter uncertainty in exposure variables in a
Monte Carlo simulation are summarized in Table 3-1. If uncertainty in only a few parameter values is of
interest, multiple 1-D MCA simulations can yield the same results as a 2-D MCA simulation, but without
the time and effort of a 2-D MCA. An example illustrating this concept is given in Table 3-2. With
multiple 1-D MCA simulations, variability is characterized in one or more variables using probability
distributions for variability (PDFv's), and uncertainty in a parameter is characterized with a series of
different point estimates from a probability distribution for uncertainty (PDFu) (e.g., 95% lower
confidence limit LCL [95% LCL], sample mean, and 95% UCL). In a 2-D MCA simulation, variability is
characterized in one or more variables using PDFv's, and uncertainty in one or more parameters is
characterized with PDFu's. With both approaches, the influence of the parameter uncertainty can be
presented as a credible interval or confidence interval (CI) around the risk distribution, depending on how
the PDFu's are defined. When only a few sources of parameter uncertainty are quantified, multiple
1-D MCA simulations are preferred over a 2-D MCA because the approach is easier to use and
communicate. However, if the goal is to explore the effect that many sources of parameter uncertainty
may have on the risk estimates simultaneously, a 2-D MCA is preferred. Iterative 1-D MCA simulations
with different combinations of confidence limits may be impractical.
Table 3-1. Methods for Characterizing Parameter Uncertainty with Monte Carlo Simulations.
Approach
Single Point
Estimate
Multiple Point
Estimates
Parametric
PDFu1
Non-parametric
PDFu
Example of Model Input
• 95% UCL
• 95% LCL
• sample mean
• 95% UCL
PDFu for the mean based on the
sampling distribution, derived from
a Student's /-distribution.
PDFu for the mean based on
bootstrap resampling methods.
Method
1-D MCA
1-D MCA
2-D MCA
2-D MCA
Example of Model Output
PDFv1 for risk, calculated using the 95%
UCL for one parameter.
Three PDFv's for risk, representing the
90% CI for each percentile of the risk
distribution.2 The 90% CI only accounts
for uncertainty in a single parameter (not
multiple parameters).
One PDFv for risk with confidence
intervals at each percentile of the risk
distribution. The CI reflects uncertainty in
one or more parameters.
Same as parametric probability distribution
for uncertainty.
'Probability distribution for uncertainty (PDFu) and probability distribution for variability (PDFv).
2The 95% UCL for the concentration term represents a 1-sided confidence interval (CI), meaning there is a 95% probability that
the value is greater than or equal to the mean. Similarly, the 95% LCL would represent the 1-sided CI in which there is a 95%
probability that the value is less than or equal to the mean. Both values are percentiles on the probability distribution for
uncertainty (PDFu), also called the sampling distribution for the mean. Together, the 95% LCL and 95% UCL are equal to the
2-sided 90% confidence interval only for cases in which the PDFu is symmetric. For example, the sampling distribution for the
arithmetic mean of a sample from a normal distribution with an unknown variance is described with the symmetric Student's
/-distribution, whereas the PDFu for the mean of a lognormal distribution is asymmetric. In order to compare the results of
multiple 1-D MCA simulations and a 2-D MCA simulation, the same methodology should be employed to define the PDFu and
the corresponding confidence limits.
It is generally incorrect to combine a PDFu for one parameter (e.g., mean of the concentration
term) with one or more PDFv's in other exposure factors when conducting a 1-D MCA for variability.
Page 3-12
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
However, distributions for uncertainty and variability may be appropriately combined in a 2-D MCA. As
discussed in Appendix D, with 2-D MCA, a clear distinction should be made between probability
distributions that characterize variability (PDFv) and parameter uncertainty (PDFu). A 2-D MCA
propagates the uncertainty and variability distributions separately through an exposure model, thereby
making it possible to evaluate the effect of each on the risk estimates.
Example: Comparison of Multiple Point Estimates of Uncertainty in 1-D MCA, and Distributions of
Uncertainty in 2-D MCA
Table 3-2 illustrates an application of the approaches presented in Table 3-1 for quantifying
variability and parameter uncertainty. This is a hypothetical example, and no attempt was made to use
standard default assumptions for exposure variables. Two sources of variability are quantified: (1) inter-
individual variability in exposure frequency (EF), characterized by a triangular distribution, and (2) inter-
individual variability in exposure duration (ED), characterized by a truncated lognormal distribution. In
addition, two sources of uncertainty are presented: (1) a point estimate for soil and dust ingestion rate,
intended to characterize the RME; and (2) an upper truncation limit of the lognormal distribution for ED,
intended to represent a plausible upper bound for the exposed population. Methods for quantifying these
sources of uncertainty are discussed below. Additional sources of uncertainty may also have been
explored. For example, the choice of a triangular distribution for a PDFv may be provocative for some
risk assessors, since there are few cases in which empirical data suggest a random sample is from a
triangular distribution. Nevertheless, triangular distributions may be considered rough, or "preliminary"
distributions (see Chapter 2 and Appendix B, Section B.2) for cases when the available information
supports a plausible range and central tendency.
The choice of distributions is a potential source of uncertainty that can be explored by rerunning
simulations with each alternative, plausible choice, and examining the effect on the RME risk.
Simulations with preliminary simulations may yield at least three different outcomes. First, this type of
sensitivity analysis can help guide efforts to improve characterizations of variability for selected variables
that have the greatest affect on the risk estimates. Second, results may provide justification to exit the
tiered process without continuing with additional Monte Carlo simulations since further effort would be
unlikely to change the risk management decision. Finally, if the major sources of uncertainty can be
clearly identified, a subset of the less sensitive variables may be defined by point estimates without
significantly reducing the uncertainty in the risk estimates.
Parameter uncertainty can be quantified for both point estimates and PDFv's. In this example,
both types of inputs (i.e., point estimates and PDFv's) are presented as sources of parameter uncertainty:
the RME point estimate for soil and dust ingestion rate (IRsd), and the upper truncation limit on a PDFv
for ED. For IRsd, assume that three different studies provide equally plausible values for the RME: 50,
100, and 200 mg/day. A uniform PDFu is specified to characterize this range of plausible values. For
ED, assume that the maximum value reported from a site-specific survey was 26 years, but surrogate data
for other populations suggest the maximum may be as long as 40 years. A uniform PDFu is specified to
characterize this range of plausible values as well.
In Cases 1-3, the impact of uncertainty in IRsd and ED was evaluated using a series 1-D MCA
simulations. Inputs for uncertain parameters associated with IRsd and ED in Case 1, 2, and 3 represent
the minimum, central tendency, and maximum values, respectively. Each simulation yields a different
risk distribution based on different combinations of point estimates for parameters. Although a PDFu was
specified for IRsd, it would have been incorrect to combine the PDFu with the PDFv's for EF and ED in a
Page 3-13
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
1-D MCA because the result would have been a single distribution of risk that co-mingled uncertainty and
variability.
In Case 4, a single 2-D MCA simulation was run using the PDFu's for uncertainty and the
PDFv's for variability. By propagating variability and uncertainty separately, the 2-D MCA yields a
series of distributions of risk, from which credible intervals can be calculated for each percentile of the
CDF.
„ , CxIRxCFxEFxED __„
Risk = x CSF,
BW x AT °ral
Table 3-2. Example of 1-D MCA and 2-D MCA.
Variable
C (mg/kg)
IRsd
(mg/day)
CF (kg/mg)
EF
(days/year)
ED (years)
BW (kg)
AT (days)
CSF
(mg/kg-day)"1
Type of
Input
pt estimate
pt estimate
PDFu for
pt estimate
pt estimate
PDFv
PDFv
PDFu for
parameter of
PDFv
pt estimate
pt estimate
pt estimate
1-D MCA
Casel
500
50
~
1E-06
triangular
min =200
mode = 250
max = 350
T-lognormal
mean = 9
stdv = 10
max = 26
~
70
25550
1E-01
Case 2
500
100
~
1E-06
triangular
min =200
mode = 250
max =350
T-lognormal
mean= 9
stdv = 10
max= 33
~
70
25550
1E-01
Case 3
500
200
-
1E-06
triangular
min =200
mode = 250
max =350
T-lognormal
mean= 9
stdv = 10
max = 40
~
70
25550
1E-01
2-D MCA
Case 4
500
see below
uniform (50, 200)a
1E-06
triangular
min =200
mode = 250
max = 350
T-lognormal
mean = 9
stdev = 10
max = PDFu (see below)
max ~ uniform (26, 40)b
70
25550
1E-01
Uncertainty in the RME point estimate, defined by a uniform distribution with parameters (minimum, maximum).
bUncertainty in the upper truncation limit of the lognormal distribution, defined by a PDFv with parameters (mean, standard
deviation, maximum) and a PDFu for the maximum defined by a uniform distribution with parameters (minimum,
maximum).
Page 3-14
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Monte Carlo Simulation Results
Figures 3-3 and 3-4 illustrate CDFs for risk produced from Monte Carlo simulations using
Crystal Ball® 2000. The 1-D MCA simulations (Figure 3-3) were run with 10,000 iterations and Latin
Hypercube sampling. The 2-D MCA simulation (Figure 3-4) was run with 250 iterations of the outer
loop (uncertainty) and 2,000 iterations of the inner loop (variability). Details regarding 2-D MCA
simulation are given in Appendix D.
Figure 3-3 shows CDFs for risk based on three simulations of a 1-D MCA simulation. Each
simulation used a different combination of plausible estimates of the RME value for IRsd and the upper
truncation limit for ED, as discussed above. The results provide a bounding estimate on the risk
distribution given these two sources of uncertainty. The 95th percentile risk, highlighted as an example of
the RME risk estimate, may range from approximately 7E-06 to 3.5E-05.
Figure 3-4 shows a single CDF for risk, representing the central tendency risk distribution. This
CDF was derived by simulating uncertainty in the risk distribution using 2-D MCA. For this example, the
2-D MCA yields 250 simulations of the risk distributions for variability, so that there are 250 plausible
estimates of each percentile of the risk distribution. In practice, more than 250 simulations may be
needed to adequately quantify uncertainty in the risk distribution. Results of a 2-D MCA can be
presented as probability distributions of uncertainty, or box-and-whisker plots of uncertainty at selected
percentiles of the risk distributions. Figure 3-4 shows the central tendency (50th percentile) estimate of
uncertainty for the entire CDF of risk. In addition, a box-and-whisker plot is shown at the 95th percentile
of the CDF. Selected statistics for the box-and-whisker plot are included in a text box on the graphic (i.e.,
minimum; 5th, 50th, and 95th percentiles, and maximum). The 90% credible interval is given by the 5th and
95th percentiles. For this example, the 90% credible interval for the 95th percentile of the risk distribution
is: [9.1E-06, 3.1E-05].
Figures 3-3 and 3-4 demonstrate that the two approaches (i.e., multiple 1-D MCA and 2-D MCA)
can yield the same results. However, when there are numerous sources of uncertainty, 2-D MCA offers at
least two advantages over multiple 1-D MCA simulations: (1) 2-D MCA allows the multiple sources of
uncertainty to be included simultaneously so the approach is more efficient than a series of 1-D MCA
simulations; and (2) multiple 1-D MCA simulations yield multiple estimates of the RME risk, but it is not
possible to characterize the uncertainty in the RME risk in quantitative terms; a 2-D MCA yields a PDFu
for RME risk, which allows for statements regarding the level of certainty that the RME risk is above or
below a risk level of concern.
The 95th percentile is a focus of this example because it is a recommended starting point for
determining the risk corresponding to the RME. Chapter 7 provides guidance to the risk decision makers
on choosing an appropriate percentile (on a distribution of variability) within the RME risk range (90th to
99.9th percentiles). The chapter also includes a qualitative consideration of the uncertainty or confidence
surrounding a risk estimate in the decision-making process.
Page 3-15
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Figure 3-3
1.00
range of
uncertainty
[7E-06, 3.5E-05]
0.00
1.0E-07 1.0E-06 1.0E-05
Risk
1.0E-04
Figure 3-4
•\ nn
I .UU
Oqn
Q) >
> .*;
u ~~ n pn
ra 5 °-bU
"= ™
f -0
C Q
3 C n yn
o a. u-'u
0 fiO
Ocn
95th %ile ^,1
* T
-
Case 4
— *—
_^
^— • —
/
/
/•
.
Uncertainty
in 95th %ile
min = "
5th = £
50th =
95th = :
max= ;
7.7 E-06
3.1 E-06
.9 E-05
3.1 E-05
3.4 E-05
^
| 90% Cl ^.
[9.1 E-06, 3.1 E-05]
O.OE+00 1.0E-05 2.0E-05
Risk
3.0E-05
Page 3-16
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
3.4.2 SCENARIO AND MODEL UNCERTAINTY
All models are simplified representations of complex biological and physical processes. As such,
they, and the scenarios to which they are applied, may introduce a significant source of uncertainty into
an exposure and risk estimate. Models may exclude important variables or important pathways of
exposure, ignore interactions between inputs, use surrogate variables that are different from the target
variables, or they may be designed for specific scenarios and not others. As a result, a model may not
adequately represent all aspects of the phenomena it was intended to approximate or it may not be
appropriate to predict outcomes for a different type of scenario. For example, a model intended to
estimate risk from continuous, steady state exposures to a contaminant may not be appropriate or
applicable for estimating risk from acute or subchronic exposure events. In any risk assessment, it is
important to understand the original intent of a model, the assumptions being made in a model, what the
parameters represent, and how they interact. Based on this knowledge, one can begin to understand how
representative and applicable (or inapplicable) a model may be to a given scenario. If multiple models
exist that can be applied to a given scenario, it may be useful to compare and contrast results in order to
understand the potential implications of the differences. The use of multiple models, or models with
varying levels of sophistication, may provide valuable information on the uncertainty introduced into a
risk estimate as the result of model or scenario uncertainty. The collection of measured data as a reality
check against a given parameter or the predicted model outcome (such as the collection of vegetable and
fruit contaminant data to compare against modeled uptake into plants) is also useful in attempting to
reduce or at least gain a better understanding of model and scenario uncertainty.
3.5 EXAMPLE OF PRA FOR HUMAN HEALTH
The following hypothetical example provides a conceptual walk-through of the tiered approach
for PRA in Superfund risk assessment. The example begins with a baseline human health point estimate
risk assessment (Tier 1) and moves to Tier 2, in which multiple iterations of a 1-D MCA are run using
default and site-specific assumptions for input distributions. The general concepts associated with the
tiered approach are discussed in Chapter 2, and a similar example for ecological risk assessment is given
in Chapter 4. The 1-D MCA results are based on simulations with Crystal Ball® 2000 using
10,000 iterations and Latin Hypercube sampling. These settings were sufficient to obtain stability (i.e.,
<1% difference) in the 95% percentile risk estimate. The example is presented in Exhibit 3-5. Tables and
figures supporting the example are given immediately following the exhibit.
Page 3-17
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
EXHIBIT 3-5
USING THE TIERED PROCESS FOR PRA
HYPOTHETICAL CASE STUDY FOR HUMAN HEALTH RISK ASSESSMENT
RI Planning/Scoping/Problem Formulation/Data Collection
Site Description: Former federal facility
Site Size: 100 acres (5 acres within spill area (ISA); 95 acres outside spill area (OSA))
Stakeholders: Refuge employees, environmental activists, etc.
Land Use: Future wildlife refuge
Receptors: Future wildlife refuge workers (i.e., ornithologists and fishery biologists)
Sampling Data: n=35 surface soil samples (see Figure 3-5 for sample locations)
Chemical of Concern: ChemX
Chemical Properties: Nonvolatile
Toxicological Properties: Carcinogen: CSForaland CSFdermal= 5.5E-02, CSFinh= 2.73E-02;
Noncarcinogenic health data are lacking
Risk Level of Concern: 1E-04 for cancer
Tier 1 Point Estimate - Baseline Risk Assessment
Exposure Unit: (see Figure 3-5) ornithologist (exposed in OSA) and fishery biologist
(exposed in ISA)
Exposure Pathways: Ingestion of soil/dust; inhalation of fugitive dust, dermal absorption
Concentration Term: 95% UCL for arithmetic mean (Table 3-3)
Risk Equations: Exhibit 3-6
Exposure Parameters: Table 3-4
Results: Table 3-5
Is the Information Sufficient for Risk Management Decisions?
Sensitivity Analysis
Discussion
Identify Data
Gaps/Needs
Communication
With Stakeholders
PRA
Discussion
Work
Planning
Collect
Data
Stakeholder meeting is convened—point estimate results are discussed and ideas are exchanged
as follows:
• Risk estimates are expected to be conservative due to the use of standard default
exposure parameters, but are the defaults representative?
• Stakeholders are concerned about risk to workers and about the consequences of
remediation (e.g., negative impacts on habitat and potential job losses).
• Stakeholders are concerned about the relevance of some nonsite-specific exposure
variables (e.g., exposure duration), but are not sure which variables to investigate
further (i.e., which is the most influential?).
• Results of the sensitivity analysis from point estimate risk assessment cannot
identify where the high end risk estimate falls on the risk distribution.
• There is sufficient information (e.g., arithmetic mean, standard deviation,
percentiles) for some of the exposure variables to develop initial probability
distributions to characterize variability.
(continued on next page)
Page 3-18
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
(continued)
Is the Information Sufficient for Risk Management Decisions? (continued)
Inside Spill Area (ISA)
Outside Spill Area (OSA)
RME risk estimate: 2.4E-04 (Table 3-5)
RME percent contribution to risk by
pathway: Table 3-5 (inhalation adds a
minimal contribution to total risk, e.g.,
<1%)
RME risk estimate is greater than the level
of concern (1E-04) by a factor of 2.4
RME risk estimate is close to the level of
concern and therefore information may not
be sufficient
RME risk estimate: 6.6E-05 (Table 3-5)
RME percent contribution to risk by
pathway: Table 3-5
RME risk estimate is less than the level
of concern (1E-04) by a factor of 0.7
RME risk estimate is sufficient for risk
management decisions because point
estimate results are protective
No
Yes
Refine Point Estimate Analysis
Onh
No further changes to the point estimate are possible without more data.
Information from a PRA may influence the risk management decision by:
- Identifying where on the risk distribution the risk estimate falls.
- Identifying data gaps through a more advanced sensitivity analysis (i.e.,
which variables would benefit from additional data collection due to their
influence on the risk estimate?)
Complete
RI/FS Process
No
Tier 2 Probabilistic Estimate - Conduct a Preliminary 1-D MCA for Variability
Exposure Unit: Inside Spill Area (Fishery biologist) (see Figure 3-5)
Exposure Pathways: Soil ingestion and dermal absorption; inhalation excluded (< 1% of total risk)
Concentration Term: 95% UCL on arithmetic mean ISA (see Table 3-3)
Probability Distributions and Parameters: See Table 3-6
Results: See Table 3-7
(continued on next page)
Page 3-19
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
(continued)
Sensitivity Analysis
Discussion
Identify Data
Gaps/Needs
Communication
With Stakeholders
PRA
Discussion
Work
Planning
Collect
Data
5MDP> Is the Information Sufficient for Risk Management Decisions?
Stakeholder meeting is convened—1-D MCA results are discussed and ideas are exchanged:
• Sensitivity analysis from the 1-D MCA demonstrates that exposure duration, soil
ingestion rate, body weight, and adherence factor are the most sensitive variables (see
Figure 3-6).
• Additional data collection efforts for exposure duration data specific to fishery
biologists is feasible.
Preliminary PRA suggests that the Tier 1 RME point estimate risk in ISA (i.e., 2.4E-04)
corresponds with the 99th percentile of the risk distribution.
PRA results show that the RME risk range (i.e., 90th to 99.9th percentile) is 1E-04 to 4E-04.
Information from a preliminary 1 -D MCA may not be sufficient for a risk management
decision as the RME risk range is sufficiently close to the level of concern to warrant further
investigation.
No
Refined PRA
Analysis Only?.
RME risk range is sufficiently close to the level of concern to warrant
further investigation.
More rigorous process for fitting distributions to selected variables (e.g.,
IR_soil, SA_skin, etc.) may influence risk management decision, and level
of effort is reasonable; therefore proceed with a refined 1-D MCA.
Yes
(continued on next page)
Page 3-20
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Yes
(continued)
Tier 2 Refined PRA
- Conduct Refined 1-D MCA and Refined Point Estimate
Exposure Unit: Fishery biologist-inside spill area (ISA) (see Figure 3-5)
Exposure Pathways: Ingestion of soil and dust, and dermal absorption
Concentration Term: 95% UCL on arithmetic mean
Probability Distributions/Parameters: see Table 3-8 for sample data and summary statistics;
exposure duration defined by lognormal PDF (arithmetic mean=14, SD=9.4, upper
truncation of 44 years)
Results: see Table 3-9
Sensitivity Analysis
Discussion
Identify Data
Gaps/Needs
Communication
With Stakeholders
PRA
Discussion
Work
Planning
Collect
Data
5MDP> Is the Information Sufficient for Risk Management Decisions?
Stakeholders meeting is convened. Refined 1-D MCA results are discussed and ideas are
exchanged as follows:
• Sensitivity analysis from refined 1-D MCA indicates that the use of site-specific data
did not significantly alter the relative ranking or magnitude of rank correlations for
input variables (similar graphic as Figure 3-6).
• Refined 1-D MCA results suggest that the refined RME point estimate risk
corresponds with the 99th percentile of the risk distribution (Table 3-9).
• Refined 1-D MCA results show that the RME range (i.e., 90th to 99.9th percentile) is
1.6E-04 to 5E-04, with 95th percentile of 2.1E-04.
• Information from refined 1 -D MCA is sufficient for risk management decision
because the RME risk (95th percentile) is above the level of concern of 1E-04 using
site specific exposure duration data, and additional data collection on IR_soil term is
not warranted. Complete RI/FS process.
Yes
Complete RI/FS Process
Stakeholders and RPM decide that the best remedial
alternative is to remove surface soil in the 5 acre spill
area and cover the refuge area with clean fill before
beginning refuge construction.
Page 3-21
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Exposure Unit
Boundary
Soil Sample
Location
Hot Spot
Boundary
Figure 3-5. Site map for future wildlife refuge showing boundaries for the exposure
unit and potential hotspot, as well as sampling locations (n=35). Sample numbers
correspond with concentration data given in Table 3-3.
'The 95% UCL was estimated using the Land method (see Appendix C).
Table 3-3. Concentrations in Surface Soil (mg/kg).
Outside Spill Area
1088
646
3943
149
3704
845
488
387
1438
2502
(n=20)
305
2787
760
149
1088
837
1295
1239
1006
283
Inside Spill Area
1934
402
4215
1121
629
2293
257
288
57
228
(n=15)
970
985
743
158
21296
Summary Statistics
Mean
Standard Deviation
95% UCL1
Outside Spill Area
1247
1121
2303
Inside Spill Area
2372
5348
8444
Page 3-22
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
EXHIBIT 3-6
RISK EQUATIONS
Soil Ingestion
Risk = Cs x CF x IRs x FI x EF X ED x Oral CSF
BWxAT
Dermal Absorption
Risk = Cs x CF x SA x AF x ABS x EF X ED x Dermal-Adjusted CSF
BWxAT
Inhalation of Fugitive Dust
Risk = Csxl/PEFxIRaxETxEFXED x Inhalation CSF
BWxAT
Total Risk = Sum of risks from each exposure pathway (soil + dermal + inhalation)
Where:
Cs = Concentration of ChemX in soil (mg/kg)
IRs = Soil ingestion rate for receptor (mg/day)
FI = Fraction ingested from contaminated source (unitless)
CF = Conversion factor (1E-06 kg/mg)
SA = Skin surface area available for exposure (cnf/event)
AF = Soil to skin adherence factor for ChemX (mg/cm2)
ABS = Absorption factor for ChemX (unitless)
IRa = Inhalation rate for receptor (m3/hr)
PEF = Soil-to-air paniculate emission factor (kg/m3)
ET = Exposure time for receptor (hours/day)
EF = Exposure frequency for receptor (days/year)
ED = Exposure duration for receptor (years)
B W = Body weight of receptor (kg)
AT = Averaging time (years)
CSF = Cancer slope factor (oral, dermal, inhalation) (mg/kg-day)"
Page 3-23
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Table 3-4. Exposure Parameters used in Point Estimate Analysis.
Exposure
Variable
IRs
FI
CF
SA
AF
ABS
IRa
PEF
ET
EF
ED
BW
AT
CTE
Value
50
0.5
1E-06
3300
0.1
0.1
1.3
1.36E+09
8
200
5
70
25550
RME
Value
100
1
1E-06
3300
0.2
0.1
3.3
1.36E+09
8
225
25
70
25550
Units
mg/day
unitless
kg/mg
cmVevent
mg/cm2
unitless
mVhr
kg/m3
hours/day
days/year
years
kg
days
Reference
CTE: U.S. EPA, 1997a, p. 4-25
RME: U.S. EPA, 2001
Site-specific
Constant
U.S. EPA, 2001, 50th percentile value for all adult
workers — exposure to face, forearms, and hands
CTE: U.S. EPA, 1998; Table 3.3, value for
gardeners
RME: U.S. EPA, 2001
U.S. EPA, 1998, default for semi-volatile organic
compounds (SVOCs)
U.S. EPA, 1997a, p. 5-24, outdoor worker hourly
average: mean and upper percentile
U.S. EPA, 2001
Site-specific
CTE: Site-specific assumption
RME: U.S. EPA, 2001
CTE: U.S. EPA, 1993, p. 6
RME: U.S. EPA, 2001
U.S. EPA, 1993, p. 7
constant
CTE = central tendency exposure; RME = reasonable maximum exposure.
Table 3-5. Point Estimate Risks and Exposure Pathway Contributions.
Risk Estimate
by Exposure Pathway
Soil Ingestion
Dermal Absorption
Inhalation
Total Risk
Inside Spill Area (n = 15)
CTE
6.5E-06 (43 %)
8.6E-06 (57 %)
9.9E-10 (< 1 %)
1.5E-05
RME
1.5E-04(60%)
9.6E-05 (40 %)
1.4E-08 (< 1 %)
2.4E-04
Outside Spill Area (n = 20)
CTE
1.7E-06 (43 %)
2.3E-06 (57 %)
2.7E-10 (< 1 %)
4.1E-06
RME
4.0E-05 (60 %)
2.6E-05 (40 %)
3.8E-09 (< 1 %)
6.6E-05
Example of % contribution: % Soil for RME risk inside spill area = (Soil risk / Total risk) x 100%
= (1.46E-04 / 2.42E-04) x 100% = 60%
Page 3-24
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
Table 3-6. Input Distributions for Exposure Variables used in 1-D MCA for Variability.
Exposure
Variable1
IR soil
SA_skin3
Absorption
Fraction
IR air
EF
ED
BW
Distribution
Type
Triangular
Lognormal
Uniform
Lognormal
Triangular
Lognormal4
Truncated
Lognormal5
Lognormal
Parameters2
0, 50, 100
18150, 37.4
0.1,0.2
1.68, 0.72
200, 225, 250
11.7,7.0
14.0, 9.4, 44.0
71.75, 14.2
Units
mg/day
cm2
mg/cm2
mVhour
days
years
years
kg
Reference
U.S. EPA, 1993, 2001
U.S. EPA, 1997a, Table 6-4
(Total male/female body surface area)
U.S. EPA, 2001; minimum truncation limit is
professional judgment
U.S. EPA, 1996, p.5-10
U.S. EPA, 2001; truncation limits are
professional judgment
U.S. EPA, 1997b, Table 15-161 and U.S. EPA,
2001
(Mean value is based on average of total median
tenure for professional specialty and farming,
forestry, and fishing)
Site-specific survey data, used in refined
1-D MCA
U.S. EPA, 1997a, Tables 7-4 and 7-5;
(Combined male/female body weight
distributions)
'All other exposure parameters are inputted as point estimates (see Table 3-4).
Parameters for lognormal PDF are X ~ Lognormal (arithmetic mean, arithmetic standard deviation) unless otherwise stated.
Parameters for triangular PDF are X ~ Triangular (minimum, mode, maximum). Parameters for uniform PDF are X ~
Uniform (minimum, maximum).
3A point estimate of 0.189 was used to adjust the surface area skin (SA_skin) distribution, which is based on total body surface
area, to account for skin exposures limited to face, forearms, and hands (U.S. EPA, 1997a, Vol. I).
4Parameters for preliminary lognormal PDF for ED were converted from a geometric mean of 10 and a 95th percentile of 25.
'Parameters for site-specific lognormal PDF for ED are arithmetic mean, standard deviation, and upper truncation limit.
Table 3-7. 1-D MCA Risk Estimates using Preliminary Inputs.
Cumulative
Percentile
50th
90th
95th
99th
99.9th
Spill Area Risk
5.7E-05
1.3E-04
1.6E-04
2.4E-04
3.9E-04
Page 3-25
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
-1
ED
IR soil
-0.28 1
AF
EF
IR_air
1 0.86
1 0.30
BW
10.15
I] 0.08
0.01
i I i i I i I
.0 -0.5 0.0 0.5 1.0
Rank Correlation
Figure 3-6. Results of sensitivity analysis for preliminary 1-D MCA (Tier 2)
showing the Spearman Rank correlations (see Appendix A and B) between input
variables and risk estimates.
Table 3-8. Exposure Duration Survey Results.
Survey Results (years)
24.9
8.4
3.0
6.8
18.5
9.1
7.2
20.3
11.7
4.7
20.9
10.6
12.7
44.2
17.2
6.5
16.5
6.0
18.8
11.7
Summary Statistics
n
min
max
arithmetic mean
standard dev
median/GM
GSD
20
3.0
44.2
14.0
9.4
11.7
1.8
Table 3-9. Refined Point Estimate and 1-D MCA Risk Estimates.
Cumulative Percentile
Refined RME
Point Estimate
50th
90th
95th
99th
99.9th
Spill Area Risk
3.1E-04
6.7E-05
1.6E-04
2.1E-04
3.2E-04
5.3E-04
Page 3-26
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 3 ~ December 31, 2001
REFERENCES FOR CHAPTER 3
Hedges, L.V. and I. Olkin. 1985. Statistical Methods for Meta-Anafysis. Academic Press, Inc. Orlando.
Oregon DEQ. 1998. Guidance for the Use of Probabilistic Analysis in Human Health Exposure
Assessments. Waste Management and Cleanup Division. Interim Final. November.
Stiteler, W.M., L.A. Knauf, R.C. Hertzberg, and R.S. Schoeny. 1993. A Statistical Test of Compatibility
of Data Sets to a Common Dose-Response Model. Regulatory Tox. Pharm. 18: 392-402.
U.S. EPA. 1989. Risk Assessment Guidance for Superfiind (RAGS): Volume I. Human Health Evaluation
Manual (HHEM) (Part A, Baseline Risk Assessment). Interim Final. Office of Emergency and
Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1992a. Final Guidelines for Exposure Assessment. EPA/600/Z-92/001. 51 Federal Register,
22888-22938. May 29.
U.S. EPA. 1992b. Guidance on Data Usability in Risk Assessment. Part A. Final. Office of Solid Waste
and Emergency Response, Washington, DC. OSWER Directive No. 9285.7.09A. NTIS
PB92-96336.
U.S. EPA. 1993. Data Quality Objectives Process for Superfiind. Office of Solid Waste and Emergency
Response. Washington, DC.
U.S. EPA. 1996. Final Soil Screening Guidance, May 17, 1996. Soil Screening User's Guide. Office of
Solid Waste and Emergency Response, Washington, DC. EPA 540/R-96/018.
U.S. EPA. 1997a. Exposure Factors Handbook, Volume 1. Office of Research and Development,
Washington, DC. EPA/600/P-95/002Fa.
U.S. EPA. 1997b. Exposure Factors Handbook, Volume 2. Office of Research and Development,
Washington, DC. EPA/600/P-95/002Fb.
U.S. EPA. 1997c. Exposure Factors Handbook, Volume 3. Office of Research and Development,
Washington, DC. EPA/600/P-95/002Fc.
U.S. EPA. 1997d. Memorandum from Deputy Administrator Fred Hansen on the Use of Probabilistic
Techniques (including Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for
Monte Carlo Analysis. Office of Research and Development, Washington, DC.
EPA/630/R-97/001.May.
U.S. EPA. 1998. Guidelines for Ecological Risk Assessment. Final. National Center for Environmental
Assessment, Washington, DC. EPA/630/R-95/002F.
U.S. EPA. 2001. Supplemental Guidance for Developing Soil Screening Levels for Superfiind Sites.
Office of Solid Waste and Emergency Response. Washington, DC. OSWER Directive
No. 9355.4-24. December.
Page 3-27
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
CHAPTER 4
PROBABILISTIC ANALYSIS IN ECOLOGICAL RISK ASSESSMENT
4.1 INTRODUCTION
4.1.1 BASIC APPROACH FOR PERFORMING ECOLOGICAL RISK ASSESSMENTS
Ecological risk assessment (ERA) is defined by the 1997 Environmental Protection Agency's
(EPA) Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting
Ecological Risk Assessments (ERAGS) (U.S. EPA, 1997a) as an evaluation of the "likelihood that adverse
ecological effects are occurring or may occur as a result of exposure to one or more stressors". The
ERAGS document is generally similar to, and consistent with the earlier framework guidance and
approach (U.S. EPA, 1992a) which was expanded upon and superceded by the Guidelines for Ecological
Risk Assessment (U.S. EPA, 1998). The EPA has developed extensive technical and policy guidance on
how ERAs should be planned and performed (see Exhibit 4-2). In general, this process has three main
elements, as shown in Figure 4-1:
Discussion
Between the
Risk Assessor
and
Risk Manager
(Planning)
Ecological Risk Assessment
PROBLEM FORMULATION
•— 1
A
N
A Characterization
Y °f
„ Exposure
S
? h
Charac
Eco
Ei
I
^
V \
— 1
terization
of
logical
Tects
1
h
1
V v
RISK CHARACTERIZATION
i
r
O
>
Sf
3!'
o'
3.
a
o"
a
=
o
Sf
rt-
o
3.
era
Discussion Between the Risk
Assessor and Risk Manager
(Results)
i
i
L
r
Risk Management
Figure 4-1. Ecological Risk Assessment Framework (U.S. EPA, 1992a)
Page 4-1
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
Problem Formulation provides a foundation for the entire risk assessment. This element
includes the specification of risk management goals and assessment endpoints, the development
of a site conceptual model with exposure pathways and receptors, and the development of a
sampling and analysis plan to collect data on exposures and measures of effects that are needed to
support the ERA. In general, problem formulation serves as the foundation of an ERA and often
is an iterative process, whereby substantial re-evaluation may occur as new information and data
are collected during the site investigations. Collection of data in subsequent iterations is often
triggered by identification of major data gaps and uncertainties in the risk characterization that
prevent confident decision making by risk managers.
Analysis includes two principal measurement steps that are based upon the problem formulation:
Assessment of exposures and assessment of ecological effects. Assessment of exposures includes
the identification of stressors at the site that may affect ecological receptors, a characterization of
the spatial and/or temporal pattern of the stressors in the environment at the site, and an analysis
of the level of contact or co-occurrence between the stressors and the ecological receptors.
Assessment of ecological effects includes identification of the types of effects which different
stressors may have on ecological receptors, along with a characterization of the relationship
between the level of exposure to the stressor and the expected biological or ecological response.
This is referred to as the stressor-response relationship.
Risk Characterization combines the exposure characterization and the effects characterization in
order to provide a quantitative likelihood or qualitative description of the nature, frequency, and
severity of ecological risks attributable to exposure to stressors at a site, as well as an evaluation
of the ecological relevance of the effects. Good risk characterizations express results clearly,
articulate major assumptions and uncertainties, identify reasonable alternative interpretations, and
separate scientific conclusions from policy judgments (U.S. EPA, 1995, 1998).
Page 4-2
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-1
DEFINITIONS FOR CHAPTER 4
Assessment Endpoint - An explicit expression of an environmental value (ecological resource) that is to be
protected, operationally defined by risk managers and risk assessors as valuable attributes of an ecological
entity.
Benchmark Dose (BMD) - The dose which causes a specified level of response. The lower confidence limit on
the BMD is usually referred to as the BMDL.
Community - An assemblage of populations of different species specified by locales in space and time.
Conceptual Model - A site conceptual model (SCM) in the problem formulation for an ecological risk
assessment is a written description and visual representation of predicted relationships between ecological
entities and the stressors to which they may be exposed, including sources and pathways of stressors.
Ecological Risk Assessment (ERA) - The process that evaluates the likelihood that adverse ecological effects
may occur or are occurring as a result of exposure to one or more stressors.
Lines of Evidence - Information derived from different sources or techniques that can be used to characterize
the level of risk posed to exposed receptors; weight-of-evidence generally refers to the quantity of
science, while strength of evidence generally refers to the quality of science.
Lowest-Observed-Adverse-Effect Level (LOAEL) - The lowest level of a stressor evaluated in a test that
caused a statistically significant effect on one or more measurement endpoints linked to undesirable
(adverse) biological changes.
Measurement Endpoint (Measure of Effect) - A measurable ecological property that is related to the valued
characteristic chosen as the assessment endpoint. Measurement endpoints (also called measures of effect)
often are expressed as the statistical or numeric summaries of the observations that make up the
measurement.
No-Observed-Adverse-Effect Level (NOAEL) - The highest level of a stressor administered in a test that did
not cause a statistically significant effect in any measurement endpoint linked to an undesirable (adverse)
biological change.
Population - An aggregate of individuals of a species within a specified location in space and time.
Receptor - The ecological entity (with various levels of organization) exposed to the stressor.
Risk Characterization (ecological) - The third and last phase of ERA that integrates the analyses of exposure to
stressors with associated ecological effects to evaluate likelihoods of adverse ecological effects. The
ecological relevance of the adverse effects is discussed, including consideration of the types, severity, and
magnitudes of the effects, their spatial and temporal patterns, and the likelihood of recovery.
Scientific/Management Decision Point (SMDP) - A time during the ERA when a risk assessor communicates
results or plans of the assessment at that stage to a risk manager. The risk manager decides if information
is sufficient to proceed with risk management strategies or whether more information is needed to
characterize risk.
Species - A group of organisms that actually or potentially interbreed and are reproductively isolated from
similar groups; also, a taxonomic grouping of morphologically similar individuals.
Stressor - Any chemical, physical or biological entity that can induce an adverse response in an ecological
receptor; Superfund considers all stressors, but focuses on chemical (toxicant) stressors.
Toxicity Reference Value (TRV) - A dose or concentration used to approximate the exposure threshold for a
specified effect in a specified receptor. A TRV is often based on a NOAEL or LOAEL from a laboratory-
based test in a relevant receptor species.
Page 4-3
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-2
ECOLOGICAL RISK ASSESSMENT GUIDANCE AND POLICY DIRECTIVES
EPA has developed extensive guidance and policies on methods and approaches for performing ERAs,
including the following:
(1) Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting
Ecological Risk Assessments ("ERAGS"), Interim Final (U.S. EPA, 1997a). This document
includes processes and steps specifically selected for use in ERAs at Superfund sites. This
document supersedes the 1989 EPA RAGS, Volume II, Environmental Evaluation Manual,
Interim Final (U.S. EPA, 1989). Supplements to ERAGS include the EcoUpdates (U.S.
EPA, 1991-present, Intermittent Bulletin Series, 1991 to present), which provide brief
recommendations on common issues for Superfund ERAs.
(2) Guidelines for Ecological Risk Assessment ("Guidelines") (U.S. EPA, 1998). This document
updates general (nonprogram specific) guidance that expands upon and replaces the earlier
Framework for Ecological Risk Assessment (U.S. EPA, 1992a). The approaches and
methods outlined in the Guidelines and in ERAGS are generally consistent with each other.
(3) Risk Assessment Guidance for Superfund (RAGS): Volume 1-Human Health Evaluation
Manual (Part D, Standardized Planning, Reporting, and Review of Superfund Risk
Assessments), (U.S. EPA, 2001). This guidance specifies formats that are required to present
data and results in baseline risk assessments (both human and ecological) at Superfund sites.
(4) Policy Memorandum: Guidance on Risk Characterization for Risk Managers and Risk
Assessors, F. Henry Habicht, Deputy Administrator, Feb. 26, 1992 (U.S. EPA, 1992b). This
policy requires baseline risk assessments to present ranges of risks based on "central
tendency" and "reasonable maximum" (RME) or "high-end" exposures with corresponding
risk estimates.
(5) Policy Memorandum: Role of the Ecological Risk Assessment in the Baseline Risk
Assessment, Elliott Laws, Assistant Administrator, August 12, 1994 (U.S. EPA, 1994). This
policy requires the same high level of effort and quality for ERAs as commonly performed
for human health risk assessments at Superfund sites.
(6) Policy Memorandum: EPA Risk Characterization Program, Carol Browner, Administrator,
March 21, 1995 (U.S. EPA, 1995). This policy clarifies the presentation of hazards and
uncertainty in human and ERAs, calling for clarity, transparency, reasonableness, and
consistency.
(7) Issuance of Final Guidance: Ecological Risk Assessment and Risk Management Principles
for Superfund Sites. Stephen D. Luftig for Larry D. Reed, October 7, 1999 (U.S. EPA,
1999). This document presents six key principles in ecological risk management and
decision making at Superfund sites.
Page 4-4
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
ERA is a key component of the remedial investigation process that EPA uses at Superfund sites.
ERAGS is a program-specific guidance for Superfund that focuses on chemical stressors released into the
environment from hazardous waste sites. This guidance refers to ERA as a "qualitative and/or
quantitative appraisal of the actual or potential impacts of contaminants from a hazardous waste site on
plants and animals other than humans and domesticated species. An excess risk does not exist unless:
(1) the stressor has the ability to cause one or more adverse effects, and (2) the stressor co-occurs with or
contacts an ecological component long enough and at a sufficient intensity to elicit the identified adverse
effect." The ERAGS document provides guidance on using an eight-step process for completing an ERA
for the Superfund Program, as shown in Figure 4-2.
SThP I: SC'RLLNINC, LhVT.L:
• Site Visit
• Problem Formulation
• loxieity [{valuation
STLP2: SCRLLNINC, LLVLL:
• Fxposure Estimation
• Risk Caleulation
Step 3: Problem Formulation
STEP 4: STUDY DESIGN AND DQO
PROCESS
• Lines of Fvidenee
• Measurement Fndpoints
Work Plan and Sampling and Analvsis Plan
STEPS : VERIFICATION OF FIELD
SAVIIM .ING DESIGN
STEP (•, : SITF INVESTIGATION AND
DATA ANALYSIS
STEP 7 : RISK CHARACTERIZATION
ST1-:PS: RISK MANAGHMHNT
Risk Assessor and
Risk \1anager
Aiireemenl
SMDP= Scientific/Management Decision Point
Figure 4-2. Eight-step Ecological Risk Assessment Process for Superfund (U.S. EPA, 1997a).
Page 4-5
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.1.2 PREDICTIVE vs OBSERVATIONAL APPROACHES
In general, conclusions about ecological hazards from environmental contamination may be based
on information derived from two different techniques: the predictive approach (a comparison of
calculated exposures with a set of toxicity reference values), and the observational approach (direct
evaluation of the range of potential exposures, coupled with site-specific toxicity testing and population
demographic estimates).
Predictive Approach: The core of all Superfund ERAs is the predictive approach, including
exposure assessment, toxicity assessment, and risk characterization. The predictive approach is
based on a comparison of calculated estimates of chemical exposure of a receptor to one or more
Toxicity Reference Values (TRVs) appropriate for that chemical and that receptor. The ratio of
exposure at the site to the TRY is referred to as the Hazard Quotient (HQ). The predictive
approach has always been used at Superfund sites because it is relatively easy to implement, and
because it can be used to evaluate not only current risks, but also risks that might exist in the
future if any important changes were to occur in the level of contamination (e.g., due to on-going
fate and transport processes), or to changes in land use (a change in land use might alter a number
of habitat factors that influence the number and identify of ecological receptors). The predictive
approach, however, has the inherent uncertainties of the assumptions in the exposure and toxicity
models which are seldom site-specific and thus can lead to either over-protective or under-
protective estimates of risk.
Direct Observation: If there is a need to reduce uncertainties in the predictive approach, direct
observations of exposure and effects can be collected at Superfund hazardous waste sites. The
predictive approach used in ERA does not negate the use of descriptive toxicological approaches
or the use of site-specific exposure data, such as toxicity testing or bioaccumulation
measurements. Site-specific observations, such as toxicity testing of invertebrates over a gradient
of site contaminant exposure levels, may be used to develop site-specific and chemical-specific
toxicological relationships. Site-specific measures of exposure or ecosystem characteristics can
be used to reduce uncertainty in the exposure assessment and aid in the development of cleanup
goals in the Baseline ERA. The direct observation of the exposure and effects on ecological
receptors does not however constitute a complete risk assessment. If field or laboratory studies
are NOT designed appropriately to elicit stressor-response relationships, direct impacts should not
be used as the sole measure of risk because of the difficulty in interpreting and using these results
to develop cleanup goals in the ERA. Furthermore, poorly designed toxicological evaluations of
environmental media from the site may not allow a definitive identification of the cause of
adverse response. For example, receptor abundance and diversity as demographic data reflect
many factors (habitat suitability, availability of food, predator-prey relationships among others).
If these factors are not properly controlled in the experimental design of the study collecting the
observational data, conclusions regarding chemical stressors can be confounded. In addition,
direct observation provides information about current risks only and not potential risks should
land use or exposure change in the future. Hence, direct observations may be used as a line of
evidence in an ERA, but should not be the sole evidence used to characterize the presence or
absence of the risks of an adverse effect in the future.
Page 4-6
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.1.3 POTENTIAL ADVANTAGES AND LIMITATIONS OF PROBABILISTIC METHODS IN ERA
Probabilistic risk assessment (PRA) is a computational tool that may help increase the strength of
the predictive evaluation of ecological risks, as well as sometimes helping to better evaluate distributions
of observational data for an ERA. The potential advantages of PRA compared to, or possible benefits in
augmentation of, the conventional point estimate approach for characterizing variability in exposure or
risk are discussed in Chapter 1 and Exhibits 1-6 and 1-7. In brief, point estimate calculations utilize
simplifications and assumptions in order to deal with the complex mathematics of combining inputs that
are inherently variable. Probabilistic models, in contrast, are designed to combine sets of information on
inputs that are expressed as probability distributions. Therefore, PRA generally can yield risk estimates
that allow for a more complete characterization of variability and uncertainty, and a potentially more
useful sensitivity analysis as compared to estimating sensitivities of inputs from point estimates (see
Appendix A). For example, sensitivity analysis can help determine major contributors to exposure factors
and sources of uncertainty that could help to design better sampling and analysis plans in later iterations
to help fill data gaps and reduce uncertainties for risk characterization.
Because of the inherent differences in the computational approach, as in the case with any
additional risk assessment information, PRA may sometimes lead to a different risk assessment outcome
and risk management decision than would be derived from the use of point estimate calculations alone.
The differences in the decisions stemming from the two approaches will vary from case to case,
depending mainly on the form of the exposure or risk model, the attributes of the distributions of the input
values, and the quality, quantity, and representativeness of the data on which the input distributions are
derived. Sometimes the differences between the two approaches will be quite large, and the information
gained from a PRA can play an important role as weight-of-evidence in communicating risks to
stakeholders and risk managers.
Even though PRA may have some advantages, it also has limitations and potential for misuse.
PRA can not fill basic data gaps and can not eliminate all of the potential concerns associated with those
data gaps. That is, if one or more of the input distributions are not well characterized and the
distribution(s) must be estimated or assumed, then the results of the PRA approach will share the same
uncertainty as the point estimate values. However, given equal states of knowledge, the PRA approach
may yield a more complete characterization of the exposure or risk distribution than the point estimate
approach.
Of course, any prediction of exposure or risk is based on the use of mathematical models to
represent very complex environmental, biological, and ecological systems. No matter how sophisticated
the computations, questions will always exist as to whether the calculated values are a good
approximation of the truth. Therefore, even when PRA is used as a supplemental tool to point estimations
(deterministic) of risks in the ERA process, a weight-of-evidence approach that combines the predictive
approach with direct observations will still provide the most appropriate basis for decision making.
A second application of PRA in ERA, besides the characterization and incorporation of
distributions of data for ERA, is the characterization of uncertainty in calculated estimates of exposure or
risk. In this application, whatever uncertainty may exist in one or more of the input distributions is
characterized, and quantitative estimates of the confidence limits around the mean, upper bound, or any
other percentile of the output distribution are calculated. This use of PRA is often especially important in
risk management decision making, since the range of uncertainty around central tendency exposure (CTE)
and reasonable maximum exposure (RME) or other upper bound estimates of exposure or risk can
Page 4-7
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
sometimes be quite large. As stated before, the point estimate approach can also provide estimates of
uncertainty, but the PRA approach often provides a more complete characterization of the uncertainty.
4.1.4 Focus OF THIS CHAPTER
This chapter focuses on the application of PRA as a tool for predicting ecological risks at
Superfund sites. Some of the methods and approaches described in this chapter are similar to those that
have been developed by U.S. EPA's Office of Pesticide Programs Committee on Federal Insecticide,
Fungicide and Rodenticide Act (FIFRA) Risk Assessment Methods (ECOFRAM, 1999a, 1999b) for
use in assessing environmental hazards of pesticide products. However, the methods described in this
chapter are specifically designed to be applicable at Superfund sites and to be consistent with other
Superfund guidance.
This chapter does not seek to provide guidance on the many basic issues that must be faced in
planning and performing any ERA. Prior to considering the use of PRA in an ERA, fundamental
concepts will already have been developed, such as a problem formulation with a conceptual site model,
selection of representative receptors, definition of exposed populations, definition of risk management
objectives and goals, selection of assessment endpoints, calculation of TRVs and development of site
sampling plans, etc. Likewise, this chapter does not repeat the presentation of basic statistical and
mathematical methods used in PRA, since these are described in other chapters and appendices of this
document. In summary:
"3° This chapter focuses on application of PRA techniques to ERA at Superfund
sites.
us- The reader is assumed to be familiar with the basic methods used in ERA at
Superfund sites, and this chapter does not address basic tactical and
technical issues in ERA.
«s° The reader is assumed to be familiar with the basic mathematical principles and
techniques of PRA as described in other chapters and appendices of this document.
4.2 DECIDING IF AND WHEN TO USE PRA IN ECOLOGICAL RISK ASSESSMENT
As shown in Figure 4-2, the ERA process for Superfund includes a number of scientific/
management decision points (SMDPs) (U.S. EPA, 1997a). The SMDP is a point of consultation between
the risk manager, EPA Regional Biological Technical Assistance Group (BTAG) coordinator, EPA
regional ecotoxicologist, and other stakeholders, and is intended to provide an opportunity for re-
evaluation of direction and goals of the assessment at critical points in the process. It is during the SMDP
discussions that it is important to decide whether or not a PRA is likely to be useful in decision making.
If so, the pursuit of distributed data is justified. Within the 8-step process of developing the ERA, PRA
could provide insight at several steps. A decision to move forward with distributional analyses should be
considered within the BTAG context during the documentation of the outcome of the SMDPs after Step 3
within the process. As a reminder, PRA is NOT intended to be a replacement for point estimate analyses;
rather PRA supplements the required presentation of point estimates of risk. It is also emphasized that the
use of PRA should never be viewed as or used in an attempt to simply generate an alternative risk
estimate or PRG, compared to that which was derived by a point estimate ERA; instead, PRA should be
Page 4-8
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
used to provide insightful information on distributions of various factors (exposure, toxicity, and hazards)
which can provide weight-of-evidence evaluations of potential risks in conjunction with a point estimate
ERA. There are a number of factors to consider in making these decisions, as discussed below.
4.2.1 TECHNICAL CONSIDERATIONS
The fundamental reason for performing any predictive risk assessment (point estimate or
probabilistic) is to provide information to risk managers in order to help support the risk management
decision-making process. As noted above, a properly performed PRA may help to yield more description
of variability in exposure and risk than can be achieved using the point estimate approach. Therefore, if
any of a site's data may be better described and evaluated by distributions, then a PRA can be applied to
any part of an ERA or even to the entire ERA for expressing risk characterization in probabilistic terms;
again, always in conjunction with the required point estimate ERA. However, when risk estimates
derived from the point estimate approach are either far below or far above a level of risk management
concern, any such potential improvements in risk characterization are not likely to influence risk
management decision making. In these cases, PRA is not likely to be as useful in decision making. Even
so, PRA may help in these situations by providing information that may be useful in better deciding
where the gradient of excess risks are reduced to acceptable levels. Rather, it is more common for a PRA
to be useful when point estimates of risks are close to the decision threshold (such that PRA-based
refinements in the risk estimates might be important in making risk management decisions). It is for this
reason that PRA may be useful to apply either during the development of the ERA after the screen
(Steps 3 to 6, U.S. EPA, 1997a), or after point estimate results from the baseline ERA have been
completed (Steps 1 to 7, U.S. EPA, 1997a).
The results of a point estimate risk assessment will normally present the range of risks based on
central tendency exposure and reasonable maximum exposure input assumptions and on the no-observed-
adverse-effect-level (NOAEL)- and lowest-observed-adverse-effect-level (LOAEL)-based TRVs (U.S.
EPA, 1992b, 1997b). The bounds for the highest HQ are derived from the ratio of the RME compared to
the NOAEL-based TRY, and the bounds for the lowest HQ are based on the ratio of the CTE compared to
the LOAEL-based TRV. These two bounded extreme estimates of risk can be used to screen out cases
where PRA is not likely to be as useful. That is, if the risk to the RME receptor is clearly below a level of
concern using the NOAEL-based TRV, then risks to the exposed population are likely to be low and PRA
analysis is likely not needed. Likewise, if risks to the CTE receptor are clearly above a level of concern
using the LOAEL-based TRV, then risks to the exposed population are likely to be of definite concern,
and a PRA may not provide as much additional useful information to the risk manager, except in the case
where uncertainties remain high and the derivation of an appropriate and realistic clean-up goal may be
difficult. If the risks are intermediate between these two bounds (e.g., risks to the CTE receptor are below
a level of concern based on the LOAEL-based TRV but are above a level of concern based on the
NOAEL-based TRV), then PRA might be helpful in further characterizing the site risks in balance with
the point estimates of risks and in supporting decision making or in deciding if additional iterations of
analyses would be needed. This concept is illustrated graphically in Figure 4-3.
Page 4-9
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
(
A
B
NOAEL
C
D
LOAEL
E
3 10 20 30 40 50 60 70
Exposure (arbitrary units)
CTE RME
80 90 100
Figure 4-3. Example of cases where use of PRA may be helpful. In cases A and E, the range of risks (CTE to
RME) estimated by the point estimate method are either well below (Case A) or well above (Case E) the likely
level of concern based on the NOAEL-LOAEL range, and PRA is not likely to alter risk management decisions
regarding the potential need for remediation. In cases B, C, and D, the point estimates of risk overlap or fall within
the range of potential concern, suggesting that PRA-based risk estimates might be helpful in supporting risk
management decisions.
The second main technical reason to consider conducting PRA is that the PRA methodology can
help characterize and quantify the degree of variability and uncertainty around any particular estimate of
exposure or risk (e.g., the CTE or RME). The purpose of the analysis would be to estimate the
uncertainty around an exposure or toxicity or risk estimate, generally with little or no additional data
acquisition. The only additional information needed to perform the analysis is an estimate of the
uncertainty in the true parameter values of the key variables in the variability model. In some cases, these
estimates of uncertainty around parameter values may be developed from statistical analysis of the
available data. Alternatively, professional judgment may be used to establish credible bounds on the
parameters, especially when relevant data are sparse.
«*" Even in the presence of data gaps, uncertainty analysis using PRA can provide useful
information. Indeed, it is when data are limiting or absent that a quantitative
probabilistic analysis of uncertainty may be most helpful.
Page 4-10
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.2.2 COST AND SCHEDULE CONSIDERATIONS
Performing a PRA can sometimes add time and cost to an ERA. As discussed in Chapter 2,
in part, the decision to progress from a point estimate assessment to a PRA reflects a belief that the
potential value of the PRA for risk management decision making outweighs the additional time and costs.
The tiered process encourages a systematic approach for both the point estimate and probabilistic
assessments, whereby the least complex methods are applied first. For example, the initial Tier 2
assessment may be conducted with a set of preliminary probability distributions for variability (PDFv),
developed with much the same information and assumptions that were applied to develop point estimates
in Tier 1. Parameter values can be estimated by setting the arithmetic mean equal to the CTE point
estimate, and the 95th percentile equal to the RME point estimate. The choice of distributions may differ
depending on the state of knowledge for a particular variable (see Appendix B). For example, unbounded
variables might be characterized with lognormal distributions while bounded distributions are
characterized by beta or Johnson Sb distributions. Certain variables may continue to be characterized by
point estimates, especially if the sensitivity analysis suggests that the chemical, pathway, and/or exposure
variables are relatively minor contributors to total exposure and risk. The decision to collect additional
data or explore alternative methods for developing probability distributions can be reexamined in an
iterative fashion by evaluating the expected benefits of the added information to the risk management
decision-making process. These concepts are presented in greater detail in Chapter 2 (see Figures 2-1 and
2-2).
4.3 PROBLEM FORMULATION
Once a decision has been made to include PRA in an ERA, the first step should be to re-visit the
problem formulation step and carefully determine the scope and objectives of the PRA. Typically, a
considerable amount of knowledge will have been gained during the screening level and baseline point
estimate evaluations, and this knowledge should be used to help focus and narrow the scope of the PRA.
That is, the PRA will generally utilize the same basic exposure and risk models used in the point estimate
approach, but the PRA will typically evaluate only a sub-set of the scenarios considered. For example,
chemicals, pathways, and/or receptors that are found to contribute a negligible level of exposure or risk
may usually be omitted from the PRA, while those factors that contribute significantly to an excess level
of risk concern in the point estimate approach should generally be retained. As noted previously, when a
chemical or pathway is omitted from a PRA analysis, this does not mean that it is eliminated from the
overall risk assessment; rather, it may be kept in the assessment as a point estimate.
The next step in problem formulation for a PRA should be to define whether the goal of the
analysis is to characterize variability alone, or to characterize both variability and uncertainty. In either
case, sensitivity analysis (as summarized in the preceding paragraph, or for more details see Appendix A)
should be used to help identify which of the input variables contribute the most to the variability in the
outputs (exposure, toxic effects, or risk), and the initial PRA should focus on defining the probability
density functions (PDFs) for those input variables. An analysis of uncertainty, if thought to provide
additional useful information, may also be included at the initial level, or may be delayed until the initial
analysis of variability is completed.
As always, problem formulation should be viewed as an iterative process, and it is reasonable and
appropriate that decisions regarding the scope and direction of the PRA should be reassessed (at SMDPs)
as information becomes available from the initial evaluations. As stressed above, the fundamental
criterion which should be used is whether or not further PRA evaluations are likely to provide additional
information to a point estimate ERA that will help strengthen and support the risk management decision-
making process.
Page 4-11
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.4 MODELING VARIABILITY IN EXPOSURE
There are two main types of descriptors of exposure that may be used in ERA: dose and
concentration. For terrestrial receptors such as mammals or birds, exposure is most often described in
terms of ingested dose (mg/kg-day). In most cases, this will be based on chemical ingested from drinking
water and/or the diet, including incidental soil ingestion, but could also include amounts of chemical
taken up across the skin or through inhalation as additional routes of exposure. The exposure levels are
most often expressed as doses, since that term tends to normalize the confounding factors of variable
daily intake rates and body weights that occur if/when one only evaluates concentrations. For aquatic
receptors, the main route of exposure is usually by direct contact and less often by ingestion, so exposure
is usually characterized in terms of concentration of contaminants in surface water, pore water and/or
sediment. Likewise, exposure of terrestrial plants and terrestrial invertebrates, such as earthworms, is
usually described in terms of concentration of contaminants in soil. In some cases, exposure of terrestrial
receptors is characterized in terms of specific tissue or whole-body concentrations of contaminants.
Examples of calculating and presenting dose-based and concentration-based distributions of exposure are
presented below.
4.4.1 CHARACTERIZING VARIABILITY IN DOSE
The general equation used for calculating the dose of a contaminant of concern in a specified
environmental medium (e.g., water, soil, air, diet, etc.) by a particular member of a population of exposed
receptors is:
DT = C x TR / BW
uijj ^j x ir^j / D Wj
where:
Dig = Average daily intake of chemical due to ingestion of medium "I" by a population
member "j" of the exposed population (mg/kg-day)
Q = Concentration of chemical in environmental medium "I" (mg/unit medium)
IRg = Intake rate of medium "I" at the site by population member "j" (units of medium
per day)
BWj = Body weight of population member "j" (kg)
Total exposure of a population member "j" is then the sum of the exposures across the different media:
In this basic equation, IRg and BWj are random variables (i.e., they have different measurable values for
different members of the exposed population) that are often correlated. For example, a receptor with a
relatively low intake rate can also be expected to have a low body weight. Some studies utilize paired
measurements of IR and BW by individual, and present a distribution of the ratio (IRg /BWj), referred to
as a body weight-normalized intake rate (mg/kg-day). This expression provides an alternative to using a
correlation coefficient to relate two input variables (see Appendix B), and can be entered into the dose
equation as follows:
(IRa
DL , = C x —^
{BWj.
where the ratio is characterized by a single probability distribution. Because the variability in this ratio is
likely to be different than the variability in the ratio of the IR and BW variables treated independently,
Page 4-12
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
accounting for the correlation can affect the distribution of dose and risk. If empirical data for
quantifying the ratio are limited but a relationship is expected, plausible ranges of correlations may be
explored as a source of uncertainty in the risk estimates.
The concentration term (Q) may be characterized by a point estimate or a probability distribution,
depending on the relationship between the geographic scales of the measurement data and receptor home
range (see Appendix C, Section C.3.1). If the home range of the receptor is small compared to the spatial
distribution of sampling locations, Q may be characterized by the probability distribution for variability
in measured concentrations. Alternatively, if the home range is large compared with the exposure area
evaluated, then a point estimate (e.g., mean or uncertainty in the mean) may be more appropriate.
In the PRA approach, PDFs should be defined for as many of the input variables as reasonable,
especially for those variables that are judged (via sensitivity analysis) to contribute the most to the
variability in total exposure. The basic principles for selecting the key variables to model as PDFs are
presented in Appendix A, and the basic methods used for selecting and fitting distributions are described
in detail in Appendix B.
Figure 4-4 shows several examples of graphical formats which may be used to present the
estimated distribution of ingested doses in an exposed population. If a single distribution is plotted (top
panel), the PDF format is usually the most familiar and useful for risk assessors and managers, but the
cumulative distribution function (CDF) format tends to be less cluttered when multiple distributions are
shown (e.g., compare the middle graph to the bottom graph). In addition, percentiles can be read directly
from a CDF format, but not from a PDF format graph. In all cases, it is very useful to superimpose the
CTE and RME point estimate ranges of exposure directly on the same graph as is used to show the
distribution of exposures estimated by PRA. This provides a convenient way to compare the results of
the two alternative computational methods, and interpret additional information that the PRA can add to
the point estimate ERA.
us- A conventional point estimate, range of exposure (CTE to RME) or toxicity
(NOAEL to LOAEL) and corresponding risk ranges should be calculated
and presented for comparison with the PRA results.
Page 4-13
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.4.2 CHARACTERIZING VARIABILITY IN EXPOSURE CONCENTRATION
As noted above, in some cases the most appropriate descriptor of exposure is concentration
(either in an abiotic medium such as water, soil, or sediment, or in the tissues of the receptor), rather than
ingested dose. Assuming that the concentration values in the medium of concern are measured rather than
modeled, PRA is not required to generate the distribution of concentrations. Rather, the available data
may be used to define an appropriate theoretical or empirical distribution function (EDF), as described in
Appendix B. If concentrations in the medium are modeled (calculated by PRA) rather than measured,
then the exposure distribution may be estimated by using distribution functions (PDFs or CDFs, rather
than using point estimates as inputs to the fate and transport model(s) and/or uptake models that predict
the concentration levels in the medium of concern. The resulting distribution(s) of concentration may be
displayed graphically using the same formats as illustrated in Figure 4-4, except that the x-axis has units
of concentration rather than dose. As above, the point estimate ranges of concentration used in the CTE
and RME calculations should be plotted on the same graphs to provide a convenient basis for comparing
the results of the two approaches and to help interpret the additional information that the PRA can add to
the point estimate outputs.
4.5 MODELING VARIABILITY IN TOXICITY
4.5.1 VARIABILITY IN RESPONSE AMONG MEMBERS OF A POPULATION
Data on the toxicity of a chemical usually comes from laboratory studies whereby groups of
organisms (laboratory mammals, fish, benthic organisms, plants, earthworms, etc.) are exposed to
differing levels of chemical, and one or more responses (endpoints) are measured (survival, growth,
reproduction, etc.). These toxicological observations define the exposure-based stressor-response curve
that is characteristic for that specific receptor, chemical, and response.
In the point estimate approach, information from the dose/stressor-response curve is generally
converted to one or more TRVs, each representing a specific point on the dose-based or concentration-
based stressor-response curve. For example, the highest dose or concentration that did not cause a
statistically significant change in a lexicologically significant endpoint is defined as either the NOAEL
dose or the no-observed-effect concentration (NOEC), while the lowest dose or concentration that did
cause a statistically significant effect on a relevant endpoint is the LOAEL dose or the lowest-observed-
effect concentration (LOEC). Generally, exposures below NOAEL- or NOEC-based TRVs are
interpreted to pose acceptable risk, while exposures above LOAEL- or LOEC-based exposures are judged
to pose potentially unacceptable risk. It is essential to note the need for high quality toxicity data to
derive reliable and confident TRVs. Strong sampling and study designs, that generate data for site
exposure factors and toxicological stressor-response relationships, are of critical importance for producing
high quality ERAs by either point estimate or PRA approaches. Shortcomings in either area could be
major data gaps or uncertainties that detract from the confidence in the risk characterization of the ERA,
and may be a basis for pursuing additional iterations of sampling or studies that are more strongly
designed to fill those critical data gaps and reduce uncertainty.
Use of the TRV approach, however, does have some potential limitations. Most important is that
the ability of a study to detect an adverse effect depends on both the range of doses tested and the
statistical power of the study (i.e., the ability to detect an effect if it occurs). Thus, studies with low
power (e.g., those with only a few test animals per dose group) tend to yield NOAEL or NOEC values
that are higher than studies with good power (those with many animals per dose group). In addition, the
choice of the TRV is restricted to doses or concentrations that were tested, which may or may not be close
Page 4-15
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
to the true threshold for adverse effects, and this uncertainty increases as the interval between doses
increases. Finally, it is not always easy to interpret the significance of an exposure that exceeds some
particular TRV, since the severity and incidence of response depends on the shape and slope of the
exposure response curve (information that is not captured in a point estimate TRV).
One way to resolve some of these stressor-response limitations is to apply uncertainty factors to
the NOAEL or NOEC and LOAEL or LOEC, which calculates an adjusted TRV that reduces the study's
exposure level of concern to account for those uncertainties, so that there is a lesser chance of overlooking
possible adverse exposures (i.e., avoiding a false negative conclusion). Another way to resolve some of
the stressor-response limitations is to fit a mathematical equation to the available exposure-response data
and describe the entire exposure-response curve. This may be done using any convenient data fitting
software, but EPA has developed a software package specifically designed for this type of effort. This
software is referred to as the Benchmark Dose Software (BMDS), and is available along with detailed
documentation and explanation of the methodology at www.epa.gov/ncea/bmds.htm.
The most appropriate mathematical form of the exposure-response model depends on whether the
endpoint measured is discrete and dichotomous (e.g., survival) or continuous (e.g., growth rate). For a
dichotomous endpoint, the result of the fitting exercise is a mathematical exposure-response model P that
yields the probability of a response in an individual exposed at any specified level of exposure (expressed
either as dose or concentration). Exhibit 4-3 shows an example of this process using hypothetical data.
Thus, for an individual with an exposure level of "x", the probability of a response in that individual is
simply P(x). In a population of individuals with exposures xl, x2, x3, ...xi, the expected number of
responses (e.g., deaths) in the exposed population is the sum of the probabilities across all individuals in
the population. Stated another way, the average fraction of the population that will experience the
response is given by the expected value of P (i.e., the average value of P(x)).
Page 4-16
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-3
MODELING VARIABILITY IN RESPONSE FOR A DICHOTOMOUS ENDPOINT
The following data are from a hypothetical study of the acute lethality (24 hour) of a chemical using
fathead minnows as the test organism:
Concentration
ug/L
0
10
20
30
40
60
Number
Tested
20
19
20
20
18
20
Survival
Dead
0
1
0
3
7
15
Alive
20
18
20
17
11
5
These data were fit to each of the dichotomous models available in BMDS. The best-fit model was
the logistic equation. A graph of the best fit curve is shown below.
Best Fit Dose-Response Model
0.0
10
20 30 40 50
Concentration (ug/L)
60
70
Basic Equation
Probability of mortality (cone) = 1 / (1 + exp(-a - b*conc))
Best fit parameters
a -4.80
b 0.101
Goodness of Fit
P
AIC
0.604
79.12
P=Chi Square Goodness of Fit test statistic
AIC=Akaike's Information Criterion
Page 4-17
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
For a continuous endpoint, the BMDS software yields equations that give the expected mean
response m(x) at a specified exposure level, along with the standard deviation s(x) that characterizes how
variable the response is among different individuals exposed at that same exposure level. The standard
deviation may be modeled either as a constant (homogeneous variance) or a function of the exposure level
(heterogeneous variance), with the choice depending on which approach yields the best agreement with
the observed variances. In most cases there will not be sufficient data to allow a meaningful analysis of
the true shape of the underlying distribution of responses at a given exposure, so the choice of the
distributional form of the variability in response will require an assumption. In the absence of any clear
evidence to the contrary, it is considered likely that the distribution of responses will not be strongly
skewed, and that the distribution may be reasonably well modeled using a normal PDF (truncated as
necessary to prohibit selection of biologically impossible or implausible values). Thus, variability in
response at dose "x" may generally be modeled as:
Response(x) ~ NORMAL[m(x), s(x), min, max]
However, if available data suggest some other distributional form is more appropriate, that form should
be used and justified.
Exhibit 4-4 shows an example of this process using hypothetical data. In this case, the mean
response was found to be well modeled by the Hill equation, and the standard deviation was found to be
best characterized as a constant (rho=0). Thus, given an exposure level "x", the mean response m(x) may
be calculated from the model, and this value along with the standard deviation may then be used as
parameters for an appropriate type of PDF (e.g., normal) to describe the expected distribution of
responses in a population of different individuals exposed at level "x". Section 4.7.2 describes methods
that may be used to characterize and quantify the uncertainty associated with this approach.
Page 4-18
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-4
MODELING VARIABILITY IN RESPONSE FOR A CONTINUOUS ENDPOINT
The following data are from a hypothetical study of the effects of a chemical on the growth of
laboratory mice. Animals were exposed to the chemical via drinking water for 21 days. The
measurement endpoint was weight gain, expressed as a percentage of the starting weight of
each animal.
Ingested dose
mg/kg-day
0
50
100
150
200
250
Number
Tested
5
5
5
5
5
5
Weight Gain (% Starting Value)
Mean
24
22
25
18
7
-8
Stdev
8
9
6
7
10
5
These data were fit to each of the continuous models available in BMDS. The best-fit model was
the Hill equation with constant variance. A graph of the best fit curve is shown below.
50
100 150 200
Dose (mg/kg-day)
250
300
Basic Equations
Mean Response(d) = int + v*d"n / (k"n + d"n)
Variance(d) = alpha'mean response(d)Arho
Best fit parameters
int
v
n
k
alpha
rho
23.70
-51.41
5.295
228.7
48.5
0 (constant variance)
Goodness of Fit
P
AIC
0.685
154.5
P=Chi Square Goodness of Fit test statistic
AIC=Akaike's Information Criterion
Page 4-19
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.5.2 VARIABILITY IN RESPONSE AMONG SPECIES
In some cases, risk management decisions may also consider community-level effects as well as
population-level or sub-populations effects. That is, a stressor might be considered to be below a level of
concern for the sustainability of a community if only a small fraction of the total number of exposed
species are affected. In this case, toxicological responses may be best characterized by the distribution of
toxicity values across species. This is referred to as a Species Sensitivity Distribution (SSD). This type
of approach is generally used for communities of aquatic receptors, since all of the different species that
make up the community (e.g., all fish, benthic invertebrates, aquatic plants, and amphibians that reside in
a stream) will be exposed to approximately the same concentration of contaminant in the water. The
process for generating an SSD consists of the following steps:
(1) Select an appropriate type of endpoint (lethality, growth, reproduction, etc.), and select an
appropriate type of point estimate from the exposure-response curve for each species. For
example, the TRV might be the LC50 for lethality or the EC20 for growth. The key
requirement is that the SSD be composed of TRV endpoints that are all of the same type, not
a mixture.
(2) Collect all reliable values for that type of TRV from the literature for as many relevant
species as possible. When more than one value is available for a particular species, either
select the value that is judged to be of highest quality and/or highest relevance, or combine
the values across studies to derive a single composite TRV for each species. It is important to
have only one value per species to maintain equal weighting across species.
(3) Characterize the distribution of TRVs across species with an appropriate CDF. Note that
there is no a priori reason to expect that an SSD will be well characterized by a parametric
distribution, so both parametric and empirical distributions should be considered.
Once an SSD has been developed, the fraction of species in the exposed community that may be
affected at some specified concentration may be determined either from the empirical distribution or from
the fitted distribution. Exhibit 4-5 shows examples of this approach. In this hypothetical case, the TRV
selected for use was the LClow (in this case, the LClow is defined as all LC values <=LC10). A total of
13 such values were located. The first graphical presentation is the empirical distribution function, where
the Rank Order Statistic (ROS) of each value is plotted as a function of the log of the corresponding
value. This may be used directly to estimate the fraction of the species in a community that will be
affected by any particular environmental concentration. For example, in this case, it may be seen that a
concentration of 10 ug/L would be expected to exceed the LClow for about 33% of the aquatic species for
which toxicity data are available. The second graph shows how the data may be characterized by fitting
to a continuous distribution. In this case, a lognormal distribution was selected as a matter of
convenience, but other distributions may also yield acceptable fits. Based on the best fit lognormal
distribution for the SSD data, it is calculated that a concentration of 10 ug/L would be expected to impact
about 31% of the exposed species. However, as noted above, there is no special reason to expect that an
SSD will be well characterized by a continuous parametric distribution, so some caution should be used in
the use of a continuous distribution to fit an SSD, especially when the SSD is based on a limited number
of species and when the purpose of the SSD is to estimate percentiles and exposures outside the observed
range. The risk assessor should always present an evaluation of the robustness of an SSD to aid in the
decision process.
Page 4-20
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-5
HYPOTHETICAL SPECIES SENSITIVITY DISTRIBUTION
Hypothetical Data
Species
a
b
c
d
e
f
9
h
i
j
k
1
m
LC|OW
2
2.5
3
5
15
26
41
55
67
81
125
220
600
ln(LClow)
0.693
0.916
1.099
1.609
2.708
3.258
3.714
4.007
4.205
4.394
4.828
5.394
6.397
Rank
1
2
3
4
5
6
7
8
9
10
11
12
13
ROS
0.07
0.14
0.21
0.29
0.36
0.43
0.50
0.57
0.64
0.71
0.79
0.86
0.93
z-score
-1.465
-1.068
-0.792
-0.566
-0.366
-0.180
0.000
0.180
0.366
0.566
0.792
1.068
1.465
Example EOF: ROS vs LC,OW (log-scale)
1.0
0.9 -
Approximately
33% affected at C = 10
ug/L
10 100
LC-low (ug/L)
1000
Example Parametric Fit: (Lognormal)
7.0
6.0 -
5.0 -
? 4.0-
I 3.0-
2.0 -
1.0 -
0.0
y = 2.06x + 3.34
Best Fit Parameters
mu = 3.34
sigma = 2.06
-2.0
-1.5
-1.0 -0.5 0.0 0.5
Z-Score
1.0
1.5
2.0
Page 4-21
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.6 MODELING VARIABILITY IN RISK
0.010
0.008 -
0.006 -
0.004 -
0.002 -
0.000
Total Daily Intake
4.6.1 VARIABILITY IN HAZARD QUOTIENT
As noted above, the most common descriptor of risk used in predictive risk assessments is the
Hazard Quotient (HQ). The HQ is the ratio of the exposure for some generalized or typical hypothetical
member of the receptor population at a site, compared to an appropriate TRV value that equates to an
acceptable level of risk for that receptor and chemical. Usually the HQ approach is not based on a single
value, but on a range of values in which different levels of exposure (CTE and RJVIE) are compared to
both the NOAEL to LOAEL benchmarks. In general, HQ values below 1 are interpreted as indicating
acceptable risk, while HQ values above 1 are interpreted as indicating the potential for adverse effects.
Because exposure varies among different members of an exposed population of receptors, HQ
values also vary among members of the exposed population. Several alternative approaches for
characterizing this variability by PRA methods are presented below.
Variability Within a Population
Figure 4-5 illustrates
the simplest approach for
summarizing variability in HQ
values among the members of
an exposed population. In this
format, the TRV values
appropriate for a particular
exposure are simply
superimposed on the graph
illustrating the distribution of
exposures. This may be done
either for a dose-based (as
shown in the figure) or for a
concentration-based exposure
parameter. This format allows
an easy evaluation of the
fraction of the population above
(HQ > 1) and below (HQ < 1)
each TRV, especially when
presented in CDF format.
However, this format does not
allow for a quantitative estimate
of the fraction of the population
with HQ values above any value
other than 1, although a similar
calculation and presentation
could be made for any multiple
of the TRVs, which would
directly equate to that multiple
of the HQ (e.g., depicting the
TRV-LOAEL
RME Dose
50
100
150 200 250 300
Ingested Dose (mg/kg-day)
350
400
450
100
Ingested Dose (mg/kg-day)
1000
Figure 4-5. Example Comparison of Exposure Distribution to TRV.
Page 4-22
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
results for a value equal to 10-times the TRV would show the fraction of the population with an HQ
greater than 10).
More directly, the distribution of HQ values may be calculated by dividing each exposure value
by one or all of the TRVs based on the NOAEL, LOAEL, BMDL, etc., as shown in Figure 4-6. Note that
dividing a distribution by a constant does not change the shape of the distribution (only its scale), so the
shape of the HQ distribution will appear identical to that of the exposure distribution. Figure 4-6
illustrates two HQ distributions; one calculated using the NOAEL-based TRV, the other using the
LOAEL-based TRV. In a case such as this where there are two or more HQ distributions, a CDF format
is generally easier to evaluate than a PDF format, since overlap between the curves is minimized. The
CDF format allows an easy quantitative evaluation of the fraction of the population above and below any
particular HQ level. For example, in the case shown in Figure 4-6, it may be seen that 83% of the
population is expected to have HQ values below 1 based on the NOAEL-based TRV, while 4% are
expected to have HQ values above 1 based on the LOAEL-based TRV. This type of description
(percentage of the population with HQ values within a specified range) is very helpful in predicting
proportions of a population exposed to specified doses of concern.
0.010
0.008
LOAEL-Based HQ
NOAEL-Based HQ
0.000
1.0
Hazard Quotient
0.0
1.0
Hazard Quotient
10.0
Figure 4-6. Example Distribution of HQ Values.
Page 4-23
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
Variability Between Species
A similar approach may be used for characterizing the variability in risks among different species
in a community. Figure 4-7 is an example that compares the distribution of concentration values in a
water body (the variability might represent either time or space) to an appropriate SSD of TRVs for
different species of aquatic receptors that might reside in that water body. Three different graphical
formats are illustrated. In the upper panel, the PDF of concentration is compared to the CDF of the SSD.
This format is easy to understand and may be interpreted visually, but is difficult to interpret
quantitatively. The middle panel shows that same information, but with both distributions presented in
CDF format. This allows for a quantitative evaluation of the fraction of the species that will be above
their respective TRVs at any specified part of the exposure distribution. For example, using a simple
graphical interpolation process (shown by the dashed lines), it may be seen that the 90th percentile of
concentration (21 ug/L) will impact approximately 24% of the exposed species. The bottom panel shows
the results when this same process is repeated (mathematically) for each of the concentration percentiles.
As seen, hazards to the community of receptor species is quite low until concentration values reach the
80th to 85th percentile, but then rise rapidly. For example, a concentration value equal to the
95th percentile (about 28 ug/L, which will occur approximately 5% of the time) is expected to impact
approximately 68% of the exposed species, and the 99th percentile (which will occur about 1% of the
time) is expected to impact nearly all of the exposed species.
Page 4-24
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
0.10
SSD of TRV values
in multiple species
20 30
Concentration (ug/L)
40
O
20 30
Concentration (ug/L)
40
100%
80% -
0%
50
50
0.70 0.75 0.80 0.85 0.90
Percentile of the Concentration Distribution
0.95
1.00
Figure 4-7. Example Presentation of Species Sensitivity Distribution.
Page 4-25
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.6.2 VARIABILITY IN RESPONSE
As noted above, HQ and Hazard Index (HI) (where appropriate) values are a convenient way to
characterize risk to ecological receptors, but interpreting the biological significance of the ranges of HQ
values greater than 1 is not always easy. One of the main advantages to the PRA approach is that
distributions of exposure may be combined with exposure-response distributions in order to generate
distributions that characterize the frequency and magnitude (severity) of responses in an exposed
population. Two examples of this approach are presented below.
Example 1: Dichotomous Response
In this hypothetical example, atoxic chemical is being transported by surface water run-off from
a Superfund site into a nearby stream. Because of short-term and seasonal variability in rainfall levels
(which influences both run-off rate and stream flow), the concentration of the chemical in the stream has
been observed to vary as a function of time. The risk manager at the site wants to know two things:
(1) How often will the concentration enter a range that can cause acute lethality in fish?; and (2) When
that happens, what percent of the fish population is likely to die? Exhibit 4-6 summarizes the
hypothetical concentration data and illustrates the basic approach. In this case, the concentration data are
most conveniently modeled as an empirical PDF. Next, assume that the acute concentration-lethality
curve is available for the chemical of interest in a relevant indicator species offish. For convenience,
assume the response function is the same as that shown in Exhibit 4-3. Then, the PDF for acute mortality
may be generated by repeated sampling from the concentration distribution and calculating the probability
of response (acute mortality) for each concentration value selected. Because this is a case where the
entire population offish at the exposure location may be assumed to be exposed to the same concentration
in water, the probability of mortality in a single fish is equivalent to the average fraction of the population
that is expected to die as a result of the exposure. The resulting PDF is shown in the graph in Exhibit 4-6.
As seen, lethality is expected to be low or absent about 95% of the time, but about 5% of the time the
concentration may enter a range where acute lethality may occur. The extent of mortality within the
exposed population is expected to range from about 20% at the 97th percentile of exposure (i.e., this is
expected to occur about 3% of the time), up to about 70% at the 99th percentile of exposure (i.e., this is
expected to occur about 1% of the time).
Page 4-26
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
Scenario
EXHIBIT 4-6
MODELING VARIABILITY IN A DICHOTOMOUS RESPONSE
Exposure of a population of fish to concentration values in a stream that vary over time
Hypothetical Concentration Data in Water
Response
PRA Simulation
Step 1
Step 2
Example Output
Value
0.5
1.1
2.5
5.1
9.2
15.8
24.7
52.6
83.1
(1/2 DL)
(max)
Percentile
0.00
0.10
0.25
0.50
0.75
0.90
0.95
0.99
1.00
Endpoint = acute mortality
Stressor-response model fit (see Exhibit 4-2)
P(c)= 1/(1+exp(4.8-0.1*c)
80% -
X 40% - -
20% -
0%
0.70
Draw a concentration at random from the empiric distribution
Calculate the probability of mortality at that dose
Track this as the forecast cell
Percentile
0.050
0.250
0.500
0.750
0.900
0.950
0.990
0.999
% Lethality
0.9%
1 .0%
1 .4%
2.0%
3.9%
9.1%
63%
96%
0.75
0.80 0.85 0.90
Percentile of Concentration
0.95
1.00
Page 4-27
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
Example 2: Continuous Response
Exhibit 4-7 provides a hypothetical example of modeling variability in response for a continuous
endpoint. In this example, assume that a toxic chemical has been released by a Superfund site and has
been transported in low levels by air to a nearby meadow. Among the receptors of potential concern in
the meadow are a number of different types of small mammal, and the field mouse has been selected to
serve as an indicator species for this group. The goal of the PRA is to characterize the effects of the
chemical on the growth of field mice in the meadow. Exposure occurs mainly by ingestion of seeds that
have been contaminated by uptake of the chemical from soil, and it has been determined that the
variability in average daily intake (DI) of chemical from the diet can be modeled as a lognormal
distribution with mean of 104 mg/kg-day, and a standard deviation of 127 mg/kg-day. Assume for
convenience that the exposure-response curve for growth inhibition in mice by the chemical is the same
as that presented previously in Exhibit 4-4. Given these inputs, the expected distribution of responses is
derived as follows:
Step 1: Draw a random value for the DI of a random member of the population
Step 2: Calculate the mean response m(d) and the standard deviation of the response s(d) for a
group of individuals exposed at that dose (d)
Step 3: Define the distribution of responses at that dose as NORMAL[m(d), s(d)]
Step 4: Draw a response from that distribution, and track this as the output variable
An example of the output for this example is shown in the two graphs at the bottom of
Exhibit 4-7. As seen, mice that are not exposed to the chemical display a range of growth rates ranging
from about +10% to +40%. Many of the mice (about 90%) residing in the contaminated field are
experiencing a range of growth rates that are only slightly decreased from rates expected for unexposed
animals. However, about 10% of the animals have weight gains that are markedly less than for
unexposed animals, ranging from about +5% to -30% (i.e., a net weight loss of 30% compared to the
starting weight).
It should be noted that the response distribution calculated in this way is what would be expected
for a large population of exposed receptors. If the actual exposed population is small, then the actual
response distribution may vary somewhat compared to the typical response shown in Exhibit 4-7. In
cases where it is important to evaluate this variability about the expected average pattern of response, this
may be done by running repeated Monte Carlo simulations using a number of trials (iterations) within
each simulation that is equal to the expected size of the exposed population. Each simulation will thus
represent a possible response distribution in the exposed population, and the range of responses across
different populations may be evaluated by comparing the multiple simulations. As noted above, the
magnitude of the variability between populations is expected to be small if the population size (number of
trials) is large, although this depends on the characteristics of the exposure and response functions.
Page 4-28
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-7
MODELING VARIABILITY IN A CONTINUOUS RESPONSE
Scenario
Exposure of a population of field mice to a chemical ingested via the food chain
Example Inputs
Exposure
Distribution of Average Dl LN(104,127)
Response (see Exhibit 4-3)
Endpoint = Growth (% increase in 21 days)
Stressor-response model fit
Mean response(dose) = 23.7 - 51.4*dose"n / (228.7A5.29 + dose"5.29)
Stdev (dose) = 7.0 (constant)
PRA Simulation
Step 1 Draw a dose at random from the lognormal distribution of dose
Step 2 Calculate the mean response [m(d)J and standard deviation of the response (s(d) at that dose
StepS Define the PDF for response at dose d: NORMAL(m(d), s(d))
Step 4 Draw a response at random from this PDF
Track this as the forecast cell
Example Output
Percent ile
0.05
0.25
0.50
0.75
0.90
0.95
0.99
Control
10.9
18.6
24.0
29.3
34.1
37.0
42.6
Exposed
-18.6
14.7
21.4
26.9
31.5
34.2
39.1
0.06
0.05 -
0.04 -
0.03 -
0.02 -
0.01 -
0.00
o 10
Body Weight Gain (%)
1.0
& 0.6 -
'H
£
o
it 0.4 -
0.2 -
0.0
-40
-30
-20
-10
0 10
Weight Gain (%)
20
30
40
50
Page 4-29
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.6.3 JOINT PROBABILITY CURVES
In this approach, if data are available to characterize the probability of a particular exposure
occurring, and an exposure-response curve is available, these may be combined to yield a curve (referred
to as a Joint Probability Curve) that shows the probability that a response greater than some specified
magnitude will occur. An example is shown in Figure 4-8. The upper panel shows a hypothetical
cumulative exposure probability distribution (plotted on the primary y-axis) along with the
exposure-response curve (plotted on the secondary y-axis). The steps needed to generate the Joint
Probability Curve are as follows:
Step 1: Select an exposure level "x" and record the probability (Px) of exceeding that exposure.
For example, in Figure 4-8, at an exposure of 12 units, the cumulative probability of exposure is
84%. Thus, the probability of exceeding that exposure is 16%.
Step 2: Find the expected response at that same exposure (RJ. In this case, the response at an
exposure of 12 is 2.2.
Step 3: Plot a data point at Rx on the
x-axis and Px on the y-axis.
Step 4: Repeat this process for many
different exposure levels, being sure
to draw samples that adequately cover
all parts of the probability scale.
The lower panel of Figure 4-8 shows the
results obtained using the hypothetical data in
the upper panel. The advantage of this format
is that it gives a clear visual display of both
the probability and magnitude (severity,
extent) of response. Further, the area to the
left of the curve is a relative index of the
population-level or community-level risk, and
comparison of this area across different
scenarios is helpful in comparing different
risk scenarios (both in risk characterization
and risk management). However, this
approach is based on the mean response at a
dose, and does not account for variability in
response between multiple individuals all
exposed at that dose. Employing a
two-dimensional Monte Carlo analysis
(2-D MCA) procedure could help to display
this variability in response between the
individuals at a given dose.
10 15 20
Exposure Level (aritrary units)
2345
Response (arbitrary units)
Figure 4-8. Example Joint Probability Curve.
Page 4-30
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
Note that unless 2-D MCA is used, this approach does not require Monte Carlo modeling.
Rather, the calculations can usually be performed in a spreadsheet format using built-in spreadsheet
functions.
4.7 MODELING UNCERTAINTY IN ECOLOGICAL RISK ASSESSMENTS
As emphasized above, one of the greatest potential benefits of the PRA approach is the ability to
combine estimates of uncertainty associated with different components of the exposure and risk models in
order to describe the overall uncertainty in final exposure or risk estimates. Some basic options for
characterizing and presenting uncertainty in exposure, toxicity, HQ, and response are presented below.
4.7.1 UNCERTAINTY IN EXPOSURE
Most estimates of dose-based exposure for terrestrial receptors (birds, mammals) are based on
calculated estimates of chemical intake using simple or complex food web models, sometimes coupled
with environmental fate and transport models that can link risk to a receptor with a source of
contamination. In cases where
receptors are exposed mainly by
direct contact rather than
ingestion (e.g., fish, soil
invertebrates, etc.), concentration-
based (as opposed to dose-based)
descriptors of exposures may be
derived using mathematical fate
and transport models. The basic
principles for modeling
uncertainty in ecological exposure
models (either dose-based or
concentration-based) are the same
as discussed in Appendix D. In
brief, probability distribution
functions of uncertainty (PDFu's)
are used to characterize the
uncertainty in the parameters of
the probability distribution
functions of variability (PDFv's)
for some or all variables in the
exposure model. Then, a
2-D MCA is used to derive
quantitative estimates of the
uncertainty around each percentile
of the variability distribution of
exposure. Figure 4-9 illustrates
the type of tabular and graphic
outputs that this approach
generates.
Variability
Percentile
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
Uncertainty Percentiles
5th
0.4
0.7
0.9
1.2
1.5
1.8
2.1
2.6
3.0
3.6
4.2
5.0
5.9
7.2
8.8
10.9
14.5
20.1
32.9
Mean
1.1
1.6
2.1
2.6
3.1
3.7
4.3
5.0
5.8
6.6
7.7
8.8
10.3
12.1
14.4
17.5
22.0
29.6
46.5
95th
2.0
2.8
3.5
4.2
5.0
5.9
6.7
7.6
8.7
9.9
11.3
12.9
14.8
17.2
20.3
24.1
30.1
39.4
60.0
o
1.0 -,
0.9-
0.8-
0.7-
0.6-
0.5-
0.4-
0.3-
0.2-
0.1 -
0.0-
Lower bound on exposure
^
Best estimate of exposure
•\
Upper bound on exposure/
0.1
1.0 10.0
Ingested Dose (mg/kg-day)
Figure 4-9. Example Presentation of Uncertainty in Exposure.
If exposure is based on
measured rather than calculated values by PRA (e.g., measured concentrations in an abiotic medium,
Page 4-31
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
measured concentrations in receptor tissues), uncertainty in the empirical or best-fit continuous
distribution through the data can be quantified using the statistical methods detailed in Appendix B.
As discussed in Chapter 1, it is important to understand that there are many sources of uncertainty
and that this approach to uncertainty analysis focuses mainly on parameter uncertainty and uncertainty in
the true shape of input variable distributions. It does not capture other sources of uncertainty relating to
the fundamental adequacy of the exposure and risk models used to describe the behavior of complex
biological systems or of sampling and analytical errors and uncertainties, so the uncertainty estimates
should always be interpreted in this light as being somewhat incomplete.
4.7.2 UNCERTAINTY IN TOXICITY
Toxicity information used for ERAs is often a source of uncertainty in the risk assessment
process. This uncertainty may arise from multiple areas and may include both quantitative uncertainty in
the dose-response data (involving toxicokinetics and study designs) and qualitative uncertainty in the
relevance of the data (involving toxicodynamics). Methods for characterizing the quantitative uncertainty
in both point estimates of toxicity (TRVs) and in full exposure-response curves are outlined below.
Uncertainty in TRVs
TRVs for a chemical are point estimates of exposure levels that do not cause an unacceptable
effect in an exposed receptor population. Ideally, all TRVs would be based on NOAEL and LOAEL
values derived from studies in which the receptor, endpoint, exposure route and duration were all matched
to the assessment endpoints defined for the site. However, such exact matches are seldom available.
Therefore, it is often necessary to extrapolate available toxicity data across route, duration, endpoint
and/or species, leading to uncertainty in the most appropriate value to use as the NOAEL or LOAEL.
There are no default methods for developing TRVs on a site. However, some options include the use of
allometric dose scaling models, physiologically-based biokinetic models, benchmark dose estimates or
other approaches based mainly on policy and/or professional judgment. Guidelines for dealing with the
uncertainty in components of the TRV derivation by uses of PRA are provided below.
Uncertainty in NOAELs andLOAELs
Uncertainty in the NOAEL or LOAEL for a chemical has two components: (1) uncertainty within
a study; and (2) uncertainty between studies, under exact specified conditions of exposure.
Assuming that a single study has been selected to provide the NOAEL and/or LOAEL values to
be used in deriving a TRV for a chemical, it is customary to define the NOAEL as the highest exposure
that did not cause a statistically significant effect, and the LOAEL is the lowest exposure that did cause a
statistically significant effect. As noted earlier (see Section 4.5.1), this approach has a number of
limitations, and there may be substantial uncertainty as to whether the observed NOAEL and LOAEL
values actually bracket the true threshold effect level. One way to quantify uncertainty in the exposure
levels that cause some specified level of adverse effect is through the use of exposure-response curve-
fitting software such as EPA's BMDS package. In this approach, the risk assessor selects some level of
effect that is judged to be below a level of concern, and another level of effect that would be of concern.
The choice of these response levels is a matter of judgment, and depends on the nature and severity of the
endpoint being evaluated. A specified level of effect is referred to as a Benchmark Response (BMR), and
the exposure that causes that response is referred to as the Benchmark Dose (BMD). Given information
on the number of test organisms in each test group and on the variability of the response in those
Page 4-32
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
organisms, the BMD software uses maximum likelihood methods to derive the 5% lower confidence
bound on the exposure that causes the BMR. This is referred to as the BMDL. This uncertainty bound
may be used to quantify the uncertainty in the BMD, and hence to characterize this source of uncertainty
in the TRY. The simplest method for approximating the uncertainty distribution around the BMD is to
assume the distribution is approximately normal, with mean equal to the BMD and standard deviation
(standard error) given by:
Stdev=(BMD - BMDL) / 1.645
For advanced analyses, a more accurate characterization of the uncertainty distribution around the BMD
may be derived by Monte Carlo simulation. In this approach, each model parameter is assumed to be
normally distributed, with mean and standard error values provided by the BMDS output. Monte Carlo
simulation is then used to select alternative model parameter sets, being sure to account for the covariance
between parameters (the covariance matrix is also provided by the BMDS output). For each parameter
data set, the BMD is calculated, and the distribution of BMD values across many iterations is a better
approximation of the uncertainty in the BMD.
Uncertainty in the effect level (NOAEL or LOAEL) for a chemical may also arise because there
is more than one study available for the chemical, and the studies do not yield equal estimates of the
effect level. It is important to note that the process of reviewing available toxicity studies, choosing the
most relevant endpoint for use in deriving a TRV, and identifying the most relevant study is a process
requiring basic toxicological expertise (not probability or statistics), and this process must be completed
both for point estimate and probabilistic risk assessments. In general, studies based on different receptors,
endpoints, exposure routes and/or durations are not equally relevant for evaluating a particular assessment
endpoint in a particular indicator species. However, in some cases, multiple studies of the same endpoint
in the same species will be available. In such a case, assuming that all the studies are judged to be equally
reliable, the best estimate of the LC50 may be derived by calculating the geometric mean of the available
alternative values (after adjustment to constant hardness). Uncertainty around the best estimate may then
be based on the observed inter-study variability, using the basic principles for choosing PDFu's as
described in Appendix B.
Uncertainty in Extrapolation ofTRVs
In general, extrapolation ofTRVs across species or endpoints is not desirable, since the
magnitude and direction of any potential error is generally not known. Sometimes, extrapolations
between species are attempted based on allometric scaling models that seek to adjust toxicity values
accounting for differences in body weight. Alternatively, physiologically-based pharmacokinetic (PBPK)
models that seek to account for differences in a number of other physiological variables (metabolism rate,
organ size, blood flow, etc.) can be used. However, the validity of these models is often not well
established. In those cases where these models are used, and where the uncertainty in the model is judged
to warrant quantitative evaluation, the primary source of the model should be consulted in order to derive
an estimate of the uncertainty in the quality of the extrapolation and in the parameters of the model. As
noted earlier, PRA may capture uncertainty associated with model input parameters, but does not usually
capture all sources of uncertainty in the model. In particular, most models of this sort are designed to
extrapolate only the average response as a function of dose, and are not intended to extrapolate variability
between individuals at a specified dose. When no mathematical model is available to support quantitative
extrapolation across species, exposure duration or endpoint, professional judgment and/or policy may be
used to select extrapolation factors to account for the uncertainty.
Page 4-33
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
The risk assessor should ensure that the risk manager understands the uncertainty associated with
any model selected and applied, and that the results of the calculations (point estimate or PRA) are
conditional upon the model selected.
Uncertainty in Parameters of the Dose-Response Models
When toxicological exposure-response data are fit to mathematical equations, the fitting software
will usually provide quantitative information on the uncertainty in the best estimates for each of the
model parameters. For example, in the dichotomous model illustrated in Exhibit 4-3, the output from the
BMDS software included the following information on the uncertainty in the parameters of the best-fit
logistic equation:
Parameter
a
b
Best Est
-4.80
0.101
Std Error (SE)
0.83
0.019
Because the uncertainty in the best estimate of each model parameter is asymptotically normally,
uncertainty in the parameters may be modeled as:
PDFu (parameter i)=NORMAL(best estimate of parameter i, SE of parameter i)
Note that the parameters of the model are generally not independent, and generally should not be treated
as such. Thus, when modeling the uncertainty in the parameters of the best-fit exposure-response model,
the PDFv's for the parameters should be correlated according to the correlation matrix or the variance-
covariance matrix, as provided by the modeling software.
4.7.4 UNCERTAINTY IN RESPONSE
If the risk characterization phase of the risk assessment focuses on an estimation of the
distribution of responses rather than the distribution of HQ values, the uncertainty in the distribution of
responses can be evaluated using two-dimensional Monte Carlo techniques using PDFu's for the
parameters of the exposure and exposure-response models derived as described above. The same
graphical output may be used for this presentation as was illustrated in Figure 4-9, except that the x-axis
is response rather than HQ. This format is illustrated in Figure 4-10 for a dichotomous endpoint (e.g.,
acute lethality). In this example, the average probability of response among the members of the exposed
population (shown in the graph by the black diamond symbols) is 8.2%, with a confidence bound around
the mean of 4.9 to 12.8%. This is equivalent to concluding that about 8.2% of the population is expected
to suffer acute lethality, but the true fraction dying could range from 4.9 to 12.8%.
Page 4-34
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
CL
o>
E
O
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0
Lower bound on
response
Best estimate of
response
Upper bound on .
response
Mean response
indicated by
black
diamonds
.1%
1.0% 10.0%
Response (Probability of lethality)
100.0%
Figure 4-10. Example Presentation of Uncertainty in Response.
4.7.3 UNCERTAINTY IN HAZARD QUOTIENT
Once the uncertainty in exposure and/or toxicity distributions has been characterized as described
above, there are a number of options for presenting the resultant uncertainty in the HQ (or HI, if
appropriate and applicable for summing HQs) distributions. Figure 4-11 shows one simple graphical
format, where the point estimate of the TRY is superimposed on the uncertainty bounds of the exposure
distribution (upper panel), or the uncertainty bounds of the TRV are superimposed on the best estimate of
exposure (lower panel). One could also superimpose the range of TRVs over the range of exposures, to
capture most of the uncertainty in the HQ. Furthermore, such distributional outputs should always show
the point estimate ranges of CTE and RME exposures in respect to the ranges of TRVs, for use in weight-
of-evidence to help interpret the PRA and point estimate results. The advantage of this format is that no
additional Monte Carlo modeling is needed to derive initial descriptors of uncertainty in risk. For
example, in the upper panel it may be seen that the best estimate of the fraction of the population exposed
at a level below the TRV is about 83%, but that this is uncertain due to uncertainty in the exposure
estimates, and the true percent below the TRV might range from 74 to 90%. Similarly, in the bottom
panel, the best estimate of the fraction of the population below the TRV is also about 83%, but due to
uncertainty in the TRV the actual value could range from 64 to 91%. Uncertainty could also be presented
by showing a combined graph with both ranges of exposure and TRVs, such as described below.
Page 4-35
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
1.0
0.9 -
O.I
0.7 -
0.6 -
0.5 -
0.4 -
0.3 -
0.2 -
0.1 -
0.0
Lower bound on exposure
Best estimate of exposure
Upper bound on exposure
0.1
1.0 10.0
Ingested Dose (mg/kg-day)
Best
estimate
of TRV
100.0
1.0
0.9 -
0
0.7 -
0.6 -
0.5 -
0.4 -
0.3 -
0.2 -
0.1 -
0.0
Best estimate of exposure
0.1
Lower bound on TRV
1.0 10.0
Ingested Dose (mg/kg-day)
Upper
bound on
TRV
Best
estimate
of TRV
100.0
Figure 4-11. Example Presentation of Uncertainty in Exposure and TRV.
Page 4-36
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
A more complete characterization of uncertainty in HQ may be achieved by using PRA to
combine the uncertainty in both the exposure and the TRY terms, resulting in the uncertainty bounds on
the HQ distribution itself (see Figure 4-12). In this example, it may be seen that 63% of the exposed
population is estimated to have an HQ below 1.0, but that this is uncertain due to uncertainty in both the
exposure distribution and the TRV, and that the true fraction of the population below a level of concern
(HQ < 1) could range from 45 to 81%.
1.0
0.9
0.8
to.7
.Q
CD
.Q
Q
a.
CD
>
is
^
E
^
O
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Lower bound
on HQ
Upper bound on
HQ
Best estimate of
HQ
0.01
0.1
10
HQ
Figure 4-12. Example Presentation of Uncertainty in HQ Estimates.
4.8 INTERPRETING RESULTS OF AN ECOLOGICAL PRA
In some cases, the information contributed by a PRA may provide a more complete
characterization of risks to a population of receptors than can be obtained by using point estimate
methods. However, whether by PRA or by point estimate or a combination, the results of the risk
assessment must be interpreted to reach a risk management decision.
In contrast to the case for human health risk assessments (where default risk-based decision rules
are well established), there are no established default decision rules for identifying when risks to
ecological receptors are and are not of concern. In the point estimate approach, EPA guidance (U.S. EPA
1992b, 1995) recommends consideration of both the RME and CTE exposure/dose estimates along with
TRVs based on both LOAELs and NOAELs (U.S. EPA 1997a) to reach a risk management decision. The
same principle applies to probabilistic ERAs.
In some cases, interpretation of an ecological PRA is relatively simple. For example, if the
distribution of HQ values calculated using an appropriate NOAEL-based TRV are less than 1.0 for nearly
all members of the population, then it is likely that risks are within an acceptable range for the population.
Conversely, if the distribution of HQ values calculated using a LOAEL-based TRV are significantly
greater than 1.0 for most members of an exposed population, then it is likely that risks are not acceptable
Page 4-37
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
for the population. However, for cases which fall between these bounding conditions (and for cases
where one needs to clearly define the boundaries of potential excess risks for a gradient of contamination
and exposures), the level of risk or response that is considered acceptable must be defined by the risk
assessor and the risk manager on a site-specific and receptor-specific basis. This evaluation should take
the following factors into account:
(1) The Risk Management Goal
The risk management objective for most Superfund ERAs is defined as population sustainability
(U.S. EPA, 1999). In this case, harm to some members of the exposed population may be acceptable, if
that harm does not lead to an overall reduction in population viability. This situation (protection of a
population rather than protection of individuals) is sometimes equated with use of the CTE (average)
receptor as the basis for risk management decision making. That is, if the HQ for the CTE receptor is
below a level of concern, it is sometimes assumed that population risks are acceptable.
However, the choice of the CTE receptor as the basis for risk management decision making may
not be sufficiently protective in all cases. For the vast majority of wild populations, the proportion of the
population that must be protected to ensure population stability will be unknown. At a small number of
sites, a population biologist may be able to provide some information. Moreover, the percentile of the
CTE receptor in the exposure or risk distribution may vary depending on the shape of the distribution.
The proportion of the population experiencing exposure greater than that of the CTE receptor could range
from less than 10% up to 50% or even higher. Also, the ecological significance of an adverse effect on
some members of a population depends on the nature of the stressors and on the life history and
population biology of the receptor species. Because of these complexities, use of the CTE as a decision
threshold for nonthreatened or endangered species may be appropriate in a small number of cases, but risk
assessors and risk managers should realize that the choice of the CTE receptor requires a species- and
endpoint-specific justification and the CTE should not be used as the default basis for a risk management
decision. Rather, for the majority of ERAs, the risk management decision should be based on the RME
receptor or an upper percentile of the distribution of variability in risk/exposure.
(2) The Toxicological Basis of the TRY
The biological significance of a distribution of variability in HQ cannot be interpreted without a
proper understanding of the nature of the TRY being used to evaluate the distribution. This includes the
nature of the toxicological endpoint underlying the TRV, its relevance to the assessment endpoint, and the
shape (steepness) of the dose-response curve. For example, an HQ of 2 based on an EC20 for reduction
in reproductive success would likely be interpreted as more significant lexicologically than an HQ of 2
based on the EC20 for an increase in liver weight. Likewise, an HQ of 2 based on an LClow for acute
lethality would be more significant if the dose-response curve for lethality were steep than if it were
shallow, since it would be easier to cause greater response with smaller increases in exposure to
contaminants.
Page 4-38
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
(3) The Characteristics of the Receptor
Ultimately, the question which must be assessed is whether an effect of degree "x" occurring in
"y" percent of the population is biologically and ecologically significant. This, in turn depends on the
attributes of the receptor being evaluated. For example, a reduction of 10% in the reproductive success of
a fecund and common species (e.g., the field mouse) might not lead to a significant reduction in
population number, while the same effect could be of concern in a species with lower fecundity and/or
lower population density (e.g., the moose). Thus, the interpretation of an analysis of variability in
exposure and/or effect often requires the input of a trained population biologist with expertise in the
receptor of concern.
Because of these issues, there is no default rule for what level of effect is and is not acceptable for
an exposed ecological population; except for the case of no potential excess risks where the RME
exposures do not exceed the TRY based on a NOAEL, assuming there is reasonable confidence in those
exposure and toxicity values. In some cases, mathematical models may be available for predicting the
population-level consequences of a given pattern of effects (e.g., see ECOFRAM 1999a for some aquatic
population models), but in general the extrapolation from a distribution of individual responses to an
estimation of population-level effects is difficult. For this reason, close consultation between the risk
manager and the ecological risk assessor is necessary for translating results of an ERA into an appropriate
and successful risk management decision.
4.9 GUIDELINES FOR PLANNING AND PERFORMING A PROBABILISTIC ERA
4.9.1 PLANNING AN ECOLOGICAL PRA
Chapter 2 provides a general discussion of the key steps that should be followed when planning a
PRA. These guidelines are equally applicable to ecological PRA as to human health PRA. Of the key
steps in the process, most important are the following:
Dialogue Among Stakeholders
As discussed in Section 4.2, the decision if and when to perform an ecological PRA is an SMDP
shared by risk assessors, risk managers, and stakeholders, including members of the public,
representatives from state or county environmental agencies, tribal government representatives, natural
resource trustees, private contractors, and potentially responsible parties (PRPs) and their representatives.
A scoping meeting should be held after the completion of the baseline risk assessment in order to discuss
the potential purpose and objectives of a PRA, and to identify the potential value of the analysis to the
risk management process. If it is decided to perform at least an initial PRA evaluation, subsequent
meetings of a similar type should occur iteratively in order to assess whether any further effort is
warranted.
Preparation of a Workplan
Any PRA beyond the simplest screening level evaluation should always be accompanied by a
workplan. The purpose of the workplan is to ensure that all parties agree on the purpose and scope of the
effort, and on the specific methods, data, and procedures that will be used in the PRA. Workplans should
be developed according to available guidance for workplans for nonprobabilistic ERA (U.S. EPA, 1992b,
Page 4-39
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
1997a) and should consider three elements: (1) the 16 guiding principles of MCA (U.S. EPA, 1997b);
(2) the eight guidelines for PRA report submission (U.S. EPA, 1997b); and (3) the tiered approach to
ERA (U.S. EPA, 1997a). Development of a workplan for PRA is discussed in greater detail in Chapter 2,
and Exhibit 4-8 summarizes the key elements of a proper workplan. The workplan must be submitted to
the BTAG coordinator and/or regional ecotoxicologist for review and for approval by the risk manager.
The EPA strongly recommends that PRPs who wish to perform PRAs of ecological risk involve the
Agency in the development of a workplan in order to minimize chances of significant disagreement, as is
required by EPA policy.
EXHIBIT 4-8
EXAMPLE ELEMENTS OF A WORKPLAN FOR ECOLOGICAL PRA
1. Introduction/Overview
Conceptual site model
Assessment endpoints
Indicator species
Measures of exposure and effect
2. Description of Exposure and Risk Models
Basic exposure models (fate and transport, uptake, food web, intake, etc.)
Basic risk models (HQ, dichotomous response, continuous response)
3. Results from a Point Estimate Assessment
CTE and RME risk estimates from baseline evaluation
4. Rationale why a PRA will be helpful
Goals of the assessment (variability, uncertainty, both)
Expected benefit to risk manager
5. Description of the Proposed PRA
Exposure scenarios to be evaluated
Output variables to be modeled in variability and/or uncertainty space
6. Proposed PDFs, and their basis
Method for performing sensitivity analysis and for selecting key variables
Data source for characterizing key variables
Approach for selecting and parameterizing key variables
Proposed list of PDFs for exposure variables (optional but desirable)
Method for dealing with the concentration term
Method for dealing with correlations
7. Proposed Software and Simulation Approach
Commercial or custom
Monte Carlo or Latin Hypercube
Number of Iterations
Method(s) for sensitivity analysis
8. Preliminary Results (optional, but helpful)
Results of a screening level evaluation
Identification of variables where more effort is needed to improve the
distribution function
Page 4-40
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
4.9.2 EVALUATING AN ECOLOGICAL PRA
When an ecological PRA is submitted to EPA for consideration, it will be reviewed in order to
determine if it has been performed in accord with sound principles of ERA (U.S. EPA, 1997a, 1998), and
with sound principles of PRA (U.S. EPA, 1997b). A general checklist that may be helpful to reviewers is
provided in Appendix F, and key features of this checklist are summarized in Exhibit 4-9. Eight specific
conditions for acceptance of a PRA submitted to EPA are provided in U.S. EPA (1997b).
At the discretion of EPA risk assessor or risk manager, the PRA report may be submitted for
additional EPA internal review and/or an external review process in accord with Agency guidelines for
conducting peer reviews (U.S. EPA, 2001). The external peer review may be used in cases where the
issues are complex or contentious and the opinions of outside expert peer reviewers can improve the
PRA.
EXHIBIT 4-9
CHECKLIST FOR INCLUDING A PRA AS PART OF THE ERA (SEE APPENDIX F)
All risk assessments should include point estimates prepared according to current Superfund national and
regional guidance.
A workplan must be submitted for review and approval by the appropriate EPA regional project manager
(RPM) and/or BTAG coordinator prior to submission of the PRA.
A tiered approach should be used to determine the level of complexity appropriate for the ERA. The
decision to ascend to a higher level of complexity should be made with the risk manager, regional risk
assessor and other stakeholders.
The eight conditions for acceptance presented in the EPA policy on PRA (U.S. EPA, 1997b) should be
clearly addressed by each PRA submitted to the Agency.
Information in the PRA should possess sufficient detail that a reviewer can recreate both the input
distributions and all facets of the analysis. This includes copies of published papers, electronic versions
of necessary data and other materials deemed appropriate by EPA.
4.10 EXAMPLE OF THE TIERED PROCESS IN ERA
As discussed in detail in Chapter 2, one of the key elements in the risk assessment process is
deciding if and when further analysis is warranted. This includes decisions regarding whether to employ
PRA calculations to supplement point estimate calculation, and if so, what level of effort to invest in
those PRA calculations. The following section presents a relatively simple hypothetical example
illustrating how the tiered approach might operate at a site where ecological risk is an important concern.
Page 4-41
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
Problem Formulation
PestCorp is a former chemical manufacturing facility that produced mainly chlorinated pesticides
10 to 20 years ago. Data collected on the PestCorp property indicate that a number of spills or releases of
chlorinated pesticides took place when the facility was in operation, and that site soils are broadly
contaminated, especially with pesticide X. This contaminated soil has lead to impacts on a nearby lake of
about 300 acres that receives surface water runoff from the PestCorp site. Samples from the lake reveal
low but detectable levels of pesticide X in water, with relatively high values in sediment and in the tissues
of a variety of aquatic organisms (crayfish, snails, benthic macroinvertebrates and fish). The
concentration values in all media (water, sediment, aquatic organisms) tend to be highest in the part of the
lake receiving runoff from the PestCorp property, with a gradient of diminishing values at locations
further away from the area where runoff enters the lake.
A BTAG committee formed by EPA to identify potential ecological concerns at the site
recognized that many different species could be exposed to the contaminants in the lake, including
aquatic receptors residing in the lake (fish, invertebrates, aquatic plants), as well as mammals and birds
that frequent the lake for food or water. Because pesticide X is lipophilic and tends to biomagnify in the
food web, the BTAG decided that the highest risks would likely occur in higher-level predators such as
mammalian omnivores, and selected the racoon as a good indicator species to represent this trophic
group. Pathways of exposure that were identified as warranting quantitative evaluation included
(a) ingestion of water, (b) ingestion of aquatic food items, and (c) incidental ingestion of sediment while
feeding or drinking at the lake. The BTAG determined that the assessment endpoint was protection of
mammalian omnivore populations.
Point Estimate Risk Evaluation
A series of iterative screening-level point estimate calculations (Steps 1 to 2 of the 8-step ERAGS
process) were performed to investigate whether or not there was a basis for concern at the site. Initial
calculations using simplified and conservative inputs (i.e., exposure based on the maximum measured
concentration in each medium, an area use factor of 1, and the most conservative available TRVs)
indicated that the HQ value for pesticide X could be quite large. Therefore, a refined screening level
evaluation was performed in which point estimates of CTE and RME risk were derived using the best
information currently available. Key elements of the approach are summarized below:
• The CTE receptor was assumed to be exposed at a location where concentration values were the
average for the whole lake, and the RME receptor was assumed to be exposed at a location where
concentrations were equal to the QS^percentile of values from the lake.
• Because only limited data were available for measured concentrations of pesticide X in aquatic
prey items, the concentration values in aquatic prey were estimated using a linear
bioaccumulation model: C(prey)=C(sed) x BAF. The BAF was estimated from the existing data
by finding the best fit correlation between the concentration values in sediment and crayfish at
7 locations in the lake: C(crayfish)=5.04 x C(sed) (R2=0.792).
• The TRV values were based on a study in mink in which the toxicity endpoint was the percent
inhibition of reproductive success.
Page 4-42
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
These inputs and the resulting HQ values are shown in Exhibit 4-10. As seen, estimated risks to the CTE
receptor approach or slightly exceed a level of concern (HQ=4.7E-01 to 1.4E+00), and risks to an RME
receptor are well above a level of concern (9.1E+00 to 2.7E+01). The chief pathway contributing to the
dose and risk is ingestion of contaminant in aquatic food web items (crayfish, fish, amphibians, etc.).
EXHIBIT 4-10
REFINED SCREENING POINT ESTIMATE INPUTS AND RESULTS
Basic model
HQ = DI(total)/TRV
Dl(total) = Dl(water) + Dl(food) + Dl(sed)
DI(i) = C(i)*IR(i)*AUF(i)
Other Assumptions
C(diet) = C(sed) * BAF
IR(sed) = IR(diet)*F(sed)
IR(diet) = IR(total)*F(diet)
Category
Inputs
Results
Variable Variable Units
Concentration Concentration in water mg/L
Concentration in sediment mg/kg
BAF (sediment to aquatic prey)
Concentration in aquatic prey mg/kg
Intake Rates Total water intake rate L/kg-day
Total food intake rate kg/kg-day
Fraction of diet that is sed
Fraction of diet that is aquatic prey
Area Use Factors Fraction of total water ingested at the lake
Fraction of total diet from the lake
TRVs LOAEL-based TRV mg/kg-day
NOAEL-based TRV mg/kg-day
Daily Intake Water ingestion mg/kg-day
Sediment ingestion mg/kg-day
Aquatic prey ingestion mg/kg-day
Total mg/kg-day
HQ (LOAEL-Based) Water ingestion
Sediment ingestion
Aquatic prey ingestion
Total
HQ (N GAEL- Based) Water ingestion
Sediment ingestion
Aquatic prey ingestion
Total
Point Est. Values
CTE RME
0.12 0.38
24 77
5 5
1 20 385
0.082 0.12
0.06 0.09
0.03 0.06
0.15 0.25
0.3 0.6
0.25 0.6
0.6 0.6
0.2 0.2
3.0E-03 2.7E-02
1.1E-02 2.5E-01
2.7E-01 5.2E+00
2.8E-01 5.5E+00
4.9E-03 4.6E-02
1.8E-02 4.2E-01
4.5E-01 8.7E+00
4.7E-01 9.1E+00
1 .5E-02 1 .4E-01
5.4E-02 1.2E+00
1.4E+00 2.6E+01
1.4E+00 2.7E+01
Page 4-43
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
SMDP 1 at Step 2 ofERAGS
The BTAG considered these results to indicate that inhibition of reproduction was possible in at
least some members of the exposed population, but that the fraction of the population that was affected
and the degree of impact on the population was difficult to judge from the point estimate calculations.
Based on this, a decision was made to conduct a screening level PRA in order to provide some additional
information on the magnitude and probability of risk.
Workplan 1
The contractor performing the risk assessment developed a brief workplan that proposed an
approach for a screening level PRA. The plan called for a Monte Carlo-based evaluation of variability in
exposure and risk among different members of the exposed mammalian omnivore (racoon) population. In
brief, all exposure inputs that were treated as constants in the point estimate approach (i.e., were the same
for CTE and RME exposure) were also treated as constants in the PRA evaluation. Because water
contributed so little to dose or HQ, this pathway was not evaluated in the PRA, but was accounted for by
adding in the point estimate values to the PRA results. All variables that are fractions (i.e, may only
assume values between zero and one) were modeled as beta distributions, and all other variables were
modeled as lognormal. For screening purposes, the parameters for all distributions were selected so that
the mean and 95th percentile values of the PDF's matched the corresponding CTE and RME point
estimates. The BTAG reviewed this proposed approach and authorized PRA work to begin.
Screening Level PRA Results
The screening level PRA inputs and the resulting estimates of the variability in HQ are shown in
Exhibit 4-11. The CTE and RME point estimates are also shown for comparison. As seen, the PRA
distribution of HQ values indicates that about 68% of the individuals in the population are likely to have
HQ values below 1E+00, while 32% have HQ values above 1E+00.
Comparison of the CTE point estimates of HQ to the mean HQ values derived by PRA reveals
the values are very close. This is expected because both depend on the mean values of the input
variables, and the same mean values were used in both sets of calculations. With regard to upper-bound
estimates, the RME point estimate values are at the 98th percentile of the PRA HQ distribution, within the
target range (90th to 99th) usually considered appropriate. Note, however, that the 98th percentile is about
5-fold higher than the 95th percentile, emphasizing the high sensitivity of the RME HQ values to the
precise percentile of the RME.
Page 4-44
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-11
SCREENING LEVEL PRA CALCULATIONS OF HQ DISTRIBUTION
Data Category
Concentrations
Intake Rates
Area Use Factors
TRVs
Variable
Concentration in water
Concentration in sediment
BAF
Concentration in aquatic prey
Total water intake rate
Total food intake rate
Fraction of diet that is sed
Fraction of diet that is aquatic prey
Fraction of total water ingested from lake
Fraction of total diet from the lake
LOAEL-based TRV
NOAEL-based TRV
Units
mg/L
mg/kg
-
mg/kg
L/kg-day
kg/kg-day
--
--
__
--
mg/kg-day
mg/kg-day
Screening Level Distribution
Type param 1 param 2
Not evaluated in PRA
LN 24 33
Const 5
Calculated
Not evaluated in PRA
LN 0.060 0.060
Beta 3.42 110.7
Beta 6.10 34.6
Not evaluated in PRA
Beta 1 .20 3.59
Const 0.6
Const 0.2
0%
0.01
1
HQ Value
10
100
TRV Basis
NOAEL
LOAEL
Central Tendency
Mean of PRA
1.44
0.48
Point Est CTE
1.42
0.47
Ratio
0.99
0.99
Upper Bound
95th of PRA
5.4
1.80
Point Est. RME
27.4
9.12
Ratio
5.06
5.06
Page 4-45
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
SMDP2
The BTAG considered these results, and decided that it was very probable that pesticide X was
causing an effect in some members of the exposed population, but decided that a final risk management
decision would be facilitated by characterizing the distribution of responses (rather than the distribution
of HQ values). The BTAG asked the contractor performing the work to develop a proposed approach for
characterizing the distribution of responses.
Workplan 2
The contractor obtained a copy of the toxicity report upon which the TRVs were based, and
determined that the study did include sufficient dose-response data to support reliable dose-response
modeling. The contractor recommended that this be done using EPA's BMDS. The BTAG approved this
proposed approach and authorized work to proceed.
PRA Refinement 1
The contractor fit the raw dose-response data (inhibition of reproduction in mink) to a number of
alternative models available in BMDS, and found that the dose-response curve could be well
characterized by the Hill Equation with nonconstant variance, as follows:
Mean Response at dose d (% decrease in reproduction)=(100 x d25)/(0.925 + d25)
Std. Dev. in Response at dose d (%)=SQRT[1.6-(mean response at dose d)1-3]
Based on this model, the point estimate LOAEL value (0.6 mg/kg-day) corresponds to an effect level of
about 27%, and the NOAEL of 0.2 mg/kg-day corresponds to an effect level of about 2%.
Using this exposure-response model in place of the point-estimate TRV values, the refined PRA
predicted a distribution of responses in the exposed population as shown in Exhibit 4-12. As seen,
approximately 81% of the population was predicted to experience an effect on reproduction smaller than
10%, while 9% were expected to have a reduction of 10 to 30%, 4% a reduction of 30 to 50%, and 6% a
reduction of more than 50%. On average across all members of the exposed population, the predicted
reduction in reproductive success was about 9%.
Page 4-46
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
EXHIBIT 4-12
SIMULATED DISTRIBUTION OF RESPONSES
Exposure-Response Model
Resp = Normal(Mean,Stdev)
Mean = a + b*x"n / (x"n + k"n)
Stdev = alpha*meanArho
x Total daily intake
a 0
b 100
k 0.9
n 2.5
alpha 1.6
rho 1.3
100%
C
10
20
30 40 50 60 70 80
Percent Reduction in Reproductive Success
90
100
Percent
Reduction
0-10%
1 0-30%
30-50%
>50%
Percent
of Population
81%
9%
4%
6%
Page 4-47
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
SMDP3
The BTAG debated the likely population-level consequences of this predicted distribution of
responses in members of the exposed population. After consulting with a field biologist with experience
in the population dynamics of mammals such as racoons, the BTAG decided that the distribution of
responses in the exposed population would cause a continued stress on the mammalian omnivore
community and that reductions in population number were likely over time. Based on this, the risk
manager and the BTAG agreed that remedial action was desirable and that a range of alternative clean-up
strategies should be investigated. This was performed using the methods described in Chapter 5 (see
Exhibit 5-5).
Page 4-48
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 4 ~ December 31, 2001
REFERENCES FOR CHAPTER 4
ECOFRAM. 1999a. ECOFRAM Aquatic Report (Draft). Ecological Committee on FIFRA Risk
Assessment Methods. Draft report available online at http://www.epa.gov/oppefedl/ecorisk.
Report dated May 4.
ECOFRAM. 1999b. ECOFRAM Terrestrial Draft Report. Ecological Committee on FIFRA Risk
Assessment Methods. Draft report available online at http://www.epa.gov/oppefedl/ecorisk.
Report dated May 10.
U.S. EPA. 1989. Risk Assessment Guidance for Superfitnd, Volume II: Environmental Evaluation
Manual. Interim Final. Office of Emergency and Remedial Response. Washington, D.C.
EPA/540/1-89/001. March.
U.S. EPA. 1991-present. Eco Update. Intermittent Bulletin Series. Office of Emergency and
Remedial Response. 1991 to present.
U.S. EPA. 1992a. Framework for Ecological Risk Assessment. EPA Risk Assessment Forum.
EPA/630/R-92/001. February.
U.S. EPA. 1992b. Policy Memorandum: Guidance on Risk Characterization for Risk Managers
and Risk Assessors from F. Henry Habicht, Deputy Administrator, February 26.
U.S. EPA. 1994. Memorandum: Role of the Ecological Risk Assessment in the Baseline Risk
Assessment. Elliott Laws, Assistant Administrator, Office of Solid Waste and Emergency
Response. OSWER Directive No. 9285.7-17. August 12.
U.S. EPA. 1995. EPA Risk Characterization Program. Memorandum from the Administrator.
March 21.
U.S. EPA. 1997a. Ecological Risk Assessment Guidance for Superfund: Process for Designing
and Conducting Ecological Risk Assessments. Interim Final. Solid Waste and Emergency
Response. OSWER Directive No. 9285.7-25. June 5.
U.S. EPA. 1997b. Guiding Principles for Monte Carlo Analyses. Risk Assessment Forum.
EPA/630/R-97-001.
U.S. EPA. 1998. Guidelines for Ecological Risk Assessment. Risk Assessment Forum. U.S.
Environmental Protection Agency, Washington DC. EPA/630/R-95/002F. April.
Published May 14. Federal Register 63(93):26846-26924.
U.S. EPA. 1999. Memorandum: Issuance of Final Guidance: Ecological Risk Assessment and Risk
Management Principles for Superfund Sites. P. Stephen D. Luftig for Larry D. Reed, Office of
Emergency and Remedial Response. OSWER Directive No. 9285.7-28. October 7.
U.S. EPA. 2001. Risk Assessment Guidance for Superfund: Volume I. Human Health Evaluation
Manual (Part D, Standardized Planning, Reporting, and Review of Superfund Risk Assessments).
Office of Emergency and Remedial Response. Washington, DC. OSWER Directive
No. 9285.7-47. December.
Page 4-49
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
CHAPTER 5
PROBABILISTIC RISK ASSESSMENT AND
PRELIMINARY REMEDIATION GOALS
5.0 INTRODUCTION
According to the National Contingency Plan
(NCP) (U.S. EPA, 1990a, 40CFR §300.430(d)(4)), risk
assessment and risk management decision making go
hand-in-hand: data from the remedial investigation are
used to characterize risk, and results of the baseline risk
assessment help to establish acceptable exposure levels
for use in developing remedial alternatives. In practice,
risk managers may identify two major objectives of risk
assessment: (1) to determine if remediation is necessary
(i.e., Is there unacceptable risk at the site?}; and (2) if
remediation is necessary, to determine a preliminary
remediation goal (PRG) (i.e., What chemical
concentrations would result in a risk estimate that will
be adequately protective of human health and the
environment?}. The answer to the first question (is there
unacceptable risk?} depends upon a number of factors,
including the measured or estimated concentration levels
of contaminants in site media, and takes uncertainty in
the measurements into account. In contrast, the answer
to the second question (what is the PRG needed to
achieve a specified level of protection?} does not
necessarily depend on any knowledge of the actual level
or pattern of site-specific concentration data, and does
not necessarily depend on the uncertainty in site
concentration data. Thus, while exposure point
concentrations (EPCs) and PRGs are closely related to
each other, they have important differences (see
Section 5.1 for further elaboration on EPCs and PRGs).
Once a risk manager has selected a PRG at a site,
determining whether a particular area meets or will meet
the PRG requires careful comparison of site data with the
PRG, including a consideration of the uncertainty in the
site data. For a further discussion on variability and
uncertainty in the concentration term, readers are urged
to consult Appendix C in this guidance.
EXHIBIT 5-1
SUMMARIES OF SOME KEY TERMS
Preliminary Remediation Goal (PRG) - initially
developed chemical concentration for an
environmental medium that is expected to be
protective of human health and ecosystems. PRGs
may be developed based on applicable or relevant
and appropriate requirements, or exposure scenarios
evaluated prior to or as a result of the baseline risk
assessment. (U.S. EPA, 1991a).
Generic PRG - a chemical concentration protective
of human health developed prior to the baseline risk
assessment that uses default exposure assumptions
representing common exposure scenarios, e.g.,
Region 3 risk-based concentrations (RBCs) or
Region 9 PRGs.
Site-specific PRG - site-specific chemical
concentration, protective of human health and
ecosystems, based on exposure scenarios in the
baseline risk assessment. Generally calculated for
the various exposure scenarios considered in the
baseline risk assessment.
Remediation Goals (RG) - site-specific chemical
concentration, protective of human health and
ecosystems, chosen by the risk manager as
appropriate for a likely land use scenario.
Remediation Action Level (RAL) - the
"not-to-exceed" level; a concentration such that
remediation of all concentrations above this level in
an exposure unit lowers the EPC sufficiently to
achieve a target risk level. The RAL will depend on
the mean, variance, and sample size of the
concentrations within an exposure unit as well as
considerations of short-term effects of the chemicals
of concern.
Cleanup Level (Final Remediation Level) -
chemical concentration chosen by the risk manager
after considering both RGs and the nine remedy
selection criteria of the NCP (U.S. EPA, 1990a).
Also referred to as Final Remediation Levels (U.S.
EPA, 199la), chemical-specific cleanup levels are
documented in the Record of Decision (ROD). A
cleanup level may differ from a PRG because risk
managers may consider details of the site-specific
exposure, various uncertainties in the risk estimate,
and implementation issues (e.g., the technical
feasibility of achieving the PRG).
Page 5-1
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
EXHIBIT 5-2
DEFINITIONS FOR CHAPTER 5
95% UCL for mean - The one-sided 95% upper confidence limit for a population mean; if a sample of size (w) was repeatedly
drawn from the population, the 95% UCL will equal or exceed the true population mean 95% of the time. It is a measure
of uncertainty in the mean, not to be confused with the 95th percentile (see below), which is a measure of variability. As
sample size increases, the difference between the UCL for the mean and the true mean decreases, while the 95th percentile
of the distribution remains relatively unchanged.
95th Percentile -The number in a distribution that is greater than 95% of the other values of the distribution, and less than
5%of the values. When estimated from a sample, this quantity may be equal to an observed value, or interpolated from
among two values.
Applicable or Relevant and Appropriate Requirements (ARARs) - Federal or state environmental standards; the NCP states
that ARARs should be considered in determining remediation goals. ARARs may be selected as site-specific cleanup
levels.
Backcalculation - A method of calculating a PRO that involves algebraic rearrangement of the risk equation to solve for
concentration as a function of risk, exposure, and toxicity.
Bootstrap Methods - Parametric and non-parametric methods for estimating confidence intervals for a statistic by resampling
directly from the data set with replacement.
Coverage - Confidence intervals are expected to enclose a true but unknown parameter according to a specified probability,
such as 90% or 95%. This is the expected coverage of the confidence interval, given a specified significance level
(alpha). The difference between the expected coverage and the actual coverage is one metric for evaluating statistical
methods that yield different confidence intervals.
Exposure Point Concentration (EPC) - The average chemical concentration to which receptors are exposed within an
exposure unit. Estimates of the EPC represent the concentration term used in exposure assessment.
Exposure Unit (EU) - For Superfund risk assessment, the geographical area about which a receptor moves and contacts a
contaminated medium during the period of the exposure duration.
Forward Calculation - A method of calculating a risk estimate that involves the standard arrangement of the risk equation to
solve for risk as a function of concentration, exposure, and toxicity.
Iterative Reduction (IR) - A method of calculating a PRO that involves successively lowering the concentration term until the
calculated risk is acceptable. This method can be applied to any medium.
Iterative Truncation (IT) - A method of calculating a PRO that involves developing an expression for the concentration term
in which higher values of concentration are removed or "truncated" to reduce the maximum concentration, and
re-calculating risks associated with the reduced concentration. The method may be repeated with consecutively lower
truncation limits until risk is acceptable.
Land Method - The conventional method for calculating uncertainty in the mean concentration (e.g., 95% UCL) when the
sample data are obtained from a lognormal distribution (U.S. EPA, 1992).
Maximum Detected Concentration (MDC) - The maximum concentration detected in a sample.
True Mean Concentration - The actual average concentration in an exposure unit. Even with extensive sampling, the true
mean cannot be known. Only an estimate of the true mean is possible. A greater number of representative samples
increases confidence that the estimate of the mean more closely represents the true mean.
Page 5-2
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
Two Office of Solid Waste and Emergency Response (OSWER) guidance documents in preparation:
(1) Draft Guidance on Calculation of Upper Confidence Limits for Exposure Point Concentrations at Superfund
Sites (U.S. EPA, 2001a), and (2) Draft Guidance on Surface Soil Cleanup at Superfund Sites: Applying Cleanup
Levels (U.S. EPA, 2001b), also address topics related to the calculation of EPCs and comparison of those EPCs to
a PRO.
In practice, calculations of risks, given concentration data, are commonly referred to as "forward
calculations", while calculations of PRGs, based on chosen target risk levels, are referred to as "back-
calculations". This terminology reflects the algebraic rearrangement of the standard risk equation needed to solve
for the concentration term when point estimates are used to characterize exposure and toxicity input variables.
For probabilistic risk assessment (PRA), the process for developing a PRG can be more complex. This chapter
presents methods and recommendations for developing site-specific PRGs within the framework of PRA.
Are there different types of PRGs?
Generic PRGs have been developed for some chemicals and exposure media using point estimates based
on standard default exposure assumptions (e.g., U.S. EPA, 1991b) and toxicity criteria available in the Integrated
Risk Information System (IRIS) or Health Effects Assessment Summary Table(s) (HEAST) or from
Environmental Protection Agency's (EPA's) National Center for Environmental Assessment. Soil Screening
Guidance levels, Region 9's PRG table and Region 3's Risk Based Concentrations (RBCs) table are examples of
generic point estimate PRGs. Generic PRGs are often used for screening chemicals of potential concern in Data
Evaluation and Hazard Identification steps of the risk assessment process.
«s° There is a clear distinction between generic PRGs, site-specific PRGs,
remediation goals (RGs), and cleanup levels. The focus of this chapter is on
site-specific PRGs.
At this time, EPA does not recommend the use of PRA to develop generic PRGs. Until the science and
policy decisions associated with the use of default assumptions in PRA have evolved, generic PRGs should only
be developed from point estimate methods, as was done in the examples listed above.
As indicated in Exhibit 5-1, site-specific PRGs generally are developed after the baseline risk assessment.
However, during the feasibility study or even later in the Superfund process, the methods described in this chapter
may be used to modify cleanup levels at the discretion of the risk manager. However, it is generally not
appropriate to use PRA for modifying cleanup levels during the feasibility study if PRA was not used in the
baseline risk assessment.
$f Risk-based PRGs are initial guidelines and do not represent final cleanup levels.
Only after appropriate analysis in the remedial investigation/feasibility study (RI/FS), consideration of
public comments, and issuance of the record of decision (ROD) does a RG become a final cleanup level. A
cleanup level may differ from a RG because risk managers may consider various uncertainties in the risk estimate.
While the two main criteria for determining a cleanup level are: (1) protection of human health and the
environment, and (2) compliance with applicable or relevant and appropriate requirements (ARARs), a cleanup
level may differ from the RG because of modifying criteria, such as feasibility, permanence, state and community
acceptance, and cost effectiveness. These and other factors are reflected in the nine evaluation criteria outlined in
the NCP (U.S. EPA, 1990a; 40CFR §300.430(e)(9)(iii)) (see Chapter 1, Exhibit 1-2).
Page 5-3
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
This chapter and Appendix C provide a comprehensive description of the issues associated with
developing site-specific PRGs with both point estimate and probabilistic approaches, including the use of
geostatistics. Because methods for calculating a 95% upper confidence limit for the mean (95% UCL) are
discussed fully in the Draft Guidance on Calculation of Upper Confidence Limits for Exposure Point
Concentrations at Superfund Sites (U.S. EPA, 2001a) and Draft Guidance on Surface Soil Cleanup at Superfund
Sites: Applying Cleanup Levels (U.S. EPA, 2001b), they are covered only briefly in this guidance. In general, this
chapter, Appendix C, and the Superfund guidance under development should be consulted by risk assessors when
developing site-specific PRGs.
5.1 GENERAL CONCEPTS REGARDING EPCs AND PRGs
PRGs developed from point estimate risk assessments and PRAs will be discussed in this section to
compare and contrast the two approaches. The PRG is a special case of the concentration term (or EPC) in the
risk equation. The intent of the EPC is to represent the average chemical concentration in an environmental
medium in an exposure unit (EU) (i.e., the area throughout which a receptor moves for the duration of exposure).
The EPC should be determined for individual EUs within a site. Because an EPC is calculated from a sample,
there is uncertainty that the sample mean equals the true mean concentration within the EU; therefore, to account
for associated uncertainty, the 95% upper confidence limit for the mean (95% UCL) is generally used for
Superfund risk assessments (U.S. EPA, 1992). For both point estimate and probabilistic approaches, the PRG is
an assumed value of the EPC that yields a risk estimate that is at or below an acceptable risk level.
«*" The EPC usually represents the average concentration within the EU estimated
from a sample; the PRG usually represents the average concentration within the
EU that corresponds to an acceptable level of risk.
The PRG may be thought of as a goal for the post-remediation EPC (see Section 5.1.2). Specifically,
after remediation is completed, the average concentration (or the 95% UCL used as a measure of uncertainty in
the average) for the EU should be sufficiently low to be protective of human health and the ecosystem. While the
methods used to calculate the pre- and post-remediation EPC may differ, the interpretation of the EPC remains
constant. For example, if the 95% UCL is used to represent the EPC before remediation, then the EPC following
remediation (e.g., the PRG) should also represent a 95% UCL (Bowers et al., 1996).
Risk assessors may consider both variability and uncertainty in the development of an EPC. The
calculation of a 95% UCL generally requires knowledge of not only chemical concentration measurements within
the EU but also the receptor's behavior. Relevant information may include the variability in concentrations in the
given sample, the sampling locations, and variability in the movement and activity patterns of receptors within the
EU. A discussion of spatial and temporal variability associated with characterizing contamination in different
exposure media is presented in Appendix C, and important sources of uncertainty in the EPC are discussed in
Section 5.1.1.
For all risk assessments, chemical concentration measurements should be collected in a manner that is
consistent with an understanding of both the source of contamination and the definition of the exposure unit. An
investment of time and resources should be made in planning, scoping, and problem formulation. Part of this
investment is to follow the Data Quality Objectives (DQO) process to obtain samples appropriate for the risk
assessment and sufficient to support the remedial decision (U.S. EPA, 1993, 1994, 2000). Using new methods of
sample collection and analysis such as dynamic workplans and real-time analysis may enable risk managers to get
the most "bang for the buck" from the resources available for site characterization. Information about these
methods and the DQO process is available from EPA's Office of Emergency and Remedial Response (U.S. EPA,
Page 5-4
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
2001c) and Technology Innovation Office (U.S. EPA, 2001d, 2001e). The world wide web address is
http://clu-in.org/charl_edu.cfm#syst_plan.
5.1.1 SOURCES OF UNCERTAINTY IN THE EPC
The 95% UCL is generally used as the EPC to represent uncertainty in the mean concentration in both the
central tendency exposure (CTE) and reasonable maximum exposure (RME) risk estimates for Superfund (U.S.
EPA, 1992). Similarly, in PRA, a probability distribution for uncertainty may be used in a two-dimensional
Monte Carlo analysis (2-D MCA) simulation (see Appendix D) to represent a source of uncertainty in the EPC.
There are numerous potential sources of uncertainty in the estimate of the true mean concentration within the EU.
The sources of uncertainty when the EPC is expressed as either a single number or a distribution are the same and
can be grouped into the following four broad categories:
(1) Uncertainty in the sample data. A limited number of measurements in the sample are used to make
inferences about the EPC and the spatial distribution of concentrations at a site. Uncertainties may
arise from many factors, including both sampling variability and measurement error. As the number
of samples increases, the uncertainty generally decreases (e.g., more information will be available to
characterize the spatial distribution and variation in concentration). In point estimate risk
assessments, the 95% UCL is generally used as the EPC to account for the uncertainty in estimating
the average concentration within an EU.
(2) Uncertainty about the location of the EU. When the size of a receptor's EU is less than the size of
the site, the placement of the EU may be a source of uncertainty, especially when the contamination is
distributed unevenly across the site and the PRA includes exposure scenarios for future land uses.
(3) Uncertainty in the behavior of the receptor. Even in the case of extremely well characterized sites, it
remains uncertain whether the receptor will contact the environmental medium in a temporal and/or
spatial distribution that can be adequately represented by the environmental samples collected.
(4) Uncertainty in chemical concentrations over time. The concentration in a given medium may
undergo temporal changes, which may introduce uncertainty in estimates of a long-term average.
Examples include the movement or attenuation of a solvent plume in groundwater; aerobic or
anaerobic degradation; the change in the average concentration in a fish population due to changes in
population dynamics; and the mixing of surface and subsurface soil overtime.
A lack of knowledge in all four categories may be considered when selecting approaches to quantify
uncertainty in the concentration term. One of the first steps in quantifying uncertainty is to define the EU, or the
geographical area in which individual receptors are randomly exposed for a relevant exposure duration.
Depending on the receptor's movement and activities, an EU may be as small as a child's play area (e.g.,
sandbox) or as large as the foraging area of an upper trophic level animal predator (e.g., an entire military base).
The relationship between the size of the EU, the movements of the target receptor, and health endpoint of concern
(i.e., acute or chronic) may dictate the appropriate use of sample data in developing an EPC. One of the
assumptions generally made for the concentration term in Superfund risk assessment is that receptors contact all
parts of an EU at random, and that measurements are obtained from a simple (or stratified) random sample. If an
individual is randomly exposed within the same EU over a long period of time, the most appropriate metric for the
EPC would be the true (but unknown) population mean of the concentrations within the EU (e.g., 95% UCL).
Page 5-5
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
Often, the scale of the EU will be different (smaller or larger) than the scale of the sample data. For
example, an ecological receptor population may have a small home range relative to the size of the entire site, or
the endpoint of concern may be acute toxicity, requiring an evaluation of a short-term exposure scenario. If the
receptors are not expected to contact all parts of the site with equal probability, then the EU may be redefined so
that only a subset of the data collected for site characterization are used to estimate the EPC. In addition, the
location of the EU may be unspecified within the site because there may be multiple areas that provide suitable
habitat for the receptor population. Departing from the assumption of random exposure within one unique
geographic area presents an additional challenge to estimating an EPC. In some cases, it may be informative to
develop multiple estimates of the EPC in a PRA. By treating the EPC as a random variable, risk assessors can
explore the effect of uncertainty in the location of the EU. A variety of modeling approaches are available to
calculate an EPC (e.g., arithmetic mean, or 95% UCL) based on the spatial variability in chemical concentrations
measured over an area larger than the EU. Methods such as geostatistics (see Section 5.5.2 and Appendix D),
Microexposure Event Modeling (MEE) (see Appendix D), and random walk scenarios (Hope, 2000, 2001) may
be used to quantify both the spatial and temporal variability in exposure to varying concentrations. Using these
methods, risk assessors may redefine the EU to be more representative of the random movement of the receptor
during the period of exposure. Because these modeling approaches may be considered more advanced methods
for quantifying the EPC, they are generally considered in Tier 3 of the PRA process (see Chapter 2).
5.1.2 PRE- AND POST-REMEDIATION EXPOSURE POINT CONCENTRATIONS
The differences between pre- and post-remediation EPCs are discussed below. In general, both estimates
of the EPC are based on the same concepts regarding the exposed population and the definition of the EU.
However, the post-remediation EPC will tend to yield lower estimates of (post-remediation) risk and can require
more advanced methods for calculating uncertainty (e.g., 95% UCL).
The pre-remediation EPC is determined based on existing site sampling at the time of the remedial
investigation, prior to remediation. By contrast, the post-remediation EPC generally is determined based on a
prediction of site conditions after remediation. For example, in surface soil, the post-remediation EPC can be
determined by substituting the nondetect level (generally, half the laboratory reporting limit) for some of the high
concentrations in the sample and recalculating the EPC. The underlying assumption in calculating a post-
remediation EPC is that remediation will have sufficiently reduced the chemical concentrations at the site, and the
risk existing after remediation is complete will be equal to or less than the target risk level of concern.
The preceding discussion is most applicable to surface soil PRGs. In general, compared with other
exposure media (e.g., groundwater, air), surface soil is stationary with relatively constant chemical concentrations
within an EU. For other environmental media, more complex approaches may be needed to estimate the
post-remediation EPC. Modeling of the remediation process may introduce additional uncertainty not
encountered in risk estimates based on the pre-remediation EPC.
5.1.3 REMEDIATION ACTION LEVELS (RALs) AND 95% UCL CALCULATION METHODS
The EPC should incorporate knowledge about the spatial distribution of contamination, the behavior of
the receptor, the location of the EU, land use, and other factors. These factors affect both the numerical value of
an EPC and uncertainty associated with this estimate. In many cases, it is presumed factors associated with land
use will not change after remediation.
The remediation action level (RAL) is the maximum concentration that may be left in place at any
location within an EU such that the average concentration (or 95% UCL as a measure of the average) will not
Page 5-6
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
present a risk above levels of concern. This RAL may be considered a "not-to-exceed" threshold or action level
for the purposes of site remediation. Using surface soil as an example, areas within the EU that have
concentrations greater than the RAL may be excavated and replaced with clean fill (e.g., nondetect surrogate
values). To obtain a post-remediation EPC, the 95% UCL is calculated after substituting the surrogate nondetect
value for all measurements located within the EU that are greater than the RAL.
When appropriate, the same statistical method of uncertainty should be used to estimate UCLs for both
the pre- and post-remediation EPCs. However, in some instances, the method used for calculating the
pre-remediation EPC will be inappropriate for calculating the post-remediation EPC, because the distribution of
contaminant concentration will have changed. For example, pre-remediation site sampling may suggest that
variability in concentrations can be reasonably characterized by a lognormal distribution, which would support
the use of the Land method for estimating the 95% UCL. The post-remediation site conditions, however, may
reflect a mixture of clean fill and contamination, resulting in a poor fit to a lognormal distribution (see Figure 5-3,
Section 5.5.3). In this case, the Land method would not be appropriate. Because of the difference in the
statistical distribution of concentration measurements used to estimate the pre-remediation EPC and post-
remediation EPC, a non-parametric (i.e., distribution free) method should be considered for calculating
uncertainty in the average concentrations in both pre- and post-remediation scenarios. In general, when the
method used to calculate the 95% UCL for a post-remediation scenario is different than that of the pre-
remediation scenario, the 95% UCL for the pre-remediation scenario should be recalculated with the post-
remediation method. Results of this change in methodology can be presented as part of a quantitative uncertainty
analysis. Specifically, this recalculation will allow for an evaluation of the effect that a RAL has on the
confidence interval for the mean. The discordance between pre- and post-remediation distributions can be
expected to increase as the degree of remediation needed to achieve a target risk level of concern increases.
In general, risk assessors should be aware of the practical and statistical issues associated with the various
methods of calculating the 95% UCL, and the application of these methods to both the pre- and post-remediation
concentration distribution. Different methods can yield very different confidence intervals, some of which are
expected to yield more accurate coverage (i.e., likelihood that the confidence interval includes the parameter)
depending on characteristics of the underlying distribution of concentrations, such as distribution shape, sample
size, and variance (Gilbert, 1987; Hall, 1988). Information about a variety of parametric and non-parametric
methods, such as bootstrap resampling, can be found in The Lognormal Distribution in Environmental
Applications (U.S. EPA, 1997), Estimating EPCs When the Distribution is Neither Normal nor Lognormal
(Schulz and Griffin, 1999) and a Superfund guidance document currently under development, Draft Guidance on
Calculation of Upper Confidence Limits for Exposure Point Concentrations at Superfund Sites (U.S. EPA,
200 la).
5.1.4 CONSIDERATION OF RISK FROM ACUTE TOXICITY
Sometimes a risk assessment will need to address more than one health endpoint of concern (e.g., cancer
and noncancer). The RAL should be sufficiently low so that it is simultaneously protective of each endpoint of
concern. Generally, when acute toxicity is a concern, the long-term average concentration across the entire EU
may not be the appropriate metric for assessing risks. For example, a single episode of a child ingesting a handful
of soil containing malathion may result in an acute toxic effect to that child. Therefore, the RAL must not only be
low enough to reduce the post-remediation EPC to acceptable long-term average levels, but also low enough that
acute toxicity will not be an issue. This consideration applies to both point estimate and probabilistic estimates of
PRGs.
Page 5-7
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
"S" For consideration of acute toxicity, the risk assessor should consult, as
appropriate, with a toxicologist in the development ofRALs.
For a small number of chemicals, toxicity values have been determined based on acute effects (e.g., nitrate in
drinking water). However, at present, EPA does not have acute toxicity criteria or guidance on acute toxicity
applied to the RAL. Hence, consultation with a toxicologist is vital.
5.1.5 CHARACTERIZATION OF UNCERTAINTY IN THE EPC: POINT ESTIMATES AND DISTRIBUTIONS
In point estimate risk assessments, the 95% UCL is typically used to characterize uncertainty in the EPC
(U.S. EPA, 1992). In PRA, either a point estimate (e.g., 95% UCL) or a probability distribution may be used to
characterize uncertainty in the concentration term. The probability distribution may characterize either variability
or uncertainty. The terms probability distribution for variability (PDFv) and probability distribution for
uncertainty (PDFu) can be used to distinguish between probability distributions for variability and uncertainty,
respectively.
The decision to use a point estimate, PDFv, or PDFu, as the input for the concentration term in a Monte
Carlo model will depend on the goals of the Monte Carlo simulation, as determined by the tiered process (see
Chapter 2). If the goal is to characterize variability in risk, in general, a one-dimensional Monte Carlo analysis
(1-D MCA) will be used and the appropriate input for the concentration term will be a point estimate that
characterizes uncertainty in the mean concentration within the EU. As explained in Section 5.1.1, risk assessors
will need to consider the relationship between the size of the EU, the movements of the target receptor, and health
endpoint of concern (i.e., acute or chronic) to determine how to use the available sample data to define the EPC.
A PDFu is typically not an appropriate choice for the concentration term in a 1-D MCA when the goal is to
characterize variability in risk. Mixing of a PDFu for the concentration term with PDFv's for other exposure
variables in 1-D MCA would yield a single risk distribution from which the relative contributions of variability
and uncertainty could not be evaluated. Use of a PDFu for the concentration term may be considered in
2-D MCA simulations (see Appendix D), where the goal may be to characterize both variability and uncertainty
in risk.
When the sample size is small and the variance is large, the 95% UCL may exceed the maximum detected
concentration (MDC). In such a case, the MDC is generally used to estimate the EPC, although the true mean
may still be higher than this maximum value (U.S. EPA, 1992). For poorly characterized sites, there may be
considerable uncertainty that site remediation will be sufficient to reduce the 95% UCL to a health-protective
level. Poor site characterization may provide an impetus for the risk manager to opt for a more health-protective
remedial alternative or to collect additional data.
To ensure that actual cleanup based on a RAL is protective generally requires post-remediation
confirmation sampling. This step in the risk management process is emphasized further in Section 5.8 on
measurement of attainment.
5.1.6 MULTIPLE CHEMICALS
Developing PRGs for multiple chemicals in one or more environmental media is particularly challenging.
When multiple chemicals are present, the total risk level should be considered for regulatory purposes with each
chemical contributing a portion of the total risk. This issue is quite complex and usually will affect both the
calculation of the risk and development of site-specific PRGs. Chemicals may exhibit different spatial and
temporal variability within the EU. Fate and transport characteristics may vary between chemicals as well as
between different areas of the site. Co-located sampling, or geostatistical techniques (e.g., co-kriging) may
Page 5-8
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
provide insights regarding relationships in spatial patterns for different chemicals (see Appendices C and D) and
the corresponding exposures for receptors.
5.2 WHEN TO USE PRA FOR DEVELOPING PRGs
Because point estimate risk assessments and PRA employ different approaches to characterize variability
and uncertainty, the resulting RME risk estimates and calculations of PRGs are often different. The magnitude of
the difference can depend on many factors, including the number of input variables described with probability
distributions in the PRA, the choice of distributions used to characterize variability or uncertainty (especially for
those variables that are highly ranked in a sensitivity analysis), the percentile of the probability distribution that
corresponds with RME point estimate for each input variable, and the choice of percentile from the PRA used to
represent the RME risk (e.g., 95th percentile). Since the results of a point estimate approach and PRA can be
expected to differ, but the magnitude of the difference is not known a priori, this can present a challenge in
deciding whether or not to conduct a PRA to develop a PRG. The potential advantages and disadvantages of both
the point estimate approach and the PRA can be factored into the decision (see Chapter 1, Exhibits 1-6 and 1-7).
In general, PRA may be appropriate for developing site-specific PRGs in cases where PRA has also been
used to estimate site-specific risks. As indicated by the tiered approach (see Chapter 2), if the risk manager
determines that quantifying variability and uncertainty may enhance risk management decision making, PRA may
be warranted. If a PRA is feasible, the risk manager should proceed to Tier 2 and employ PRA to complete the
RI/FS process. Usually, embedded in a site-specific PRG are all of the exposure assumptions and toxicity metrics
used in the risk assessment. Hence, introducing the use of PRA for PRGs in the feasibility study (or any time
after the remedial investigation and baseline risk assessment are complete) would, in effect, undermine the tiered
approach.
«*" If only point estimates were used in the risk assessment, probabilistic methods
should not be used for PRG development.
If additional data have been collected to conduct PRA, the point estimate risk assessment should be
revisited with the new data as well. As discussed in Chapter 2, a point estimate risk assessment (Tier 1) should
always accompany a PRA. PRA is intended to enhance risk management decision making, and should not be
viewed as a substitute for point estimate approaches. Using the tiered approach, a risk assessor can determine the
appropriate level of complexity that is supported by the available information to conduct the risk assessment and
to calculate a PRG.
5.3 METHODS FOR DEVELOPING PRGs
Risk assessors may use PRA to quantify sources of uncertainty and variability in the calculation of PRGs
as well as risks. Two of the common methods for calculating PRGs in PRA include: (1) backcalculation (see
Section 5.4), which is equivalent in concept to the point estimate calculation of a PRG; and (2) iterative forward
calculation methods, including iterative reduction and iterative truncation (see Section 5.5). Backcalculation can
be used in PRA when the target risk and concentration terms are expressed as point estimates. Iterative methods
can be more involved, but unlike backcalculation, there are no constraints on their application to PRA. The two
approaches yield the same result when the same assumptions are used in the risk assessment.
Page 5-9
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
5.4 BACKCALCULATION
Traditionally, risk is calculated as a function of multiple exposure variables, including the concentration
term, and toxicity value (Equation 5-1). If one or more of the exposure variables is described by a PDF, a Monte
Carlo simulation will yield a distribution for risk (see Chapter 1).
Backcalculation methods can be envisioned as setting a target risk level (e.g., RME risk equal to 10~6 or
Hazard Index equal to 1) and then algebraically reversing the risk equation to solve for the concentration term
(Equation 5-2). A Monte Carlo simulation using Equation 5-2 will yield a distribution of concentrations that
reflects the combination of distributions from all other exposure variables.
——— = Intake
BWv, AT
Intake x Toxicity = Risk
CxV = Risk
C=
Equation 5-1
Equation 5-2
where,
Toxicity =
C
V
IR
AT
BW
ED
EF
toxicity term representing either the cancer slope factor (CSF) or reference dose
(1/RfD) for the chemical in the exposure medium
concentration term
algebraic combination of the toxicity term with all exposure variables except C
ingestion or inhalation rate
averaging time
body weight
exposure duration
exposure frequency
This calculation produces a distribution of PRGs that represents the same sources of variability as a
forward calculation of risk. Each percentile of the PRG distribution (i.e., the a percentile) corresponds to the
1-a percentile from the distribution of risk estimates. For example, if the 95th percentile of the distribution of risk
estimates was chosen to represent the RME individual, the 5th percentile (1-0.95=0.05) would be the
corresponding concentration value from the distribution of PRGs (Bowers, 1999). The correspondence between
the risk distribution and the PRG distribution is intuitive—just as selecting a higher percentile on the risk
distribution is more protective, a lower percentile on the PRG distribution is more protective. The RME range for
the risk distribution 90th to 99.9th percentile is analogous to an RME range for the PRG distribution of
0.1st to 10th percentile.
Backcalculation has been a familiar method of developing PRGs and may be appropriate in some
situations for the sake of clarity and transparency due to the general understanding of this method among risk
assessment practitioners. Once a backcalculation has been performed to determine a PRG, the PRG should be
used as the concentration term in a forward calculation to ensure that the risk at the PRG is acceptable.
Page 5-10
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
5.4.1 DIFFICULTIES WITH BACKCALCULATION
There are limitations in the use of backcalculation in PRA (Person, 1996). Simple rearrangement of
Equation 5-1 does not suffice when the variable (i.e., the concentration or risk term) that is backcalculated is
represented by a probability distribution (Burmaster et al., 1995; Person, 1996). The difficulty for PRA arises
because each risk estimate from an MCA that uses the familiar "forward-facing" risk equation represents a
combination of random values selected from the input distributions. Therefore, the output can be considered
conditional on all of the inputs. Rearranging the risk equation does not maintain the same conditional
probabilities; therefore, the distribution for risk estimated as a function of the distribution for concentration in
Equation 5-1 does not return the same distribution for concentration when applied in Equation 5-2. While there
are techniques that can maintain the dependencies and correlations between exposure factors when the risk
equation is rearranged (e.g., deconvolution), they are complex and beyond the scope of this guidance.
Backcalculation methods may also be difficult to implement in situations in which complex
fate-and-transport considerations are present. Leaching of soil contamination to groundwater, bioconcentration of
chemicals at higher trophic levels, and other multimedia processes that result in exposure via several
environmental media are situations in which backcalculation may not be useful. Note that these difficulties are
not unique to backcalculation. Uncertainty in fate-and-transport considerations makes any type of PRO
determination challenging.
Further, the backcalculation approach only provides information on the EPC that corresponds to a risk
level of concern; it does not specify an RAL that would achieve this EPC. For example, when a risk equation is
algebraically solved for concentration (see Equation 5-2), a PRG is developed without a corresponding RAL.
Thus, there is no information associated with the PRG value to indicate the highest concentration in the EU that
must be removed so that the average concentration (or 95% UCL) within the EU is at or below the PRG. Hence,
additional efforts are needed. In addition, post-remediation concentrations may need to satisfy more than one
regulatory constraint. For example, the average (or 95% UCL) concentration within an EU may need to be less
than a concentration associated with chronic toxicity or cancer and simultaneously, the RAL concentration may
need to be less than a concentration that might cause acute toxicity.
In spite of these caveats, backcalculation methods may be appropriate for some sites. For example, when
the target risk is specified by a single numerical value and the risk manager has chosen a percentile of variability
to represent the RME individual, then a backcalculated PRG can be derived from a PRA.
Although backcalculation methods may be appropriate for some sites, risk assessors should be familiar
with their limitations. Because of these limitations, this guidance recommends iterative forward calculations as
the primary method for calculating PRGs when performing a PRA. Iterative methods avoid difficulties associated
with applying MCA to a backcalculation, and can provide more information for the risk manager.
5.5 ITERATIVE METHODS
Iterative methods simply involve calculating risk with the "forward-facing" equation (see Equation 5-1) a
number of times (iteratively) using progressively lower values for the concentration term until the risk is
sufficiently protective. This iterative method has also been called the "repeated runs" method. Note that iterative
methods for calculating a PRG are not uniquely applicable to PRA. Iterative methods also may be used to
develop PRGs in point estimate risk assessments.
Page 5-11
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
$f EPA recommends
iterative simulations as a general approach
for calculating PRGs from probabilistic risk
assessments.
Most often, iterative forward
calculations are performed using a systematic
trial-and-error method until the percentile of
variability in risk chosen to represent the
RME individual is at or below acceptable
risk levels. Sometimes, a short cut can be
used to reduce the number of simulations
needed with the trial-and-error method. If
successive "guesses" of the EPC are plotted
with the corresponding risk estimate, the
exact solution can be determined from the
best-fit line, thereby significantly reducing
the effort required to implement this method.
An example is given in Figure 5-1. For
many risk equations, the relationship
between the EPC and the RME risk will be
approximately linear. Nevertheless, the final
estimate of the EPC should be checked by
running another simulation for risk with this
estimate.
2.5
2.0
OJ , 5
•O 1-S
a
0.0
y=0.0014x+ 0.0518
R2 = 0.9943
PRO <= 657 ppm
500
1000
EPC (ppm)
1500
2000
Figure 5-1. A hypothetical example of the use of iterative methods to
determine the EPC that corresponds with a target RME Hazard Index (HI)
of 1.0. Assume that the EPC is represented by the 95% UCL and the
RME HI is the 95th percentile of the output distribution. In this case, four
separate Monte Carlo simulations were run with iteratively decreasing
values for the EPC. The least-squares, best-fit line to these four data
points suggest that a reasonable PRO would be approximately 660 ppm.
A possible and significant advantage of iterative forward calculations over back-calculations is that the
method is intuitive and yields a distribution of risks rather than a distribution of PRGs (as with a back-calculation
method). The distribution of risks will be more familiar to the public and other stakeholders, and thus, both the
method and the resulting output may be easier to communicate to senior level managers and stakeholders (see
Chapter 6).
Two general types of iterative methods are described in more detail in Sections 5.5.1 and 5.5.2. The main
difference between the methods is in the interpretation of the concentration term that is being reduced. With
iterative reduction, the concentration is assumed to be the post-remediation EPC, whereas with iterative
truncation, it represents the RAL needed to achieve a post-remediation EPC.
5.5.1 ITERATIVE REDUCTION
Iterative reduction can be applied to any medium. Generally, a point estimate representing the EPC (e.g.,
95% UCL) is successively lowered, each time repeating the Monte Carlo simulation of variability in risk. When
the EPC is reduced until the endpoint of concern (e.g., RME risk corresponding to the 95th percentile) is at or
below an acceptable level of risk, the PRG is set at the corresponding EPC. The goal is to identify the point
estimate that corresponds to a target risk level. Note that the PRG is not the same as the RAL. The RAL is the
maximum concentration that may be left in place within an EU to achieve the PRG.
The concentration at which the risk is acceptable defines the PRG. Therefore, the PRG bears the same
uncertainties as the EPC. For example, assume that a risk assessor examined the carcinogenic effects from
Page 5-12
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
chronic consumption of a chemical in groundwater, then the exposure unit may be determined by the long-term
average concentration at any well that potentially draws drinking water from the contaminated groundwater.
Uncertainty in the long-term average concentration can reflect a number of factors that contribute to spatial and
temporal variability, including the direction of groundwater flow, natural attenuation, and other fate and transport
variables. Remediation by a pump-and-treat system for a prolonged period of time may be used to lower the
concentrations at the wells. Even though the remediation strategy may be complicated by spatial and temporal
variability, iterative reduction can be used to establish a PRG. A remediation strategy may be considered a
potential candidate if it can achieve the PRG by reducing the average concentration at each of the well locations.
The concept of "hot-spot" removal, or truncation of the highest concentrations first, would not be an option under
this scenario (see Section 5.5.2).
5.5.2 ITERATIVE TRUNCATION
Iterative truncation is a method of calculating a PRG that involves developing an expression for the
concentration term in which higher values of concentration are removed or "truncated" to reduce the maximum
concentration. These higher values are replaced by the surrogate nondetect value. The risk is recalculated for
each successive reduction in the highest value. The method is repeated with consecutively lower truncation limits
until risk is acceptable.
Iterative truncation is most applicable to surface soil cleanup as the spatial variability over time is
minimal compared to other media (e.g., surface water). With each iteration of the risk equation (e.g.,
Equation 5-1), the highest concentration value is truncated corresponding to a different RAL. In this way
a"not-to-exceed" level is specified and the PRG is recalculated the same way in each iteration. The process
continues until the risk distribution yields risk estimates at or below the level of concern.
Iterative truncation can be applied to either the empirical distribution function (EDF) for the
concentration term, or a fitted distribution for variability in concentrations within the EU. Applied to the EDF,
the maximum detected concentration within the EU is replaced with a surrogate value for a nondetect (e.g., half
the reporting limit or the background value for some chemicals), and the EPC (e.g., 95% UCL) is recalculated for
this altered data set. If this new EPC yields unacceptable risk, then the two highest detected concentrations are
replaced by the nondetect value and the EPC is recalculated. In the third iteration, the three highest detections are
replaced, and so on, until the target risk level is achieved. Alternatively, the sample data may be fit to a
probability distribution for variability, and the process would be repeated with decreasing values in the high-end
tail of the continuous distribution.
When the concentration term is a distribution representing uncertainty in the mean concentration, then,
similar to the recalculation of the point estimate 95% UCL described above, this distribution of uncertainty in the
mean concentration should be determined anew each time a datum is replaced with the nondetect value.
When a distribution of variability in concentration is used for the EPC, for example, in an ecological risk
assessment where sampling may be sparse relative to the foraging area of a small home range receptor (see
Appendix C), then the distribution developed in an identical way with the high values replaced by the surrogate
nondetect value should be used in the iterative determination of a PRG.
The decision to apply iterative truncation should be made after considering a variety of characteristics of
the sample data and post-remediation scenario (see Exhibit 5-3). For example, small sample size may result in
high uncertainty in the 95% UCL, thereby limiting the use of iterative truncation. Quantitative criteria regarding
these factors are not provided in this guidance given that the level of certainty required for decision making will
Page 5-13
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
vary on a case-by-case basis. Use of geostatistical methods (Appendices C and D) may aid in interpreting site
data or improving sampling design. Geostatistics is capable of describing the spatial distribution of a contaminant
in a quantitative fashion. These methods establish a correspondence between the actual sampling locations and
the locations a receptor would be expected to frequent. Additionally, it enables the estimation of concentrations
in unsampled locations. Hence, for determination of concentrations at specific locations at a site or within EUs of
various sizes and shapes, geostatistics may provide an invaluable tool. Geostatistics has applications both to
developing the EPC and PRG and has been recommended and used at some sites for characterization of soil and
groundwater contamination (U.S. EPA, 1990b, 1991c).
Although the consideration and use of
geostatistics is encouraged, a full consideration of
geostatistics is beyond the scope of this guidance. Those
interested in greater detail than provided in Appendices C
and D are urged to consult the Superfund guidance
document currently under development, Draft Guidance
on Surface Soil Cleanup at Superfund Sites: Applying
Cleanup Levels (U.S. EPA, 200Ib), for additional
discussion of how geostatistics can be used to quantify the
concentration term or the PRG.
Generally, iterative truncation methods fail to
produce adequate cleanup strategies when site
characterization is incomplete. This problem, however, is
not specific to PRA. Both point estimate and probabilistic
methods are sensitive to poor site characterization.
Risk assessors should realize that application of
iterative truncation may result in areas on-site that have
concentrations higher than the PRG. This is because the
PRG will reflect an average concentration (or 95% UCL)
from a distribution of concentrations in which the
maximum is truncated at the RAL. For example,
Figure 5-3 (see Section 5.5.3) shows how the
concentration distribution can be truncated at an RAL,
while still leaving behind concentrations greater than the
PRG.
5.5.3 EXAMPLE OF ITERATIVE METHODS
The iterative truncation method is easiest to think
about with regard to soil cleanup when contaminated soil
is removed and replaced with clean fill soil. This
replacement would reduce both the mean and 95% UCL.
In most cases, risk assessors may assume that the
concentrations of chemicals in clean fill soil can be
represented by the surrogate nondetect value (e.g., half
1.
EXHIBIT 5-3
CRITERIA FOR ITERATIVE TRUNCATION
Sample size («) is sufficient. Small
sample sizes lead to large estimates of
uncertainty in the concentration term.
Small sample size may cause the risk
assessor to overlook some sources of
uncertainty.
2. Concentration distribution is not highly
skewed. A highly skewed distribution
may yield unreliable estimates of
uncertainty, especially for small sample
sizes.
3. Sampling design yields a representative
distribution of measurements within the
exposure unit. Simple random sampling
may fail to represent a patchy spatial
distribution of contaminants. Similarly,
hotspot (e.g., cluster) sampling may fail to
represent random movement of receptors.
To evaluate potential biases in sampling,
analyses with both standard statistical
methods and geostatistical methods may be
required.
4. Assumptions about the post-remedial
distribution of concentration are
reasonable. If these assumptions are
shown to be incorrect by subsequent
sampling events, the process for
developing a PRG may need to be repeated
and additional remedial activities may be
required.
the detection limit). Alternatively, the fill may be
sampled so that the measured concentrations in the fill dirt may be used to calculate the post-remediation
Page 5-14
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
concentration term. Generally, metals and other inorganic chemicals will be present in clean fill, albeit at lower
concentrations than on site.
A simple example using the 95% UCL as a point estimate for the EPC is given in Exhibit 5-4. In this
example, background concentrations of chemical X were very low and hence, the fill was assumed to have a
concentration of half the detection limit. The risk management objective is to identify a PRG in which the 95th
percentile risk estimate is below 1E-04 and to determine the RAL necessary to achieve this PRG. This example
illustrates how iterative truncation is applied to the empirical distribution function, rather than fitting the
concentrations to a parametric distribution.
Assume that iterative reduction of the 95% UCL demonstrated that a post-remediation EPC of no greater
than 33 mg/kg is needed to achieve a RME risk of 1E-04. What is the RAL that yields this EPC? The risk
assessor recognizes that the post-remediation concentration distribution is very often a mixed distribution,
consisting of a group of nondetect values and a truncated parametric distribution. Because of the complex nature
of mixed distributions (Roeder, 1994), non-parametric methods for calculating the 95% UCL of the arithmetic
mean (e.g., bootstrap resampling) were determined to be appropriate (U.S. EPA, 1997; Section 5.1.3).
Page 5-15
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
EXHIBIT 5-4
EXAMPLE OF ITERATIVE METHODS
Scoping and Problem Formulation
Chromium contamination was present at a 12-acre industrial facility. In scoping and problem
formulation, all stakeholders agreed that the facility would maintain itself and the current land use
would continue into the foreseeable future. Most of the facility area was maintained as green space
and as a buffer with the surrounding community. Surrounding the facility to the fence line were
lawns and ornamental shrubs tended by landscape workers. These landscape workers were
considered to be the high risk group as they would move freely and randomly over the entire area of
the facility outside the buildings. Hence, the landscape workers would be exposed to an average
concentration over the entire area of the facility outside the buildings. The management of the facility
was very cooperative and concerned about their workers. Nonetheless, the facility management did
not wish to bear more cost than necessary.
Site Characterization - Soil Sample (n=70)
Seventy surface soil samples were obtained using a sampling grid placed over all 12 acres. Five
or six sampling locations were placed in each acre. None of the samples was composited. The
grid-based sampling permits a rough estimation of the percentage of the site that would need active
remediation. The detection limit for the chromium was 1 mg/kg. Four of the samples were
nondetects. Sampling results are shown in Table 5-1. Although the samples from the site appeared to
occur in a lognormal distribution (Figure 5-2), the presumed post-remediation distribution would be a
mixed distribution, consisting of a truncated lognormal distribution and a group of data at the
surrogate nondetect value.
Figure 5-2. Lognormal probability plot of soil
concentrations, including 4 nondetects.
Table 5-1. Soil sample (n=70) (mg/kg).
0.5
0.5
0.5
0.5
6.8
7.2
7.8
8.0
8.2
9.3
9.7
10.6
10.8
11.0
11.8
12.0
13.7
13.9
14.7
15.0
16.2
17.1
17.4
17.9
18.4
18.6
19.7
19.8
22.0
22.8
25.1
25.4
26.4
26.9
27.1
28.2
28.3
30.3
30.9
31.1
34.0
34.0
36.5
43.3
43.3
45.3
46.4
48.2
49.3
52.6
54.1
57.8
60.2
65.7
66.1
71.8
82.7
84.7
98.1
107.7
120.6
122.2
140.7
211.9
224.1
235.6
266.8
284.0
361.2
486.6
8.0
-2.0
-3.0 -2.0 -1.0 0.0 1.0 2.0 3.0
z-score
Page 5-16
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
In this example, a series of iterative truncations showed that removal of all sample results greater than
100 mg/kg (n=l 1) and replacement of these with the nondetect surrogate of 0.5 mg/kg yielded a 95% UCL of
33 mg/kg and RME risk below 1E-04. Table 5-2 summarizes the results of the calculations for the three
conditions: (1) pre-remediation concentrations; (2) post-remediation concentrations using iterative truncation to
achieve an RAL of 100 mg/kg; and (3) post-remediation concentrations assuming the 95% UCL calculated is used
as the RAL. Note that if the PRO of 33 mg/kg was applied as a "not-to-exceed" level (i.e., RAL), the resulting
remediation effort would increase from 15 to 40% of the site, yielding a 95% UCL of 14 mg/kg. While this
would be a protective decision, other information was used to support the selection of the second scenario instead.
A toxicologist was consulted, who indicated that acute exposure to the workers at levels of 100 mg/kg would not
present a health risk. To build additional protectiveness into the remedy, the management also indicated
scheduling for the landscape workers would be performed so the areas tended would be rotated among all the
workers.
Table 5-2. Pre- and Post-Remediation EPCs (95% UCLs) for Chemical X in Surface Soil Samples.
Remediation Scenario
1. Pre-remediation
2. Post-remediation using the PRO as
the 95% UCL
3 . Post-remediation using the PRO as
the RAL (i.e., "not-to-exceed")
RAL (mg/kg)
NA
100
33
EPC (mg/kg)
95% UCL
93
33
14
Percent of Site to be
Remediated
NA
15%
40%
NA=not applicable for a pre-remediation scenario.
Figure 5-3 shows a conceptual framework for considering the post-remediation distribution as a mixture
between a group of nondetects and a distribution of contamination truncated at the RAL. Prior to remediation, the
EPC exceeds a level that would be protective of human health and ecosystems. If the high-end soil concentrations
are removed and the soil is replaced with clean fill, the resulting distribution will be bimodal, with one peak
occurring at the nondetect concentration, and the second occurring near the mean of the post-remediation
distribution.
5.5.4 MULTIPLE EXPOSURE UNITS AND ITERATIVE METHODS
When multiple EUs are present at the site, there may be a small number of samples within a given EU and
the uncertainty in the concentration term generally will be large. It may be possible to use knowledge of the
mechanism of how the contamination occurred along with spatial patterns in the sampling results in other nearby
EUs to quantify uncertainty. Geostatistical techniques for estimating the mean concentration may provide useful
insights into the importance of accounting for spatial relationships among the sample data. Appendix C also
provides a discussion of the situation of multiple EUs within a larger site.
Page 5-17
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
VI
4*
Q
PRG =
Post-remediation
EPC (95% UCL)
Pre-re mediation
EPC (95% UCL)
RAL
Nondetects
Concentration (ppm)
Truncated Values
Figure 5-3. Hypothetical example of a mixed, bimodal distribution that represents a combination of the pre-
remediation distribution truncated at the remediation action level (RAL) and a uniform distribution representing
clean fill at the surrogate nondetect concentration. Shaded portions represent equal areas. In this example, the
PRO is defined by the post-remediation EPC (95% UCL).
5.6 PRGs FOR GROUNDWATER
For some chemicals encountered at hazardous waste sites, chemical-specific ARARs may exist, and may
be considered as PRGs. ARARs may be selected as site-specific cleanup levels. The maximum contaminant
levels of the Safe Drinking Water Act are examples of ARARs.
$f For groundwater contamination, ARARs should be applied as RALs if they are
protective.
Of course, for cases in which an ARAR is less protective than a remediation goal determined from a risk
assessment, then a risk-based PRG may be developed in accordance with the NCP (U.S. EPA, 1990a).
As an exposure medium, groundwater is the opposite of soil in that groundwater is not static, and
receptors are usually exposed at one location (i.e., the well head). Often, a single well can be considered the EU
when assessing risks associated with either the residential or industrial/occupational scenarios. The EPC may still
reflect the concept of averaging over a long time period (e.g., years) due to potential changes in concentrations in
Page 5-18
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
well water over time. For example, chemical fate and transport modeling may suggest that concentrations are
decreasing overtime. Similarly, there may be temporal and spatial variability depending on the seasonal
fluctuations of the water table. Ideally, the risk assessment would focus on individuals who may be exposed at
locations nearest to the center of the contaminant plume, where concentrations are likely to be highest (Freeze and
Cherry, 1979; Sposito, etal., 1986).
Because of the uncertainty in the movement of groundwater and the necessity of sampling the medium at
fixed locations, identifying a meaningful RAL needed to achieve a given PRO is difficult. In most cases, ARARs
will be applicable as RALs or "not-to-exceed" levels.
5.7 PRGs FOR OTHER CONTAMINATED MEDIA
Iterative truncation techniques are generally applied to a static medium, such as soil, rather than dynamic
or fluid media such as water and air. This is simply because it is difficult to design a method that will selectively
remove high concentrations from a fluid medium. Iterative reduction may be more relevant than iterative
truncation when an RAL cannot be developed. These issues are discussed below with respect to sediment, surface
water, and fish.
Sediment
Sediment may be transported over time more readily than soils. If it can be assumed that the sediment
remains in place, then iterative truncation techniques may be applied. However, at some sites, sediment may be
considered a fluid medium. For example, sediment may be resuspended by the movement of water craft, waves,
changing tides, or erosion. Similarly, the depth of the contaminated sediment may change over time as new layers
of sediment are deposited above more contaminated sediment.
Exhibit 5-5 gives an example of the use of iterative truncation to evaluate alternative RALs for sediment
of a lake contaminated by pesticide runoff. In this example, the RAL is related to both the ecological endpoint of
concern (i.e., reduction in reproductive success of mammalian omnivores at the lake) and the fraction of areal
extent of the lake that would require remediation at that RAL.
Page 5-19
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
EXHIBIT 5-5
EVALUATION OF ALTERNATIVE RALs USING ITERATIVE TRUNCATION
Risks to a population of mammalian omnivores residing near a lake contaminated with pesticide
"X" were judged to be sufficiently high that a reduction in population number over time was
expected (see Chapter 4, Exhibit 4-12). The primary reservoir of pesticide X in the lake is
sediment. The STAG committee decided to use the iterative truncation method to estimate the
beneficial effects of a series of different Remedial Action Levels (RALs). PRA was used to predict
the distribution of responses (percent reduction in population success) and the areal extent of the
lake requiring remediation as a function of RAL. The results are summarized below.
RAL in
Sediment
None
100 ppm
80 ppm
60 ppm
40 ppm
20 ppm
Reduction in
Mean
8.9%
7.6%
7.0%
5.9%
4.4%
1 .9%
Reproductive Success
90th 95th
31 % 59%
24% 48%
27% 45%
18% 36%
12% 26%
4.7% 1 0%
Fraction
of Lake
0%
3%
5%
8%
16%
37%
The STAG reviewed these results and concluded that while an RAL of 20 ppm would be needed to
provide nearly complete protection of the exposed population, an RAL of 40 ppm would provide a
good reduction in effect level while tending to minimize the areal extent of the lake that required
remediation, which in turn would tend to minimize disturbance of the ecosystem during
remediation. Based on this, the risk manager identified 40 ppm as the RAL and initiated a
feasibility study to investigate ways of achieving this objective.
0.05
RAL = 20 ppm (37% of lake)
,RAL = 40 ppm (16% of lake)
Existing distribution of
concentration levels in
sediment
40 60 80 100
Concentration in Sediment (ppm)
120
140
Page 5-20
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
Biota (Fish, Aquatic Invertebrates, Plants)
Biota, such as fish, aquatic invertebrates, and plants can serve as bioindicators or indirect estimators of
contamination in other exposure media that would be targets for remediation. The concentration of chemicals fish
may reflect a combination of exposures via sediment, the water column, and food source (e.g., prey). Therefore,
the use of bioindicators to develop PRGs in other media introduces a sources of uncertainty. If there is a high
correlation between concentrations in fish and sediment, then sediment concentrations may be considered when
developing PRGs to protect the receptor population. The EU, in this case, is the area where the angler population,
or ecological predator population, harvests fish. However, in risk assessments that include a fish ingestion
exposure pathway, there may be high uncertainty about the true concentration term. Concentrations may be
affected by many factors, including changes in the fish population and changes in fish preferences, which may be
difficult to address in risk assessments. The choice offish species consumed by a given individual may also
affect the concentration term.
Fish population studies and fate and transport considerations of the contaminants may indicate if and
when a fish population will reach a calculated cleanup level. For many sites, it may be difficult to obtain this
level of site-specific data due to resource and time constraints.
Although remediation may not immediately reduce contaminant concentrations in biota, the determination
of a cleanup level can serve as a target for any future decline in concentrations. In general, iterative reduction
methods are applicable for developing PRGs to protect aquatic ecosystems; however, under some conditions
iterative truncation may also be used. For example, if contamination is correlated to relatively static sediment,
and the home-range of the fish is relatively small (e.g., nonmigratory) then iterative truncation may be applicable.
Surface Water
The development of PRGs for surface water is also difficult with iterative truncation. For fluid media
(e.g., groundwater or surface water), iterative reduction can be performed using a range of EPCs to determine a
PRG with acceptable risk at the target RME percentile.
5.8 MEASUREMENT OF ATTAINMENT
The NCP (U.S. EPA, 1990a) provides for continued monitoring for groundwater cleanups to ensure
attainment of the remedial action objectives. In addition, it is common practice among remedial project managers
to conduct confirmation sampling after completing a remedy for soil contamination. However, completion of the
remedial action according to this strategy does not necessarily mean that risks within EUs at the site have been
reduced to levels specified in the ROD. The degree of uncertainty about whether the remedial action at the site
has achieved the cleanup level should determine whether confirmation sampling is warranted. In general,
confirmation sampling following cleanup activities is recommended. Sampling after the remedial investigation is
complete may show additional areas needing remediation (i.e., where additional contamination exists).
If additional sampling is conducted after the remedial investigation, the concentration term and
corresponding estimates of risk should be recalculated. The PRG developed in the remedial investigation may not
be health-protective in light of the additional contamination. The same concepts that relate the concentration term
to the PRG should be applied in this situation.
Page 5-21
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
Confirmation sampling activities are included in remedial design/remedial action plans to ensure the
remedy is successful. In addition, the five-year review presents a second opportunity to ensure that any
contamination left on site does not pose an unacceptable risk.
$f If confirmation sampling Indicates an Insufficient reduction In risk, a more
extensive remediation effort may be needed. Possible reasons for not achieving
remedial action objectives can Include Inadequate site characterization or the
discovery of unknown contamination.
For post-remediation sampling, the DQO process should generally be followed. If the post-remediation
risk associated with the confirmation sample indicates risk exceeds a level of concern, then additional remediation
may be warranted.
Page 5-22
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
5.9 SUMMARY OF RECOMMENDED METHODS
Table 5-3 summarizes the possible methods for developing PRGs for various environmental media. It
should be noted that iterative reduction (IR) can be used in all cases, whereas iterative truncation (IT) is limited to
situations where the highest concentrations can be identified and removed. Backcalculation may be applicable in
all cases, but because of caveats noted in Section 5.4.1, iterative approaches are generally recommended in this
document.
Table 5-3. Summary of Potential Methods for PRG Development by Environmental Medium.
Potential Exposure
Medium
Soil
Sediment
Biota (Fish, Aquatic
Invertebrates, Plants) -
bioindicators of
contamination in
sediment
Surface Water
Groundwater (GW)
Home-grown produce,
milk, livestock, other
food items
Back-
calculation
X
X
X
X
X
X
Iterative
Reduction
(IR)
X
X
X
X
X
X
Iterative
Truncation
(IT)
X
X
SA
NA
NA
SA
Explanations for IT
Applicable if soil is relatively fixed.
Applicable if sediment is relatively fixed.
In some situations, sediment transport may
be a better assumption due to current
velocity, tides, resuspension, etc.
Depends on home-range offish relative to
the scale of the sampling design. If
contamination is correlated to relatively
static sediment, and the home-range of the
fish is relatively small (e.g., non-migratory)
then IT may be applicable.
Not applicable as surface water is a fluid
medium.
Not applicable as GW is a fluid medium.
Generally, ARARs must also be satisfied.
Depends on relative contributions of soil
uptake (applicable) vs. foliar deposition (not
applicable).
X=applicable
NA=not applicable
SA=sometimes applicable
Page 5-23
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
REFERENCES FOR CHAPTER 5
Bowers, T.S., N.S. Shifrin, and B.L. Murphy. 1996. Statistical Approach to Meeting Soil Cleanup Goals.
Environ. Sci. Technol. 30:1437-1444.
Bowers, T.S. 1999. The Concentration Term and Derivation of Cleanup Goals Using Probabilistic Risk
Assessment. Hum. Ecol Risk Assess. 5(4):809-821.
Burmaster, D.E., K.J. Lloyd, and K.M. Thompson. 1995. The Need for New Methods to Backcalculate Soil
Cleanup Targets in Interval and Probabilistic Cancer Risk Assessments. Hum. Ecol. Risk Assess.
Person, S. 1996. What Monte Carlo Methods Cannot Do. Hum. Ecol. Risk Assess. 2:990-1007.
Freeze, R.A. and J.A. Cherry. 1979. Groundwater. Prentice Hall, Inc., NJ.
Gilbert, R.O. 1987. Statistical Methods for Environmental Pollution Monitoring. VanNostrand Reinhold, NY.
Hall, P. 1988. Theoretical Comparison of Bootstrap Confidence Intervals. Ann. Statist. 16:927-953.
Hope, B.K. 2000. Generating Probabilistic Spatially-Explicit Individual and Population Exposure Estimates for
Ecological Risk Assessments. Risk Anal. 20(5):573-589.
Hope, B.K. 2001. A Case Study Comparing Static and Spatially Explicit Ecological Exposure Analysis Methods.
Risk Anal. 21(6): 1001-1010.
Roeder, Kathryn. 1994. A Graphical Technique for Determining the Number of Components in a Mixture of
Normals. J. Amer. Stat. Assoc. 89(426):487-495.
Schulz, T.W. and S. Griffin. 1999. Estimating Risk Assessment Exposure Point Concentrations When the Data
are not Normal or Lognormal. Risk Anal. 19: 577-584.
Sposito, G., W.A. Jury, and V.K. Gupta. 1986. Fundamental Problems in the Stochastic Convection-Dispersion
Model of Solute Transport in Aquifers and Field Soils. Water Res. 22(l):77-88.
U.S. EPA. 1990a. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40 CFR 300:
5 5 Federal Register, 8666-8865, Thursday, March 8.
U.S. EPA. 1990b. Geostatistics for Waste Management . A Users Manual for the GEOPACK Geostatistical
Software. EPA/600/8-90/004, January.
U.S. EPA. 1991a. Risk Assessment Guidance for Superfund (RAGS), Volume I: Human Health Evaluation
Manual (HHEM), Part B, Development of Risk-Based Preliminary Remediation Goals. Office of
Emergency and Remedial Response, Washington, DC. EPA/540/R-92/003 . NTIS PB92-963333.
U.S. EPA. 1991b. Risk Assessment Guidance for Superfund (RAGS), Volume I: Human Health Evaluation
Manual (HHEM), Supplemental Guidance: Standard Default Exposure Factors, Interim Final. Office of
Emergency and Remedial Response, Washington, DC. OSWER Directive No. 9285.6-03. June.
Page 5-24
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 5 ~ December 31, 2001
U.S. EPA. 1991c. GEO-EAS 1.2.1 Users Guide. EPA/600/8-91/008. April.
U.S. EPA. 1992. Supplemental Guidance to RAGS: Calculating the Concentration Term. Office of Solid Waste
and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-081.
U.S. EPA. 1993. Data Quality Objectives Process for Superfund: Interim Final Guidance. Office of Research and
Development, Washington, DC. EPA/540/R-93/071.
U.S. EPA. 1994. Guidance for the Data Quality Objectives Process (EPA QA/G-4). Office of Research and
Development, Washington, DC. EPA/600/R-96/055. September.
U.S. EPA. 1997. The Lognormal Distribution in Environmental Applications. Office of Research and
Development, and Office of Solid Waste and Emergency Response, Washington, DC.
EPA/600/R-97/006. December.
U.S. EPA. 2000. Data Quality Objectives Process for Hazardous Waste Site Investigations. Office of
Environmental Information, Washington, DC. EPA/600/R-00/007. January.
U.S. EPA. 2001a. Draft Guidance on Calculation of Upper Confidence Limits for Exposure Point Concentrations
at Superfund Sites. Office of Emergency and Remedial Response, Washington, DC.
U.S. EPA. 200 Ib. Draft Guidance on Surface Soil Cleanup at Superfund Sites: Applying Cleanup Levels.
Office of Emergency and Remedial Response, Washington, DC.
U.S. EPA. 2001c. Integrating Dynamic Field Activities into the Superfund Response Process: A Guide For
Project Managers. Final Draft. Office of Emergency and Remedial Response, Washington, DC. OSWER
Directive No. 9200.1-40. December.
U.S. EPA. 2001d. Improving Sampling, Analysis, and Data Management for Site Investigation and Cleanup.
Technology Innovation Office. EPA/542/F-01/030a. April.
U.S. EPA. 2001e. Resources for Strategic Site Investigation and Monitoring. Technology Innovation Office,
Washington, DC. EPA/542/F-01/030b. September.
Page 5-25
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
CHAPTER 6
COMMUNICATING RISKS AND UNCERTAINTIES IN
PROBABILISTIC RISK ASSESSMENTS
6.0 INTRODUCTION
The Environmental Protection Agency (EPA) has developed a guidance document, Risk
Assessment Guidance for Superfitnd: Volume I-Human Health Evaluation Manual, Supplement to Part
A: Community Involvement in Superfund Risk Assessments (U.S. EPA, 1999a) and two videotapes,
"Superfund Risk Assessment and How You Can Help, An Overview " (10 minutes) (U.S. EPA, 1999b) and
"Superfund Risk Assessment and How You Can Help" (40 minutes) (U.S. EPA, 2000b), to improve
community involvement in the Superfund risk assessment process. The videotapes (available in both
English and Spanish) show examples of how regions have involved communities in the risk assessment
process at several Superfund sites. The guidance document and videotapes, along with the Superfund
Community Involvement Handbook and Toolkit (U.S. EPA, 1998), should serve as a primary community
involvement resource for risk assessors and remedial project managers (RPMs). The Handbook and
Toolkit offers the following specific guidance:
• Provides suggestions for how Superfund staff and community members can work together
during the early stages of Superfund remedial investigation and feasibility study (RI/FS) and
later cleanup
• Identifies where, within the framework of the human health risk assessment methodology,
community input can augment and improve EPA's estimates of exposure and risk.
• Recommends questions the site team (risk assessor, RPM, and community involvement
coordinator [CIC]) should ask the community.
• Illustrates why community involvement is valuable during the human health risk assessment
at Superfund sites.
This chapter provides guidance and suggestions on how to deal with risk communication issues
that arise during a probabilistic risk assessment (PRA). Specifically, the concepts of uncertainty and
variability may present additional communication challenges for PRA. For example, whereas discussions
of uncertainty for point estimate risk assessments are often qualitative in nature, PRA opens the floor for
discussion and presentation of quantitative uncertainty analysis. Concepts associated with quantitative
characterizations of uncertainty may be more difficult to communicate and may not be well received due
to stakeholder desires for certainty (Slovic et al., 1979). As such, this chapter highlights appropriate
stakeholder involvement and principal risk communication skills that are effective for communicating
PRA concepts and risk information. Key factors for successful communication of PRA include early and
continuous involvement of stakeholders, a well-developed communication plan, good graphics, a working
knowledge of the factors that may influence perceptions of risk and uncertainty, and a foundation of trust
and credibility.
Page 6-1
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
EXHIBIT 6-1
DEFINITIONS FOR CHAPTER 6
Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in a population,
usually considered to be the mean or median of the distribution.
Community Advisory Group (CAG) - A group formed to provide a public forum for community members to present
and discuss their needs and concerns related to the Superfund decision-making process. A CAG serves as the focal
point for the exchange of information among the local community, EPA, State regulatory agency, and other
pertinent Federal agencies involved in the cleanup of a Superfund site.
Community Involvement Coordinator (CIC) - As a member of the CAG and site team, the CIC coordinates
communication plans (i.e., the CIP) and addresses site-specific CAG organizational issues.
Community Involvement Plan (CIP) - A plan that identifies community concerns and the preferences of the community
for the communication of site-related issues.
Confidence Interval - A range of values that are likely to include a population parameter. Confidence intervals may
describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95th percentile risk).
When used to characterize uncertainty in a risk estimate, it is assumed that methods used to quantify uncertainty in
the model inputs are based on statistical principles such as sampling distributions or Bayesian approaches. For
example, given a randomly sampled data set, a 95% confidence interval for the mean can be estimated by deriving
a sampling distribution from a Student's t distribution.
Credible Interval - A range of values that represent plausible bounds on a population parameter. Credible intervals may
describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95th percentile risk).
The term is introduced as an alternative to the term confidence interval when the methods used to quantify
uncertainty are not based entirely on statistical principles such as sampling distributions or Bayesian approaches.
For example, multiple estimates of an arithmetic mean may be available from different studies reported in the
literature—using professional judgment, these estimates may support a decision to describe a range of possible
values for the arithmetic mean.
Cumulative Distribution Function (CDF) - Obtained by integrating the PDF, gives the cumulative probability of
occurrence for a random independent variable. Each value c of the function is the probability that a random
observation x will be less than or equal to c.
Hazard Quotient (HO) - The ratio of estimated site-specific exposure to a single chemical from a site over a specified
period to the estimated daily exposure level, at which no adverse health effects are likely to occur.
Hazardous Substance Research Centers (HSRC) - Research centers providing free technical assistance to communities
with environmental contamination programs through two distinct outreach programs: Technical Outreach Services
for Communities (TOSC) and Technical Assistance to Brownfields Community (TAB).
Histogram - A graphing technique which groups the data into intervals and displays the count of the observations
within each interval. It conveys the range of values and the relative frequency (or proportion of the sample) that
was observed across that range.
Monte Carlo Analysis (MCA) or Monte Carlo Simulation - A technique for characterizing the uncertainty and
variability in risk estimates by repeatedly sampling the probability distributions of the risk equation inputs and
using these inputs to calculate a distribution of risk values. A set of iterations or calculations from Monte Carlo
sampling is a simulation. For example, a single iteration for risk from ingestion of water may represent a
hypothetical individual who drinks 2 L/day and weighs 65 kg; another iteration may represent a hypothetical
individual who drinks 1 L/day and weighs 72 kg.
Parameter - A value that characterizes the distribution of a random variable. Parameters commonly characterize the
location, scale, shape, or bounds of the distribution. For example, a truncated normal probability distribution may
be defined by four parameters: arithmetic mean [location], standard deviation [scale], and min and max [bounds].
It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithmetic mean
ingestion rate).
Percentile - A number in a distribution such that X % of the values are less than the number and 1-X % are greater. For
example, the 95th percentile is a number in a distribution such that 95% of the values are less than the number and
5% are greater.
Page 6-2
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
EXHIBIT 6-1
DEFINITIONS FOR CHAPTER 6—Continued
Point Estimate Risk Assessment - A risk assessment in which a point estimate of risk is calculated from a set of point
estimates for exposure and toxicity. Such point estimates of risk can reflect the CTE or RME, depending on the
choice of inputs.
Potentially Responsible Party (PRP) - Individuals, companies, or any other party that is potentially liable for
Superfund cleanup costs.
Preliminary Remediation Goal (PRG) - Initially developed chemical concentration for an environmental medium that
is expected to be protective of human health and ecosystems. PRGs may be developed based on applicable or
relevant and appropriate requirements (ARARs), or exposure scenarios evaluated prior to or as a result of the
baseline risk assessment. (U.S. EPA, 1991a, 1991b).
Probabilistic Risk Assessment (PRA) - A risk assessment that yields a probability distribution for risk, generally by
assigning a probability distribution to represent variability or uncertainty in one or more inputs to the risk
equation.
Probability Density Function (PDF) - A function or graph representing the probability distribution of a continuous
random variable. The density at a point refers to the probability that the variable will have a value in a narrow
range about that point.
Rank Correlation (Spearman Rank Order Correlation Coefficient) - A "distribution free" or nonparametric statistic r
that measures the strength and direction of association between the ranks of the values (not the values
themselves) of two quantitative variables.
Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site (U.S.
EPA, 1989). The intent of the RME is to estimate a conservative exposure case (i.e., well above the average
case) that is still within the range of possible exposures.
Remedial Investigation/Feasibility Study (RI/FS) - Studies undertaken by EPA to delineate the nature and extent of
contamination, to evaluate potential risk, and to develop alternatives for cleanup.
Sensitivity Analysis - Sensitivity generally refers to the variation in output of a model with respect to changes in the
values of the model's input(s). Sensitivity analysis can provide a quantitative ranking of the model inputs based
on their relative contributions to model output variability and uncertainty. Common metrics of sensitivity
include:
>• Pearson Correlation Coefficient - A statistic r that measures the strength and direction of linear association
between the values of two quantitative variables. The square of the coefficient (r2) is the fraction of the
variance of one variable that is explained by the variance of the second variable.
>• Sensitivity Ratio - Ratio of the change in model output per unit change in an input variable; also called
elasticity.
*• Spearman Rank Order Correlation Coefficient - A "distribution free" or nonparametric statistic r that
measures the strength and direction of association between the ranks of the values (not the values
themselves) of two quantitative variables. See Pearson (above) for r2.
Stakeholder - Any individual or group who has an interest in or may be affected by EPA's site decision-making
process.
Technical Assistance Grant (TAG) A federal grant that is intended to provide a community with the opportunity to
hire independent experts to help evaluate and explain the results of a risk assessment.
Technical Outreach Services for Communities (TOSC) - A service of the HSRC with the aim to provide independent
technical information and assistance to help communities with hazardous substance pollution problems.
Uncertainty - Lack of knowledge about specific variables, parameters, models, or other factors. Examples include
limited data regarding the concentration of a contaminant in an environmental medium and lack of information
on local fish consumption practices. Uncertainty may be reduced through further study.
Variable - A quantity that can assume many values.
Page 6-3
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Section 6.1 discusses the need for early and continuing stakeholder involvement. Section 6.2
recommends a seven-step process for communicating PRA results to stakeholders, and Sections 6.3
and 6.4 provide guidance on specific techniques for communicating information. The success of risk
communication efforts will depend on the extent to which the communication strategy addresses the
needs of a diverse audience, with different perceptions of risk and uncertainty (Section 6.5), and the
degree of trust and credibility that is established from the outset of the process (Section 6.6).
Section 6.7 provides a discussion of risk communication issues that are uniquely relevant to RPMs.
6.1 STAKEHOLDER INVOLVEMENT
Many stakeholders may be interested in a
risk assessment (see Exhibit 6-2). It is generally
important to involve and engage interested
stakeholders early and continuously throughout
the decision-making process (U.S. EPA, 2001).
Public involvement activities should be
tailored to the needs of the community and
described in the site communications strategy.
The CIC should coordinate these first steps
through the development of a Community
Involvement Plan (CIP). Coordination between
the RPM, risk assessor, and CIC is needed to
determine the appropriate points in the RI/FS
process to communicate with the community, and
plan for the appropriate level of communication.
The CIP should identify community concerns and
the preferences of the community for the
communication of site-related issues. The CIP
may be updated during the RI/FS as needed.
EXHIBIT 6-2
STAKEHOLDERS POTENTIALLY INVOLVED IN THE
DECISION-MAKING PROCESS FOR PRA
• EPA risk assessors and managers
• Members of the public
• Representatives from state or county
environmental or health agencies
• Other federal agencies (e.g., health agencies,
Natural Resources Damage Assessment
(NRDA), trustees, etc.)
• Tribal government representatives
• Potentially responsible parties (PRPs) and their
representatives
• Representatives from federal facilities (e.g.,
Department of Defense, Department of Energy,
etc.)
Examples of outreach activities include
giving oral presentations and poster sessions at public meetings, coordinating group meetings or focused
workshops, conducting interviews with community members on specific issues, and distributing fact
sheets.
Ideally, the public and other interested stakeholders would be involved early in the site-specific
decision-making process. If the community has not been previously involved, efforts should be made, in
coordination with the CIC, to identify and communicate with the appropriate individuals in the
community prior to the Agency's receipt of the PRA workplan. The public and other stakeholders should
be given the opportunity to provide input to the workplan for a PRA (see Chapter 2, Section 2.1).
The initial community meeting can serve to establish a rapport between EPA and the community
and facilitate the exchange of information needed to support a PRA. This information may include policy
decisions associated with both point estimate and probabilistic approaches, as well as technical details
regarding the conceptual exposure model and the selection of distributions. A discussion of these topics
may increase certainty about the assumptions made in the risk assessment. For example, the community
may be able to offer insights regarding site-specific activities and sources of exposure data not readily
Page 6-4
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
available to the risk assessor. This type of discussion should allow for the free exchange of information
with the public and sets the stage for future discussions. It is important that an appropriate level of detail
be presented at the first meeting. Instead of overloading the audience with information, it is generally
better to coordinate several meetings so that complex policy and technical concepts can be broken down
into smaller discussion topics.
Following the approval of the PRA workplan, the public and other interested stakeholders should
be involved in various stages of the PRA development, including providing and/or reviewing data,
reviewing the selected distributions (e.g., selected creel survey) and commenting on PRA documents as
appropriate during public comment periods. On-going community involvement may require
consideration of EPA's resources including the availability of personnel and contractor support. Other
considerations include EPA's compliance with provision in the National Contingency Plan (NCP) for
involving the community. The appropriate level of community involvement in the PRA should be based
on a number of factors including the nature and extent of contamination at the site, the expressed interests
of the community members, the complexity of the PRA, and the role of PRA in site-specific remediation
or cleanup decisions.
6.2 COMMUNICATION AND PRESENTATION
Communication is a two-way process that should involve the transfer of information between the
Agency and the stakeholders, as well as active listening by the Agency to the stakeholder's ideas and
concerns. The goals of risk communication are to present risk information in an understandable manner
through an open, honest, frank, and transparent presentation and discussion of risks, including
uncertainties. In meeting these goals, it is important that the RPMs and risk assessors be sincere and
direct in their presentation of the results of the PRA, accept the public and other interested stakeholders as
valuable contributors to the process, and listen to the concerns and ideas that are raised.
One goal of communication should be to respect the stakeholder's concerns. The public and
other interested stakeholders should have the opportunity to understand the PRA and its effects on the
decision-making process. Technical Assistance Grants (TAGs) may be one way to advance this goal by
providing the community the opportunity to hire independent experts to help evaluate and explain the
results of the PRA. Alternatively, the RPM and risk assessor may use the tools outlined in Sections 6.3 to
6.6 to present PRA concepts and the results of the PRA to the community in a manner that is easily
understood. This may require significant up-front planning, testing, and post-evaluation to identify the
appropriate messages to communicate and to determine how well this information was communicated.
The site-specific PRA communication plan should be consistent with the NCP's provisions on
community involvement. It is important to recognize that community involvement is part of a regulatory
process and that EPA generally will consider all timely public input, but may not implement all of it.
Ultimately, EPA must meet the legal requirements of the Superfund law in making decisions regarding
remedial actions.
A vast body of literature exists regarding risk communication. Since the early 1980's, a number
of researchers have developed models for communicating risk to the public. These models are available
in the scientific literature, and a list of supplemental references is provided at the end of this chapter.
Page 6-5
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
6.2.1 COMMUNICATION OF PRA WITH CONCERNED CITIZENS, OTHER STAKEHOLDERS, AND
MANAGERS: AN OVERVIEW
Before the decision to conduct a PRA is made, a CIP should be in place. Generally, when a
decision is made to conduct a PRA, an important step should be to work with citizens to develop a
communication strategy for PRA and its application within the Superfund process (see Chapter 1). The
initial introduction of the community to the RI/FS process should include a discussion of the principles of
risk assessment. This discussion may be best presented in an informal setting such as a public availability
session. Because of the potentially complex nature of PRA and quantitative uncertainty analysis, a small
group meeting may be an appropriate forum in which to discuss issues and facilitate an exchange of ideas.
If there is interest among a large group of stakeholders, multiple small group sessions may be scheduled.
Such meetings may provide the foundation for building trust and credibility (see Section 6.6).
In general, it is important to identify whether a Community Advisory Group (CAG) should be
formed. The purpose of a CAG is to provide a public forum for community members to present and
discuss their needs and concerns related to the Superfund decision-making process. The CIC is an
important member of the team and may coordinate communication plans, hand-out materials, and address
site-specific organizational issues.
A number of resources may be available to the community to aid in understanding technical
material in a PRA. In addition to the TAG program, which provides funds for qualified citizens' groups
affected by a Superfund site to hire independent technical advisors, another program is the Technical
Outreach Services for Communities (TOSC), which uses university educational and technical resources to
help communities understand the technical issues involved in hazardous waste sites in their communities.
This is a no-cost, non-advocate, technical assistance program supported by the Hazardous Substance
Research Centers.
The tiered approach for PRA presented in Chapter 2 (Figures 2-1 and 2-2) encourages risk
assessors and RPMs to participate in discussions with stakeholders early in the process of developing
point estimate and probabilistic approaches. If a decision is made to perform a PRA, a continuing
dialogue should be useful to evaluate interim results of the PRA and determine if additional activities are
warranted (e.g., data collection, further modeling). These on-going discussions should help assure that
RPMs are aware of the details of the PRA analysis and are comfortable with the material that will be
shared with the community, other interested stakeholders, and senior managers.
Page 6-6
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
6.2.2 STEPS FOR COMMUNICATION OF THE RESULTS OF THE PRA
The complexity of a PRA will vary depending on the site-specific nature of the assessment
performed. For example, PRAs may include an analysis of variability, uncertainty, or both. Some
analyses may involve simulations to evaluate temporal variability (e.g., Microexposure Event analysis)
and spatial variability (e.g., geostatistics). The challenge for presenters is to identify the critical
information and level of detail to be presented to various audiences that may be involved in the Superfund
decision-making process (e.g., senior risk managers, concerned citizens, congressional staff, and PRPs).
The 7-step process, described below (and summarized in Exhibit 6-3), may be repeated many
times during the performance of a PRA. For communication purposes, a PRA normally will involve more
interaction with stakeholders than a point estimate risk assessment because PRA concepts and results are
often more difficult to communicate.
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(1) Identify the Audience
The first step should be to identify
the audience of potentially interested
stakeholders. Strategies for presenting PRA
information normally will be tailored to the
audience. Participants in the audience may
change during the tiered process depending
on the complexity of the PRA (see Chapter 2)
and the specific site-management decisions
being made.
(2) Identify the Needs of the Audience
The second step should be to identify
the needs of the audience. The relevant
information and the appropriate level of detail
will vary depending on the audience. For
example, some participants may be well informed about PRA concepts and will not need much
introductory PRA information. For other audiences, PRA concepts may be new, so it may be beneficial
to hold an informal meeting to discuss the general objectives and methods used to conduct a PRA. Once
introductory PRA concepts have been discussed and are understood by the audience, more advanced
discussions may be warranted on topics such as the sources of data used in the PRA, the most critical
variables in the PRA (identified during the sensitivity analysis), the selection of distributions, and the
level of characterization of uncertainty (see also Section 6.5). The risk assessor should select the key
information for each topic and discuss the significance of this information based on the intended
audience.
EXHIBIT 6-3
IMPORTANT STEPS FOR
COMMUNICATING PRA RESULTS
Identify the audience
Identify the needs of the audience
Develop a communication plan
Practice to assure clarity of presentation
Present information
Post-meeting review of presentation and
community feedback
Update information as needed for future
assessments and presentations
Page 6-7
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
(3) Develop a Communication Plan
The third step should be to develop a plan
to communicate significant information to the
public in an easily understandable format
(Exhibit 6-4). Adequate planning in the
presentation of PRA information is essential. A
thorough understanding of the design and results
of the PRA will help to place the information in
proper context and understandable format (U.S.
EPA, 1994). Even more importantly, the risk
assessors and RPMs should clearly identify the
main messages to be presented.
EXHIBIT 6-4
KEY CONSIDERATIONS IN DEVELOPING
UNDERSTANDABLE MATERIAL
Identify main messages
Place information in appropriate context
Use clear formats
Use examples and graphs
Provide handouts and glossaries
Present information with minimum jargon
Section 6.4 provides examples of graphics that may be useful in presentations of PRA. Handouts,
glossaries, and other materials may complement a presentation and provide information for discussion
following the meetings. In addition, examples designed to help demonstrate concepts unique to PRA
(e.g., using one probability distribution to describe variability and a second distribution to describe
parameter uncertainty) may help facilitate the flow of communication and increase the level of
understanding. One useful technique in public meetings is to involve members of the audience to
illustrate a concept. For example, the topic of discussion may be the method used to select and fit a
probability distribution used to characterize variability in a PRA. To demonstrate this concept, a risk
assessor can draw a bell-shaped curve on a flip chart and label the x-axis, "number of liters of water
consumed per day", and the y-axis, "number of people who consume a specific amount of water in a day".
Next, each meeting participant can be asked to identify their own consumption pattern, perhaps by
holding up a 0.5 liter bottle and asking how many such bottles are consumed on an average day. This
community-specific information can then be plotted on a new graph in the form of a histogram and the
bars can be connected to form a curve or distribution similar to the one first drawn. The resulting
distribution (for an example, see Figure 6-1) can then be used to discuss the following PRA concepts in
more detail:
• Variability (between individuals)
Shape of the distribution and plausible range of values
• Central tendency exposure (CTE) and reasonable maximum exposure (RME) estimation
Uncertainty in the distribution (sample size, potential response bias, differences in activity
patterns)
Uncertainty in a parameter estimate (difference between the 95% upper confidence limit
(UCL) for a mean and the 95th percentile)
Using this information as a basis, the risk assessor can compare the results from the community
analysis with data from various geographic areas in the U.S. where water consumption patterns may
differ. The risk assessor can then lead a discussion with the community regarding the various sources of
uncertainty in selecting and fitting exposure distributions, including:
(a) Extent of Representation - Are the available data representative of the target population?
For example, would the data on water consumption collected during the meeting be
representative for various population groups?
Page 6-8
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
(b) Data Quantity - What sample size is needed to develop a distribution? This discussion will
introduce the concept that uncertainty in both point estimates and probability distributions
may be reduced by increasing the sample size
(c) Data Quality - Are the data collected using acceptable study protocols? Is the information
available from the peer-reviewed literature? An example can be made of the data collected
during the meeting to highlight issues associated with survey design, and methods for
controlling for potential bias or error. For example, if the survey data were to be used in a
risk assessment for a drinking water scenario, the data quality may be improved by repeat
sampling over time
Other exposure variables that can be used in this distribution example include: fish consumption
rates, chemical concentrations in soil, and fraction of time spent indoors. In general, examples should
focus on variables that may be of interest, are easily illustrated, and are unlikely to make participants
uncomfortable divulging personal information such as age.
(4) Practice to Assure Clarity of Presentation
The fourth step should be to practice the presentation to assure that the information is presented
clearly to the intended audience. Staff from communication groups or public information offices within
EPA regional offices may help to determine whether or not the presentation addresses the needs of
various audiences. Also, practicing the presentation with co-workers who are unfamiliar with the site can
help assure that the appropriate messages are being conveyed, and will help the team prepare for potential
questions that will arise during the meeting.
(5) Present Information
A number of factors should be considered when developing a plan to present the PRA in a
meeting. Although the size of the public meeting can sometimes be unpredictable, typically individuals
will feel more comfortable asking questions and expressing opinions in small, informal settings. For any
audience, it is usually helpful to have general fact sheets on PRA available for distribution. The fact
sheets may contain information that describes the PRA process, how information from the PRA will be
used at the site, and how the community may comment on the PRA report. The meeting team should
usually include the CIC, RPM, Risk Assessor, and additional support as necessary.
Audio-visual materials and equipment should be checked prior to the start of the meeting. For
example, overheads should be viewed from the audience seating to assure that information is accessible
and readable. Presentations using portable computers can be effective for showing how the results of the
PRA may differ with changes in modeling assumptions.
(6) Post-meeting Review of Presentation and Community Feedback
At the end of a meeting, it can be helpful to encourage participants to provide feedback regarding
effective and ineffective communication techniques. Not only can this information be used to improve
presentations offered to similar audiences in the future, it also provides a sense for how well the main
messages and specific technical issues were communicated.
Page 6-9
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
(7) Update Information as Needed for Future Assessments and Presentations
Shortly after the meeting or briefing, modifications should be made to the materials for future
presentations where appropriate. In addition, if information is obtained that is relevant to the risk
assessment, this information may be included in a subsequent analysis, and the process would be
repeated.
6.3 COMMUNICATING DIFFERENCES BETWEEN POINT ESTIMATE AND PRA
One method for effectively explaining the PRA approach to quantifying variability and
uncertainty is to employ comparisons to the more easily understood point estimate methodology. These
comparisons can focus on either the inputs or the outputs associated with the two approaches. The
communicator may focus on a specific input variable, such as drinking water intake, and explain that with
the point estimate methodology, a single average or high-end value (e.g., 2 liters per day for adults)
normally is used to quantify exposure, whereas with PRA, a probability distribution (e.g., lognormal) is
used to characterize variability in exposure among a population. In addition, the outcomes (e.g., cancer
risk estimates) can be compared by showing where the point estimate(s) of risk fall within the distribution
of risks generated with PRA.
When communicating results from point estimate and PRA models, an important concept to keep
in mind is that both methods yield risk estimates with varying degrees of uncertainty. Continuing with
the above example, concepts associated with uncertainty (e.g., representativeness, data quantity, and data
quality) can be introduced by asking the audience if their estimate of water consumption on a specific day
would be equal to their average daily consumption rate over a 1-year period. This example highlights a
common source of uncertainty in exposure data (i.e., using short-term survey data to estimate long-term
behavior). Section 6.5 discusses different perceptions of uncertainty.
It is common to accept output from quantitative models without fully understanding or
appreciating the corresponding uncertainties and underlying assumptions. One challenge in presenting
PRA results is to determine the most effective way to communicate sources of uncertainty without
undermining the credibility of the assessment (see Section 6.6). For example, it may be counterintuitive
that the more sources of uncertainty that are accounted for in a PRA, the wider the confidence intervals
tend to be in the risk estimates (see Section 6.4.2). The audience may question the utility of a method that
appears to introduce more complexity in a risk management decision. It may be useful to point out that
many sources of uncertainty are present, and methods available to acknowledge and quantify them may
differ in point estimate and probabilistic risk assessments.
The basic concepts of PRAs described in Chapter 1 may be used in developing presentations.
Exhibits 1-5 and 1-6 in Chapter 1 summarize some of the advantages and disadvantages of point estimates
and probabilistic approaches that should be considered when evaluating differences in the risk estimates
of the two approaches. For example, point estimates of risk do not specify the proportion of the
population that may experience unacceptable risks. In contrast, PRA methods allow statements to be
made regarding both the probability of exceeding a target risk, and the level of confidence in the risk
estimate.
When summarizing results of PRA, graphs and tables generally should include the results of the
point estimates of risk (e.g., CTE and RME). It may be informative to note where on the risk distribution
each of the point estimates lies. By understanding the assumptions regarding the inputs and modeling
Page 6-10
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
approaches used to derive point estimates and probabilistic estimates of risk, a communicator will be
better prepared to explain the significant differences in risk estimates that may occur. Special emphasis
should be given to the model and parameter assumptions that have the most influence on the risk
estimates, as determined from the sensitivity analysis (see Appendix A).
6.4 GRAPHICAL PRESENTATION OF PRA RESULTS TO VARIOUS AUDIENCES
Graphics can be an effective tool for communicating concepts in PRA. As the old adage goes, "A
picture is worth a thousand words." A graphic usually can be most easily understood by a diverse
audience when it conveys a single message. It is generally a good idea to keep the graphics simple so that
the message is clear. In general, each graphic should be developed and modified depending on the type
of presentation and the intended audience.
f The key to presenting graphics in PRA effectively is to select a relatively small
number of appropriate messages, and to find a balance between meaningful
information and overwhelming detail.
Points to consider when developing graphics for public meetings, senior staff, and the press are
presented below. Certainly, recommendations for presenting clear and informative graphics are
applicable to all three forums. Practical recommendations for graphical analysis techniques and tips for
successful visual displays of quantitative information are given by Tufte (1983) and Helsel and Hirsch
(1993).
6.4.1 PUBLIC MEETING
For a public availability session (or meeting), care should be taken to assure that the graphics are
of appropriate size and the lettering is easy to read. For example, a graphic on an 8 1A x 11 inch sheet of
paper, or a font size smaller than 18 pt in a computer presentation, may not be easily seen from the back
of a large auditorium. It may be appropriate to present information using large posters, spaced so that the
audience may move among them and discuss the posted results with the risk assessor or RPM. Handouts
and a glossary of terms may also be used. Using slides with too much text should be avoided, since the
information may be difficult to read and understand. Pre-planning and pilot testing the graphics before
the presentation may be helpful in assuring that the message is accurately portrayed to the community.
Consistent with EPA's guidance on risk characterization, the CTE and RME cancer risks and
noncancer hazards, and EPA's decision point should be highlighted on graphics. The discussions
accompanying the graph should emphasize that these values represent risks to the average and high-end
individuals, respectively, and serve as a point of reference to EPA's decision point. The distribution of
risks should be characterized as representing variability among the population based on differences in
exposure. Similarly, graphics that show uncertainty in risk can be described using terms such as
"confidence interval", "credible interval", or plausible range. The graphics need not highlight all
percentiles. Instead, selected percentiles that may inform risk management decisions (such as the S^SO*,
90th, 95th, and 99th percentiles) should be the focus. Figure 6-1 shows an example of a PDF for variability
in risk with an associated text box for identifying key risk percentiles.
Page 6-11
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
0.06
CD
Q
0.05 -
0.04 -
= 0.03 -
CO
O
ol
0.02 -
0.01 -
0.00
O.OE+00
99th %ile = 1.8E-06
95th %ile = 1.2E-06
90th %ile = 9.2E-07
50th %ile = 4.1E-07
5.0E-07 1.0E-06 1.5E-06 2.0E-06 2.5E-06 3.0E-06
Risk
1.00
0.00
99th %ile = 1.8E-06
95th%ile = 1.2E-06
90th %ile = 9.2E-07
50th %ile = 4.1E-07
O.OE+00
5.0E-07
1 .OE-06
1.5E-06
Risk
2.0E-06
2.5E-06
3.0E-06
Figure 6-1. Hypothetical PRA results showing a probability density function (PDF) (top
panel) for cancer risk with selected summary statistics for central tendency and high-end
percentiles. This view of a distribution is useful for illustrating the shape of the distribution
(e.g., slightly right-skewed) and explaining the concept of probability as the area under a curve
(e.g., most of the area is below 1E-06, but there is a small chance of 2E-06). Although
percentiles can also be overlayed on this graphic, a cumulative distribution function (CDF)
(bottom panel) may be preferable for explaining the concept of a percentile.
Page 6-12
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Figure 6-2 gives two examples of graphics that can be used to display results of a sensitivity analysis
from a Monte Carlo Analysis (MCA). While both graphics are likely to be understood by non-technical
audiences, the pie chart may be more familiar. The pie chart (Figure 6-2A) suggests that the results
should sum to 1.0, which may not be true if there are correlations among one or more variables, or if only
a subset of the variables are displayed (e.g., those that contribute at least 1%). The available data can be
normalized so that the squared correlation coefficients do sum to 100%, and this approach has been
adopted by some commercial software available to run Monte Carlo simulations (e.g., Crystal Ball® by
Decisioneering, www.decisioneering.com). The benefit of showing the squared correlation coefficient
(r2 or r-square, also called the coefficient of determination), rather than the correlation coefficient (r) is
that r-square is proportional to the total variation in risk associated with specified input variable.
Therefore, one can use the r-square to describe, in quantitative terms, the contribution of the input
variable to the total variance in the risk distribution. In this example, exposure duration (ED) contributes
approximately two-thirds (64%) to the total variance in risk.
A more technical graphic is the tornado plot (Figure 6-2B). In addition to showing the relative
magnitude of the correlations (r-square), it illustrates the direction of influence a specific variable has on
the final risk estimate. Bars that extend to the right indicate a positive correlation (e.g., high risk
estimates correspond with high values for the variable), whereas bars that extend to the left indicate a
negative correlation (e.g., high risk estimates correspond with low values for the variable.) In this
example, the exposure duration (ED) has the largest positive correlation with risk, while body weight
(BW) has the largest negative correlation with risk.
The graphics shown in this chapter are a small fraction of the graphics that might be used to
communicate concepts related to PRA. Numerous additional examples are given throughout this
guidance document. Table 6-1 provides a summary of cross references to other figures that were
developed for this guidance document to convey specific concepts regarding variability and uncertainty.
Page 6-13
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Table 6-1. Examples of Graphics for Communicating PRA Concepts in this Guidance Document.
General PRA Topic Area
Conceptual Diagrams for Fundamental Concepts
Monte Carlo Analysis
Tiered process for PRA
PDFs and CDFs
Input variable(s)
Risk distribution with selected percentiles
highlighted
Comparing RME risk (e.g., 95th percentile) with
risk level of concern
Selecting and Fitting Probability Distributions
Fitting distributions - frequency distribution
overlaid by a PDF
Lognormal probability plot
Sensitivity Analysis
Sensitivity analysis - tornado plot of Spearman
rank correlations
Sensitivity analysis - pie chart
Joint probability curve
Variability in toxicity
Species sensitivity distribution
Iterative Simulations
CDFs from multiple 1-D MCA simulations to
convey uncertainty in the risk distribution
PRG Selection
Estimation from best-fit line for RME risk and
EPC
RME risk ranges corresponding to alternative
choices of PRG
90% credible interval for RME risk (95th
percentile) corresponding to alternative choices of
PRG
Location
Figure 1-2
Figure 2-1, 2-2
Figure 1-1, 4-4, 4-5,
4-6
Figure 6-1
Figure 1-3, 4-3, 7-2,
Figure 3-1
Figure 5-2
Figure 3-6, 6-2b
Figure 6-2a
Figure 4-8
Figure 4-7
Figure 3-3
Figure 5-1
Figure 7-4
Figure 7-5
Variability Uncertainty
X
X X
X
X
X
X
X
X
X
X
X
X
X
X
X
Page 6-14
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Bi-model distribution for concentration showing
pre-remediation EPC, post-remediation EPC,
remediation action level, and uniform distribution
for clean fill
2-D MCA Results
Illustration of tabular and graphic outputs of a 2-D
MCA
Confidence intervals (or credible intervals) on a
risk distribution
Box-and-whisker plot for results of 2-D MCA
Horizontal box-and-whisker plots with multiple
CDFs
Figure 5-3
Figure 4-9
Figure 1-4, 4-10,
4-11,4-12
Figure 3-4, 7-3
Figure 6-3
X X
X
X
X
X X
Page 6-15
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
SA_skin
1%
A. Pie Chart
IR_soil
18%
B. Tornado Plot
-1
ED
IR_soil
AF
EF
SA_skin
-0.1 0|H
1 0.64
10.18
IO.OG
0.01
0.01
BW
.0 0.0 1.0
Square of Rank Correlation (r2)
Figure 6-2. Results of a sensitivity analysis shown as a pie chart (A) and tornado plot (B). Both graphics illustrate
the concept of the relative contribution to variance for exposure variables that contribute at least 1% to the variance
in risk. The pie chart suggests that the sum of the squared rank correlations equals 1.0, which is true only if the
results are normalized to 100%. The tornado plot gives both the magnitude and direction (positive or negative) of
the correlation. ED=exposure duration, IR_soil=soil ingestion rate, AF=absorption fraction, EF=exposure
frequency, SA_skin=surface are of skin, and BW=body weight.
Page 6-16
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
6.4.2 EPA SENIOR STAFF
For communicating PRA with EPA's senior risk managers (e.g., EPA Section Chiefs, EPA
Branch Chiefs, or EPA Division Directors), an executive summary or executive briefing package may be
appropriate. This presentation should highlight major findings, compare point estimate and probabilistic
results, provide sensitivity analysis results, and state uncertainties addressed in the PRA.
1.00
0.75
5th 25th 50th 75th 95th
Percentiles
TO
.Q
3
o
0.50
0.25
0.00
2.0
Figure 6-3. The results of a 2-D MCA. The graphic shows a method of presenting variability as a cumulative
distribution function and uncertainty as box plots at the 25th, 50th, and 95th percentiles of variability. The CDF of
the 50th percentile is represented by the solid line and the CDFs given by the dotted lines represent the 5th and
95th percentiles of uncertainty for each percentile of variability.
EPA senior level risk managers would generally be most interested in the risk estimates at the
50th, 90th, 95th, and 99.9th percentiles (i.e., a CTE risk estimate and the RME risk range). EPA senior
managers may also wish to know the uncertainty surrounding each of the percentiles of risk. This
uncertainty can be described in a table (e.g., confidence intervals around the 95th percentile risk) or a
graphic (e.g., box-and-whisker plots). It is advisable for the risk assessor to have this information on
hand during the briefing to respond to questions. Presenting distributions of uncertainty along with
distributions of variability can create a very busy figure or table—it is best to keep things simple.
Page 6-17
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Figure 6-3 shows cumulative distribution functions (CDFs) for the Hazard Quotient (HQ) for a
single chemical, representing variability in HQ. One method of displaying uncertainty is to use
box-and-whisker plots. In this example, the horizontal box and whiskers represent uncertainty around
selected percentile estimates of variability. Specifically, the three box-and-whisker plots correspond to
the 25th, 50th, and 95th percentiles of the distribution for variability in HQ. The box shows the 25th and
75th percentiles (i.e., interquartile range) of uncertainty, whereas the whiskers show the 5th and
95th percentiles of uncertainty. In this example, uncertainty in the 95th percentile HQ is quantified by the
box-and-whiskers plot in which the 5th percentile of uncertainty is 1.1, the 50th percentile is 1.3, and the
95th percentile is 1.4. This suggests that despite the uncertainty in the estimate of the 95th percentile of
variability, an HQ of 1.0 is likely to be exceeded. Sometimes such results are said to describe the
90% confidence interval in the 95th percentile HQ. The term "confidence interval" is used loosely in this
context to convey information about uncertainty; however, it is not the same as a statistical confidence
limit that one might obtain by estimating a population parameter from a sample. An alternative term that
may be more appropriate in this case is "credible interval".
The three curves represent similar information on uncertainty across the complete range of
percentiles for variability. The solid line shows the CDF for all of the 50th percentiles of uncertainty,
whereas the dotted lines show the 5th and 95th percentiles of uncertainty.
The box-and-whisker plot is simple to produce, conveys information about the symmetry and
width of the confidence interval, and is easy to interpret (Tufte, 1983). In general, box-and-whisker plots
are useful for summarizing results from two-dimensional Monte Carlo (2-D MCA) simulations. The
methods and inferences associated with 2-D MCAs are discussed further in Appendix D. The results of a
2-D Monte Carlo simulation represent a range of possible estimates for the percentile given one or more
sources of uncertainty that were included in the simulation. If the target audience for this graphic has a
greater understanding of statistics, it may be less confusing if alternative phrases are used to describe the
results, such as "credible interval" or "probability band".
Graphics that show probability density functions for uncertainty (PDFu's) are generally more
meaningful to a technical audience of risk assessors and uncertainty analysts. Alternative graphics may
be needed to communicate other sources of uncertainty in risk estimates (e.g., use of alternative
probability models for exposure variables, effect of changes in the model time step, application of spatial
weighting to concentration data, etc.). Additional information on communicating risks to senior EPA
managers is given by Bloom et al. (1993).
The results from the sensitivity analysis may be useful to the senior managers in deciding whether
additional sampling is necessary. One issue that may be important to address with risk managers and
senior staff is that the width of the credible interval (e.g., 5th to 95th percentiles of uncertainty) will be
determined in part by the number of sources of uncertainty that are quantified. As additional sources of
uncertainty are quantified and included in the model, the interval around the risk distribution will tend to
widen. This situation may appear to be counterintuitive for those managers who expect confidence to
increase as uncertainty is quantified. However, by uncovering and quantifying the sources of uncertainty,
the benefits in the risk communication and decision-making process should become clear. The results of
the sensitivity analysis should help to focus discussions, data collection efforts, and analyses on the more
significant sources of uncertainty. In addition, by developing estimates of credible intervals of
uncertainty in risk estimates, the decision-making process using the tiered approach may become more
transparent.
Page 6-18
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
6.4.3 PRESS RELEASES
For a press briefing presentation, care should be given to identify messages and develop
publication quality graphics with clear descriptions that can be provided in press packages. It is usually a
good idea to provide the graphics in both color and black and white so that the press can choose the most
appropriate presentation style for the story. The RPMs generally should work with the CIC, the press
staff in the Communication Division, and senior managers to develop press materials. Adequate time
should be left for the preparation of materials and internal Agency review and approval before
information is released.
6.5 PERCEPTION OF RISK AND UNCERTAINTY
The purpose of this section is to present current thinking about how people view risk and
uncertainty. This section should provide useful information for planning risk communication and
addresses the first step in the seven step process (Section 6.2.2), "Identify the Audience."
There are many individual differences in the way people regard the risks and hazards that are
present in modern life. These differences have their roots in the differences in perception of risk and
uncertainly of the individual human mind (Slovic, 1986). The risk assessor and/or risk communicator
should keep in mind the general perceptions about risk held by different groups. Communications should
be tailored to the specific audience. This section summarizes some of the criteria used to judge risks in
the absence of scientific data and the direction of the potential bias that may be expected by applying
these criteria. Additional publications on this issue are identified in the reference section at the end of this
chapter.
In the absence of scientific data, the general public evaluates risks using inferences of judgment
as described below (Slovic et al., 1979):
• Availability: People tend to judge risks as more likely if they are easy to recall.
Overconfidence: People tend to be overconfident about the judgments they make based on
the use of heuristics.
Desire for Certainty: People tend to misgauge risk/benefit conflicts in favor of the benefits
as a result of a desire for certainty and anxiety about uncertainty.
Page 6-19
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Slovic et al. (1979) identified nine characteristics of risk that may influence perceptions. These
nine dimensions may provide a perspective on whether a health risk is perceived as "more risky" or "less
risky", as described in the table below.
Dimension of Risk
Voluntariness
Immediacy of the effect
Exposed persons' knowledge about
risk
Sciences' knowledge about risk
Control over risk
Newness
Chronic/Catastrophic
Common/Dread
Severity of the consequences
More Risky
Involuntary
Delayed
Low
Low
Low
Unfamiliar or New
Catastrophic
Dreaded
High
Less Risky
Voluntary
Immediate
High
High
High
Familiar
Chronic
Common
Low
The presentation of uncertainty in a risk estimate can be interpreted with vastly different
conclusions depending on the audience and their perceptions. For example, a thorough scientific account
of multiple sources of uncertainty presented to a group of interested risk assessors and environmental
scientists may be clearly understood. Such a group will likely conclude that the assumptions made in the
risk assessment were appropriate and that the results can be used with confidence as a decision support
tool. In contrast, a similar scientific presentation given to the community may be misunderstood, and the
perceived risk may be greater. Citizens are often more concerned about the potential impact to their
personal situation, than to the uncertainty in the risk estimate. Consequently, the community may react
negatively to a long, highly scientific presentation on uncertainty. A good rule of thumb is to limit the
presentation to no more than 15 minutes.
Focusing heavily on uncertainty may cause citizens to conclude that the risk must be high. They
may also conclude that the presenter is incompetent because he or she is not sure of anything, or that the
presenter is trying to hide something by cloaking the information in technical jargon, or even that the
presenter is intentionally avoiding the public's issues of concern. To the extent possible, technical jargon
during the presentation should be avoided or explained.
A helpful presentation generally should incorporate the following steps: (1) present information
about the conclusions that can be drawn from the risk assessment; it is extremely frustrating for
decision-makers to receive detailed information on uncertainty without conclusions (Chun, 1996);
(2) describe the certainty of the information that supports these conclusions; (3) address the uncertainty
and its implications for the conclusions; and (4) present the information without jargon and in a frank and
open manner. Section 6.4 provides examples of graphics that may be useful in presentations of PRA.
Page 6-20
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
6.6 TRUST AND CREDIBILITY
The single most important quality a presenter may need to possess in order to communicate to
others is a sense of trust and credibility. Trust and credibility are based on working with the community
and providing thoughtful, accurate responses to questions and concerns raised by the community.
Building trust and credibility is important, whether communicating to a high-level technical audience, a
RPM/decision-maker who wishes to have the "big picture," or the public.
Credibility can best be established through a long history of frank and open discussions with the
community. In addition, a presenter can gain credibility if he or she has the ability to restate the available
information so that it addresses the concerns and interests of an audience. The ability to garner trust and
credibility comes from knowing the audience, respecting their opinion, and communicating at an
appropriate level (U.S. EPA, 1994).
6.7 COMMUNICATION ISSUES FOR RPMs
Following the RPM's decision to conduct a site-specific PRA, the level of stakeholder
involvement in the development and review of the PRA should be evaluated. Establishing the appropriate
level of stakeholder involvement may include input from the CIC, risk assessor and appropriate senior
managers (e.g., Section Chief, Branch Chief, etc.). The level of stakeholder involvement may vary
depending on the site complexity and the interest of the community. As an initial step, it may be
appropriate to conduct an exploratory session where letters are sent to various stakeholders (e.g.,
environmental groups, CAG, etc.) inviting their participation in a general meeting on the topic of PRA. If
there is a strong interest among the stakeholders, then a more involved communication plan may be
appropriate including, but not limited to the following steps:
• Providing stakeholders with an introduction to the principles of PRA in an informal session
(e.g., public availability session).
Providing a draft Scope of Work (SOW) to interested stakeholders followed shortly thereafter
by an availability session to discuss comments on the document.
• Providing a period of time for the stakeholders to review and comment on the selected
distributions, including an availability session for discussions with EPA staff where the
community may help to identify key site-specific information such as exposure factors and
receptor behavior.
Providing the opportunity for EPA risk assessor to meet with the TAG grantee (if
appropriate) and stakeholders to ask questions regarding the SOW.
• Providing a revised SOW including a response to stakeholder comments.
Page 6-21
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Providing an overview of the final PRA at a public meeting and providing appropriate
supporting PRA documents in the repositories for stakeholder review and comment. This
session may be part of the general session regarding the remedial investigation when the risk
assessment is discussed. Based on the complexity of the PRA, it may be appropriate to hold a
public availability session where the stakeholders (including the TAG grantee), if
appropriate, are able to meet with EPA staff to ask questions and offer suggestions regarding
the document.
Providing a response to comments from stakeholders regarding the PRA.
If the level of interest is low, then a less extensive CIP may be appropriate. In this case, fact
sheets (in plain language) describing the general principles of PRA to the stakeholders and the key
findings of the PRA may be provided (U.S. EPA, 2000a). At public meetings where the risk assessment
is discussed, a short discussion of the PRA findings and their significance may be appropriate. The PRA
document should be made available in the repositories for review and comment by the stakeholders.
For sites with medium interest, a combination of the activities identified above may be
appropriate. For example, it may be appropriate to have a public availability session on the principles of
PRA and then make the documents available for review and comment.
The RPM should consider a number of administrative issues in developing the plan for involving
the stakeholders in the PRA. Issues to consider include: staff resources, funds for obtaining meeting
space, availability of contractor support, significance of PRA in decision making, and the length of time
required to complete the RI/FS. To aid in reducing costs, it may be appropriate to combine meetings
regarding PRA and point estimate risk assessment based on the close links between the documents.
Page 6-22
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
REFERENCES FOR CHAPTER 6
Bloom, D.L. et al. 1993. Communicating Risk to Senior EPA Policy Makers: A Focus Group Study. U.S.
EPA Office of Air Quality Planning and Standards.
Chun, A. 1996. Strategies for Communicating Uncertainty to the Public. IBM Risk Conference
Proceedings, October 31.
Helsel, D.R. and R.M. Hirsch. 1993. Statistical Methods in Water Resources. Elsevier Science.
Amsterdam.
Slovic, P., B. Fischoff, and S. Lichtenstein. 1979. Rating the Risks. Environment 21(3): 14-20 and
36-39.
Slovic, P. 1986. Informing and Educating the Public About Risk. Risk Anal. 6(4):403-415.
Tufte, E.R. 1983. The Visual Display of Quantitative Information. Graphics Press. Chesire, CT.
U.S. EPA. 1989. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health
Evaluation Manual (HHEM) (Part A, Baseline Risk Assessment). Interim Final. Office of
Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1991a. Risk Assessment Guidance for Superfund (RAGS), Volume I: Human Health
Evaluation Manual (HHEM), Part B, Development of Risk-Based Preliminary Remediation
Goals. Office of Emergency and Remedial Response, Washington, DC. EPA/540/R-92/003.
NTISPB92-963333.
U.S. EPA. 1991b. Role of the Baseline Risk Assessment in Superfund Remedy Selection Decisions.
Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive
No. 9355.0-30.
U.S. EPA. 1994. Seven Cardinal Rules of Risk Communication. Office of Policy Analysis. Washington,
DC. EPA/OPA/87/020.
U.S. EPA. 1998. Superfund Community Involvement Handbook and Toolkit. Office of
Emergency and Remedial Response, EPA 540-R-98-007.
U.S. EPA. 1999a. Risk Assessment Guidance for Superfund: Volume I-Human Health Evaluation
Manual. Supplement to Part A: Community Involvement in Superfund Risk Assessments.
EPA/540/R-98/042, March.
U.S. EPA. 1999b. Superfund Risk Assessment and How You Can Help: An Overview. Videotape.
September 1999 (English version) and August 2000 (Spanish version). English Version:
EPA-540-V-99-003, OSWER Directive No. 9285.7-29B. Spanish Version (northern Mexican):
EPA-540-V-00-001, OSWER Directive No. 9285.7-40. Available through NSCEP: 800.4909.198
or 513.489.8190.
Page 6-23
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
U.S. EPA. 2000a. El SuperfundHoy Dia. La Estimation de Reisgos: Como Lograr La
participation del la Comunidad. ^Que es la Estimation del Riesgopara la SaludHumana?
OSWER Directive No. 9200.2-26K. Enero. (fact sheet)
U.S. EPA. 2000b. Superfund Risk Assessment and How You Can Help. Videotape. (English version
only). EPA-540-V-99-002, OSWER Directive No. 9285.7-29A. Available through NSCEP:
800.4909.198 or 513.489.8190, September.
U.S. EPA. 2001. Early and Meaningful Community Involvement. Office of Solid Waste and Emergency
Response. Washington, DC. OSWER Directive No. 9230.0-99. October 12.
Supplemental References Regarding Risk Communication and Public Perception
Connelly, N.A. and B.A. Knuth. 1998. Evaluating Risk Communication: Examining Target Audience
Perceptions About Four Presentation Formats for Fish Consumption Health Advisory
Information. Risk Anal. 18:649-659.
Covello, V.T. 1987. Decision Analysis and Risk Management Decision Making: Issues and Methods.
Risk Anal. 7(2): 131-139.
Deisler, P.E. 1988. The Risk Management-Risk Assessment Interface. Last in a Five-Part Series on
Cancer Risk Assessment. Environ. Sci. Technol. 22:15-19.
Fischhoff, B. 1995. Risk Perception and Communication Unplugged: Twenty Years of Process.
Risk Anal. 15(2): 137-145.
Fischhoff, B. 1998. Communicate unto others. Reliab. Eng. Syst. Saf. 59:63-72.
Fischhoff, B., A. Bostrom and M.J. Quadrel. 1997. Chapter 34. Risk Perception and Communication.
In: Oxford Textbook of Public Health, Vol. 2, pp 987-1002. London: Oxford Univ. Press (Ed. R
Defels, et al.).
Hora, S.C. 1992. Acquisition of Expert Judgment: Examples from Risk Assessment. J. Energy Eng.
118(2):136-148.
Ibrekk, H. and M.G. Morgan. 1987. Graphical Communication of Uncertain Quantities to
Non-Technical People. Risk Anal. 7:519-529.
Johnson, B.B. and P. Slovic. 1995. Presenting Uncertainty in Health Risk Assessment: Initial
Studies of its Effects on Risk Perception and Trust. Risk Anal. 15:485-494.
Kaplan, S. 1992. 'Expert Information' Versus 'Expert Opinions.' Another Approach to the Problem of
Eliciting/Combining/Using Expert Knowledge in PRA. Reliab. Eng. Syst. Saf. 35:61-72.
Morgan, M.G., A. Bostrom, L. Lave and C. J. Atman. 1992. Communicating Risk to the Public.
Environ. Sci. Technol. 26(11):2048-2056.
Ohanian, E.V., J.A. Moore, J.R. Fowle, et al. Workshop Overview. 1997. Risk Characterization: A Bridge
to Informed Decision Making. Fundam. Appl. Toxicol. 39:81-88.
Page 6-24
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 6 ~ December 31, 2001
Thompson, K.M. and D.L. Bloom. 2000. Communication of Risk Assessment Information to Risk
Managers. J. Risk Res. 3(4):333-352.
Page 6-25
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
CHAPTER 7
ROLE OF THE PRA IN DECISION MAKING
7.0 INTRODUCTION
When deciding whether or not to remediate a hazardous waste site, the risk manager needs to
know if an unacceptable risk is present, and if so, what cleanup level to apply to the contaminated media.
For this information, the risk manager should turn to the risk assessor for help in interpreting the results of
the risk assessment. This chapter provides guidance on how to interpret the results of a probabilistic risk
assessment (PRA) to help determine if an unacceptable risk is present, and the criteria to consider when
deriving a risk-based preliminary remediation goal (PRO) and a final remedial goal.
7.1 GENERAL PRINCIPLES OF RISK-BASED DECISION MAKING IN SUPERFUND
Under Agency policy, an individual with reasonable maximum exposure (RME) will generally be
the principal basis for evaluating potential human health risks at Superfund sites (see Risk Assessment
Guidance for Superfund (Section 6.1.2 of U.S. EPA, 1989) and the National Contingency Plan's (NCP)
Preamble (U.S. EPA, 1990)). The RME is defined as the highest exposure that is reasonably expected to
occur at a site, and is intended to estimate a conservative exposure case (i.e., well above the average case)
that is still within the range of possible exposures. In general, where cumulative carcinogenic risk to the
RME individual is less than 1E-04, and the non-carcinogenic Hazard Index (HI) is less than or equal to 1,
remedial action is not warranted under Superfund unless there are adverse environmental impacts, or the
applicable or relevant and appropriate requirements (ARARs) are not met. As discussed in Section 7.2.4,
the RME receptor is often (although not always) an appropriate basis for evaluation of risks to ecological
receptors, as well.
Once a determination of unacceptable risk to humans and/or ecological receptors has been made,
the risk managers will typically ask the risk assessor to develop site-specific PRGs. PRGs are generally
defined as health-based chemical concentrations in an environmental media for which the risks (cancer or
noncancer) to the RME receptor would not exceed some specified target level. For systemic or
noncarcinogenic toxicants, the target risk level is generally a HI of unity (1). This is considered to be a
threshold concentration to which the human population (including sensitive subgroups) and ecological
receptors may be exposed without adverse effect during less-than-lifetime (i.e., chronic, subchronic, or
short-term) exposures. For carcinogens, the target risk level used to derive the PRG typically represents a
cumulative lifetime cancer risk to an individual of between 1E-06 and 1E-04 (equivalently expressed as
10~6 and 10~4). For carcinogenic risks, less-than-lifetime exposures are converted to equivalent lifetime
values (U.S. EPA, 1989). The 1E-06 risk level is specified in the NCP as a point of departure for
determining remediation goals when ARARs are not available or not sufficiently protective. It is
important to remember that risk-based PRGs are initial guidelines and do not represent final cleanup or
remediation levels. Remediation levels are finalized after appropriate analysis in the remedial
investigation/feasibility study (RI/FS) and record of decision (ROD). A final cleanup level may differ
from a PRG based on the risk manager's consideration of various uncertainties in the risk estimate, the
technical feasibility of achieving the PRG, and the nine criteria outlined in the NCP (see Chapter 1,
Exhibit 1-2).
Page 7-1
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
EXHIBIT 7-1
DEFINITIONS FOR CHAPTER 7
Applicable or Relevant and Appropriate Requirements (ARARs) - Federal or state environmental standards; the NCP
states that ARARs should be considered in determining remediation goals. ARARs may be selected as
site-specific cleanup levels.
Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in a population,
usually considered to be the mean or median of the distribution.
Confidence Interval - A range of values that are likely to include a population parameter. Confidence intervals may
describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95th percentile
risk). When used to characterize uncertainty in a risk estimate, it is assumed that methods used to quantify
uncertainty in the model inputs are based on statistical principles such as sampling distributions or Bayesian
approaches. For example, given a randomly sampled data set, a 95% confidence interval for the mean can be
estimated by deriving a sampling distribution from a Student's t distribution.
Credible Interval - A range of values that represent plausible bounds on a population parameter. Credible intervals
may describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95th percentile
risk). The term is introduced as an alternative to the term confidence interval when the methods used to
quantify uncertainty are not based entirely on statistical principles such as sampling distributions or Bayesian
approaches. For example, multiple estimates of an arithmetic mean may be available from different studies
reported in the literature—using professional judgment, these estimates may support a decision to describe a
range of possible values for the arithmetic mean.
Hazard Index (HI) - The sum of more than one Hazard Quotient for multiple substances and/or multiple exposure
pathways. The HI is calculated separately for chronic, subchronic, and shorter-duration exposures.
Hazard Quotient (HQ) - The ratio of a single substance exposure level over a specified time period (e.g., subchronic)
to a reference dose (or concentration) for that substance derived from a similar exposure period.
Preliminary Remediation Goal (PRO) - Initially developed chemical concentration for an environmental medium
that is expected to be protective of human health and ecosystems. PRGs may be developed based on applicable
or relevant and appropriate requirements, or exposure scenarios evaluated prior to or as a result of the baseline
risk assessment. (U.S. EPA, 1991a, 1991b).
Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site (U.S.
EPA, 1989). The intent of the RME is to estimate a conservative exposure case (i.e., well above the average
case) that is still within the range of possible exposures.
Remedial Investigation/Feasibility Study (RI/FS) - Studies undertaken by EPA to delineate the nature and extent of
contamination, to evaluate potential risk, and to develop alternatives for cleanup.
RME Range - The 90th to 99.9th percentiles of the risk distribution generated from a PRA, within which an RME risk
value may be identified. The 95th percentile is generally recommended as the starting point for specifying the
RME risk in a Superfund PRA.
RME Risk - The estimated risk corresponding to the reasonable maximum exposure.
Page 7-2
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
7.2 INTERPRETING A RISK DISTRIBUTION
7.2.1 WHAT Is A DISTRIBUTION OF RISK AND WHAT DOES IT LOOK LIKE?
In the traditional point estimate risk assessment approach, risks to the RME individual are
characterized as single point values (e.g., HI=2, or cancer risk=lE-05). In the PRA approach, the output
of the risk assessment is an estimate of the distribution of risks across all members of the population. An
example is shown in Figure 7-1.
1.00 -,
0.80 -
_>,
05 0.60 -1
.0
CL
® 0.40 -]
E
D
O
0.20 -
0.00
99th%ile = 1.8E-06
95th%ile = 1.2E-06
90th %ile = 9.2E-07
50th%ile = 4.1E-07
O.OE+OO
5.0E-07
1.0E-06
1.5E-06
Risk
2.0E-06
2.5E-06
3.0E-06
Figure 7-1. Hypothetical PRA results showing a cumulative distribution function (CDF) for lifetime excess
cancer risk.
In this example, the x-axis of Figure 7-1 represents the excess lifetime cancer risk level and the y-axis
represents the cumulative probability of the cancer risk level within the hypothetical population. The
graph also shows various landmarks along the distribution curve such as the 50th percentile, the 90th, 95th,
etc. In this illustration, the 95th percentile corresponds to a cancer risk of 1.2E-06.
Page 7-3
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
7.2.2 WHAT is THE RME RANGE?
Given a risk distribution such as shown in Figure 7-1, what part of the risk distribution should a
risk manager be concerned about? As explained above, the risk to the RME receptor is a key factor in
making decisions regarding the need for action at a Superfund site. EPA's Guidelines for Exposure
Assessment (U.S. EPA, 1992) states that the "high-end" (or RME) of exposure for a population occurs
between the 90th and 99.9th percentiles, with the 99.9th percentile considered a bounding estimate.
Similarly, PRAs developed to support RME risk estimates for Superfund should reflect this approach.
«s" In this guidance, the 90th to 99.9th percentiles of the risk distribution are
collectively referred to as the recommended RME range.
In utilizing PRA results to determine if an unacceptable risk is present and to develop a PRO
which is sufficiently protective, risk managers should address two questions:
(1) What percentile of the risk distribution will be selected to represent the RME receptor?
(2) How will information on uncertainty in the high-end risk estimates be used in this process?
The risk manager may consider a number of factors in choosing a specific percentile to represent
the RME individual. This may include both quantitative information and professional judgment. In
particular, risk managers may need to understand what sources of variability and uncertainty are already
explicitly accounted for by the modeling approach and inputs (i.e., point estimates and/or probability
distributions) used to estimate the risk distribution, and what sources may be present but are not
quantified. Approaches for selecting an appropriate percentile in human health and ecological risk
assessments are described below.
7.2.3. RELATING THE RISK DISTRIBUTION TO THE RISK MANAGEMENT GOAL FOR HUMAN HEALTH
In most cases, a recommended starting point for risk management decisions regarding the RME is
the 95th percentile of the risk distribution. The 95th percentile for the risk distribution is an appropriate
description of high-end exposure as identified by the Presidential/Congressional Commission on Risk
Assessment and Risk Management (1997).
is" In human health PRA, a recommended starting point for risk management
decisions regarding the RME is the 95* percentile of the risk distribution.
Figure 7-2 illustrates this approach for a site where cancer risks are the risk driver. Assume the
risk manager has selected an excess cancer risk of 1E-05 as the risk management goal, and the
95th percentile as the definition of the RME. If line B on the graph represents a 1E-05 probability of
cancer, a no-action decision may be warranted because the 95th percentile of the risk distribution is below
the cancer risk level of concern. Conversely, if we were to assume that the 95th percentile is above the
risk level of concern (i.e., line A on the graph represents 1E-05), remedial action may be warranted.
Page 7-4
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
c
0)
a
0.06
0.05
0.04
A ,h B
95th
Percentile
= 0.03
0.02
0.01 -
0.00
Risk
Figure 7-2. Example of a probability distribution for risk illustrating the 95th percentile and
two different risk levels of concern (A and B). Assuming the 95th percentile corresponds to
the RME, the need for remedial action depends on how the RME risk compares with the risk
level of concern. For Case A (RME > level of concern), remedial action may be warranted.
For Case B (RME < level of concern), remedial action may be unnecessary.
Although the 95th percentile is recommended as a starting point for defining the RME in the
majority of human health risk assessments conducted within the Superfund program, the risk manager
may use discretion in selecting a different percentile within the RME range (90th to 99.9th percentiles). In
situations where the risk manager believes that a sufficient amount of site-specific information has been
collected to indicate that the risk estimates are much more likely to be high (e.g., overestimated due to
multiple health protective inputs), the risk manager may choose a lower percentile within the
recommended RME risk range (e.g., the 90th) as the most representative of the RME estimate at the site.
Conversely, when the risk manager believes that the risk estimates may tend to underestimate true risks,
or if there is substantial uncertainty in the accuracy of the risk estimates, the risk manager may choose a
percentile higher than the 95th in the recommended RME risk range (e.g., the 98th or the 99th). There are a
variety of factors that can be considered when making this decision, such as the qualitative and
quantitative uncertainty in the exposure assessment calculations, the uncertainty in the toxicity values,
and the presence of biological or measured data (in contrast to modeled data). These factors are discussed
below in Section 7.3. It is highly recommended that the risk manager consult with the site risk assessor
when applying these factors to determine an appropriate percentile in the RME risk range.
Page 7-5
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
7.2.4 RELATING THE RISK DISTRIBUTION TO THE RISK MANAGEMENT GOAL FOR ECOLOGICAL
RISK ASSESSMENT
For ecological risk assessments, the choice of the percentile of the variability distribution for
exposure or risk that will be protective depends on the receptor that is being considered as well as the
nature of the endpoint used to establish the level of concern. For most species, the risk management
objective will generally be to ensure population sustainability, even if some individual members of the
population (those at the upper end of the exposure or risk distribution) may experience a higher risk of
adverse effects. The risk management goal of population stability does not necessarily correspond to
protection of the central tendency receptor at or below the regulatory level of concern.
As indicated in Chapter 4, without knowledge of the proportion of the local population that must
survive and reproduce for the population to be stable, the choice of the central tendency exposure (CTE)
receptor as the basis of the risk management goal may not be protective. Sustainability of a local
population often depends upon the amount of "reserves" within that subpopulation to fill in ecological
niches left voided by lexicologically impaired individuals. At a very small number of sites, a population
biologist may be able to provide information about the level of effect associated with a decrease in
population sustainability. At the majority of sites, the use of the CTE receptor by risk management as the
basis for adequate protection of local populations of ecological receptors cannot be supported. Therefore,
in the absence of such species-specific (trophic level) information, it is prudent and appropriate to base
PRGs and cleanup levels on the upper end of the distribution of variability in the Hazard Quotient (HQ)
to provide greater confidence that the receptor population of concern will be protected.
For threatened or endangered species, it will normally be appropriate to provide protection to as
high a percentile of the distribution (i.e., the RME receptor) as is practicable (e.g., high-end of the RME
range of 90th to 99.9th percentiles), since injury to even a single individual is undesirable.
7.3 FACTORS TO CONSIDER IN CHOOSING THE PERCENTILE FOR THE RME
Risk assessments (both point estimate and PRA) should be based on the best quality data
available. A key component of the risk management process is a careful review and evaluation of the
potential limitations in the quality and relevance of the data that are used in the risk assessment (i.e.,
qualitative and quantitative uncertainties) in order to evaluate the strengths and weaknesses of the
assessment (U.S. EPA, 1993). Communication between risk managers, risk assessors, and other technical
team members is vital at this stage. The main question to be answered is, "How well do the inputs to the
risk assessment represent exposure pathways and behaviors at a given site?" The answer to this question
can be expressed qualitatively (e.g., high, medium, or low) or quantitatively (e.g., confidence intervals or
credible intervals). Some examples of these types of evaluation are illustrated below.
Use of Default Exposure Distributions
When site-specific data are not available, the best available information on some exposure
parameters most likely will be from studies at other sites (e.g., in other parts of the country). In both
point estimate risk assessment and PRA, the use of surrogate data to support input parameters raises
questions about representativeness for both current and future land use scenarios. A specific example of
potentially poor representativeness would be the use of national data for estimating the exposure
frequency of adult workers when the receptor of concern is a railroad worker. Railroad workers may
typically be on the site for only 100 days/year. If the risk assessment were based on the national default
assumption of 250 days/year, this choice would give a high bias to the risk estimate.
Page 7-6
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
Another example of a site-specific exposure factor that may vary considerably among different
locations is fish ingestion rates. At sites where ingestion offish contaminated with metals poses a
concern, tissue concentrations from fish fillets collected on site are often used to determine the
concentration term. However, a cultural practice of people harvesting fish on site may include consuming
some of the internal organs of the fish in addition to the fillets. If the metal contaminants selectively
accumulate in the internal organs instead of the fillet tissues, use of data only on fillets contaminants
would give a low bias to the risk estimate.
Other Factors that Influence Site-Specific Exposures
Exhibits 7-2 and 7-3 list other types of factors that may be important to consider when evaluating
the representativeness of an exposure or risk model. Given the source of the available data, the risk
assessor should identify potential uncertainties and discuss the likelihood that the values used may under-
or overestimate actual site-specific exposures. The risk manager should consider this information in
decision making throughout the tiered process for PRA (see Chapter 2).
EXHIBIT 7-2
EXAMPLES OF DEMOGRAPHIC, CULTURAL, AND BEHAVIORAL FACTORS THAT CAN AFFECT EXPOSURE
• Subsistence fishing, hunting, or ingestion of home-grown produce
• Exposures to cultural foods or medicines that contain contaminants
• Preparation of foods in containers that contain contaminants that may leach out into food or beverage
• Hobbies and other personal practices resulting in exposure to contaminants
• Age of the population (e.g., children may have greater exposure and susceptibility than adults (U.S.
EPA, 1995b, 1996)
EXHIBIT 7-3
EXAMPLES OF PHYSICAL OR GEOGRAPHICAL FACTORS THAT CAN AFFECT EXPOSURE
Geographical features that limit or enhance accessability (e.g., slopes, valleys, mountains)
Land use, including where exposure occurs within the exposure unit, and the current or future manner in
which the receptor contacts the contaminated media
Availability of contaminated medium for exposure (e.g., grass vs. bare soil)
Depth of contamination (e.g., surface soil is of greatest concern for direct contact)
Bioavailability of contaminant from media or water (e.g., physiochemical factors that enhance or reduce
absorption)
Water quality and distribution systems, including water hardness and use of lead-soldered pipes
Temporary barriers (e.g., fences, ground cover, and concrete) that affect current (but not necessarily
future) exposures
Page 7-7
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
For example, the features of a potentially exposed population and the physical and geographical
factors at a site can increase or decrease exposure to contaminated media. These factors should be
considered in defining exposure pathways and characterizing exposure variables in the risk assessment.
Such site-specific information may support a decision to evaluate the entire RME range (90th to
99.9th percentile) before selecting the percentile that represents RME risk. A departure from the
95th percentile would depend on whether or not qualitative or quantitative factors suggest an increased or
decreased exposure, and hence, risk. In practice, multiple and sometimes competing factors may need to
be balanced in order to determine an appropriate percentile for the RME risk (see hypothetical example in
Section 7.5).
Subpopulations may be at increased risk from chemical exposures due to increased sensitivity,
behavior patterns that result in high exposures, and/or current or past exposures from other sources.
Environmental health threats to children are a particular concern (U.S. EPA, 1995b, 1996). Once
identified, a subgroup can be treated as a population in itself, and characterized in the same way as the
larger population using similar descriptors for population and individual risk (U.S. EPA, 1995a). This
principle applies to both point estimate risk assessments and PRA.
Use of Biological Data
Biological monitoring data and/or other biomarker data can be useful sources of information for
evaluating uncertainty in an exposure or risk assessment. These data can provide an indication of the
magnitude of current or past exposures and the degree to which the exposures are correlated with
contaminated site media. Examples of biological data that are useful in human health assessments include
lead in blood, trichloroethylene and its metabolites in blood or urine, arsenic or methyl parathion
metabolites in urine, and poly chlorinated biphenyls (PCBs) or dioxins in blood or fat tissue. Tissue
burdens of contaminants are also widely useful as biomarkers of exposure in ecological risk assessments.
Just as air or groundwater monitoring data can provide increased (or decreased) confidence in the results
of predictive air or groundwater models, biomarkers can be used in a similar manner to evaluate how
much confidence should be placed in predictive exposure assessment models. Biological data can be
subject to the same shortcomings as other exposure data in terms of data quality and representativeness.
The design and performance of the biological data collection effort generally should be carefully
evaluated for these factors (e.g., low, medium, and high quality or confidence; low or high bias, etc.)
before using the results in the risk decision. Currently, collection of biological monitoring data is limited
at Superfund sites and requires coordination with appropriate agencies outside of EPA.
Issues Related to Toxicity Factors
A variety of factors may affect the magnitude of adverse responses expected to occur in similarly
exposed individuals such as age, physiological status, nutritional status, and genotype. In general, these
sources of inter-individual variability, and related uncertainties, are taken into account in the derivation of
toxicity values (e.g., reference concentration (RfC), reference dose (RfD), and carcinogenic slope factor
(CSF)) used in human health risk assessments. Thus, human health toxicity values usually are derived to
be health-protective for the most sensitive populations.
f Sources of variability or uncertainty are often accounted for in the derivation
of toxicity values. The level ofprotectiveness afforded by the toxicity value
may be an important factor in deciding on the appropriate RME risk
percentile to use.
Page 7-8
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
Risk managers, in collaboration with risk assessors, should carefully consider whether the
toxicity value is representative of the population of concern. For example, the toxicity value may be
based on oral exposures to drinking water, whereas exposure to a site population being evaluated may be
via soil ingestion. Similarly, the toxicity value could be based on effects in a healthy worker population,
whereas the site population encompasses all ages and a range of individual health conditions. Uncertainty
in toxicity values may reflect insufficient data to evaluate developmental toxicity concerns or to account
for in utero exposures. Also, it may be unclear whether the population of concern has similar
characteristics to the sensitive population accounted for in the derivation of the toxicity value. This
determination may require coordination with a toxicologist to review the basis for the derivation of the
toxicity values in question. Even then, in most cases, the determination will be very difficult, because our
understanding of human variability in toxicologic responses is very limited for many chemicals. When
data are insufficient to support a more quantitative representation of these sources of inter-individual
variability an uncertainty factor may be used in the derivation of non-cancer human health toxicity values
(RfD, RfC).
Some of the same factors that should be considered when employing toxicity values to estimate
risk are also relevant to the use of toxicokinetic and toxicodynamic modeling in risk assessment. For
example, a toxicity assessment for methylmercury used a technique called benchmark dose modeling
(BMD) to relate the levels in maternal blood to adverse developmental effects, based on data from a large
epidemiology study of Faroes Islanders (Grandjean et al., 1997; Budtz-J0rgensen et al, 2000). The RfD
determined is well-supported by the other large human studies from the Seychelles (Davidson et al., 1995,
1998) and New Zealand (Kjellstrom et al., 1986, 1989) as well as a physiologically-based
pharmacokinetic (PBPK) model based on the Seychelles data (Clewell et al., 1999). The RfD obtained
with benchmark dose modeling (BMD) was 1E-04 mg/kg-day. The PBPK model incorporated variability
in toxicokinetics to obtain a range of acceptable intakes of methylmercury between 1E-04 and 3E-04
mg/kg-day. Although the PBPK model was not used in the derivation of the benchmark dose value, it
was used to support the choice of uncertainty factors in the derivation of the RfD.
At the time this guidance was finalized, the understanding of this type of toxicity information
(i.e., human variability) was not well
developed. Although such information was
not used to characterize variability in human
health risks, the estimates of variability from
the PBPK model did provide additional
information on uncertainty. For decision
makers, the toxicity data and the choice of
the endpoint (e.g, neurodevelopmental
effects in the case of methylmercury) can
guide qualitative risk management choices
regarding the percentile representing the
RME (within the 90th to 99.9th percentile
range) and/or the appropriate level of
confidence in the RME estimate.
Exhibit 7-4 lists some of the issues to
consider when evaluating the uncertainty in
a toxicity value.
EXHIBIT 7-4
EXAMPLES OF TOXICITY CONSIDERATIONS
How severe is the effect?
Is the effect reversible?
How steep is the slope of the dose-response
curve at low dose?
Is the contaminant persistent in the
environment or in receptors?
Does the contaminant bioconcentrate as it
moves through the food chain?
How bioavailable is the contaminant?
Page 7-9
-------
RAGS Volume 3 Part A -
Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
Use of Quantitative Uncertainty Estimates
PRA methods such as a two-dimensional Monte Carlo analysis (2-D MCA) may be used to
quantify the uncertainty or confidence surrounding risk estimates, and this information may be helpful in
selecting the RME risk percentile. Figure 7-3 provides hypothetical results of a 2-D MCA where a
credible interval has been quantified for a 95th percentile of variability in noncancer HI. In exposure units
(EU) 1 and 3, the credible intervals for the 95th percentile are fairly narrow, which suggests a high degree
of confidence that the risks in EU1 are negligible and that the risks in EU3 are unacceptable. Conversely,
the relatively wide credible intervals in EU2 and EU4 give less confidence in the results, but suggest that
the 95th percentiles likely exceed a target HI of 1 in both cases. Further efforts to reduce or characterize
uncertainties may affect the risk management decision in these two areas.
5.0
~ 4.0
HH
a
•| 3.0
^
£ 2.0 H
0.0
95th%tile
75th%tile
50th%tile
"]j25th%tile
5th%tile
EU1
EU2
EU3
EU4
Exposure Units (EU)
Figure 7-3. Box and whisker plots characterizing uncertainty in the RME risk estimates (95th percentile of the
Hazard Index) at four locations. The box represents the inter-quartile range (25th to 75th percentiles) while the
whiskers represent the 90% credible interval (5th to 95th percentiles).
Page 7-10
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
Summary: Multiple Criteria Form the Basis of the Remedial Decision
Final risk management decisions should be based on a weighted consideration of all of the
relevant factors that influence confidence in the risk distribution. For example, a risk manager may be
presented with a risk assessment for a heavy metal in residential soil in which the distribution of cancer
risk estimates in the RME range (i.e., 90th to 99.9th percentiles) overlaps the risk range of concern
(1E-06 to 1E-04). The risk manager then should proceed with the site technical team to evaluate the data
available to define inputs for the risk assessment, as well as the site-specific factors, and the available
biological monitoring data. Assume that several factors that are likely to increase the confidence in the
risk estimates were noted: (1) the soil collection and analysis effort was well-designed; (2) the
predominant chemical and physical forms of the metal in the soil are characterized by relatively low
bioavailability; (3) all of the yards in the residential neighborhood are covered with grass lawns, a feature
generally expected to reduce direct exposure to soil; and (4) biomonitoring data from the site are all
within normal physiological ranges, suggesting little, if any, excess contaminant exposure occurred at the
site. In addition, generic national data were used in the absence of site-specific information on two input
variables that ranked highest in the sensitivity analysis, thereby reducing confidence in the risk estimates.
In this example, the consideration of these factors collectively suggests that the results of the risk
assessment are likely biased towards an overestimate of risk, and this information may be used in a risk
management selection of a percentile of the risk distribution to represent the RME receptor (e.g., less than
or equal to the 95th percentile).
7.4 UNCERTAINTY ASSOCIATED WITH THE USE OF THE 99.9TH PERCENTILE
As previously stated, this guidance adopts the 90th to 99.9th percentiles of the risk distribution as
the recommended RME risk range for decision-making purposes, consistent with EPA's Guidelines for
Exposure Assessment (U.S. EPA, 1992). A cautionary note should be added about the selection of the
higher percentiles within that range, especially the 99.9th percentile. The extreme percentiles ("tails") of
an input distribution are understandably the most uncertain part of a PDF, since the number of data values
in these ranges are less abundant than in the center of the range. This uncertainty in the tails of the input
distributions leads in turn to greater uncertainty in the tails of the calculated exposure or risk distribution,
and the magnitude of this uncertainty increases rapidly at the very high percentiles. In many cases,
estimates at the extreme tails, such as the 99.9th percentile, may be neither accurate nor plausible. For that
reason, great care should be taken when evaluating an RME risk in the upper percentiles of the risk range.
7.5 MOVING FROM A PRG To A REMEDIAL GOAL
As discussed above, where an unacceptable risk is identified, the risk assessor is typically asked
to develop site-specific PRGs (see Chapter 5 for discussion on derivation of PRGs). PRGs may be
developed using a probabilistic approach much in the same manner as they are developed using a point
estimate approach. The target risk level should be set for a specified percentile (corresponding to the
RME receptor), and the concentration in contaminated media which corresponds with that target risk level
should be calculated. It is important to understand that the PRG is an early step, not the last step, in the
selection of a final cleanup level. During the RI/FS, the risk manager should evaluate the remedial
alternatives using the nine criteria described in the NCP (U.S. EPA, 1990) (Chapter 1, Exhibit 1-2).
Achieving a target level of protection for human and/or ecological receptors is one of the primary factors,
but this objective should be balanced by criteria such as feasibility, permanence, state and community
acceptance, and cost. Indeed, there may be times when a purely risk-based PRG may be impracticable as
a final cleanup goal. In cases such as this, it is important to remember that the RME is not a single, fixed
percentile on the risk distribution, but instead represents the portion of the risk distribution curve between
Page 7-11
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
the 90th and 99.9th percentiles. Depending on the specific exposure and toxicity information available at a
site, a PRO developed using the 90th percentile of risk may be sufficient to protect the reasonably
maximum exposed individual. Alternatively, at some sites, the risk manager may feel that a PRG
developed using even the 95th percentile of risk is not sufficiently protective of the RME individual and
thus may choose to develop a PRG using a higher percentile.
*s° Selection of final remediation or cleanup levels during the RI/FS and ROD
may be an iterative process, and may consider a range of factors in addition
to the initial PRG estimate.
For example, at a former nuclear energy site, a PRG of 200 picocuries/gram (pCi/g) was
developed for plutonium in soil based on a one-dimensional Monte Carlo analysis (1-D MCA) and the
recommended starting point of the 95th percentile for the RME individual. At this particular site, the
surrounding communities were strongly opposed to this PRG as a cleanup level. They felt it was not
adequately protective, and as a result, limited progress occurred in remediating the site over the years.
The communities pointed out to the risk manager that many of the exposure assumptions used in the PRA
were not site-specific, and some members of the community felt that exposures occurred more often (i.e,
with higher frequency) and for a longer period of time (i.e., for a greater duration) than were assumed.
Based on the exposure parameters recommended by the community, the PRG would have been 75 pCi/g.
At this point, the risk manager could have chosen to either go back and collect sufficient site-specific
demographic and exposure data to refine the risk calculations and the PRG derivation, or evaluate the
feasibility of a PRG associated with higher percentiles on the risk distribution curve (e.g., 99th percentile).
In this particular example, the risk manager compared the costs associated with the cleanup that would be
required to satisfy the community concerns with the costs associated with collection of additional data
and recalculation of the risk and PRG. The risk manager decided that the additional cost of cleanup was
manageable and expected that the PRG based on the 99th percentile would be accepted by the community.
In addition, remedial activity could begin quickly without more investigation. When the risk manager
presented these findings to the community, the citizens quickly agreed with this approach and remediation
activities moved forward.
How does Variability and Uncertainty in Risk Relate to the Choice of a PRG?
An effective approach for communicating the results of a probabilistic analysis to risk managers
is to develop graphics that relate variability and uncertainty in risk to the choice of a PRG. Two graphics
are illustrated in Figures 7-4 and 7-5, based on the concept of iterative simulations presented in Chapter 5
(Section 5.5). Continuing the PRG example discussed above, assume that multiple 1-D MCA simulations
are run with PRGs for plutonium ranging from 25 pCi/g to 250 pCi/g in increments of 25 pCi/g. As the
concentration term is changed to correspond with a PRG, each Monte Carlo simulation yields a different
distribution of risk. Figure 7-4 focuses on the RME range of percentiles from the risk distribution (i.e.,
90th - 99.9th percentiles). A risk manager might use this graphic to evaluate how the PRG could change
based on the choice of the percentile used to represent the RME. A hypothetical risk level of concern of
1E-05 corresponds with the 90th percentile at a PRG of approximately 125 pCi/g, whereas 1E-05
intersects the 95th percentile line at a PRG of approximately 75 pCi/g. Therefore, when variability in risk
is the focus of the decision, the difference between an RME set at the 95th percentile instead of the 90th
percentile is 50 pCi/g.
Figure 7-5 presents information on uncertainty, rather than variability. This graphic could be
used to summarize results of a 2-D MCA (see Appendix D), or a series of 1-D MCA simulations (see
Page 7-12
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
Chapter 3, Section 3.4) applied to the same range of PRGs evaluated in Figure 7-4. In this case, the
results yield a 90% credible interval (CI) for the risk distribution. Figure 7-5 highlights the 90% CI for
the 95th percentile, assuming that a risk manager selects the 95th percentile to represent the RME risk, and
she is interested in the uncertainty in the risk estimates. Using the same hypothetical risk level of concern
(1E-05), the 90% upper CI for the 95th percentile corresponds with 1E-05 at a PRG of approximately 25
pCi/g. The risk manager may need to consider the cost and feasibility of achieving a PRG as low as 25
pCi/g. In addition, the 90% lower CI corresponds to a PRG of 250 pCi/g. The risk manager may
determine that this range of uncertainty (i.e., an order of magnitude) is too wide to set a PRG, and that
further steps are needed to reduce identify the major sources (i.e., sensitivity analysis).
Variations on Figures 7-4 and 7-5 can be developed to focus on different percentiles of the risk
range. This information, together with the results of the sensitivity analysis which highlights the major
sources of variability and uncertainty, should help to guide the selection of final remediation or cleanup
levels, or continued data collection and analysis following the tiered process for PRA.
Page 7-13
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
1 .OE-03
Variability: RME Range
{90th- 99.9th °/dles}
(0
1 .OE-04
1 .OE-05
1 .OE-06
yields 95th %ile = 1E-05
99.9th %iles
95th %iles
90th %iles
0 50 100 150 200 250 300 350
PRG (pCi/gram)
Figure 7-4. Example of graphic showing variability in risk (i.e., RME range, or 90th to
99.9th percentiles) associated with different choices of PRG for plutonium in soil (pCi/g).
The hypothetical risk level of concern (1E-05) corresponds to a 90th percentile risk at a a
PRG of ~ 100 pCi/g, and a 95th percentile at a PRG of ~ 75 pCi/g. In this example, all of
the 99.9th percentiles exceed 1E-05, leaving no choices for PRG at the high end of the
RME range.
1. OE-03
1. OE-04
.52 1. OE-05
1. OE-06
1.0E-07
Uncertainty: Credible Interval for 95th %ile
{90% LCI, 90% UCI}
90% UCI
95th %ile
90% LCI
PRG <= 25 pCi/g
yields 90% UCI <1E-05
0 50 100 150 200 250 300 350
PRG(pCi/gram)
Figure 7-5. Example of graphic showing uncertainty in 95th percentile risk associated with
the same choices of PRGs given in Figure 7-4. Uncertainty is given by the 90% upper and
lower credible interval (CI). The hypothetical risk level of concern (1E-05) corresponds with
the 90% upper CI at a PRG of ~ 25 pCi/g, and the 90% lower CI at a PRG of ~ 250 pCi/g.
Page 7-14
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
REFERENCES FOR CHAPTER 7
Budtz-J0rgensen, E., P. Grandjean, N. Keiding, et al. 2000. Benchmark Dose Calculations of
Methylmercury-AssociatedNeurobehavioral Deficits. Toxicol. Lett. 112-113:193-199.
Clewell, H.J., J.M. Gearhart, P.R. Gentry, et al. 1999. Evaluation of the Uncertainty in an Oral Reference
Dose for Methylmercury Due to Interindividual Variability in Pharmacokinetics. Risk Anal.
19:547-558.
Davidson, P., G. Myers, C. Cox, et al. 1995. Longitudinal Neurodevelopmental Study of Seychellois
Children Following in Utero Exposure to Methylmercury from Maternal Fish Ingestion:
Outcomes at 19 and 29 Months. NeuroToxicology 16:677-688.
Davidson, P.W., G.J. Myers, C. Cox, et al. 1998. Effects of Prenatal and Postnatal Methylmercury
Exposure from Fish Consumption on Neurodevelopment: Outcomes at 66 Months of Age in the
Seychelles Child Development Study. JAMA 280:701-707.
Grandjean, P., P. Weihe, R. White, et al. 1997. Cognitive Deficit in 7-year-old Children with Prenatal
Exposure to Methylmercury. Neurotoxicol. Teratol. 20:1-12.
Kjellstrom, T., P. Kennedy, S. Wallis, et al. 1986. Physical and Mental Development of Children with
Prenatal Exposure to Mercury from Fish. Stage 1: Preliminary Test at Age 4. Natl. Swed.
Environ. Protec. Bd, Rpt 3080 (Solna, Sweden).
Kjellstrom, T., P. Kennedy, S. Wallis, et al. 1989. Physical and Mental Development of Children with
Prenatal Exposure to Mercury from Fish. Stage 2: Interviews and psychological tests at age 6.
Natl. Swed. Environ. Prot. Bd., Rpt 3642 (Solna, Sweden).
Presidential/Congressional Commission on Risk Assessment and Risk Management. 1997. Risk
Assessment and Risk Management in Regulatory Decision Making. Final Report, Volume 2.
U.S. EPA. 1989. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health
Evaluation Manual (FIHEM) (Part A, Baseline Risk Assessment). Interim Final. Office of
Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40
CFR 300: 55 Federal Register, 8666-8865, March 8.
U.S. EPA. 1991a. Role of the Baseline Risk Assessment in Superfund Remedy Selection Decisions.
Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive
No. 9355.0-30.
U.S. EPA. 1991b. Risk Assessment Guidance for Superfund (RAGS), Volume I: Human Health Evaluation
Manual (HHEM), Part B, Development of Risk-Based Preliminary Remediation Goals. Office of
Emergency and Remedial Response, Washington, DC. EPA/540/R-92/003. NTIS PB92-963333.
U.S. EPA. 1992. Final Guidelines for Exposure Assessment. EPA/600/Z-92/001. 57 Federal Register,
22888-22938, May 29.
Page 7-15
-------
RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment
Chapter 7 ~ December 31, 2001
U.S. EPA. 1993. Data Quality Objectives Process for Superfimd. Office of Solid Waste and
Emergency Response. Washington, DC.
U.S. EPA. 1995a. Memorandum from Carol Browner on Risk Characterization. Office of the
Administrator. Washington, DC. February 22.
U.S. EPA. 1995b. Memorandum from Carol Browner on Policy on Evaluating Health Risks to Children.
Office of the Administrator. Washington, DC. October 20.
U.S. EPA. 1996. Memorandum from Carol Browner on EPA 's Report, Environmental Health Threats to
Children. Office of the Administrator. Washington, DC. September.
Page 7-16
------- |